Forms: Flexible Object Reconstruction from Multiple Silhouettes

Thomas J. Cashman and Andrew W. Fitzgibbon

This MATLAB code gives the implementation for our paper 'What Shape are Dolphins? Building 3D Morphable Models from 2D Images'. You will need the MATLAB Optimization Toolbox to run it.

To try the code on one of our datasets, load a Forms 'project', and then use the function 'forms' to run our optimization. For example:
>> bananas = read_project('.\projects\bananas.fpj');
>> banana_model = forms(bananas, 2);

You can then compare the fit of the model before and after optimization, by using
>> plot_modelfit(bananas, 1);                % Show banana 1 before optimization
>> plot_modelfit(bananas, 1, banana_model);  % Show banana 1 after optimization

Or take a look at the basis shapes plotted as a colour map by using
>> plot_basisshapenorm(bananas, banana_model, 1);  % First basis shape norm
>> plot_basisshapenorm(bananas, banana_model, 2);  % Second basis shape norm


To reproduce our results, use the weights and normal noise estimates described in the paper, and use the function 'align_project' to find the camera translation and scale parameters automatically, i.e.
>> bananas = read_project('.\projects\bananas.fpj');
>> bananas = align_project(bananas);
>> banana_model = forms(bananas, 2, 0.5, 0.25, 4);

>> pigeons = read_project('.\projects\pigeons.fpj');
>> pigeons = align_project(pigeons);
>> pigeon_model = forms(pigeons, 7, 0.25, 0.05, 5);

>> bears = read_project('.\projects\bears.fpj');
>> bears = align_project(bears);
>> bear_model = forms(bears, 10, 0.25, 0.25, 4);

Note that you should expect some of these optimizations to take a long time. This is particularly true for the dolphins project, which can be used to build a a model from 32 dolphin instances:
>> dolphins = read_project('.\projects\dolphins.fpj');
>> dolphins = align_project(dolphins);
>> dolphin_model = forms(dolphins, 8, 0.5, 0.25, 10);

The dolphin dataset is available here with manual segmentations, rather than the grab cut silhouettes from Powerpoint 2010. This is the reason we suggest sigma_norm = 10 above, rather than the value 40 / 3 that appears in the paper: the manual segmentations have more reliable normals, so we can use this slightly lower noise estimate. However, there is no significant difference between the manual and automatic segmentations; the results are very similar.

Last edited Jan 9, 2012 at 12:40 PM by TomCashman, version 2

Comments

No comments yet.