I have been experimenting with 123D Catch and VisualSFM to try to see if I can manage to reliably create a mesh by photographing an object on a rotating platform. As shown in this earlier post, I have had a few very successful experiences. I have also had many failures. Reading other peoples posts, I see that I am not alone in my frustration. A colleague suggested that a better way to test these programs would be to use Blender to generate input images. This allows me to exactly control camera placement, arbitrarily choose an object shape and texture, and have no background in any image. After some fairly significant effort, I was able to create a series of images using Blender. “Render Animation” produces an image sequence if the output selected is an image rather than a video format. I made two cameras, each with a co-located point source light. I textured the object (a monkey head) with a two color pattern. Success of 123D catch depends on the skin pattern, light placement, camera pointing, and surface finish options specular and diffuse. I also made the object faceted instead of smooth.
I use an animation sequence in Blender where the object rotates between frames. I have used this previously to make videos of rotating meshes. This time I used it to generate image sequences.
Here is an example rendering used for this experiment. I made 80 of these images covering 360 degrees with one camera above the center line of the object and one below. The key point is that with no background, two fixed cameras with two fixed light sources and a rotating object is the same as a fixed target and two cameras and lights revolving around it to take pictures. There can be no appreciable background in any picture, and there can only be light sources co-located with the cameras. That’s probably why using the flash on my digital SLR worked well previously.
I submitted 80 images to Autodesk 123D Catch with the following result.
The program properly identified both sets of cameras. With only one camera height, some parts of the mesh were missing in the final result.
I also ran these files through the open source program VisualSFM. It also identified all of the cameras, but made two separate point clouds. Here is the result.
After running the CMVS tool to generate a higher density mesh, I exported the result to a PLY file and opened it in Meshlab. Here is the result.
The quality is clearly below that of 123D catch, and I would also need to try to merge these two meshes. This is a process I have not had much luck trying so far.
Here is the exported OBJ mesh from 123D catch opened in Blender. It is a pretty good representation of the original.
And here is the rendered result.
This gives me hope that If I manage to duplicate the virtual camera and object rotation that I had in the Blender simulation, I might be able to also make it work in real life with real objects. I hope to show that success in a forthcoming post.
Smooth, monochromatic objects will likely fail to produce meshes using photogrammetry. Unfortunately, we want to copy small parts, and those are usually smooth and monochromatic… I might experiment with IR photography for some objects to see if it brings out features that can’t be seen with visible light. I can also put reference markers on objects with small bits of tape.
If I make this work with a person’s face, I will need to produce a structure they can lean their head against to keep them steady. I don’t think I will try to make a full head scan using this technique. A person just can’t be expected to hold still for that long.