A team at Microsoft Research wants to make your mobile phone a 3D scanner and modeling tool.
The Microsoft brain trust are working on a mobile app to let users – simply by moving their phones around an object – create a digital 3D-print-ready model and file.
"Only professional photographers – with professional equipment – can build 3-D content. The common user cannot. But with the popularity of the 3-D printer and 3-D games, the need for 3-D content is now emergent," Cai says. "If we can find an easy way for common users to create 3-D content by themselves, we can imagine that, in the next five years, everybody will be able to create 3-D content and the whole world may change."
Based on a client-through-cloud-to-client strategy, the Microsoft project begins with a user performing a fast scan using a mobile phone to circle the intended subject and collect image data.
"As long as you have an RGB camera, you can create a 3D image," Cai said. "The computational cost (of 3D scanning) is heavy, so we do the reconstruction in the cloud. After the computation is done, we transfer the data back to the user's device in around half a minute."
The most common methods of translating 2D images to 3D files uses image recognition decisions by "pose estimation." One problem? Noise left over from the large number of target objects involved. Calling it a "scalability bottleneck," the researchers are focusing on creating efficient 2D-to-3D through a correspondence filtering approach. That approach combines a light-weight, neighborhood-based step with a finer-grained pairwise step to remove spurious correspondences based on 2D to 3D geometric cues.
They say that with a filtered dataset of 300 3D objects, their solution shows a speed increase of up to 10 times over the baseline and a comparable recognition accuracy.
The researchers say a modular, client-focused solution will open up a world of possible applications for customizing products.
"If you have scanned somebody's face, you can print out a cup with the face, for yourself or as a gift for a friend," Gu said. "If you go to a furniture store and see something nice, you can use a mobile phone to scan the furniture and put it into your home environment to see if it would fit."
Intent on solving the problem of 3D object reconstruction and recognition, the research team is aiming at developing algorithms and systems to bring down the barrier of 3D reconstruction for everyday users. As they look to remove barriers to correspondences and make better recognition decisions as a way of eliminating unwelcome factors in RANSAC-based pose estimation, Microsoft says their solution is a likely way forward.