Do you need to create digital twin of a real object and you don't know how to do it? There are several options. You can install a 3D design softare and start modeling. The smarter way is to use a 3D scanner, but it is quite costly and moreover not every object may be scanned either for its dimension or transportability. Fortunately, there is an easy and cheap way to create 3D digital twins of real physical objects - photogrammetry.
- What is photogrammetry?
- Why do photogrammetry in MeshRoom?
- What camera to use and with what settings?
- What objects, when and how to take pictures of them?
- Standard, augmented or live reconstruction - which one to choose?
- How to set individual reconstruction options?
What is photogrammetry?
Photogrammetry is a field of science dealing with the reconstruction of shapes, measuring dimensions and determining the position of objects from photographic images. There are more types of photogrammetry, for the purpose of 3D scan we usually talk about multi-image photogrammetry. Its purpose is to get a digital twin of a real object captured on multiple photographs from all needed angles.
Photogrammetry can be performed using a variety of paid or free software. The software selection is quite important and decides about time required for the process of photogrammetry and the final quality of the 3D scan. Most users have been used to use free software such as Colmap, 3DF Zephyr (limited by maximum of 50 source photos in the free version) or Visual SFM for common scans with photogrammetry, today, our favorite is the MeshRoom software.
Why do photogrammetry in MeshRoom?
Meshroom is free open-source software (source code is available on Github) with simple and intuitive user interface. This software uses photogrammetric computer vision framework AliceVision - a product of cooperation of multiple universities, labs and French Post-Production company Mikros Image. See more information on AliceVision website.
Generating photogrammetry models in Meshroom is very easy, literally consists of marking the source photos, dragging them into the software window, clicking on the start / compute option and waiting for several hours during which the software recognizes objects on the photos and convert them into a high-quality 3D digital model. In addition to this standard reconstruction mode, Meshroom allows 2 more reconstruction options - augmented and live - both designed to bring options to increase time and computational efficiency of processing and quality of the final 3D model.
Meshroom software is available for both Windows and OSX / Linux operating systems that use nVidia graphics card (AliceVision requires CUDA library support). AMD users are forced to either modify source codes to remove dependency on CUDA or use an alternative software for photogrammetry. Installation instructions, as well as installation files for Meshroom are available on the software page.
What camera to use for photogrammetry and with what settings?
Photographing photos for photogrammetry can be performed using any camera, including a video camera, with the subsequent division of individual frames into separate images. It is advisable to use photos of the highest possible quality, because the quality of the resulting model depends on the quality of the photos.
A mobile phone camera is usually sufficient for most applications. We do not recommend the use of a camera with the subsequent use of individual frames, the resulting sharpness is significantly worse compared to the method with the use of photographs. It is optimal to use an DSLR camera with an aperture to at least 5-6. Ideally if you can set shooting into the RAW file format.
What objects, when and how to take pictures of them?
Photogrammetry can form the digital twin of any physical 3D object. The problem is usually with glossy and transparent surfaces. Such a surface needs to be treated before photography - either by applying paint or at least adhesive tape to prevent glare. The ideal surface for photogrammetry is a rough with a texture, this surface is for example on most statues.
If you take pictures of an object located outside, it is advisable to photograph in constant weather and light, without sharp shadows. Ideally when it's under a cloud.
During normal shooting outside the laboratory designed for this purpose, never move or rotate with the "scanned" object. Instead, move the camera around the object.
Minimum number of taken photos for photogrammetry is dependent on the object, usually it is not lower than middle tens. See following rules based on which you will find out how taking pictures work and how many photos you will probably must to make for the complete scan of the object.
- It's necessary to photograph all the parts that have to be included in the digital model
- Each additional photo must contain at least 60-80% overlay of the previous photo
- The scanned object must occupy a substantial portion of each image
- Photogrammetry software may not accept all uploaded photos (for example, if it is not able to recognize what part is on the image).
- Meshroom is capable to process photographs taken with different settings - zoom / sharpness / different camera etc. Despite this, try keep most photographs taken with same settings.
Once you have photographs to be used as source data and installed Meshroom software, it's finally time to start photogrammetry.
Standard, augmented or live reconstruction - which one to choose?
- Standard reconstruction - The standard reconstruction option is suitable in cases when you have a set of photographs without the possibility of taking and adding more photographs anytime later (e.g. you did the shooting on a trip so returning back to take more photos of the object is complicated)
- Augmented reconstruction - You have set of photographs of the "scanned" object, the object is still located on the same place, at the same environment and you can anytime return back to take more photos. The benefit of augmented reconstruction is that once you see that some place on the rendered digital model does not seem to have sufficient quality (either due to some rejected pictures by the software, bad quality of some pictures, or any human mistake), you can simply move back to make more pictures of the required place of the object and add these new photos among others to the software. Meshroom compares these additionally uploaded images with all previous images for all new pairs with other pictures (which may result in acceptation of previously rejected images in the process of reconstruction) as well as include newly added pictures between sources for generation of the model.
- Live reconstruction - Live reconstruction is gradual model creation driven by photographs placed in the source folder. This option is beneficial if you are not sure how many photographs is needed for the creation of the quality enough model - you simply start with e.g. 10-20 first photographs covering general shape of the scanned object and then adding additional smaller groups (e.g. 5 photographs) capturing detail of a certain part of the model. This mode is fun and looks cool, if you wish to enjoy some photogrammetric fun, do not hesitate to try it.
How to set individual reconstruction options?
- Standard reconstruction
- Ensure that all photos intended for the photogrammetry are in 1 folder on the computer disk
- Select all photos and move to the Meshroom window
- Save the project. Selected project folder will be destination of files associated with the final generated 3D digital twin of the object from photographs
- In the bottom bar, navigate to the "StructureFromMotion" node, make right mouse button click on it and select "compute". The calculation takes a few hours. You can interrupt it at any time by pressing the stop button. After completing the calculation, you will see the final model and information which photos were used for the creation. The 3D model files themselves are in the Texturing and MeshFiltering folders on the computer disk.
- If it happens that the resulting 3d model was created using only a few photos (most were rejected by the software), it is necessary to add either more photos, or try to adjust the reconstruction setting.
- Augmented reconstruction
While using augmented reconstruction, we recommend to work always only with partial calculations of a certain block which is more time optimal then running calculation above whole model. Which block should be recalculated you simply decide in the bottom bar by running "calculate" option above the block. Complete calculation (launched by "calculate" option from lowest texturing level) then is optimal to run for the final calculation process.
The selection of a live reconstruction requires initial setup, this is done as follows:
- In the tab "View", choose option "Live reconstruction"
- Select the source folder (where you plan to upload photos) and the minimum number to process in each step (e.g. 5)
- In the Live reconstruction Panel, click the "Start" button.
- Copy the first set of photos to the source folder (ideally 10-20 photos focused on the general shape of the object)
- Modify the interconnection of nodes - it is necessary to replace the interconnection of the "PrepareDenseScene" node with the "StructureFromMotion" node, which is located twice in the given bar (in the middle and at the end). In a live reconstruction, it is necessary to connect the "PrepareDenseScene" node to the last "StructureFromMotion" node instead of the middle one. This is done by removing the existing binding and then creating a new one.
- Start the calculation of model generation by selecting "compute" from the "Texturing" node.