In VR representations of real-world locations, we can walk all the way around objects! How is it possible to bring 3D real-world objects into the virtual world?
Seeing a real-world object from all angles in VR requires a special process called photogrammetry. While video game artists build virtual objects by sculpting geometric shapes, photogrammetry starts with cameras.
When you browse images of the front and back of the Brandenburg Gate, you don’t feel its depth or scale. For an immersive VR experience, then, just a few camera angles won’t do the job.
Photo by Alberto Capparelli
Photogrammetry requires taking hundreds (sometimes thousands!) of photos from every conceivable angle, sometimes also taking depth readings using technologies like LiDAR. Then, software fits the photos together to virtually reconstruct the object somewhat like an enormous jigsaw puzzle. Anyone can do this from a mobile phone or quadcopter/drone using software like Reality Capture. Higher-quality productions use specialized equipment that costs thousands of dollars. There are even setups where objects can sit on a platform surrounded by separate cameras that all snap at once. This instantaneous technique is faster than moving one camera around the object, and it can capture VR images of things that might move, such as animals.
In practice, once all of the images are captured and merged together, there is always work to clean up the result. VR designers may need to correct angles that weren’t captured quite right, weirdness from uneven lighting, and errors due to reflective surfaces. At this stage, designers can also add details like doors that open, rocks that can be picked up, or birds that fly by.
When the photogrammetry process is complete, you end up with a model. Like a model of a tree for a train set, the virtual model is an isolated object. The final step, then, is to place the modeled object in a setting. The setting could be a virtual structure like a museum building, or it could be a skybox: flat simulated ground with a sky that surrounds the object like a dome.
Once an object is photographed, it can take some time to analyze all the photos and do the number crunching, but the virtual results can be spectacular. Instead of seeing an artist’s interpretation of a scene, you can see the detail of every brick, board, or rock as it actually appears. Advanced photogrammetry is what makes the best VR experiences look so good.
For small static objects, there are handheld scanners that you can use to capture every angle by just moving around it. This is how a museum might capture sculptures or other physical objects. Products like Revopoint scanners are made to be hand-held and greatly simplify the process of going from object to 3D model. It’s much less work than large-scale photogrammetry and provides accurate depth without separate LiDaR equipment.
Now you know the basics of photogrammetry. Next time you’re walking around a historic building or a natural landmark in VR, you can imagine the work of the designers who made that experience possible… or at least understand why you see some glitches!
