Intro
Exploring real-world locations is one of the things I love best about VR. Here at VR Voyaging, we spend a lot of time reviewing apps and experiences that transport you to incredible places. Traditionally (a fun word given how new VR is), scenes are meticulously captured from every angle and converted into meshes (think of wireframes) that are covered with textures (the color images from the original objects). This process, known as photogrammetry, creates one or more 3D objects (models) that approximate the original scene. While photogrammetry can produce impressive results, it has limitations, such as requiring a huge number of source photos and long processing times. It also struggles considerably with transparent/translucent and shiny/reflective surfaces, which can limit what you can scan and how realistic your scans look.
How do we make it more real?
Converting your scenes to meshes can provide high detail (and even let you make measurements from your models), but let’s do better. Another method, Gaussian splats use an entirely different method to represent and render a scene. This method requires fewer source images, shorter processing time, and can handle complex lighting and reflections. Like with photogrammetry, if you get too close, things generally look worse, and you may see weird artifacts in areas that weren’t captured well. However, when done right, Gaussian splats take things to the next level by recreating reality with much more realism. When the source data isn’t as high-quality though, the result tends to be dreamy, like a surreal haze or an impressionistic painting, rather than like stepping into a photo.
There’s also some great work on creating simpler meshes and wrapping them in animated textures. Imagine animating a mannequin and projecting a real person over it from all angles. If everything lines up, it’s going to look pretty realistic! This technique allows us to create fully 3D videos, also called volumetric video. It’s crazy to walk around one of these videos and watch someone talking or see a BMX rider from any angle in your kitchen!
Let’s Get Started!
And that’s what brings us here! Banished from the realm of pure research, here are some solid apps for exploring places, scenes, and even people in your headset with the magic of splats and volumetric video. Obviously, not every capture is amazing or of equal quality, but there are some incredible examples that really showcase what the technology is capable of. Step into an artist’s studio, walk around a futuristic cyberpunk exhibit, visit archaeological sites, and try yoga with an instructor captured from every direction and placed in your own space.
These apps are all free, and some even allow you to create your own captures to share. Give them a try and let us know what you think. We hope you enjoy these as much as we do!
Scaniverse
The first entry on this list is the only one without a dedicated app. Using WebXR technology, it runs in the browser so you can just visit the link and get started! Currently intended for Meta Quest headsets, you’ll need to visit the link from your Meta Browser. I’m not clear why they have this restriction, but I expect it will be expanded to other headsets in the future.
Scaniverse is unique on the list in that it consists of the VR-enabled web site, along with Android and iOS apps. Use the phone apps to capture your own scenes. Just walk around a room, or down a path, or otherwise in an area to capture lots of views (you can’t just stand in place and turn around). Once done, the app processes the images and creates an immersive scene. Once your scene is complete, you can upload it to the Scaniverse and share it with others!
I love that the main interface in VR is a globe (like how VR Voyaging has our VR world map!). Lots of pushpins designate where people have shared captures, so you can click on a pin, get a preview of the spot, then click the preview to enter. It’s easy to use and there’s some really nice places to see!
Downsides include not being able to upvote in VR (the button is grayed out), and no traditional teleport option. You can use your controllers to slide around or up and down, but there’s no concept of the ground. Quality is as good as the source material, so some people do a better job than others. See what fun places you can visit!
More details on our app page
Gracia
Gracia started out as a desktop app only. You could view a small set of captures from the company or browse to your own files to view. They stepped things up when they started adding much more content, and create a Meta Quest app. You still get higher quality from PCVR, but being able to render everything directly on the Quest is really impressive! There’s no dedicated app to capture scenes for Gracia, but if you use Polycam or Luma (other others), you can do captures and view them here. Note that currently only the desktop version lets you view your own content.
Not only does Gracia have some really great scenes, they also have some volumetric video. The quality can be quite astounding, and it really shows off the technology. You can watch a dancer from any angle as she spins around. It’s how I always imagined holograms would be.
More details on our app page
Meta Horizon Hyperscape Demo
Hyperscape is interesting in that it’s actually an official app by Meta. It currently contains five different scenes, most being different artists’ studios. The scale is correct, so it really feels like you’re standing in these rooms. You can look up, down, and all around and teleport or walk around to get better views. Certain items have hot spots you can activate to learn more about them, but mostly it’s just a way to feel like you can step inside these places. The app is free and looks like they might add additional scenes from time to time. These scenes are fairly big so they require a fast internet connection. I suspect they may be doing some sort of streaming since there isn’t the long delay of initial loading that I’d otherwise expect. It’s an impressive demo!
More details on our app page
Voluverse
Voluverse is an app published by the content studio Volucap. They’ve worked on some interesting projects including Matrix Resurrections, 1899 on Netflix, and a Roman forum VR experience. Voluverse is an app for Meta Quest with a small library of volumetric video clips including an exercise instructor, BMX bike riding, medical training, sports, and news. It’s somewhat of a demo reel for the company as they seek partners to distribute content on their platform. Their technology is quite impressive and it’s fun to check out the clips, even if the download sizes are pretty massive!
More details on our app page
Teleport by Varjo
A recent entry in the space, Teleport is a service for viewing high-quality scenes contributed by other users. Using Gaussian splatting, quality varies based on how well the capture was done and the angles you view it from but can look really good. Many scenes from Europe at the moment but will expand as more users sign up. Basic usage is free, but being able to contribute scans requires a fee and a special iPhone app. It will be interesting to see how much it expands given its rather high cost (~$30/month), but it’s clearly geared toward larger organizations in general despite their broad pitch toward general consumers to view. Companies that pay for scans may or may not actually decide to make the end result public, so perhaps it’s all more about demonstrating the platform for paying customers. Currently you can only view in 3D within the browser with a custom Windows app being required for VR use (requiring PCVR or Meta Quest with Link). Despite these limitations, it’s nice to have another service for viewing these real-world scenes from your headset.
More details on our app page
Closing
For now, many apps and web sites will continue to focus on photogrammetry since it’s something well understood and well optimized. Modern GPU’s (graphics cards) can render them efficiently, even in VR, as long as the models aren’t too complex. Still, it’s exciting to see more apps start to focus on Gaussian splats since they are faster to process and can be more realistic. Between splats and advances in various types of volumetric video, VR is getting more and more realistic all the time!
