Naturally, you can work in a mixed-reality environment with a connected Bluetooth keyboard and mouse, and if you want to focus you can place yourself in an immersive environment, or sure You can leave see-through mode on to make sure your coworkers aren’t there. Taking photos and laughing while you wear a ridiculous headset to get things done. It wasn’t clear whether you’ll be able to connect the headset to a laptop to bring your work into mixed reality, a feature available on the Apple Vision Pro.
Gemini in XR
A tap on the side of the headset brings up an app launcher, and this is where you can toggle Gemini on if you want it to be constantly “on.” Once it’s turned on, there’s an icon at the top of the virtual space so you know everything you say and see is being registered by Gemini.
In see-through mode, you can walk up to an object and ask Gemini about it—a Googler demoing the headset (before I tried it) walked up to another wearing an FC Barcelona shirt and Gemini asked the team to find “its position.” Gemini quickly entered the team name and pulled up the search results along with league standings and recent match scores.
You can ask Gemini anything like this and it will respond with visual results displayed in the headset. I asked it to “take me to Peru,” and it opened a 3D version of Google Maps. I was able to move around and focus on Lima, and in cities where the map already has a lot of 3D models, you can explore areas in more detail. You can continue to talk to Gemini in these experiences, so I asked questions like when would be the best time to meet and got an immediate response.
In another example, I peeked inside a restaurant in New York City to take a virtual tour of the space. Google says it can use AI to gather images of the interior of a place and display it so it feels like you’re there. It did a great job, and I asked Gemini if the place took reservations, without specifically mentioning the name, because I was looking at the name of the restaurant. This does Take reservations, but Gemini couldn’t really make one for me. (This integration may come later.)
Next, I watched some videos on YouTube, where 2D content looks sharp and colorful. Stereoscopic content was even better; My senses felt overwhelmed. I saw some hikers walking along the trail and asked Gemini where it all was, and he said, “New Zealand.” I wasn’t able to confirm this, but it seemed like the right answer. I noticed some more spatial playback of 2D videos because Virtual Player added depth and layering to make them feel 3D. I turned on the Google TV app and enabled a “cinema mode,” which launches a virtual theater for watching movies and shows on other VR headsets.
Circle to exploreThe feature was launched by Google earlier this year On Android phonesAlso available in Android XR. Just walk up to a physical object near you, press the top button on the headset, and then circle and drag around the object you want to learn more about. You will get a Google search page with results.
Smart glasses
Project Mohan feels a lot like Google and Samsung catching up to the rest of the VR market, though the Gemini integration gives their efforts a unique layer. However, I will admit that I was much more excited to try the smart glasses, where the Gemini seems to be even more helpful. They did not disappoint. I went into another room and there were several pairs of glasses in front of me. Some had sunglasses, others had clear lenses. Like the headset, you can load them with your prescription. Google did not provide a name for the prototype glass.
