Google Lens Gets a Video Search Boost: Now You Can Ask Questions About What You See

Google Lens, the visual search tool, has just gotten a significant upgrade, expanding its capabilities beyond images and text to encompass video searches. This innovative feature allows users to capture a 20-second video clip and then ask questions about it using voice input, opening up a whole new realm of possibilities for visual information retrieval.

How Does Video Search Work?

The process is straightforward and intuitive. Users simply need to open the Google Lens app on their iOS or Android device, press and hold the "Search" button to begin recording, and then speak their question aloud. For example, while watching a video of fish swimming together, a user might ask, "Why are those fish swimming together?" Google Lens, powered by Gemini AI, will then analyze the video and provide a summary of relevant information from various websites, along with links for further exploration.

The remarkable aspect of this new functionality lies in Google Lens's ability to decipher a sequence of images within a video, effectively understanding the context of the visual information. Gemini, a specially designed AI model, processes these frames in order, extracting insights from the web to provide a comprehensive response. This advancement represents a significant leap forward in visual search capabilities.

Enhanced Shopping Experience

The "Circle to Search" feature within Google Lens has also been enhanced, now offering not only image recognition but also access to product reviews, prices, availability, current deals, and relevant shopping ads. This integration creates a seamless shopping experience, empowering users to make informed decisions with greater ease.

Key Features and Considerations

It's important to note that while Google Lens video search currently supports English only, it's still under development and accessible through Search Labs. This means users can sign up for Search Labs to participate in testing and gain early access to this innovative feature. Google is actively refining its voice recognition technology, though bird calls still pose a challenge for identification. To ensure user privacy, recorded videos are deleted after analysis by Gemini and will not be used for training the model in the future.

This latest development underscores Google's commitment to pushing the boundaries of visual search technology. By enabling users to query videos, Google Lens opens up a world of possibilities for exploring and understanding the visual world around us. Whether you're curious about a natural phenomenon, seeking information about a product, or simply exploring the world in a new way, Google Lens is poised to become an indispensable tool for both learning and entertainment.