Skip to main content

Google is bringing AI superpowers to your smartphone camera, and it looks ridiculous

Google Astra on a phone.
Google
Sundar Pichai stands in front of a Google logo at Google I/O 2021.
This story is part of our complete Google I/O coverage

It seems AI assistants are antique, or that’s what Google wants you to believe, for we are in the era of AI agents — and Google I/O 2024 has quickly proven that. Say hello to Project Astra, a generative AI agent with vision, text, and speech capabilities, with a sprinkling of memory and spatial awareness capabilities in tow.

Think of it as eyes for your phone that can make sense of the world around you. Point it at a mathematical equation, and it will solve it. Pointing the camera at a cat? Astra will suggest an apt name for the feline meow-ster. Ask it where you left your earbuds, and if the camera sensor has seen them, it will say something like, “You left them on the sofa.”

Recommended Videos

Astra can make sense of code appearing on a screen, identify objects and explain what they do, identify buildings, and more. Think of it as Google Lens but for the entire world, and can make sense of most anything in front of the camera’s lens.

Unlike Google Assistant, you don’t need to prompt it. Just point the camera at anything, utter the audio query, and Astra will explain in a natural language response. Google says Project Astra will be rolled out via the Gemini app later this year.

An all-seeing AI agent?

Project Astra demonstration on a phone.
Google

If Project Astra sounds familiar, that’s because OpenAI demoed a similar feature for ChatGPT — powered by the new GPT-4o model — just a day ago. OpenAI’s tool is currently under the red-teaming phase for safety testing and will be released in a phased manner, starting with ChatGPT Plus subscribers.

Google won’t say whether Astra will have a price tag. But given the near spontaneity of responses, the kind of visual data crunching involved, and the generative chops required to offer a meaningful response, it’s unlikely Astra will be served as a free perk owing to the compute requirements.

The best example would be the Google One AI Premium subscription, which already hides some of the new Gemini-powered experiences behind a paywall. But so far, Astra looks like the most amazing AI innovation Google has showcased ever since it wowed the world with Duplex’s capabilities a few years ago.

Nadeem Sarwar
Nadeem is a tech journalist who started reading about cool smartphone tech out of curiosity and soon started writing…
The entire Google Pixel 9 series just leaked. Here’s what the phones look like
NCC photo of Pixel 9 Pro Fold.

While Google’s August 2024 event isn’t until next month, we may already have a good look at what’s to come with the Google Pixel 9 series. That’s because the entire Google Pixel 9 lineup has just leaked through the NCC regulatory agency (the Taiwanese version of the FCC) with some revealing photos of all models, including the Pixel 9 Pro Fold, and other important details, as found by Android Authority.

This isn’t the first time that the Pixel 9 phones have leaked, but this could be the most informative one yet. Along with the photos, we have information on the battery sizes, new charging speeds, and more.

Read more
The Google app on your iPhone just got a hidden new feature
iPhone display showing Google image in black on white

Here's some exciting news for users of the Google Search app on iPhone and iPad. According to 9to5Google, you can now personalize the home screen icon of the Google Search app to reflect your mood better.

The traditional Google app icon features red, yellow, blue, and green on a white background. However, you can now customize it to have a dark background or opt for a black icon on a white background or a white icon on a black background.

Read more
Your Google Chrome app is getting 5 helpful new features
An Android phone showing Android 15.

The Google Chrome app for iOS and Android is introducing numerous new features to make searching in Chrome easier than ever. These features are either going live now with the latest Google Chrome app update or launching on the iPhone 15, Galaxy S24, and other devices later this year.

Google is introducing additional Chrome Actions to help you find local places. When searching for a restaurant, you will see shortcut buttons in the search results that allow you to call the restaurant, get directions, and access reviews. Android users can access this new feature in Chrome now, while iOS users must wait for a fall update.

Read more