Kerry Wan/ZDNET
Google’s I/O 2024 keynote just wrapped up, and the company spent basically the entire time talking about AI (as to be expected). During the keynote, the company detailed how artificial intelligence would integrate with Android moving forward, with Gemini stepping into the same virtual assistant role as the Google Assistant but in a more integrated, contextual way.
Also: Everything announced at Google I/O 2024: Gemini, Search, Project Astra, and more
After releasing the Gemini app back in February, Google has begun fleshing out the experience of Gemini on Android with a suite of new features that infuse AI into more aspects of your everyday life.
Google tried pitching these changes as a ground-up rethinking of Android, but at least for now, they’re more complimentary to the larger Android experience.
Gemini itself is quite literally that. Its design is being tweaked so that it floats on top of whatever you’re doing, rather than take up your entire screen like it does now. This is the same way Google Assistant used to present itself. Only this time, you get a huge text field to type your prompts into with a lessened focus on voice prompts.
Kerry Wan/ZDNET
The new overlay is meant to represent deeper integration with whatever app you’re using and give you contextualized controls. One example was when watching a video on YouTube; pull up Gemini and you’ll see a button that says “Ask this video,” allowing you to use the knowledge base of the video to ask questions or summarize it. You can also do it with PDFs, so long as you subscribe to Gemini Advanced, with its longer context window.
Also: 3 new Gemini Advanced features unveiled at Google I/O 2024
Gemini will also flow more easily within apps, one example being drag-and-drop. During the keynote, Google demoed how you could ask the chatbot to generate an image, and once they’re ready, you could drag it into a messaging app, drop it, and send it to your friend.
Over time, Google says Gemini will become more contextually aware of each app on your phone and make it easier to navigate them with Dynamic Suggestions.
Circle to Search, which is already available on over 100 million Android devices, is getting upgraded to help with problem-solving. Specifically, it’ll be able to help students with their homework by helping them better understand complex physics and math word problems they’re stuck on. You’ll get a detailed breakdown of how to work through the problem and never need to touch your digital info sheet or syllabus.
Also: Google just teased AR smart glasses, and you can already see how the software works
The new ability is powered by Google’s LearnLM model, which aims to make learning easier with AI. Over time, Google says you’ll be able to use Circle to Search to solve more complex problems involving things like symbolic formulas, diagrams, and graphs.
Circle to Search is already available across major smartphones, including the Samsung Galaxy S24.
Kerry Wan/ZDNET
Google also announced that Gemini Nano, the model built directly into Android (albeit on very few devices), will receive an upgrade called “Gemini Nano with Multimodality.” The updated LLM will allow you to use various media inputs such as text, photos, videos, and voice to interact with Gemini and get answers to questions, information on queries, and more.
Also: Meet Gemini AI Live: Like FaceTiming with a friend who knows everything
The model will power features like TalkBack for text descriptions of images and real-time spam notifications during phone calls (which is helpful if, for some reason, you believe the person calling from the unknown number is actually the prince of Egypt and you must wire him one million dollars).
These are just some of the AI goodies that Google will be bringing to Android 15 and beyond over time. Some of them will launch first on Pixel, while others will be available for anyone who downloads the Gemini app. How exactly it’ll all shake out remains to be seen, but it’s clear that if you own and use an Android phone, it’s about to get a whole lot more powerful.