Google I/O 2024: Android to Get Support for Scam Call Detection, Circle to Search for Homework using On-Device Gemini AI

Google I/O kicked off on Tuesday, the first day of the company’s annual developer conference. During the keynote event, the company’s executives also took the wraps off noteworthy new features coming to Android smartphones over the coming months — strangle enough, the company didn’t unveil Android 15 features during the keynote event. These announcements have come ahead of the company’s next major smartphone operating update that is expected to arrive in the second half of 2024. As expected, Google is set to introduce new features powered by artificial intelligence (AI), building on the path it took with Android 14 last year.

Dave Burke, VP of Engineering at Google highlighted new AI-powered features coming to Android smartphones in the coming months, at the Google I/O keynote event. One of the most impressive features showcased is an upgrade to Circle to Search — the company’s visual lookup feature currently exclusive to select Samsung Galaxy and Google Pixel phones — that will allow students to ask for help with their homework. Google says that its LearnLM technology will allow users to circle a prompt to solve “problems involving symbolic formulas, diagrams, graphs and more” — the feature is expected to arrive later this year.

Gemini, the company’s AI model that can also run on Android smartphones, is also getting upgrades. The company says that Gemini on Android will soon be able to provide information about YouTube videos, while AI generated images can be quickly added to Gmail and Messages. Meanwhile, users will be able to utilise Gemini Advanced to get answers from PDF documents — without poring through multiple pages — with a new “Ask this PDF” option, according to the company.

Google Pixel smartphones will also gain support for Gemini Nano with Multimodality, the company’s latest model for on-device AI processing. In addition to the existing ability to process text input, Pixel phones will be able to process contextual information like visuals, audio, and even spoken information. 

AI is also making its way to Google’s most basic smartphone app — the dialler. In a demo shown off at the Google I/O 2024 keynote event, the company demonstrated the ability to detect a potential scam call asking a user to transfer bank information or reveal banking information over the phone. The feature was shown to work in real-time and works on-device to protect user privacy. There’s no word from Google on when these features will make their way to users, or which (presumably Pixel) smartphones will be supported.

Google TalkBack, the accessibility feature on Android smartphones will further enhanced with Gemini Nano’s multimodal features, according to the company. The service will enable visually impaired users to get more detailed information about images, by filling in information about unlabelled images. Google says that these new features will work on-device, which means that users will be able to take advantage of the improved functionality even when they don’t have internet access. There’s no concrete launch date for the improved Google TalkBack service powered by Gemini Nano with Multimodality, but the company says it is arriving “later this year”.