Google I/O 2024: This is all the pieces Google simply introduced

Tuesday’s Google I/O ran for 110 minutes, however Google managed to reference AI a whopping 121 instances throughout (by its personal depend). CEO Sundar Pichai referenced the determine to wrap up the presentation, cheekily stating that the corporate was doing the “arduous work” of counting for us. Once more, it was no shock, we have been prepared for it. Learn extra

Generative AI for studying

Additionally at present, Google unveiled LearnLM, a brand new household of generative AI fashions “fine-tuned” for studying. It’s a collaboration between Google’s DeepMind AI analysis division and Google Analysis. LearnLM fashions are designed to “conversationally” tutor college students on a variety of topics, Google says.

Although it’s already out there on a number of of Google’s platforms, the corporate is taking LearnLM by means of a pilot program in Google Classroom. It is usually working with educators to see how LearnLM would possibly simplify and enhance the method of lesson planning. LearnLM may assist academics uncover new concepts, content material and actions, Google says, or discover supplies tailor-made to the wants of particular scholar cohorts. Learn extra

Quiz grasp

Talking of training, new to YouTube are AI-generated quizzes. This new conversational AI device permits customers to figuratively “elevate their” hand when watching academic movies. Viewers can ask clarifying questions, get useful explanations or take a quiz on the subject material. 

That is going to be some aid for many who have to observe longer academic movies, comparable to lectures or seminars, resulting from Gemini mannequin’s long-context capabilities. These new options are rolling out to pick Android customers within the U.S. Learn extra

Gemma 2 updates

One of many prime requests Google heard from builders is for an even bigger Gemma mannequin, so Google can be including a brand new 27-billion-parameter mannequin to Gemma 2. This subsequent era of Google’s Gemma fashions will launch in June. This measurement is optimized by Nvidia to run on next-generation GPU and may run effectively on a single TPU host and vertex AI, Google stated. Learn extra

Google Play

Google Play is getting some consideration with a brand new discovery characteristic for apps, new methods to amass customers, updates to Play Factors and different enhancements to developer-facing instruments just like the Google Play SDK Console and Play Integrity API, amongst different issues.

Of explicit curiosity to builders is one thing referred to as the Have interaction SDK, which is able to introduce a means for app makers to showcase their content material to customers in a full-screen, immersive expertise that’s customized to the person person. Google says this isn’t a floor that customers can see presently, nonetheless. Learn extra

Detecting scams throughout calls

Tuesday, Google previewed a characteristic it believes will alert customers to potential scams in the course of the name. 

The characteristic, which can be constructed right into a future model of Android, makes use of Gemini Nano, the smallest model of Google’s generative AI providing, which will be run solely on-device. The system successfully listens for “dialog patterns generally related to scams” in actual time. 

Google provides the instance of somebody pretending to be a “financial institution consultant.” Frequent scammer techniques like password requests and reward playing cards may even set off the system. These are all fairly effectively understood to be methods of extracting your cash from you, however loads of individuals on the earth are nonetheless susceptible to those types of scams. As soon as set off, it’ll pop up a notification that the person could also be falling prey to unsavory characters. Learn extra

Ask Images

Google Images is getting an AI infusion with the launch of an experimental characteristic, Ask Images, powered by Google’s Gemini AI mannequin. The brand new addition, which rolls out later this summer season, will permit customers to look throughout their Google Images assortment utilizing pure language queries that leverage an AI’s understanding of their photograph’s content material and different metadata.

Whereas earlier than customers may seek for particular individuals, locations, or issues of their images, due to pure language processing, the AI improve will make discovering the precise content material extra intuitive and fewer of a guide search course of.

And the instance was cute, too. Who doesn’t love a tiger stuffed animal/Golden Retriever band duo referred to as “Golden Stripes?” Learn extra

All About Gemini

Gemini in Gmail

Gmail customers will be capable of search, summarize, and draft their emails utilizing its Gemini AI expertise. It can additionally be capable of take motion on emails for extra complicated duties, like serving to you course of an e-commerce return by looking your inbox, discovering the receipt and filling out a web based type. Learn extra

Gemini 1.5 Professional

One other improve to the generative AI is that Gemini can now analyze longer paperwork, codebases, movies and audio recordings than earlier than.

In a non-public preview of a brand new model of Gemini 1.5 Professional, the corporate’s present flagship mannequin, it was revealed that it could actually soak up as much as 2 million tokens. That’s double the earlier most quantity. With that degree, the brand new model of Gemini 1.5 Professional helps the most important enter of any commercially out there mannequin. Learn extra

Gemini Dwell

The corporate previewed a brand new expertise in Gemini referred to as Gemini Dwell, which lets customers have “in-depth” voice chats with Gemini on their smartphones. Customers can interrupt Gemini whereas the chatbot’s chatting with ask clarifying questions, and it’ll adapt to their speech patterns in actual time. And Gemini can see and reply to customers’ environment, both through images or video captured by their smartphones’ cameras.

At first look, Dwell doesn’t look like a drastic improve over present tech. However Google claims it faucets newer methods from the generative AI subject to ship superior, much less error-prone picture evaluation — and combines these methods with an enhanced speech engine for extra constant, emotionally expressive and life like multi-turn dialogue. Learn extra

Gemini Nano

Now for a tiny announcement. Google can be constructing Gemini Nano, the smallest of its AI fashions, immediately into the Chrome desktop consumer, beginning with Chrome 126. This, the corporate says, will allow builders to make use of the on-device mannequin to energy their very own AI options. Google plans to make use of this new functionality to energy options like the present “assist me write” device from Workspace Lab in Gmail, for instance. Learn extra

Gemini on Android

Google’s Gemini on Android, its AI alternative for Google Assistant, will quickly be benefiting from its skill to deeply combine with Android’s cellular working system and Google’s apps. Customers will be capable of drag and drop AI-generated photos immediately into their Gmail, Google Messages and different apps. In the meantime, YouTube customers will be capable of faucet “Ask this video” to seek out particular info from inside that YouTube video, Google says. Learn extra

Gemini on Google Maps

Gemini mannequin capabilities are coming to the Google Maps platform for builders, beginning with the Locations API. Builders can present generative AI summaries of locations and areas in their very own apps and web sites. The summaries are created primarily based on Gemini’s evaluation of insights from Google Maps’ group of greater than 300 million contributors. What’s higher? Builders will not have to jot down their very own customized descriptions of locations. Learn extra

Tensor Processing Items get a efficiency enhance

Google unveiled its subsequent era — the sixth, to be precise — of its Tensor Processing Items (TPU) AI chips. Dubbed Trillium, they may launch later this yr. Should you recall, saying the following era of TPUs is one thing of a convention at I/O, even because the chips solely roll out later within the yr. 

These new TPUs will characteristic a 4.7x efficiency enhance in compute efficiency per chip when in comparison with the fifth era. What’s possibly much more essential, although, is that Trillium options the third era of SparseCore, which Google describes as “a specialised accelerator for processing ultra-large embeddings frequent in superior rating and advice workloads.” Learn extra

AI in search

Google is including extra AI to its search, assuaging doubts that the corporate is dropping market share to rivals like ChatGPT and Perplexity. It’s rolling out AI-powered overviews to customers within the U.S. Moreover, the corporate can be trying to make use of Gemini as an agent for issues like journey planning. Learn extra

Google plans to make use of generative AI to prepare the whole search outcomes web page for some search outcomes. That’s along with the present AI Overview characteristic, which creates a brief snippet with mixture details about a subject you have been trying to find. The AI Overview characteristic turns into typically out there Tuesday, after a stint in Google’s AI Labs program. Learn extra

Generative AI upgrades

Google introduced Imagen 3, the newest within the tech large’s Imagen generative AI mannequin household.

Demis Hassabis, CEO of DeepMind, Google’s AI analysis division, stated that Imagen 3 extra precisely understands the textual content prompts that it interprets into photos versus its predecessor, Imagen 2, and is extra “artistic and detailed” in its generations. As well as, the mannequin produces fewer “distracting artifacts” and errors, he stated.

“That is [also] our greatest mannequin but for rendering textual content, which has been a problem for picture era fashions,” Hassabis added. Learn extra

Challenge IDX

Challenge IDX, the corporate’s next-gen, AI-centric browser-based improvement surroundings, is now in open beta. With this replace comes an integration with the Google Maps Platform into the IDE, serving to add geolocation options to its apps, in addition to integrations with the Chrome Dev Instruments and Lighthouse to assist debug functions. Quickly, Google may even allow deploying apps to Cloud Run, Google Cloud’s serverless platform for operating front- and back-end providers. Learn extra

Veo

Google’s gunning for OpenAI’s Sora with Veo, an AI mannequin that may create 1080p video clips round a minute lengthy given a textual content immediate. Veo can seize completely different visible and cinematic types, together with photographs of landscapes and time lapses, and make edits and changes to already-generated footage.

It additionally builds on Google’s preliminary business work in video era, previewed in April, which tapped the corporate’s Imagen 2 household of image-generating fashions to create looping video clips. Learn extra

Circle to Search

The AI-powered Circle to Search characteristic, which permits Android customers to get on the spot solutions utilizing gestures like circling, will now be capable of clear up extra complicated issues throughout psychics and math phrase issues. It’s designed to make it extra pure to interact with Google Search from anyplace on the telephone by taking some motion — like circling, highlighting, scribbling or tapping. Oh, and it’s additionally higher to assist youngsters with their homework immediately from supported Android telephones and tablets. Learn extra

Firebase Genkit

There’s a brand new addition to the Firebase platform, referred to as Firebase Genkit, that goals to make it simpler for builders to construct AI-powered functions in JavaScript/TypeScript, with Go help coming quickly. It’s an open supply framework, utilizing the Apache 2.0 license, that allows builders to rapidly construct AI into new and present functions.

Among the use circumstances for Genkit the corporate is highlighting Tuesday embrace lots of the customary GenAI use circumstances: content material era and summarization, textual content translation and producing photos. Learn extra

Pixel 8a

Google couldn’t wait till I/O to indicate off the newest addition to the Pixel line and introduced the brand new Pixel 8a final week. The handset begins at $499 and ships Tuesday. The updates, too, are what we’ve come to anticipate from these refreshes. On the prime of the listing is the addition of the Tensor G3 chip. Learn extra

Pixel Slate

Google’s Pixel Pill, referred to as Slate, is now out there. Should you recall, Brian reviewed the Pixel Pill round this time final yr, and all he talked about was the bottom. Apparently sufficient, the pill is offered with out it. Learn extra

We’ll be updating this publish all through the day …

We’re launching an AI publication! Enroll right here to start out receiving it in your inboxes on June 5.