Google is plugging AI search into a mobile phone! less than 600MB, with photos, videos, and audio di

  It's the first time in the history of the world.

On October 6, Google launched Embedding Gemma 2, a search engine that can search for images, videos, and audio, and embedded it in a small model with less than 600MB of memory!

Text, code, images, video, and audio can all be searched in a single word, and this is the first time in Google's open source model.

740M parameters, mobile phones, laptops, and browsers can all run offline.

Google CEO Sundar Pichai has also personally introduced Annie: This is Google's first native multimodal open-source embedded model.

One of its breakthroughs was to allow mobile phones to search locally for images, videos, and sounds. In the presentation, as soon as you enter the phrase "birds are singing", you can find a picture of the bird and a recording of the bird.

Just over an hour after its release, Hugging Face's Victor M made it run in the browser.

He taped "birds singing" in the search box, and the grids on the screen were immediately rearranged: the front row with both photos of the birds and a recording of the wave-shaped birds.

The entire query took only 22 milliseconds, no server running, and no API tuning.

Once you have access to the app, you can let the AI help you find photos, recordings and videos on your phone without having to upload files to the cloud.

Embedding this stuff is not usually visible to ordinary users, but many AI searches and RAG queries rely on it to find information.

It works by transforming a piece of content into a series of numbers, which is equivalent to arranging a coordinate for each piece of content in a huge space.

When searching, the system searches for the content closest to the problem.

The previous generation of Embedding Gemma only handled text. To search for images and sounds, they often accessed other models or converted them into text.

Embedding Gemma 2 puts the content in the same 768-dimensional space.

For example, the word "cat" and the cat's cry can be synonymous in this space.

So you can use a single word to search for images or videos, or you can use a voice to find the corresponding clips in the video.

It can also put text, images, and videos together, allowing you to search for all the information that contains them at once.

Google gave an example of this in its model car.

A commodity page for a pair of sneakers, containing a text introduction, two detailed charts, and a video of the anti-slip test of the shoes on wet sandstone.

The model can convert this set of contents into a vector to match such searches as "waterproof shoes for cross-country running".

There is also more content that can be processed in a single instance. The context window on 8K is 4 times that of the previous generation.

Only a single type of content can accommodate up to 5.5 minutes of audio, 29 images, or 58 tons of video. It also supports more than 100 languages.

Google put it together with several models of the same scale in a round, with the biggest gap being watching charts and watching videos.

EmbeddingGemma 2 is clearly ahead of the Jina v5 Omni-Nano in image and video retrieval.

In a test published by Google, Embedding Gemma 2 scored an image recovery score of 57.3, which was only 31.6 percent higher than its comparator Gina v5 Omni-Nano.

Video retrieval also drove a gap of 50.7 points to 31.2 points, nearly 20 points ahead.

In multilingual text testing, it is basically the same as the previous generation.

So much for the real thing, it's all in the 740M parameters.

The text section is 270M, the visual encoder 170M, and the audio encoder 300M, which can be loaded on demand.

In Google's Pixel 11 Pro test, the quantified pure-text model had a minimum of about 191 MB of memory and a full multi-mode model of about 567 MB.

It also supports the search index "skinny": with less digital recording of the meaning of each paragraph, the storage space occupied can be reduced by two thirds.

In Google's multilingual text retrieval test, scores fell by less than 1 point.

Google itself has created several app templates:

Instant Media Search in AI Edge Gallery allows you to search for photos in a photo gallery at will.

Video Moments Finder is more intuitive, in a word, to locate the corresponding clips in the video.

Another application, Foresight, combines it with Gemma 4 to retrieve offline files and conference records on Mac.

On the developer side, llama.cpp is already supported, and Unsloth has also released a quantified version, which makes it easy to deploy the model locally.

According to the observations published by the Mac application Nativ, on the M5 Max, the vector generated by the 8-bit quantum version is 0.9997, with the analogue resemblance of the original version reaching 817 bits per second.

Testing by the Turkish developer Avenox is closer to everyday.

His notes were mostly written in English, but the questions were usually in Turkish. In the past, keywords matched, and the words on both sides were incorrect, so the relevant notes could not be searched.

He asked Claude to spend an hour setting up a testbed, picking out 40 notes, then changing the words, asking questions in Turkish to see if the correct notes would fit the top 18 candidates.

According to his published results, the keyword matching hit rate was only 15%; it rose to 97% in exchange for Embedding Gemma 2.

He also took 30 regular hand-input tests of real messages with the wrong words. The number of corresponding notes found by the model was twice the number of keyword matches.

There's more to it than that.

Nick Lo, the developer, put it in a Nano development board.

It takes the model about 15 milliseconds to enter a sentence and convert it into a sequence of numbers to search for. After finding the corresponding image, it is given to a single piece of ESP32-S3 and the image is drawn in stages.

In addition to multi-mode, code retrieval is also significantly improved.

On the code retrieval benchmark MTEB Code, it jumped from 68.76 points to 78.68 points from the previous generation, a decrease of almost 10 points, Google said, leading the way in models of the same size.

This is useful for code-writing agents.

Before a tool like Claude Code, Codex, can work, it is necessary to find the relevant sections in the entire code library.

With the pure text version of 270M, developers can create a local index for the code library and then search for the corresponding code in plain English. Both the indexing and retrieval steps can be done locally.

In March, Google released the multi-modal Gemini Embedding 2 in the cloud, which was paid for by API calls.

More than half a year later, Google introduced multi-mode retrieval into an open-source small model that can run on mobile phones. Photos, audio and video, with the option of remaining locally retrieved.

One of Google's uses is to pair it with Gemma 4 as offline, privacy-priority RAG.

One is responsible for finding the information, one is responsible for answering based on the information, and both searches and queries can be done on the device.

In the past, in order to search for photos, videos and recordings using AI, the content was often handed over to the cloud.

Now, this retrieval capability, with only a few hundred MB, has the opportunity to fit into everyone's pocket.