Google suddenly released the "most powerful speech-to-text model": Is it an outdated product in the Agent era, or a key puzzle piece?

Google Releases Its So-Far "Most Powerful" Speech-to-Text Model

On August 26, Google launched the new-generation speech-to-text model Gemini 3.5 Transcribe, which can automatically process filler words such as "um" and "ah", verbal slips and repeated expressions during transcription, and supplement punctuation, capitalization and text formatting.

Google calls it "the most accurate speech-to-text model to date", and Google CEO Pichai announced the model on X.

Different from traditional speech recognition models that only pursue verbatim recording, Gemini 3.5 Transcribe aims to convert original speech directly into text closer to the finished draft, reducing users' subsequent workload of organizing meeting minutes, interview shorthand and call content.

For example, when a user says "We will have a meeting on Tuesday — no, change it to Wednesday", the model can understand that the second half is a correction to the previous content, and retain the correct result in the final text instead of mechanically recording the entire verbal slip.

Gemini 3.5 Transcribe supports more than 85 languages and regional variants, and can automatically determine the language currently in use. Users do not need to set the language in advance, and the model can continue to transcribe even if the language is switched in one sentence or the same paragraph of dialogue.

This capability is mainly for scenarios such as cross-border meetings, multilingual interviews, customer service calls, and scenarios where Chinese and English technical terms are mixed together.

In real recordings, what affects transcription accuracy is often not just accents, but also background noise, multiple people speaking at the same time, and professional vocabulary. Google stated that Gemini 3.5 Transcribe is optimized for noisy environments, different accents and multilingual dialogues, and enhances the recognition ability of mixed alphanumeric information such as phone numbers, postal codes and order numbers.

Gemini 3.5 Transcribe Supports Real-time Language Switching and Seamless Streaming Transcription

The model also supports custom vocabulary lists. Developers can provide up to 1000 professional terms, abbreviations, person names or product names to guide the model to prioritize recognizing these contents. Google said that in actual use, the effect is usually better when the custom vocabulary list is controlled within 100 words.

With the Gemini 3.5 transcription function, you can eliminate speech disfluencies through the smart transcription feature.

This may be more practical for technical interviews. For example, users can add Kubernetes, BigQuery, as well as company and product names in advance to reduce the situation where the model recognizes professional terms as words with similar pronunciation.

Gemini 3.5 Transcribe also provides "speaker diarization" capability, which can identify up to 8 speakers and assign different voice segments to corresponding speaker tags. However, Google also reminded that the recognition of more than three speakers is still an experimental capability at present.

For pre-recorded audio, the model can also output word-level timestamps, marking the start and end time of each word in the recording, which is convenient for users to make subtitles, retrieve original recordings or locate original interview quotes. After enabling speaker diarization or word-level timestamps, the maximum length of a single audio recording is 30 minutes; ordinary audio transcription supports a maximum length of 1 hour per session.

The 3.5 transcription function provides transcription with multi-speaker attribution and word-level timestamps.

Word Error Rate of Streaming Transcription Reaches 4.0%

According to the Artificial Analysis evaluation data cited by Google, the average word error rate of Gemini 3.5 Transcribe in real-time streaming scenarios is 4.0%, and the average word error rate for non-streaming audio processing is 2.6%.

The word error rate is usually used to measure the proportion of substituted, deleted and added words in the speech recognition results. The lower the value, the closer the transcription result is to the original speech. However, the language, noise and accent distribution of different test sets will affect the final results, and this set of data cannot directly represent the accuracy of the model under all recording conditions.

Compared with Google's previous-generation speech transcription model Chirp 3, the final transcription latency of Gemini 3.5 Transcribe is reduced by 70%. The real-time version can continuously receive speech through WebSocket and return phased transcription results with a latency of less than one second.

Google offers two versions this time: `gemini-3.5-transcribe-live` is for real-time subtitles, voice input and Voice Agent; `gemini-3.5-transcribe` is for processing pre-recorded audio such as interview recordings, meetings and call records, supporting speaker diarization and word-level timestamps.

Developers can currently call the model through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. The comprehensive price for real-time transcription is about $0.009 per minute, and non-real-time transcription is about $0.005 per minute, with a free test quota available.

Speech Transcription Is Evolving From "Dictation" to "Organization"

The most noteworthy part of Gemini 3.5 Transcribe is not just the further reduction of word error rate, but that speech transcription tools are changing their output goals.

In the past, automatic speech recognition mainly pursued faithful recording: whatever the speaker said, the system tried to write it down verbatim. But real spoken language is full of filler words, repetitions, pauses, temporary corrections and incomplete sentences. Even if the recognition is completely accurate, the output results are often still not directly readable.

The Smart Transcription added to Gemini 3.5 Transcribe will actively clean up filler words and repeated content, handle self-corrections according to the context, and convert spoken language into text with punctuation and structure. It also supports "inverse text normalization", for example, directly converting the English dictated "twenty six million dollars" into "$26M".

This means that what the model outputs is no longer just original shorthand, but a piece of text that has been initially organized.

However, this kind of "intelligent organization" also brings new risks. For meeting minutes and daily voice input, deleting filler words usually does not affect the information; but in scenarios such as news interviews, legal evidence collection and scenarios that require verbatim verification, automatically deleting repeated expressions or rewriting numbers may change the speaker's original wording.

To this end, Gemini 3.5 Transcribe also provides a Verbatim mode, allowing developers to obtain text that is closer to verbatim records. In actual use, whether to enable filler word cleaning and intelligent formatting should be determined according to the scenario, and "neater" cannot be simply regarded as "more accurate".

At present, this technology has also begun to be integrated into Google's own products. The Rambler function of Gboard on Android can convert dictated content into formatted text, and allow users to continue modifying wording and style through voice; in the Gemini app for macOS, the model can also recognize file names and current documents combined with the screen context, and call other Gemini models through voice commands to complete document summarization, text rewriting and image generation.

From this perspective, Gemini 3.5 Transcribe is not just a new dictation model. Google is turning speech transcription into the entry point of the AI operating system: users do not need to organize language into standard instructions first, but can directly state their needs with pauses, verbal slips and temporary modifications, and then the model understands the real intention and continues to execute the task.

Can't Google See the Overall Situation?

In the general environment where the industry is fiercely competing in multimodal and Coding Agent, on the surface, Google's separate release of a speech-to-text model at this time is easy to give people the impression that it is "still solving the problems of the previous generation". Therefore, on X, some users commented that Google "can't see the overall situation clearly".

It can be said that there are a huge number of unfavorable voices on the Internet.

A more pointed criticism comes from a Gemini Ultra user.

He said he already regretted upgrading, because the so-called extra features only make sense when model capabilities and product iterations truly keep up with the industry progress. He once hoped that Google could win this competition, and this confidence largely came from Demis Hassabis and DeepMind under his leadership. But as Demis is no longer in charge of DeepMind's daily operations, he feels that "the soul of DeepMind is gone". In his view, after losing a clear research direction and leader, Google is left with a naked competition in model capabilities, and it is currently falling behind in this competition.

The release of Gemini 3.5 Transcribe further reinforces this disappointment: when peers are emphasizing stronger Agents, end-to-end voice interaction and complex task execution, Google pushes a speech-to-text model to the forefront, which easily gives paying users the feeling that the product rhythm is out of alignment with the industry focus. It needs to be pointed out that this more reflects users' dissatisfaction with Google's overall AI progress and product experience, and we cannot judge that its technical route has fallen behind only by a single transcription model.

So, is Google really in a very bad situation as the above netizens said?

In fact, the goal of Gemini 3.5 Transcribe is not to reinvent the dictation tool, but to fill the most basic and most easily overlooked layer before voice Agents enter the production environment: to let machines stably understand human speech first.

In scenarios such as customer service, meetings, interviews, and medical records, background noise, multiple people speaking, temporary corrections, mid-sentence language switching, and complex information such as order numbers will still directly affect the subsequent Agent execution.

Automatically removing "um, ah", understanding self-corrections, and outputting formatted text is essentially organizing speech from raw input into structured context that is more suitable for the model to process further.

Therefore, this product is not as narrativeally attractive as end-to-end speech models, and the author believes that it is not necessarily outdated.

The real problem is that if Google only regards it as an isolated transcription service, its value is really limited. Only by embedding it into Gboard, Gemini, Chrome and enterprise Agent workflows to make voice an entry point for invoking tools and executing tasks can it demonstrate strategic significance.

References:

https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5-transcribe/

https://ai.google.dev/gemini-api/docs/pricing?hl=en

This article is from the WeChat official account "AI Front" (ID: ai-front), written by Dong Mei, authorized for release by 36Kr.