Google is bringing artificial intelligence closer to solving one of technology’s most meaningful accessibility challenges. At its Made by Google event, the company unveiled Sign-Language-to-Text (SL2T), a new AI model that translates sign language directly into written text using a smartphone camera.
The feature is arriving on the Pixel 11 series through Gboard and Live Transcribe, allowing users to communicate through sign language across messaging apps, emails, web searches, documents, and even AI assistants like Google Gemini. More importantly, it represents a shift toward making smartphones more accessible for millions of Deaf and hard-of-hearing users who rely on sign language every day.
Rather than serving as another AI showcase, Google’s latest accessibility feature addresses a real-world communication barrier—one that has long required interpreters, specialised software, or manual typing.
TL;DR
- Google has introduced Sign-Language-to-Text (SL2T), an AI model that converts sign language into written text.
- The feature is launching in Gboard and Live Transcribe on the Pixel 11 lineup.
- It currently supports American Sign Language (ASL) translated into English.
- The model was trained on more than 100,000 hours of data spanning over 50 sign languages.
- Google says the system processes body landmarks on-device rather than sending raw video to cloud servers, improving user privacy.
- SL2T currently leads performance on the FLEURS-ASL benchmark, according to Google.
What is Google Sign-Language-to-Text?
Google Sign-Language-to-Text (SL2T) is an AI translation model that converts sign language directly into natural written language.
Unlike traditional systems that depend on multiple processing steps, SL2T uses a smartphone’s camera to observe signing and instantly generate text. The goal is to make communication faster and more natural across digital services.
Initially, the technology supports:
- American Sign Language (ASL)
- English text output
Google has indicated that the model’s multilingual training lays the foundation for supporting additional sign languages in the future.
How does Google’s Sign-Language-to-Text work?
A smartphone camera becomes the input device
Instead of requiring gloves, wearable sensors, or specialized cameras, SL2T works with a phone’s existing camera.
As a user signs, the camera captures movement while AI analyzes:
- Hand shapes
- Finger positions
- Arm movement
- Body posture
- Facial expressions
- Spatial relationships between gestures
The resulting translation appears as written text.
This allows users to communicate without switching between signing and typing.
Built for everyday communication
Google says SL2T can be used throughout Android’s communication ecosystem, including:
- Text messaging
- Email composition
- Web searches
- Document editing
- Google Gemini prompts
- Face-to-face conversations through Live Transcribe
For example, instead of typing a response during a conversation, a Deaf user can simply sign toward their phone and have the response displayed as text.
Why skipping “glosses” matters
One of the biggest technical breakthroughs behind SL2T is what it avoids.
Traditional systems use an extra translation layer
Many earlier sign language translation models first convert signs into simplified symbolic labels known as glosses.
Those glosses are then translated into natural language.
While effective in some situations, this extra step often removes important information.
Sign languages rely heavily on:
- Facial expressions
- Eye gaze
- Movement intensity
- Spatial positioning
- Body orientation
These non-manual markers carry meaning that cannot always be represented through glosses alone.
Google’s direct translation approach
SL2T translates directly from movement data into written text.
By removing the intermediate gloss layer, Google says the system preserves more of the linguistic information embedded within sign language.
That results in translations that better capture meaning rather than simply matching individual signs to words.
How Google trained the AI model
Training sign language AI is considerably more difficult than training speech recognition systems.
According to Google, SL2T was trained using:
- More than 100,000 hours of sign language data
- Over 50 different sign languages
- American Sign Language making up roughly 25% of the training dataset
The multilingual approach helped the AI identify shared grammatical structures and movement patterns across different sign languages instead of learning each language entirely in isolation.
Although the first release focuses on ASL, this broader training could help Google expand support to additional languages over time.
Privacy is built into the design
Accessibility technologies often raise privacy concerns because they rely on cameras.
Google says SL2T minimizes those concerns by processing movement differently.
Landmark tracking instead of video uploads
Rather than sending raw camera footage to Google’s servers, the system uses an on-device MediaPipe Holistic model.
This software identifies key points on the signer—including hands, face, and body—and converts them into geometric landmark coordinates.
Only those mathematical representations are used by the translation model.
In practical terms, that means:
- Raw video remains on the device.
- Less personal visual information is processed externally.
- Translation can happen with greater privacy.
This approach aligns with Google’s broader push toward on-device AI processing across Pixel devices.
How accurate is Google’s Sign-Language-to-Text?
Google says SL2T currently achieves the strongest results yet on the FLEURS-ASL benchmark, a widely used evaluation for sign language translation.
The company reports:
- 70 BLEURT zero-shot score
According to Google, that score substantially outperforms previous systems on the same benchmark.
While benchmark scores don’t always reflect real-world performance, they offer an indication of how effectively an AI model generalizes to new signing examples without additional training.
Independent testing after launch will provide a clearer picture of how the technology performs in everyday use.
Why this announcement matters
Accessibility announcements rarely receive the same attention as flagship AI features, but they often have a greater impact on people’s daily lives.
For many Deaf users, communicating with hearing individuals can involve:
- Typing responses
- Using interpreters
- Switching between communication methods
- Waiting for transcription services
Reducing even part of that friction can make digital communication significantly easier.
If the technology proves reliable, it could help users communicate more naturally in workplaces, schools, customer service interactions, healthcare settings, and everyday conversations.
Challenges still remain
Despite Google’s progress, sign language translation remains one of AI’s most difficult language problems.
Several challenges remain:
Every sign language is unique
American Sign Language is not a universal language.
Countries and regions use different sign languages with distinct grammar and vocabulary, including:
- British Sign Language (BSL)
- Indian Sign Language (ISL)
- French Sign Language (LSF)
- Japanese Sign Language (JSL)
Supporting one language does not automatically translate to supporting another.
Context matters
Many signs change meaning depending on:
- Facial expression
- Conversation context
- Previous sentences
- Regional signing styles
Building AI that understands those nuances consistently remains an ongoing research challenge.
What’s next for Google’s accessibility AI?
Google has increasingly integrated accessibility features into Android, from Live Caption to Live Transcribe and AI-powered image descriptions.
SL2T builds on that strategy by combining advances in computer vision, machine learning, and on-device processing.
Future improvements could include:
- Support for additional sign languages
- Faster real-time translations
- Two-way conversations between signed and spoken language
- Integration into more Android applications
- Expanded support across devices beyond Pixel smartphones
Whether those capabilities arrive soon will depend on further research and collaboration with Deaf communities.
The bottom line
Google’s Sign-Language-to-Text model is one of the most practical AI announcements from its latest hardware event. Instead of showcasing AI for novelty, the company is applying it to a long-standing accessibility challenge.
By translating sign language directly into written text, preserving important visual cues, and processing data with privacy in mind, SL2T has the potential to make smartphones more inclusive for Deaf users.
The initial release is limited to American Sign Language and Pixel 11 devices, but its underlying technology suggests a broader future for AI-powered accessibility—one where communication barriers become smaller, not larger.
