4 min read 4 min
The new Live Captions feature on the Google Pixel 4 and Pixel 4 XL is an underrated addition to Mountain View’s flagship phones this year. It allows you to caption any video directly on the device, whether it’s a YouTube video or footage embedded in your Twitter feed. Here’s how to use it.
Jump to section:
- Why use Live Captions?
- How to activate Live Caption on your Pixel 4
- How to adjust Live Caption settings
- How to use Live Captions for phone calls
Why use Live Captions?
Ever been sat on a train or in the office and someone sends you a cool video to check out? You can of course just turn up the volume on your smartphone and play the video for all around you to hear, but that can be anti-social in some cases. You could also reach for your headphones, but that can also be a pain if they’re in the bottom of your bag or worse, you don’t have any to hand at all.
Live Caption is a perfect solution for those moments. I don’t see this as a feature for watching longer videos or full movies, although you could if you wanted to, but for those situations where you just need to understand the dialogue without messing around with volume rockers and headphones, it’s perfect! Live Caption can add subtitles to any video media playing on your phone. It’s surprisingly accurate, too, and works quickly in real-time. It also works for video calls too, which is something that was not possible before.
Live Captions on a YouTube video on the Google Pixel 4 / © NextPit
How to activate Live Caption on your Pixel 4
There are a couple of ways to turn on Live Caption on your Pixel 4 smartphone. The quickest and easiest way is via the volume rocker, actually. If you have a Pixel 4 running on Android 10, then simply follow the steps below:
- Press the volume rocker, either up or down is fine
- At the very bottom, you will see an icon that looks like a box with text in (see screenshot below), that’s the Live Caption icon
- You can toggle it on or off quickly and easily from here
- When it’s off, it will have a line diagonally through the box
Live Caption can be turned on via the volume rocker / © NextPit
You can also turn Live Caption on and off using the Quick Settings Menu. You’ll need to add the icon to your Quick Settings if you have not already done so. Follow the steps below:
- Drag down from the top of your screen to open the Quick Settings Menu
- Tap on the pencil icon to edit the tiles you have in there
- You will find the same Live Caption icon we saw in the volume rocker menu
- Add it to your Quick Settings Meu for easy access
You can also add Live Caption to Quick Settings Menu / © NextPit
Finally, you can also activate Live Caption from the regular settings menu. It falls under the Accessibility settings. Scroll down until you see the option ‘Live Caption: automatically caption media’.
How to adjust Live Caption settings
Now that you know how to turn Live Caption on and off, you might want to dig into the settings menu to tweak the controls to your liking. You can either search your settings for “Live Caption” or find it under the Accessibility menu as described above.
You can tweak the control to your liking in the Live Caption settings. / © NextPit
From this screen, you can change the language (only English is available at the moment but this is where the setting will be when more languages launch soon), hide profanity and choose whether to introduce sound labels for things like laughter, applause and music, or whether you can live without them.
There is also the option to take away the Live Caption icon from your volume control if you don’t like using this method to toggle it on and off. You can also find information and tips for using Live Caption in this menu. Touching and holding the captions allows you to move them, for example, whilst double-tapping a caption will expand it.
Google is bringing Live Captions to phone calls
Thanks to the Android 11 Developer Preview 3, we now know that Google is planning on bringing Live Captions to phone calls as well as video content. This would mean that you would be able to essentially have subtitles for your conversation partner!
The guys over at XDA Developers analyzed version 2.13.302920511 of Device Personalization Services, the app that runs Live Captions. They found strings of code suggesting that users will be able to turn on Live Captions during phone calls on Android 11.
Have you tried Live Caption on your Google Pixel 4 smartphone? What do you use it for? Share your experiences in the comments section below.
Live Caption automatically adds captions for any audio playing on your phone, which can be incredibly useful in many situations. We’ll show you how to use it on any Google Pixel smartphone, including the Pixel 2 or newer.
At this writing, the Live Caption feature only supports English, but it works with videos, podcasts, phone and video calls, and more. (It doesn’t work with music.)
The first thing to do is check and see if you have Live Caption on your phone. To do so, swipe down from the top of the screen twice, and then tap the Gear icon to open the “Settings” menu.
Scroll down and tap “Live Caption.” If you don’t see this, you don’t have the Live Caption feature on your phone.
There are two ways you can use Live Caption. If you want it to automatically display captions any time audio is detected, toggle-On the “Live Caption” option.
If you want it to be off, but easily accessible whenever audio is playing, toggle-On the “Live Caption in Volume Control” option.
You can also use Live Caption during phone calls. To enable this, tap “Caption Calls” and select when and how you want it to work. It’s not intended to be used for calls with more than one other person.
In the image below, you see what Live Caption looks like when you watch a YouTube video. You can drag the caption box around the screen by tapping and holding it.
To manually turn Live Caption on or off, press one of the Volume buttons on the side of your phone to open the “Volume” controls. Tap the Live Caption icon to quickly turn the feature on or off.
If you selected “Ask Every Time” for Caption Calls, you’ll see the popup shown below whenever you answer a call. Select either “Caption Call” or “Don’t Caption Call.” You can also select the “Don’t Ask Again” checkbox if you don’t want to see this popup in the future.
That’s it! Depending on what you chose, Live Caption will either start every time you play audio on your phone, or you can toggle it on or off, as needed.
· 5 December 2019
– It’s now live on Pixel 4, 4 XL 3, and 3A
(Pocket-lint) – Those of you who are hearing-impaired or love to watch videos with captions on will enjoy Google’s new Live Caption feature now rolling out.
What is Google Live Caption?
At Google I/O in May 2019, Google introduced a new automatic captioning system called Live Caption.
The idea is it’ll automatically serve up captions in real-time for all audio on your mobile device (excluding voice and video calls). It’s an accessibility feature for the 466 million people in the world who are deaf or hard of hearing. But it can also come in handy for people who are on a loud commuter train, for instance, or trying not to wake a baby while they’re listening to a podcast.
Live Caption is supposed to work on videos, podcasts, and audio messages – all without Wi-Fi or data needed.
How does Google Live Caption work?
Live Caption works entirely on your Android mobile device, so data or a Wi-Fi connection is not required. To enable Live Caption, you’ll see a prompt that pops up after you press the volume button on your device. Simply tap that prompt. You can then drag the Live Caption text that appears around on your screen, or you can expand it with a double-tap to see more text. Easy peasy.
Keep in mind transcriptions might have inaccuracies if the sound quality isn’t great, but Google promises to keep improving the tech.
Which languages does Live Caption support?
Live Caption supports English, but Google said there are plans to support more languages “in the near future”.
Which devices offer Live Caption?
Live Caption launched first on Google’s own Pixel 4. It’s supposed to roll out to Pixel 3, Pixel 3 XL, Pixel 3a and Pixel 3a XL by late 2019. As of December 2019, Live Caption has also arrived for the Pixel 3 and Pixel 3a smartphones.
Google said it is “working closely with other Android phone manufacturers to make it more widely available in the coming year”.
Published: August 5, 2020
Captioned voice and video calls are now available on Google’s Pixel smartphones starting with the Pixel 4a; the device will caption phone calls made with the native dialer using automatic speech recognition. It will also caption calls, including video calls, using other apps, such as WhatsApp, Viber, Skype… etc..
A Brief History of Telecommunications Relay Service
Prior to the Americans with Disabilities Act of 1990 (the “ADA”) millions of Americans who were deaf, hard of hearing or deaf blind could not make a simple phone call. Telecommunications networks and equipment manufacturers were incapably of generating captions for phone calls. As a result, at the direction of the ADA, and the ADA’s amendment of the Communications Act of 1934, the Federal Communications Commission (“FCC”) adopted rules to establish telecommunications relay service (“TRS”) to ensure that individuals with hearing loss can access functionally equivalent voice communications services. The government stepped in where technology and the private sector could not.
August 2020 marks another historic moment in telecommunications services for individuals with hearing loss. Google Pixel smartphones, starting with the new Google Pixel 4a, will caption voice and video calls for free using automatic speech recognition. For the first time, voice and video calls will be captioned at the initiative of a private company, at no cost to the government. It took thirty (30) years, but the technological gap that led to the establishment of TRS has narrowed.
The Benefits of Live Caption
The Android “service” that captions voice and video calls on Google Pixel smartphones is called Live Caption. Live Caption offers a number of benefits compared to traditional TRS. Here are just four:
(1) TRS can only be used on a smartphone through a third-party app; the user cannot use the native dialer. Native dialers are typically feature rich and offer capabilities that TRS apps do not. For example, the Google Pixel dialer offers a feature called “Call Screen” to screen calls from possibly faked numbers in order to prevent unwanted telemarketing calls. TRS apps do not offer this and other capabilities.
(2) Google’s Live Caption service captions voice and video calls made or received on any app. Thus, for the first time, someone who is severely hard of hearing can participate in a voice or video call over the native dialer, WhatsApp, Facebook Messenger, Google Duo or any other communications app. This flexibility is unprecedented and does not exist with TRS.
(3) Live Caption works with voice or video calls, while popular TRS apps, such as Innocaption and CaptionMate support voice calls only.
(4) Live Caption is free to the user, as well as state and federal governments, who have historically paid billions of dollars to subsidize TRS.
Of course, TRS continues to play an important role. Many seniors use landline phones, for which Live Caption is not available. Neither is Live Caption a substitute for video relay service. And caption quality can vary depending upon the quality of the connection.
Nevertheless, Live Caption is unquestionably a significant step forward and offers unprecedented benefits and flexibility.
Live Caption Available on the Pixel 4a and other Pixel Devices
Live Caption and its ability to caption voice and video calls is available with the new Pixel 4a, which can be purchased through RAZ Mobility. It will also be made available with other older Pixel phones, such as the Pixel 2, 3, 3a and 4. It is unclear when this feature will be available in other android devices. The iPhone and iPad do not have this capability.
The captions will be generated on the device itself, so there will be complete privacy. Google will not have access to the call audio or captions. The captions will not be saved on the device.
Google Pixel Features for People Who are Deaf or Hard of Hearing
Here is a summary of some of the Google Pixel’s key accessibility features for individuals with hearing loss:
- Caption in-person conversations through Live Transcribe and the Google Recorder
- Caption all media content (video, podcasts and internet radio) through Live Caption
- Caption voice and video calls through Live Caption
- Built-in amplification of media audio (video, music, podcasts and internet radio) through Sound Amplifier
- Hearing aid support
- Real time text
- For those who cannot speak, the Google Assistant can speak to 911 emergency dispatchers the user’s location and the nature of her emergency
4 Top Cell Phones for People Who are Blind or Visually Impaired
3 Best Phones for People with Dementia or Alzheimer’s
Google first announced its “Live Captions” accessibility feature at Google I/O 2019 last year. Using three on-device machine learning models, this feature detects English-language speech in media and generates captions automatically. The feature was first made available on the Pixel 4 but later expanded to the Pixel 3a, Pixel 3, Pixel 2, Samsung Galaxy S20, OnePlus 7T, and OnePlus 8. Since the feature launched, however, it hasn’t received any updates to its functionality. That could change soon, though, as we spotted evidence that Google will allow Live Caption to work over phone calls.
An APK teardown can often predict features that may arrive in a future update of an application, but it is possible that any of the features we mention here may not make it in a future release. This is because these features are currently unimplemented in the live build and may be pulled at any time by the developers in a future build.
In Android 11 Developer Preview 3 for the Google Pixel 4, we analyzed version 2.13.302920511 of Device Personalization Services, the application responsible for Live Captions. In it, we spotted strings that state users will be able to turn on Live Caption over a phone call.
Enabling the feature over a phone call will announce to all participants that the call is being transcribed into captions. An audio file will be played in the phone call, and this audio file simply says the following line:
Currently, the API that Live Captions relies on, AudioPlaybackCaptureConfiguration, does not allow for capturing voice call audio. However, it’s possible that a new system-only permission added in Android 11 Developer Preview 3 will allow the feature to bypass this restriction.
Thanks to PNF Software for providing us a license to use JEB Decompiler, a professional-grade reverse engineering tool for Android applications.
Live Caption is one of the more innovative features we’ve seen yet, using on-device machine learning to generate captions for local and web videos.
The function was originally touted as an Android 10 feature but it’s limited to the Pixel 4 series for now. Fortunately, XDA-Developers has uncovered a way to bring Live Caption to non-Pixel phones.
The process requires a rooted Android 10 phone and sees users installing/updating the Device Personalization Services app and installing specific Magisk modules (visit the source link to view all the steps). XDA warns users that they shouldn’t update the aforementioned app via the Play Store, as they risk losing Live Caption.
Do be warned that rooting your phone (if you haven’t done so already) usually voids the warranty. You’re also doing this process at your own risk, as system tinkering carries the risk of bricking your device.
In any case, Live Caption is expected to hit Pixel 3 and Pixel 3a devices before the end of the year, so people with these phones simply need to sit tight for a few more weeks. Hopefully Google starts pushing this feature out to non-Pixel phones soon after the Pixel 3 and Pixel 3a receive the function.
Have you had success getting Live Caption via this method? Let us know in the comments!
This AI-powered captioning service is seriously adaptable.
By David Nield | Published Apr 06, 2021 9:38 AM
Google Live Caption works across any app, using a high level of machine learning to put text captions on audio and video files in real time. It’s useful for the hundreds of millions of people who are deaf or hard of hearing, but it can also help anyone riding in a loud subway car, sitting in a silent library room, or listening to something they simply can’t understand.
This feature, launched in 2019, stays and runs on whatever device you’re using, so it’ll seamlessly caption everything from video calls to podcasts—instantly. It also doesn’t need to ferry data to and from the cloud, so it works offline, too. It’s quick, simple, and secure.
Use Live Caption on your phone
Live Caption first appeared as an exclusive for Google Pixel phones and it currently works on the 2017 Pixel 2 and any Pixel phone launched since. The feature now also works on a select number of other Android phones, including the Galaxy S series from Samsung and the flagship OnePlus phones, but it’s not available on the iPhone.
If you’re using a Google Pixel, open up the main Android Settings app and choose Accessibility, then Live Caption. On Samsung handsets, it’s under Accessibility, Hearing enhancements, then Live Caption in Settings, while on OnePlus phones you’ll pick System, Accessibility, and Live Caption from Settings. If your Android phone offers Live Caption, it should be somewhere in the accessibility settings.
Beyond turning Live Caption on and off, you can configure various options like choosing to replace profanity with asterisks or whether or not sound labels (describing music, laughter, and applause) appear on screen. You can also decide if you want a Live Caption icon to appear in the volume control pop-up that appears when you adjust it—it will show up as a small icon underneath the volume slider, and offers another way to toggle the feature on or off.
Once Live Caption is enabled, it should show up automatically whenever audio or video is playing in any of the apps on your phone. Double-tap on the captions overlay to switch between a larger and a smaller window size, or tap and drag on the captions to move them to a different position on screen.
You can adjust the size and color of the captions, but this is under a separate menu: Caption preferences in Accessibility. You can pick one of the suggested color combinations for the subtitles, or create your own, and as you make changes to the look of the subtitles you can see a preview of how they’ll actually look on screen
Use Live Caption on your computer
Recently, Google added Live Caption as a feature in the latest version of its Chrome web browser (version 89 and later). That means you can use it for audio and video playing inside Chrome on Windows, macOS, and Linux. At the time of writing the feature isn’t available on Chrome OS, but it should be arriving soon.
From inside Google Chrome, click the three dots in the top right corner, then choose Settings, Advanced, and Accessibility. Turn on the Live Caption toggle switch, and Chrome will download the necessary speech recognition files to your computer
Click Caption preferences to set the size and appearance of the subtitles that show up on screen—you’ll actually get sent to the captions setting screen for whatever operating system you’re using, so you’ll be editing the appearance of the captions for every program on your computer, not just Chrome.
Load up any video or audio file in Chrome, and you’ll see the automatic captions appear during playback. Click the arrow at the bottom of the captions box to switch between a smaller and a larger window, or click and drag the window around to reposition it. To hide the captions, click the X in the corner of the subtitles window.
Live Caption is still a work in progress, and the subtitles can be a bit hit or miss—especially with accents or if music is playing. The extra processing it requires can also drain your device’s battery faster. All that said, it’s still an impressive and polished technology that can come in handy in a wide variety of scenarios.
David Nieldis a tech journalist from the UK who has been writing about gadgets and apps since way before the iPhone and Twitter were invented. When he’s not busy doing that, he usually takes breaks from all things tech with long walks in the countryside.
Google’s Live Caption accessibility feature received the ability to transcribe phone calls with the launch of the Google Pixel 4a earlier this month. At the time, the company had revealed that the new functionality would soon make its way to older Pixel devices with Live Caption support, including the Pixel 2, Pixel 3, Pixel 3a, and Pixel 4. Now, the feature has finally started showing up on some of these devices.
Our Editor-in-Chief Mishaal Rahman has received the feature on his Pixel 4 and, as you can see in the attached screenshots, it can be enabled by tapping on the new ‘Caption calls’ option within the Live Caption settings. Tapping on the option brings up a pop-up menu which lets you define the behavior of the call transcriptions. You can either choose to leave the feature off, have it on for all calls, or have your phone prompt you for each call.
The option to use Live Caption over phone calls now shows up on my Pixel 4. This feature first launched with the Pixel 4a. I was wondering when this would start to roll out for other Pixel phones! pic.twitter.com/GgwEg8jkRH
In response to Mishaal’s tweet, a few other Pixel users have confirmed that the feature has also made its way to the Pixel 3 XL and the Pixel 3a. In case you’re not familiar with the feature, Live Caption essentially captures audio from your device and runs it through three on-device machine learning models to generate captions from any English-language speech. The feature works with a variety of media and phone calls, including ones made over third-party apps like WhatsApp and Telegram, but it currently only supports English-language speech.
Along with the aforementioned devices, Live Caption in phone calls should also make its way to the Pixel 2 series considering that the devices do offer Live Caption support. It’s also likely to roll out to other non-Google devices that offer Live Caption support, like the Samsung Galaxy S20 series, OnePlus 7T series, OnePlus 8 series, and the OnePlus Nord in the near future.
It looks like Google has begun rolling out live caption for phone calls in more of its Pixel phones. This was first revealed in a tweet by Mishaal Rahman.
This is a good news story which the Pixel phones need given reports that the Pixel 5 may be shaping up to be a disappointment.
This feature first arrived with the launch of the Pixel 4a at the start of August. Alongside the Galaxy S20, the Pixel 4a was the only device to offer the live caption feature for calls. This made it all the more impressive when Google first announced this feature would come to Pixel devices as old as the Pixel 2 earlier this month.
Live captions captures audio from your device and runs it through on-device learning models. This works to generate captions from any English-language speech.
It works on a variety of media and calls on the device. This includes third party apps such as WhatsApp and Telegram. It does currently only support English language speech. However, it is still a very useful accessibility feature.
Live captions coming to older Pixel phones
Now that claim is coming to live with Google rolling out live captions older phones. As reported by the XDA Developers reported the feature has finally begun turning up in some devices.
As the below screenshot indicate this feature can be enabled by tapping on the new ‘Caption calls’ option. This is located in withing the Live Caption settings.
Once you have tapped on the ‘Caption calls’ option a pop up menu arises. This allows you to define when you want this feature to be used. You can choose from, ‘off’, ‘always’ and the ability to be asked whenever you are on a call.