YouTube has introduced a new AI-powered feature aimed at breaking down language barriers for users worldwide. The video-sharing platform, in collaboration with Google’s Area 120 incubator, announced the integration of an AI dubbing service called Aloud at the VidCon event. This service aims to make it easier for creators to dub videos in different languages.
According to Amjad Hanif from YouTube, the tool is currently being tested with “hundreds” of creators and supports a few languages, including English, Spanish, and Portuguese. However, the company plans to expand the language options in the future.
The process of using Aloud involves transcribing the video content, which is then presented to the user for review and editing. Afterward, the tool translates and generates the dubbing for the video. The Aloud team has provided a video demonstration to illustrate the functionality.
For creators, Aloud can be a valuable tool, even with its current limited language support. It enables them to add multi-language dubs to their videos, facilitating broader accessibility and audience reach.
YouTube is actively working on enhancing its AI dubbing feature to make translated audio tracks sound more natural and expressive, closely resembling the creator’s voice. The goal is to improve lip sync and overall audio quality. These new features are expected to be introduced by YouTube in 2024.
In a recent development, YouTube has also begun utilizing Google DeepMind’s AI technology to generate descriptions for YouTube Shorts. The visual language model known as Flamingo is used to analyze the initial frames of a video and generate relevant descriptions. This helps improve the accessibility and discoverability of YouTube Shorts content.
