AMC Using AI-Powered ‘Visual Dubbing’ to Premiere Swedish Movie In English

AMC Theaters, always one to hop on the latest trend, is premiering a Swedish movie using a new form of AI-based dubbing that makes it appear as if the actors are speaking in English. Instead of merely replacing the voice tracks with ones recorded by actors in another language, the movie Watch the Skies will additionally see actors’ facial movements altered to make it appear as if they are speaking in English.

The original actors in Watch the Skies voice the new English-language track, which is in part why the movie is compliant with SAG-AFTRA rules meant to protect members from the threat of AI. Hollywood talent has pushed back strenuously against the use of AI out of fear that they will someday be replaced entirely. But because the English dialogue was recorded by the actors themselves rather than generated by AI, and with full consent, it does not violate union contracts.

The company behind the “visual dubbing,” as it is called, is Flawless AI. By digitally altering the film’s images to make it appear natively filmed in English, the thinking goes that more moviegoers put off by dubbing will give the film a chance. Translating into another language can sometimes break the lip sync; this new dubbing alleviates that issue by altering the mouth movements themselves.

Some may not be happy about this new AI-based dubbing in film, as it is almost tantamount to declaring English is the superior language, erasing other cultures and dialects in the process. It also comes at a time when foreign language films have become increasingly popular in the United States and viewers have become more accustomed to using subtitles. Hollywood already dominates the global entertainment industry, the thinking goes, and automatically dubbing into English could be seen as an encroachment at a time when other countries have only just started to see their local film industries take off globally.

Critics might argue that much of the world does not speak English and should not be forced to bend to the will of Americans, but this seems like a minor case of translating a small budget movie intended for theatrical release in English-speaking markets.

There is also concern that something could be lost when a movie is not watched in its native tongue—the voice acting is a crucial part of any film, and actors switching to non-native English from Swedish might lose some of their emotional delivery or translate dialogue imprecisely.  But voice actors have gotten better at dubbing over the years, and proponents of dubbing say that people should watch films any way they prefer.

Companies beyond AMC have been implementing AI-based dubbing in order to expand the audience for content. The popular YouTube creator Mr. Beast made headlines several years ago for dubbing his videos in other languages like Spanish and Mandarin, but at the time he was using a more manual and costly process for doing so. YouTube late last year introduced a tool that lets creators instead automatically create AI voiceovers in other languages.

While the true value of new AI technology remains in question, there is hope in Hollywood that generative AI will be able to speed up the production process and reduce costs for visual effects. Companies like Netflix have disrupted the industry significantly by offering boatloads of cash for content only to pull back, causing a lot of instability and belt-tightening. Accessibility of AI could lead to its negligent use, however, as companies cut corners and do not use it with intentionality. Tools like OpenAI’s Sora can generate short video clips entirely from text prompts, but the product was criticized after release for its poor quality. Others from companies like Runway and Google have received better reception, though studios still face significant pushback from actors over their use.