Disney's age-altering AI that could transform Hollywood

INA-  sources 

Disney researchers are out to transform Hollywood with a new artificial intelligence tool that could eliminate the need for special effects to alter the age of actors playing a character in a movie.

Using the face re-aging network (FRAN), filmmakers can now make actors look as young as 20 or as old as 80 years old just by inputting the person's headshot into the system that then predicts which parts of the face should be altered by age.

It then adds effects like wrinkles or smoothed skin as a layer on the actor's face in the film or television show without requiring a skilled artist to alter frame-by-frame manually, which typically causes facial identity loss.

Disney touts its AI tool as being the first solution of its kind that can automatically alter an actor's age on video despite expressions, lighting conditions and viewpoints.

Digitally altering the age of actors' faces has become popular over the years, but current models seem to fail at convincing viewers.

A recent use of de-aging technology was seen in Netflix's 'The Irishman,' which was filed by director Martin Scorsese who spent millions of dollars on digital de-aging effects, only for results to be criticized by viewers.

The gangster film cost around $159 million, but most of the funds went to making Robert De Niro look in his 20s - he was 76 years old when the movie was shot.

Many viewers noted the results were unstable across the video frames.

Between how he walks, moves, talks, it's so obvious it's an old man pretending to be 30-40 years younger. It just looks SO bad, and honestly makes me wish they used multiple actors to play different portions of Robert's character.'

This issue seems to be the top reason for Disney's FRAN.

'Although research on facial image re-aging has attempted to automate and solve this problem, current techniques are of little practical use as they typically suffer from facial identity loss, poor resolution, and unstable results across subsequent video frames,' reads Disney'sstudy that was conducted in partnership with the University of Wisconsin-Madison.

The work began with a database of 2,000 synthetically generated faces, each with 14 different ages ranging from 18 to 85 years, thus providing a total of 196 training pairs per sampled identity.

Researchers then used a 'proven' neural network to alter the ages of the synthetic faces, leading to the design of FRAN.

The neural networks used, according to the researchers, are well known for preserving the spatial layout of the input.

It is now capable of 'unprecedented temporal stability and preservation of facial identity across variable expressions, viewpoints, and lighting conditions,' according to the researchers.

And the temporal smoothness over the input video frames naturally contributes to the good temporal consistency in FRAN’s output. 

'Combined, these factors make FRAN an excellent, production-ready solution for re-aging real faces on video,' the researchers wrote in the study.

To re-age images, FRAN is asked to individually output images (frames in the movie or television show) from ages 20 to 80, done in 10-year steps.

The team notes that this method allows FRAN to handle different head poses and lighting, along with altering faces that may appear blurry in some frames.

However, the study notes this technology only focuses on adults and does not change scalp hair or the person's face shape.

While Disney has not shared how it plans to use FRAN, it has used such technology in movies like Ant-Man.

The Marvel film included a flashback scene set decades prior to the rest of the story, a young Hank Pym is shown leaving SHIELD due to their undisclosed attempts at replicating his shrinking tech.