Over the past five years a group led by assistant professor Ira Kemelmacher-Shlizerman has constructed a machine learning algorithm that can piece together a digital persona of anyone from just a few hundred photographs. Celebrities, being frequently photographed, are a perfect target for this treatment. They can be mimicked using only publicly available photos.
The team’s latest breakthrough makes it possible to manipulate the expressions of anyone scanned by the algorithm. It can even transfer expressions from one individual to another. This technique is demonstrated in a number of disturbing yet hilarious videos. One clip shows Daniel Craig speaking alongside virtual representations of Ian McKellen and Tom Hanks uttering the same words. Another clip puts the words of former president George W. Bush in the mouth of current president Barack Obama.
As you might expect, the juxtaposition isn’t entirely convincing. The team is working to prove the concept, so the models it uses to replicate various celebrities don’t have the resolution and texture detail you’d find in a cutting-edge game or movie CGI.
Still, the 3D puppets get the broad strokes right, which means facial expressions are transferred with remarkable accuracy. Seeing the smile of George W. Bush on Barack Obama’s face is a bit hair-raising, to say the least.
The work is part of an ongoing effort that the researchers hope will provide the ability to fully recreate a person, to a high degree of detail, using photos alone. It’s already possible to recreate celebrities using computer animation, of course, but the techniques involved require the person to be present and involve tens or hundreds of man-hours to properly capture, animate and detail the digital persona. A machine learning algorithm, on the other hand, might be able to generate similar results with a fraction of the effort.
It might also open privacy concerns. Most people have several hundred photos uploaded to Facebook. That’s potentially enough to create an avatar using this technique. But don’t fret just yet. The technology is still in the realm of academic research, with no timeline for its completion.
