Popular apps like Prisma turn photographs into different styles of paintings using AI programs trained to mimic certain styles. But, those apps are trained on what is called paired data — the AI is created using so many sample images that the system has already learned the difference between a landscape and a selfie.
The research group at UC Berkeley, however, uses unpaired data. In other words, it can take two photographs and transfer the styles to the other without a reference image. To do that, the team had to teach the program to learn the relationship between two photos. By training the network using many photos and checking the results with both software and actual people, the team developed a program that could “successfully” transfer styles without that reference image.
“Successfully” is used loosely since the researchers’ knowledge that systems using that reference image still had superior results. The goal, however, was to build a system that works without the reference since getting that reference data can be expensive or difficult in a number of scenarios.
Some images transferred better than others — while transferring objects with a similar shape like a horse to a zebra created some impressive results, objects with different shapes did not work, like trying to change a dog into a cat. An attempt to turn an iPhone photo into a DSLR photo is also included among the team’s list of failed image transfers.
While the real-world applications for turning an apple into an orange is questionable, because the system doesn’t need that reference image, the program is widely varied on the types of style transfers it can tackle. The program was successfully able to create a shallow depth of field from a reference photo and reimagine what historic paintings would look like if the painter viewed the scene in an entirely different season. The program was also able to create style transfers for specific artists instead of only a single painting, like turning a photograph into a Monet but not necessarily Starry Night.
