Coloring monochrome photographs is a practice that dates back to the earliest days of photography. With color photography unavailable to most consumers until the 1940s, members of the public who wanted to enhance the reality of photos often experimented with hand-coloring as an alternative. It was an incredibly time-consuming effort involving watercolors, oils, or pastels — and the results, while certainly distinctive, don’t necessarily look quite as realistic as their creators may have hoped.
Today, turning black-and-white images into full color photographs is a whole lot easier — and the results far, far better. How easy? All you need to do is post, or spot, a monochrome image on Twitter and reply to it with @colorize_bot. Like summoning a colorizing Captain Planet, Colorize_bot — as it is known — will instantly spring into action with its digital Crayolas, leaving the user with a freshly multicolored image and a cheery, “Of course, nice to help you!” This all occurs in the space of just a few seconds.
And you’ve got a 21-year old computer science and engineering student from Ecuador — and some nifty A.I. tools — to thank for it.
A pandemic project
“Colorize_bot is without a doubt my best project,” creator Geovanny Zambrano told Digital Trends.
As with many projects now doing the rounds, Colorize_bot sprang to digital life as a result of the pandemic. Finding himself stuck at home with “a lot of free time” on his hands, Zambrano decided to create a Twitter bot to practice his JavaScript skills.
These simple software tools, of varying usefulness, can be called on to perform an assortment of autonomous actions on Twitter — whether it’s tweeting links to free e-books on Amazon or mashing together news headlines for comical effect. “I had many ideas,” he said. “From a bot that publishes a motivational phrase every hour to a bot ‘hour translator.’ The objective of the project was never to gain followers. My goal was just to create [something useful.]”
At this point, he remembered a YouTube video he had seen seven months earlier that detailed how machine learning can be used to remaster old photographs. Zambrano came up with the concept of building on some of these A.I. tools, which were freely available online, and transforming them into a colorizing tool. He started on October 26 and worked for a month, putting in about three or four hours each day.
The first version of the bot — a limited one that could only cope with one single image per tweet — was posted on November 28, 2020.
How Colorize_bot works
Whenever a user tags Colorize_bot in a tweet, it is activated instantly using a webhook that monitors for summons 24/7. Once a mention is captured, processed, and validated, it then gathers the monochrome images and passes them onto another colorization A.I. tool. This one was not created by Zambrano, but is rather an open-source model developed by researcher Jason Antic.
As described on Github, the model uses a variation of a Generative Adversarial Network (GAN), the discriminator and generator A.I. system that has previously been used to create everything from fake human genetic code to A.I. paintings. The NoGAN tool can be used to colorize both still images and video, although the latter unsurprisingly takes a bit longer. As Antic explains in a post about the model, even he isn’t totally sure of how it extracts the kind of data it does for colorizing images. It just learns this from massive amounts of data, which can then be prodded in the right direction by using the proper algorithms.
“My best guess is that the models are learning some interesting rules about how to colorize based on subtle cues present in the black-and-white images that I certainly wouldn’t expect to exist,” Antic writes. “This result leads to nicely deterministic and consistent results, and that means you don’t have track model colorization decisions because they’re not arbitrary. Additionally, they seem remarkably robust so that even in moving scenes the renders are very consistent.”
The completed image is then passed back to Colorize_bot to post on Twitter. The entire system of capturing an initial mention through responding to a tweet takes only 10 seconds in terms of processing. However, the bot is programmed to respond only every 30 seconds so as not to break Twitter’s rules about spamming. It also only responds to one mention per hour, per user as a way to save on infrastructure costs that, as a student, Zambrano can’t easily afford.
“Currently, I pay around $30 a month to keep the project active,” he said. “This has been financed by myself, using my savings. In the future when I can no longer pay this, I am thinking of asking for some type of donation or sponsorship from an institution that’s interested.”
A measure of success
The bot isn’t perfect, and nor is it a miracle worker. Low-resolution images, for instance, give low-resolution results. More notably, Zambrano said, Colorize_bot fares poorly when it comes to coloring manga images, which people frequently ask it to do. “This, at a technical level, is due to the fact that the colorization model was trained with real images,” he said. When it’s asked to color images of a different type from the ones it was trained on, the results are less than perfect.
Nonetheless, it’s proven impressively accurate in many cases, enough so that Colorize_Bot has already picked up more than 30,000 followers on Twitter in just a few short months. A quick search for people calling it into service reveals that it’s being asked for at least every couple of minutes.
“The best stories I have as a result of the development of this project have been the people who write to me, thanking me for coloring old photos of their relatives,” Zambrano said. One person even sent him a photo of their mom, holding a framed photo of a colorized image of an ancestor. It’s hard to work out the value of a project such as this in monetary terms. But in terms of feeling like those months of coding were well spent? These stories are an indicator that he was onto a winner.
“What a moment I lived that night, it was a feeling of satisfaction, joy, and optimism,” he said. “Seeing that beautiful mother holding her printout made me know that, many times, the things that we do simply for the purpose of helping other people can have a great impact.”
