My Heritage In Color

AI Transforming Old Photos

My dad strictly keeps a schedule of Throwback Thursdays. An uploaded family photo inspired me to pick up a colorizer tool. Here is my post honoring my family’s history.

Photos by the author. My grandmother, Eulalia Santos, and my grandfather, Jesus Valerio take a photo. From left to right, my uncles, Rodel, Arthur, Abner, and my dad, Ford.

I used Jason Antic’s DeOldify, which is an AI-based way to restore old photos. A quick intro: It’s done using deep learning, which are massive neural networks that can take images and videos as inputs, and transform them to specification. In his method, he used Convolutional Neural Networks, GANs (Generative Adversarial Networks), self-attention (inspired from NLP), and Two Time-Scale Update Rule. He elaborates more in his Github page. I’ll be reading the paper when it comes up, Jason!

With a few lines of code, one can transform an entire folder’s worth of images and videos. For the non-programmer, you can also check out MyHeritage’s site, here. From there, you can use traditional computer-based, “Photoshop” enhancements like denoising, histogram equalization, etc.

There are three models freely available. Stable, artistic, and video. The artistic model yields more vibrant and detailed models, while the stable model yields less colorful images. In the following, I used the stable model since the artistic model created an overly warm image.

Photos by the author, transformed using DeOldify.

Beautiful right? The AI model got most things undoubtedly right. The faces, hair, lips, shoes, and the plant in the background look sharp. However, some details are debatable, like the color of grandfather’s polo (could be blue), grandmother’s ribbon (dark-red?), and the kid’s shorts (could be dark-brown). I asked Dad and he doesn’t remember.

More Photos

With this success, I transformed more images. In the following, I used either artistic or stable, depending on my subjective assessment of quality.

You may notice some incorrect details, most especially bluish-gray arms. I don’t know the reason for sure so I’ll hazard a guess. Biases are inherent in most datasets so I think that the arms are being mistaken for sleeves. Another possibility is that limbs are usually occluded so it’s very difficult to recover the original skin tone.