Data Science Institute Meeting / NUI Galway / 28th November 2019

1. Old Ireland in Colour
(OIIC)
http://bit.ly/oiicslides
Professor John G. Breslin
VistaMilk and Insight SFI Research Centres
Data Science Institute
NUI Galway

2. For most DSI peeps, you have probably never heard of her
She was an Irish author and storyteller born in Kerry
Most non-millennial Irish people studied her in school (the Irish
Times says that “her mournful memoir was required reading for
a generation of Irish school students”)
Previously, all we had was that sad book and some black and
white photos of her
This is Peig Sayers

3. This is DeOldify
A deep learning model to colourise (old) black and white photos
Created by Jason Antic @citnaj

4. This is Peig Sayers with DeOldify [plus
TorchWarp (DL, morphs two images)]

7. And it’s not just photos… You can
DeOldify video too!

8. Who made DeOldify?
Made by Jason Antic, deep learning researcher/programmer
Took on the idea of using DL for image colourisation as a
hobby/project when he was undertaking the fast.ai course in
Spring/Summer 2018
Met up with Dana Kelley (collaborator on the project, left) and
Jason Antic when I was in San Diego in June 2019

9. ● GAN requires setting up the training of a colourising model
that involves a second model — the “critic” — that basically
is there to “criticize” the colourisations and teach the
“generator” to produce better images
● Since the “critic” model is also a neural network, it can pick
up on a lot of the nuances regarding what makes something
look more “realistic” (that simpler methods just cannot do)
● Output is often more colourful and convincing compared to
previous efforts (i.e. deep learning models for colourisation)
How does it work? (GAN — Generative
Adversarial Network) [more here]

10. More on how I got into this
“Donegal family research leads to amazing colour update for
black & white photos”
https://www.donegallive.ie/news/features/495528/donegal-fam
ily-research-leads-to-amazing-colour-update-for-black-white-p
hotos.html

11. Some favourites
1910 1911

12. 1902 1896 1914

13. 1963 Galway ~1942 UPenn

14. 1906
1882

15. The Stoneys, Aula Maxima, Alice Perry
Edith Stoney = first female medical physicist in the world
Florence Stoney = first female radiologist in UK or Ireland
George Johnstone Stoney = originator of the term “electron”, NUI Galway professor
Alice Perry = first female engineering graduate in UK or Ireland

16. 1902

17. 1932 (two Galway men)

22. Often requires colouring in of arms,
legs or faces: sometimes a lot of them!

23. Before you ask, historical accuracy is a
challenge (“Golden” Gate Bridge)
But more about how to tackle that later...

24. Mass colourisation using DeOldify
● Running on my MSI Aegis Ti3 8th desktop
● Using a local installation of DeOldify’s Docker/API instance
on Ubuntu with NVIDIA Docker and CUDA (there’s also a
Jupyter interface)
● Used FlickrDump to get images from NLI, PRONI and wget
from Dúchas
● Processes a photo every second or so
● 20,000 done in a number of hours
● Downloaded larger image banks for future retraining

25. 2729 photos

26. 2778 photos

27. 3267 photos

28. 11254 photos

29. Social media: >15k followers in three
months (Facebook, Twitter, Instagram)

30. 2.2M engagements on Twitter in 3
months

31. Facebook has the most followers
(nearly 8k likes)

32. Instagram is quieter but still busy

33. Inspiration for new “countries in
colour” accounts from various people
around the world
Pakistan, New Zealand, Argentina...

34. Back to the historical accuracy
● Some knowledge of the clothes someone/some group
wore, paintings, traditions, etc. (e.g. red Galway shawls)
● Usually requires some additional touchup or manual
adjustment
● New techniques allow you to spot touch areas with a dot of
colour and then spread/apply that to a region, or even to
the same region across video frames
● But the most standard way is to fire up PhotoShop

35. Countess Markievicz

36. Amelia Earhart

37. Titanic

38. DeOldify links
DYK DeOldify only kicked off as a project in the summer of 2018!
https://github.com/jantic/DeOldify (code, etc. from Jason Antic)
http://bit.ly/deoldifyimages (DeOldify images on Google Colab)
http://bit.ly/deoldifyvideos (DeOldify videos on Google Colab)
www.colorize.ml (commercial app, uses DeOldify, third party)
http://bit.ly/deoldifytutorial (by John)

39. DeepRemaster: Temporal Source-Reference Attention Networks
for Comprehensive Video Enhancement
iizuka.cs.tsukuba.ac.jp/projects/remastering/en/index.html
github.com/satoshiiizuka/siggraphasia2019_remastering
Interactive Deep Colorization
github.com/junyanz/interactive-deep-colorization/
See also

40. Thank you
Any questions?
(Irish photos from Dúchas used under a CC BY-NC 4.0 license,
most other Irish photos used are public domain)