Google Arts & Culture has launched a new interactive hub called “Klimt vs. Klimt – The Man of Contradictions,” which will offer visitors insight into Gustav Klimt’s biography, artistic inspiration, legacy, and more, along with scholarly articles penned by experts at Austria’s top institutions, including the Belvedere and the Wien Museum.
As part of its efforts, Google also digitally recreated three of Klimt’s lost works from 1899, using artificial intelligence technology to colorize black-and-white photography of the works.
The three paintings-Medicine, Philosophy, and Jurisprudence-together form the “Faculty Paintings,” which was commissioned by the University of Vienna. Upon the works’ unveiling, university officials deemed them “pornographic” and “perverse.” They were ultimately sold to a private buyer.
In 1945, during the final days of World War II in Europe, the paintings were lost in a fire, and they now survive only in the form of black-and-white photographs. (In a post on the hub, the Austrian National Library looks at the mysterious circumstances of the fire that claimed the works.)
With the support of Google, Klimt scholars and AI researchers came together to bring color to these enigmatic paintings. Franz Smola, a curator at the Belvedere, worked with Emil Wallner, a resident at the Google Arts & Culture Lab, to develop an algorithm that could accurately recolor the “Faculty Paintings.”
First, Smola combed through various archives to find documentation describing the works, especially their coloration. He then looked for motifs present in the “Faculty Paintings” that could also be found in similar paintings of Klimt’s from the same time period. For example, the three women in Jurisprudence are wrapped in snakes, similar to the artist’s Beethoven Frieze (1902), in which a woman is also surrounded by serpents colored in his trademark gold leaf. These parallels helped Smola and his team to vaguely sketch the color profiles used in parts of the paintings.
Meanwhile, Wallner began putting together an algorithm that was trained to do a statistical analysis of Klimt’s coloring style. The obstacles and breakthroughs of this machine learning process are described in an article by Wallner.
Using a huge data set (1 million pictures of the real world and 91,749 image of artworks by various artists), the algorithm ascertained a sense of skin tones and the hues of the sky from the real world images, and a sense of composition, object boundaries, and textures from the paintings. Added to that initial data set were 80 full-color reproductions of Klimt paintings, meant to key the algorithm in to the Vienna Secession artist’s style. The resulting AI-colored images provide what might be the closest we will ever get to seeing a complete image of those lost paintings.