CE38 - Révolution numérique : rapports au savoir et à la culture

Augmented Artwork Analysis. Computer-aided interpretation device for art images – AAA

Submission summary

The project seeks to produce a tool for assisted interpretation of artistic images. Taking the form of a prototype adapted for three museum settings (Museum of Fine Arts of Lyon, Lille Palace of Fine Arts and the MNHA of Luxemburg), the device will allow the study of the different levels of organization of an artwork articulating its observation in situ with an open access corpus so as to elucidate genealogies and intertextual dialogs. Two concepts – the augmented artwork and the complementary museum – provide the foundation for a novel instrument for guided visits, pedagogical meetings, and research. Indeed, the user’s skills, knowledge and objectives are essential design parameters for the development and the usability of the tool. The scientific challenge consists of joining (i) recent advances in visual studies, (ii) affordances of augmented reality, and (iii) methods of detection and recognition of objects based on deep learning to produce a semiotic reconception of a history of artistic forms. The pedagogical significance of the project is thus a guided personal experience of analysis: a significant contribution to fill a gap in compulsory primary and secondary school education, whose curriculum is nearly devoid of programs devoted to visual culture – which is, after all, the culture of our time. From the Computer Science’s point of view, the idea is to bridge the gap between the intention and the semantic in image retrieval (the distance between the user’s search target and what is found) by working on pattern similarity, using high-level and low-level attributes in machine learning, as well as taking into account the user’s interaction to help guide the system to the desired result. From digital art history standpoint, the project of a history of forms is linked to Henri Focillon. He conceived forms as (i) germs of structural dynamics within a work of art and (ii) origins of families of transformations across artworks. This research framework, in spite of important predecessors and reprises, was never applied on a large scale. This is due for at least two technical obstacles: to process big corpora of images and to detect significant patterns that cut across disparate corpora. The study of diachronic changes in forms (plastic or figurative) and their genealogical ramifications are constrained by the media (materials and formats) or by the genre (motifs). This heterogeneity has to be cross-linked with a plurality of hermeneutic tools: internals (metavisual aspects, a kind of reflexivity of the images) and externals (paratexts). So a digital device will provide a homogeneous representation where one can (i) observe if images structure their own meaning guiding the viewer on the manner in which they are constructed and they ask to be seen; (ii) ascertain if the process of explaining an image can proceed by means of other forms of discourse (such as literary, musical, theatrical sources); (iii) elucidate if the process of explaining an art image depends on emphasizing its sensory / perceptual / experiential dimension. In other words, AAA project aims to understand any given image on 1) intratextual and 2) intertextual levels at the same time. Automatic processing is necessary on both levels, especially for consulting archives. Finally, the most innovative aspect is probably the experience that an image can be provably explained by a multiplicity of other images and not by a subsuming and reductive verbal discourse. AAA project is far removed from the kind of augmented reality built upon existing encyclopedic knowledge. The reading of images cannot be reduced to extracted information; on the contrary, it takes place as a process that must be actively learned and practiced over time. As a matter of fact, the AAA will articulate the close reading of a museum collection with the distant reading of some visual databases in open access.

Project coordination


The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.


IRMA Université du Luxembourg / Institut d’Études Romanes, Médias et Arts (IRMA, UR IPSE)
C2DH Université de Luxembourg / Luxembourg Centre for Contemporary and Digital History
DL3BFR FNRS / Département des Langues et Littératures Liège Belgique Françaises et Romanes, Université de Liège
CACP Université catholique de Louvain / Centre d’Analyse Culturelle de la Première Louvain-la-Neuve Belgique Modernité

Help of the ANR 399,829 euros
Beginning and duration of the scientific project: February 2021 - 48 Months

Useful links

Explorez notre base de projets financés



ANR makes available its datasets on funded projects, click here to find more.

Sign up for the latest news:
Subscribe to our newsletter