What role did representing material properties and surface qualities play in visual communication strategies in the 14th and 15th centuries? Which materials appear more frequently in certain subjects, and what changes and trends can be discerned in the 15th century? Annotation data are needed to analyse the developments and patterns of material representation in medieval painting and graphic art on a broad source basis and to identify individual phenomena. Since the manual generation of such data is very complex and time-consuming, the KIKI subproject at the Department of Artificial Intelligence and Human Interfaces is testing and developing computer vision methods with and without deep learning for the automatic recognition of depicted material. The goal is the creation of larger annotation data pools for future analysis in the field of digital art history.
This will allow factors such as the position and proportion of the image area of the materials to be taken into account, as well as texture comparisons. Subsequently, correlations to specific subjects or geographical, genre or context-specific features canbe analysed. Connections to image annotation data, such as those available in REALonline for people, objects, actions, etc., can also be identified.
In conjunction with qualitative art-historical studies – among others on the Concordantiae Caritatis (Lilienfeld, Stiftsbibliothek, cod. 151) and the Salzburg Missal (Munich, BSB, Clm 15708-15712 ) – KIKI will work out how and by the use of which categories painted surface properties and material textures can best be recorded for art historical analysis. In a further step, a standardisation process for data collection with these developed categories will be created in the form of a guideline to allow for consistent results. Manual annotation of a selected corpus will be performed in the next work package. On the one hand, these annotations provide ground truth for computer vision analyses. Secondly, these annotations shall be analysed using quantitative methods from the digital humanities, in the sense of a distant viewing that allows the comparison of these image elements in many visual sources. The results of the latter – the patterns and peculiarities found in the data material – will in turn form the starting point for further qualitative art historical analyses.
