The transformational potential of 3D reconstruction for research on historic contexts and cultural heritage is widely recognized but rarely realized. In terms of standards, the field remains the ‘wild-west’ of Digital Humanities, with competing technologies, national research silos, and divergent humanistic approaches ensuring an overall absence of sustainability.

While visualizations of various types have become a staple of the Digital Humanities approaches deployed in art and architectural history, a fundamental concern for researchers remains how analogue data, like artworks and drawings, archival documents etc., can be integrated within such born-digital research outputs. Without embedded and transparent metadata, these outputs struggle to gain acceptance as valid vehicles for research alongside conventional peer-reviewed articles and books, and the opportunities offered by these digital tools lie dormant in humanities research.

The issue is especially pronounced in the field of 3D modeling where there are, as yet, no comparable common standards for creating models or integrating underpinning data within them. The need for shared protocols is becoming more pressing as tools and practices associated with modeling technologies are increasingly deployed to a range of art historical visualization projects focused on specific places and eras. It is this need that can be addresses only through an integrated and collaborative approach that draws on the research expertise of an interdisciplinary community and the emerging solutions and standards in ‘serious’ industries concerning 3D and data engineering/modeling.

Interest from the humanities in computing as an aid for data processing and analysis (especially in linguistics) was already evident in the 1950/60s,1 but significant engagement by art history dates from the 1980s, driven by the needs of image and object classification projects, especially those promoted by the Getty Foundation.2 An important role was played by the iconographic indexing of image collections in a formalized, computerized language and the emergence of controlled vocabularies.3 With the development of digital images and computer graphics (especially CAD software), the increase in computing power, and the spread of PCs, the first attempts at computer-assisted visualization in object-based research caused a sensation from the mid-1980s on. Early projects like the 3D reconstruction of Westminster in the UK or Cluny III in Germany pioneered what would become, by the 1990s, a boom in digital 3D reconstruction projects,4 with an ever-increasing demand for computer-generated 3D models.5 The history of art and architecture recognized early on the potential of computer-aided 3D reconstruction and the digital 3D model. The undeniable advantages of the 3D digital over the 2D analogue approach include the holistic view, the stronger spatial reference, as well as the increase in interpretive power and knowledge resulting from 3D modeling and visualization.6

The advent of the Internet - and the increased engagement with 3D modeling it engendered - saw the emergence of critical reflection on the part of art and architectural historians regarding computer-generated visualization and the relationship between digital and analogue research outputs. Already in 2001, Hubertus Günther recognized that access to an unprecedented number of publications would often not be useful for traditional research and that the 3D model in connection with a database (here still understood as a separate software application) would constitute a significantly more effective form of publication or access to the research object.7 Günther’s predictions have been amply born out over the subsequent two decades. The quantity and complexity of data now generated by humanities research is testing the limits of analogue print publication. Even at building-scale, monographic publications often extend to multiple volumes, an increasingly uneconomical model with no potential for revision or integration of new research data.8

Also in 2001, Stephan Hoppe used the Altenberg Cathedral project to show the potential of the 3D model as an interactive knowledge space and introduced the idea of a “digital footnote of the model.”9 The Virtue of Models: CAD and the new spaces of art history10 conference in 2000 and the resulting publication raised hopes for the new medium at the same time as identifying outstanding challenges for the application of 3D modeling and visualization as a working tool and methodology in the history of art and architecture.11 The lack of transparency regarding primary sources, the interpretation of those sources, and the modeling process itself led to widespread skepticism within object-based disciplines, reinforced by lack of access and interoperability. Practice fell far short of recommended scientific practice (DFG, 2013),12 and fostered a negative impression of 3D models as an ‘unserious’ field in terms of scientific research that persists today.

EU projects such as EPOCH, Carrare, and 3D Icons addressed these challenges, giving rise to the London Charter13 of 2006, the most widely-cited call for standards and best-practice in the field. The Charter argued that 3D visualizations should incorporate their underpinning data and also the processes used in their own creation, introducing a new concept of paradata for creative interpretation alongside the existing understanding of metadata. However, both the London Charter and its successor, the 2011 Seville Principles,14 were unable to establish workable solutions, and their impact has been largely aspirational, raising awareness of continued shortcomings. There is still no consensus regarding technical and methodological approaches, and little agreement on standards for their implementation. As compared to progress made in 2D with IIIF, the 3D visualization of cultural heritage remains the ‘wild-west’ of digital humanities in which lack of transparency results in models that “only convey a small percentage of the knowledge that they embody” and are “highly resistant to peer review.”15

At this point, it is worth looking over the disciplinary boundary wall into the neighboring fields of the object-related engineering sciences and their handling of 3D modeling, which trace their origins back to the emergence of computer graphics in the 1960s. IT-supported automation using digital 3D models was pioneered in mechanical engineering. As early as the 1980s, standardized data exchange formats in the 3D field (STEP)16 were agreed, triggering pushes towards digitization (notably, for example, in the automotive industry), which brought substantial increases in productivity. In this arena, the ‘seriousness’ of the 3D models is founded on object-based 3D modeling, which can be enriched with alphanumeric information. The 3D model represents a digital twin of the real object as a modeled solid (object-based) with its properties (information) embedded. The standardized 3D data exchange format guarantees effective, interdisciplinary collaboration and minimizes information loss in the development and production processes. Since the late 1990s civil engineers have sought to catch up with their mechanical engineering peers in order to drive digitization in civil and urban engineering as well as long term building operation (smart buildings) and urban management (smart cities).

In the construction industry, the term Building Information Modeling (BIM) for a collaborative planning method using the Industry Foundation Classes (IFC) data exchange format has been gaining acceptance since the mid/late 1990s. In regional and urban planning, City Geography Markup Language (CityGML), a data exchange format for storing and exchanging 3D virtual city models, has come to dominate over the past two decades. Digital 3D models can occur in different Levels of Information Need in BIM/IFC17 or Levels of Detail in CityGML18 depending on the project phase or the degree of elaboration. In addition to geometry and appearance, the semantics (meaning) and topology (relationships) of the objects are also described regarding the project specifications defined in the “Employer's Information Requirement” (EIR). The EIR is an important element of a Project BIM Implementation and is used to set out clearly to the Project Team or Supplier what information (models, documents and data) is required generally and at each project stage.

While BIM applications in construction are well-documented (Penn State University, 2013)19, the absence of standardized international workflows for integrating these advanced techniques with Cultural Heritage Assets persists. Standardizing Cultural Heritage (CH) metric documentation and Heritage/Historic Building Information Modeling (HBIM) presents challenges, as each CH asset requires distinct information across its lifecycle, encompassing physical and intangible attributes like history and culture. Tasks involving CH, such as knowledge acquisition and restoration, demand specific data inputs. Though guideline books offer general proposals, they merely represent initial standardization efforts.20 Consequently, HBIM has emerged as a novel scientific field focusing on digitally representing CH assets. Ongoing research evaluates methods and templates for standardized procurement workflows, streamlining documentation of historic structures and heritage assets while cooperating with preservation authorities.21 Utilizing diverse measurement techniques, registering point clouds, transferring geometry to BIM-compliant 3D models, and sustainable data storage are vital components of this project.

In parallel to the aforementioned efforts to create ‘serious’ digital datasets in industry and the engineering sciences around production, construction, and operation, we need to acknowledge the development of the internet and its data structure. Since Tim Berners-Lee’s presentation of the Semantic Web, a seminal call for meaningful online data being published alongside unstructured documents,22 there has been increased interest in the formalization of knowledge, networking, and use of distributed Linked Data resources of the kind Lutz Heusinger foresaw in his 1983 theses.23 CIDOC CRM (ISO 21127:2014) has been developed since the mid-1990s as a reference ontology for the human- and machine-readable representation of knowledge and controlled exchange of information in the cultural heritage field. Today, the fundamental cornerstone of the Semantic Web and Linked Data technologies is a technical approach developed by the W3C, the Resource Description Framework (RDF), for formulating statements about arbitrary things (resources), whereupon a logical language (OWL) based on formal ontologies is placed.

From 2010 onwards, we can observe a growing number of interdisciplinary projects in the context of historical 3D reconstruction of architecture. A comprehensive overview on the use of digital 3D technologies in the humanities is given by Münster in his postdoctoral thesis.24 The use cases which incorporate human- and machine-readable documentation includes projects from archaeology using the Games Engine, such as the Oplontis project,25 and from art history, such as the documentation of the reconstruction processes (paradata) of the Zwinger palace in Dresden in the DoKuVis project (Bruschke and Wacker, 2016)26 and the documentation of the transformation of the old town of Nuremberg in the TOPORAZ project.27

Special mention needs to be made here of the leading role of the project at the Herder Institute for Historical Research on East Central Europe in developing the first CIDOC CRM referenced application ontology for mapping underpinning sources and the processes involved in the 3D reconstruction of royal castles in the former German territories of East Prussia (Kuroczyński/Hauck/Dworak 2016). As a result, a Virtual Research Environment (VRE) developed in the DFG infrastructure project (WissKI)28 was adapted for the documentation of 3D reconstruction projects and the prototype presented for a web-based Virtual Museum.

The experiences from the project on royal castles were further developed at the Institute of Architecture at Mainz University of Applied Sciences. Here the use of BIM was evaluated for the art and architectural history examination of the 1938 destroyed New Synagogue in Breslau, in present day Wrocław (Poland), in conjunction with a VRE and an adapted application ontology. The project provided new insights regarding the requirements for object-oriented 3D modeling, alphanumeric information storage including the BIM internal classification system, and the use of controlled vocabularies and authority files.29 In parallel, the same approaches were applied to 3D reconstruction documentation of the mediaeval cities of Mainz, Worms, and Speyer, exhibited in a Digital Urban History Lab at the Landesmuseum Mainz.30

This preliminary work took particular account of the requirements of good scientific practice, especially with regard to documentation, but also flagged fundamental questions about object-based 3D modeling and the seriousness of the research data generated in light of the FAIR Principles proclaimed in 2016.31

The proceeding discussions pushed by The Virtue of Models conference and the emerging community, here decisively represented by the Working Group Digital 3D Reconstruction32, also highlighted the importance of critical reflexivity, drawing attention to the epistemological implications of digitizing, modeling, rendering, and reconstructing art historical objects and architectural spaces, and how these sit within art history’s established practices of critical interrogation of media.33 Digital visualization invites us to consider how the materiality of objects is handled in the 3D model and to acknowledge the agency of aesthetics (in the sense of design) in the modeling process. Critical analysis helps us to deconstruct the models’ seductive promise of reality, while prompting us to reconsider our established research questions and the very ways we do research. As we reflect on the model and its underpinning data, so we are obliged to rethink the way evidence is assembled, and the heuristics of our disciplinary practices.

A model is a research environment for testing data and reflecting on the certainty of hypotheses derived from the interpretation of data, while also serving as an invitation to consider complexity, varieties of interpretation and the risks associated with the compelling visual presentation of a photorealistic reconstruction. This capacity for critical reflection is often lacking in Digital Humanities work, but within an interdisciplinary framework provides an essential foil to the positivist language routinely deployed in engineering fields, where - to cite one telling instance - a BIM model is often termed a ‘Single Source of Truth’. A source that holds a generally valid data set of the project and that claims to be accurate. One can rely on the fact that this data is correct. Information about the data (metadata and paradata) provides the user with information about how the data was obtained (data quality) and thus increases the validity of the data.

The special significance of the 3D model in the context of construction industry is based on the desire to efficiently overcome the greatest challenge of the construction site, namely communication, by means of object-related information. The so-called integrated coordination model becomes the working and communication medium in the planning and construction process and ultimately the digital twin of the building in the operation phase. If we transfer this basic idea behind the digital planning method BIM to the requirements of object-oriented research, we can speak of a Scientific Reference Model fig. 4, which can become the object of communication and discussion and subsequently become a research space itself. The basic assumption of the Scientific Reference Model is the need to establish a standardized working method that ensures flexibility, sustainable provision, findability, interoperability and reusability (in Edition).34

According to the concept, the Scientific Reference Model is fulfilled only when a 3D model resulting from a research project is made available online in standardized data exchange formats accompanied by structured human- and machine-readable metadata and paradata. For this purpose, the data exchange formats of BIM/IFC and CityGML mentioned above are suitable, which, depending on the specific project requirements, incorporate different degrees of elaboration with regard to the geometry and the stored information.35

Thereby, the great advantages of a flexible handling of the information in the 3D models can be used efficiently (cf. fig. 1). Depending on the defined Level of Information Need, the model can meet the different requirements of the research projects. By using established and standardized formats, the model can be shared, and further used independently of the software and across disciplines fig. 5, fig. 6. The raising of awareness with regard to a common working method in the sense of the Scientific Reference Model and the resulting provision of the work results strengthens the seriousness of the 3D models in the direction of the acceptance as 'valid vehicles for research' postulated at the beginning.

The above-mentioned concerns with standards, transparency, uncertainty and sustainability were caught up at the international conference Visualizing Complexities: Practices and Heuristics of Digital Models in Art History36 in Rome in December 2020 and were refined in the conference The Future of the Virtual Past: New Directions and Shared Standards in the Reconstruction of the Medieval-Renaissance Built Environment37 in Cambridge in June 2022.

Discussion at the Cambridge conference cemented the consensus that BIM, and more especially its historical/heritage extension HBIM, represented the best pathway towards sustainable standards and potential interoperability across academic research and the cultural heritage sectors. It sharpened the focus on ‘Serious 3D’ with the capacity to go beyond the digital conservation and documentation of extant structures, to encompass the modeling of non-extant structures, and the integration of building-scale and object-scale data with attendant para- and metadata, acknowledgement of uncertainty and hypothesis.

Our discussions parallel a series of infrastructure projects at national level currently under development, like Consortium for Research Data on Material and Immaterial Cultural Heritage (NFDI4Culture)38 and the Specialized Information Service BAUdigital39 in Germany. The national efforts meet the requirements of the scientific community regarding the ‘New Data Culture’ proclaimed by German Council for Scientific Information Infrastructures (RfII) “Ensuring sufficient data quality over the entire data life cycle requires a joint effort. Each of the producers, processors, and various reusers of research data must assume responsibility for ensuring the quality of the data. This requires a new culture of open data and data sharing, but also agreements on standards.”40

It is now time to define the digital 3D reconstruction as a research method and introduce the digital 3D model as an established research space. The idea behind the Scientific Reference Model and the resulting Serious 3D for art and architectural history provides an important step and is currently being explored at five universities within the European project 'Computer-based Visualization of Architectural Cultural Heritage' (CoVHer).41

The following article is based on a proposal submitted to the UK-German Funding Initiative in the Humanities (AHRC-DFG) under the same title in February 2023 in collaboration with Donal Cooper (University of Cambridge) and Fabrizio Nevola (University of Exeter). Thus, I would like to thank the colleagues from the United Kingdom for their good cooperation, further hints, and some eloquent formulations. Furthermore, I extend my gratitude to Tanja Michalsky and Elisabetta Scirocco from the Bibliotheca Hertziana - Max Planck Institute for Art History for inviting me to participate in the Rome 2020 conference, which provided an excellent setting for further discussions and the starting point for the Cambridge conference in 2022.