Photo: (Andaloro 2006).

In synthesis, the Atlas:

The Atlas, therefore, allows us to visually re-read pictorial documents within an integrated system.

Our aim was not to advance ideal or evocative reconstructions, but to create 2D and 3D graphic elaborations designed to emphasize the link between the wall paintings and the architectural contexts as they were originally designed.

We started from a 3D model derived from the direct survey of the architectural structures (at first using a total station and subsequently a 3D surface point cloud).4 When the monuments were destroyed or significantly modified by subsequent interventions, we used indirect surveys on the basis of historical documentation, rebuilding exclusively the internal space of the churches.

Onto the latter we then added images of:

These reconstructions were originally conceived for printed volumes, but this is only the first step of many, which aim to visually transmit Rome’s medieval heritage to a wider audience, using mainly the Web. We now aim to integrate 2D and 3D reconstructions with visual and textual content in a specific geolocated digital environment. Our intention is to offer visual paths through which diverse audiences can enjoy the paintings of the Roman Middle Ages within their original architecture in and around the city. The aim for the near future is to join a digital platform such as DynASK-Dynamic Atlas of knowledge developed in the Department of Architecture of the Roma Tre University (scientific coordinator: Antonio Pugliano). This digital information system on a geographical basis aimed at deepening the knowledge of Rome and its Metropolitan Area by protecting the urban fabric from exploitation while at the same time forecasting seismic and hydrogeological risks.5 The platform already comprises several cartographies of Rome ranging from the Forma Urbis to Nolli’s map, from the Gregorian Land Registry, to both Lanciani’s and Muratori’s maps. All geographical and historic information are collected in a database both textual and graphic; the information in the GeoAI system is redrawn in AutoCAD and implemented into a GIS. At the moment the system manages over two thousand thematic information layers.6 In this frame, our 3D models of churches (from 4th to 15th century) will be linked to the maps of medieval Rome, such as Muratori’s, which, in the future, will allow the larger audience to explore the city and discover its pictorial treasures (fig. 2).

To this end, the 2016 exhibition Santa Maria Antiqua. Between Rome and Byzantium, was a crucial test for us. The exhibition was curated by Maria Andaloro, Giuseppe Morganti and me on the occasion of the extraordinary reopening of the church after a restoration phase which lasted fifteen years.7 The church of S. Maria Antiqua at the Roman Forum was discovered in 1900 on the west slope of the Palatine Hill. Its surviving wall paintings represent a unique heritage in the Christian world between the 6th and 10th centuries. The pictorial decoration testifies to the development in the Middle Ages of both painting in Rome, and — due to the destruction brought upon sacred images by Iconoclasm — that of the entire Byzantine civilization.8 The 'palimpsest wall', considered by many scholars a milestone in the history of medieval painting, bears witness to the development of the decorative phases of the church through the overlapping of eight layers of murals (fig. 3).9

The exhibition exposed to a wider audience a multimedia path marked by video mapping and light mapping, used for the first time in the Roman Forum, alongside videos aimed at displaying, through an emotional journey, the history and complexity of the pictorial phases of the monument. The Santa Maria Antiqua video, projected inside the monument on a video wall, allowed one to browse, with the aid of dynamic images, the extraordinary pictorial palimpsest preserved on the apse wall of the church and to follow, by means of virtual reconstructions, the metamorphoses undergone by the pictorial decoration on the walls of the monument over five centuries (fig. 4).10

4 Figure 4.

© Katatexilux

After a long and often painstaking physical restoration of the frescoes, a virtual restoration was used. This enabled the general public to ‘read’ the iconographic fabric in its entirety, as opposed to its fragmented surviving state. Based on the 3D reconstructions proposed in the Atlas, the video mapping, then, allowed to project — with an immersive approach — directly onto the walls of the chapels on the sides of the presbytery, the complete pictorial fabric as well as the decoration in opus sectile. The projection started with a line-drawing and finished with a complete virtual restoration of the painted decoration (fig. 5).11 The installation was enormously successful and is still open to the public as part of the special route offered to visitors of the Archaeological Park of the Colosseum. The exhibition was conceived as a temporary installation, but became a permanent attraction. This will require a review in the short term, giving the opportunity to reflect in greater detail around the necessity of tailoring and creating bespoke experiences for different and often diverse users.

With this new problem in mind we took part in the project called EHEM.12 Enhancement of Heritage Experiences: the Middle Ages. Digital Layered Models of Architecture and Mural Paintings over Time, funded by EU Horizon 2020, JPI CH Conservation, Protection and Use (JPICH-0127).13

EHEM is a three-year project, managed by a consortium of the Universities of: Barcelona,14 Roma Tre,15 Tuscia (Viterbo),16 and the CYENS Center of Cyprus.17 The EHEM team comprises art historians, CS engineers, restorers, and scientists.18

The project aims to obtain virtual reconstructions of medieval artistic heritage — architecture with mural paintings — over the centuries by incorporating art historical knowledge and the diachronic perspective of heritage, as an instrument for researchers, restorers and heritage curators and to improve the visitor's perceptions and experiences.

In the 3D models elaborated we intend:

We selected three case studies:

By working at the same time on the three monuments we will be able to understand the benefits and limitations of adopting an ontology that drives workflows and standardizes methods.

In relation to Santa Maria Antiqua, one of the aims of the project is to display and showcase each and every phase tied to the ten layers of fresco that we mapped inside the church. We will create 3D digital models that incorporate this information in layers in the form of a database that allows us to collect archaeological, architectural, art historical documentation, textual and graphic data, together with the current state of knowledge on the site. According to the EHEM approach, the lifecycle of the building will be digitally reproduced in all its temporal phases, like a time machine. Layer upon layer we want to piece together the pictorial decoration of the church through its iconography.

The 3D model will also incorporate information on the different materials and techniques linked to the ten pictorial phases gathered in more than fifteen years of surveys and restoration campaigns.

The project is structured into four blocks (fig. 8). Block 1 specifies the actions required to acquire the data and associated information of the temple; Block 2 embraces the different research activities that deal with generating the faithful and complete models; Block 3 consists in storing and enriching the models in a common open access platform; Block 4 is about making results available to different users. We organized the project into nine work packages (fig. 9). WP3-7 refer to scientific activities; the others are functional: for example, management (WP1); specifications (WP2); evaluation (WP8); and dissemination (WP9).21 WP3 encompasses the tasks from the acquisition of images, to the placement of the paintings onto the digital model of the building.22 These tasks include: acquisition of faithful color specification using a Hypercolorimetric Multispectral Imaging (HMI) technique;23 digital color restitution; virtual restoration of murals; collection within the digital model of the detached paintings stored in different locations. WP4 foresees the digitization of buildings and their geometric processing in order to obtain a mesh adapted to the needs of the project (intended to interact with WP3).24 WP5 involves providing the simulation of the lighting suited to each period and to associate it with distinct visualization phases (WP7). It includes natural lighting through openings, simulated according to weather conditions and seasons, and artificial lighting produced by different sources such as candles and oil lamps. To simulate the lighting, Unreal Engine 5, a modern, high-performance game engine in combination with Radiance (lighting simulation tool) and Lumen (lighting software by Epic Games) are being used in the 3D models of each site.25 WP6 concentrates on storing in an adequate platform that enables one to analyze, document, share and enrich the models produced by WP3 and WP4. We foresee to annotate the models. For annotations requiring simple regions, we plan to use the Annotorious JavaScript library, as far as targets with complex boundaries, we have developed a tool prototype (based on TagLab).26 WP6 will provide feedback to WP3 and WP4. WP7 is intended to present the results to the different targeted users (general public; CH restorers and curators, the academic community) designing and implementing the suited GUI to each community, guided by the evolutionary approach of the project.27 WP7 provides feedback to WP6.

10 Figure 10.

Photo: CYENS Photo Archive.

I would like to conclude with a reflection that occurred to me while observing the visitors at Santa Maria Antiqua and which touches on our anxiety to produce 3D models that are increasingly realistic and seductive from an aesthetic point of view. In the Theodotus chapel spectators enter, watch the video mapping and leave without dwelling on what is really painted on the walls. A few months ago, I read a small but illuminating book entitled Bassa risoluzione (Low resolution) by Massimo Mantellini, a leading expert on the Italian Internet network.28 Mantellini states that although we live in an era of high resolution, low-resolution contents are passed off to the audience. The Internet has radically changed our approach to depth, information, social relations, markets and culture. Therefore, the choice of low resolution is a choice of reduction, a renunciation of complexity. Working on the virtual restitution and storytelling of contexts found in medieval wall paintings, which are almost always fragmentary and difficult to understand, I wonder if the real cultural challenge that awaits us is, paradoxically, that of using all available technology to activate the general public’s imagination. In this manner, we might seek out tools that help them read history, contexts, and, in my specific field, architecture with paintings in their fragmented and complex nature, and thus we would forgo presenting them with depictions that are intact and mimetically restored in a virtual world.

I believe that elaborating high-resolution, realistic 3D models is more useful to the academic community, than for the public. The latter are likely to experience the models as spectators, and often they passively take in our visualizations of a monument without stimulating their imagination of how it could have been in the past.