Semantic models and services for conservation and restoration of cultural heritage: A comprehensive survey

3.1.Ontology of paintings and preservation of art

Twentieth Century Paint Project1919^,2020 (20thCPaint) explored the preservation of 20th C. paintings in Asia and the Pacific [52]. Key objective of the project was the development of an online knowledge base that allows painting conservators to access integrated and structured information about conservation of 20th c. paintings and facilitates information exchange between painting conservators and other preservation experts.2121

In this context, [52] proposes the Ontology of Paintings and Preservation of Art (OPPRA), a semantic model specialized in CnR of paintings. The ontology was developed in order to: i) capture CnR documentation data, ii) capture data from published papers and iii) integrate the aforementioned data that derive from both internal and external data sources so that reasoning can be performed over them.

The development of OPPRA was organized in five stages [87]: i) conceptual modelling of information relevant to conservation of paintings, ii) reuse of existing models, iii) reuse of existing controlled vocabularies, iv) extension and refinement of the reused classes and relationships, v) evaluation of ontology applicability to services developed in the context of the 20th CPaint Project. The ontology is implemented in Web Ontology Language Description Logics2222 (OWL DL) using the Protégé2323 software (for further information see [80]). Also, logical rules were defined using OWL 2 RL2424 (implemented in OWLIM -currently GraphDB-2525 OpenRDF Sesame triple store2626).

OPPRA combines and reuses the CIDOC CRM [25], the Open Archives Initiative Object Re-use and Exchange standard (OAI-ORE) [61] and the OreChem model [60]. Certain classes of OPPRA extend classes from CIDOC CRM [25] and OreChem (see [87]). Furthermore, OPPRA reuses the following controlled vocabularies: Getty Art and Architecture Thesaurus (AAT) [47], Visual Glossary of the Australian Institute for the Conservation of Cultural Material (AICCM) [4], International Network for the Conservation of Contemporary Art Database for Artists’ Archive (INCCA) [87], RUG Spectral Database (from now on RUG) [53], Conservation and Art Materials Encyclopedia Online (CAMEO) [72] and US National Institute of Standards and Technology Chemistry WebBook (from now on NIST) [86]. OPPRA consists of 2325 classes and 181 relations (either new or reused) [87].

OPPRA classes and relations model information related to the following thematic clusters [52,87]: i) Painting, which represents the identity of the conservation object (e.g., title, artist, period, technique, genre, condition, owner, custodian, provenance), ii) Paint, which represents the production material(s) and technique(s) of the conservation object (e.g., paint name, type, chemical property, physical property, pigment, additive), iii) Paint Decomposition, which represents alterations of the conservation object (e.g., type, cause, physical/chemical process/reaction), iv) Paint Analysis Method, which represents the analysis of the conservation object (e.g., SEM, TEM, FTIR, Raman), v) Paint Conservation/Preservation Treatment, which represents the remedial or preventive conservation of the conservation object (e.g., cleaning, environmental conditions control), vi) Experiment, which represents experiments that either simulate a damaging process or test a potential conservation treatment, vii) Publications, which represents published works related to CnR processes, experiments, and case studies. Table 1 depicts the correspondence between the CnR aspects (see Section 2), and the thematic clusters covered by OPPRA.

Based on OPPRA, a system that consists of an online knowledge base and intelligent services, specialized in CnR and Conservation Science (CS)2727 was developed [87]. The system allows the import of documentation and description of experiments conducted by conservators and material scientists, and the uploading of the experimental data to the knowledge base. Furthermore, it allows automatic extraction of structured data about past research and experiments from publications and websites that are relevant to art conservation. The extracted data from published papers are integrated into the knowledge base, together with data from external databases, allowing their linking to the imported user data and consequently a seamless unified search over critical information about conservation, art history and materials science. The knowledge base and services do not appear to be open for access, while it is not clear if they are still in use.

OPPRA was evaluated by assessing its applicability to the aforementioned services [87]. The evaluation showed that conservators and scientists were able to document and link data using OPPRA, thereby achieving more precise data retrieval. Additionally, the automatic extraction of structured data from relevant publications achieved high accuracy. Finally, the semantic search of integrated cross-disciplinary data hosted in the knowledge base allowed more complex queries, compared to traditional data integration tools.

OPPRA used to be online available, though the url is no longer accessible.2828 The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.2.Monument damage ontology

The main objective of the Monument Damage Information System (MONDIS) project2929 was the development of the MONDIS system. MONDIS is a knowledge-based system dedicated to damage diagnosis and possible conservation interventions of historical buildings [20].

MONDIS project developed Monument Damage Ontology (MDO) in order to efficiently integrate, organize and process diverse information related to the domain of interest, and eventually support documentation and monitoring of historical buildings damages, as well as potential intervention planning/application [11,20].

The development of MDO was organized in three phases [20]: i) distinguishing the requirements of damage documentation according to literature and international standards, ii) establishment of the relations among damage factors based on CnR methodologies and workflows, iii) validation of the ontology with experts. MDO was implemented in OWL 2 using OpenRDF Sesame triple store.

MDO is divided in two parts: i) the core, which represents knowledge about damages of immovable CH and ii) special taxonomies which provide particular vocabularies for the documentation and analysis of damages and interventions [11]. MDO integrates (either partially or fully) the following taxonomies, thesauri, and glossaries: International Database and Gallery of Structures (from now on IDGS) [103], Material and component classification from Strufail (MCCS) [37], Taxonomy of Building Components for Performance-Based Earthquake Engineering (TBCPBEE) [94], RTS: Trıdnık stavebnıch konstrukcı a pracı (RTS) [11], ICOMOS illustrated glossary on stone deterioration patterns (ICOMOS) [111], On Site For Masonry Standard Damage Catalogue and List of Structural Typologies and Related Requirements (OSM) [11]. The core ontology consists of 59 classes and 119 relations,3030 while the version of the ontology with the special taxonomies consists of 1258 classes and 121 relations.3131

The core part of MDO is divided in five thematic clusters [11,19]: i) Component and Construction Description, which represents the physical and functional characteristics of a conservation object ii) Events, which represents occurrences that can influence the condition of a conservation object, iii) Damage Diagnosis and Intervention, which represents alteration processes that affect a conservation object, as well as remedial or preventive conservation and restoration activities that confront the alteration processes and their effects, iv) Risk Assessment, which represents the analysis of a potential event or effect that can potentially harm a conservation object, v) Measurement Assessment, which represents the measurement of the magnitude of alteration factors and effects. Table 2 depicts the correspondence between the CnR aspects (see Section 2) and the thematic clusters covered by MDO.

Based on the MDO, MONDIS knowledge-based system provides a set of tools for data import, editing, integration, processing, and visualization [18]. More specifically MONDIS includes the inputting applications i) MONDIS mobile/desktop app and ii) Ontomind profile, as well as the visualizing and supporting tools iii) MONDIS explorer, iv) knowledge matrix and v) terminology editor. MONDIS application allows the documentation (on-site for the mobile version) of the condition of historical buildings based on measurements and observations about examined damages [18,19]. The data are uploaded to the MONDIS server, and after being validated by the users -in order to verify the quality of the record- they become accessible through the MONDIS explorer. Once shown in the MONDIS explorer records can be integrated with extra information which was not collected and documented during the on-site examination of a building, via Ontomind profile. Ontomind visualizes the ontological mapping of records to the MDO as a simple tree-like structure. The records available in the MONDIS are semantically linked to their diagnosis and possible interventions which are visualized and presented to the user through the knowledge-matrix web-based application. Finally, the MONDIS terminology editor facilitates the browsing or editing of the taxonomies and term lists used in MONDIS software tools. The software tools are available online.3232

Each section of MDO was validated by conducting public workshops and internal meetings [20]. Furthermore, MDO has been populated with records which were used for the presentation of MONDIS software tools functionality [18].

Both the core MDO ontology and the MDO version with the taxonomies are available online. According to the bibliographic search, MDO has been re-used by [118] for the development of a new ontology within the CnR domain (see Section 3.9).

3.3.Color and space in cultural heritage knowledge representation

The Color and Space in Cultural Heritage (COSCH)3333 project aimed to enhance the mutual understanding of tangible CH documentation between the various experts of CnR and Preservation3434 of the CH domain [14].

An important outcome of the COSCH community research is COSCH Knowledge Representation (COSCH^KR), a semantic model which encodes experts’ knowledge on good practices for visual documentation and analysis (with non-invasive techniques) of tangible CH.

COSCH^KR was developed through an iterative process where the gathered knowledge was first verified by groups of experts [55,114]. The experts participated in the specification of relevant terms and vocabularies as well as in discussions over three representative case studies, contributing to the development of COSCH^KR class structure and dependencies as well as the specification of inference rules [114]. The ontology was implemented in OWL 2.

The top-level structure of COSCH^KR consists of five classes interrelated through five properties [14]. In general, the COSCH^KR ontology contains more than 750 classes, while its taxonomic hierarchy has on average five levels [114]. COSCH^KR subclasses are associated with inference rules, which cut across the top-level classes (e.g., Revelation of Underdrawing (subclass of CHApplications) has Requirement 2D_Data (subclass of Data)). While the COSCH^KR does not reuse some existing ontology, in the context of its development there was the intention to keep it in line with CIDOC CRM in order to maintain information interoperability.

The core of COSCH^KR consists of five top-level classes [14]: i) Physical Thing, which represents a conservation object to be measured in terms of its physical and visual characteristics, ii) CH Applications, which represents CH research questions applying to spectral or spatial data, iii) Data, which represents digital/analogue data and document types that are either generated or used to process existing/generated data, iv) Technologies, which represents technical processes, measurement principles, tools/instruments and the way they are set up to generate or process data, and v) External Influences, which represents limitations of a documentation/analysis project (in terms of location, budget, environmental conditions etc.). Table 3 depicts the correspondence between the CnR aspects (see Section 2) and the top-level classes of COSCH^KR.

Based on COSCH^KR, [114] describes a recommender system that enables experts from different subdomains of CnR and preservation of tangible CH to put forward their queries and get answers related to documentation/analysis strategies for CH objects and applications without worrying about the complexity of the backend model [55]. Particularly, the proposed system would allow seeking answers to queries of varying complexity and invokes the model to infer underlying facts and heuristics. First of all, the system aims to help users to identify useful factors for different documentation/analysis actions as well as factors that cannot be satisfied, using implicit reasoning. Afterwards, based on those factors, the proposed system would be able to recommend solutions, providing experts with an overview of optimal spectral and spatial recording strategies according to their needs.

The three representative case studies were not only used to enrich the knowledge model but also to evaluate the inference mechanism and results [114]. The evaluation results [114] showed that the inferences of the model are satisfying, though it is highlighted that COSCH^KR is still under development, and therefore the CH applications, inference rules and corresponding recommendations that are to be added in the future will improve the model and its performance.

The ontology is not yet available online, but it is intended to be documented and made publicly available [114]. Additionally, the OWL file may be provided (as noted in the project website3535). The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.4.DOC-CULTURE

The Development of an integrated information environment for assessment and documentation of conservation interventions to cultural works/objects with nondestructive testing techniques (DOC-CULTURE)3636 project explored non-destructive testing and evaluation techniques (NDT&E) in three axes: i) the NDT&E usage in assessment of conservation object condition and conservation interventions effects, ii) the documentation of NDT&E data through metadata and conceptual frameworks and iii) the implementation of an information system for the recording and storage of NDT&E data [57,58].

In the context of the DOC-CULTURE project a model which is based on CIDOC CRM [25] and Dublin Core (DC) [38] models together with a CIDOC CRM extension are proposed for facilitating the complete representation of CnR processes and stages, including NDT&E techniques [58,109].

For developing the CIDOC CRM extension, the intended user groups, the documentation requirements and the different CnR processes and stages were defined, and the main entities and properties of NDT&E were specified [58,109]. Next, different standards for modelling data related to the CH domain (namely Metadata Object Description Schema (MODS) [71], Resource Description and Access (RDA) [99], PREMIS [95] and Muse Meta [59]) were studied and employed for the representation of DOC-CULTURE entities and properties. Furthermore, extra classes required in order to better represent the NDT&E field were added. Particularly, based on the study conducted in the context of the DOC-CULTURE project, the extension of three CIDOC CRM classes (namely E7_Activity, E11_Modification and E4_Period) were proposed [58].

The five main entities of DOC-CULTURE model are [109]: i) Artifact, which represents the identity, materials and technique and preservation state of the conservation object, ii) Conservation, which represents CnR interventions, iii) Measurement, which represents measurement actions, iv) Equipment, which represents equipment used for CnR and measurement actions and v) Digital Documentation, which represents both the digital representation of the conservation object (e.g., digital images) and any digital file produced during CnR or measurement actions. Table 4 depicts the correspondence between the CnR aspects (see Section 2) and the main entities of DOC-CULTURE.

The proposed model has been used for modelling data related to conservation interventions provided by the National Archaeological Museum of Athens [59]. Additionally, the model has been deployed in an information system built for the purposes of the DOC-CULTURE project. The system consists of the following subsystems, which provide the respective services [59]: i) image processing, which allows the application of filters on images of conservation objects, ii) numerical analysis, which provides functions for the identification of past conservation processes and detects lesions on the surface of a conservation object, iii) metadata management, which allows the matching of metadata elements to database columns, iv) image annotation, which enables the addition of annotation marks on an image together with information about previous CnR processes and v) documentation management functionality, which implements the proposed metadata scheme as well as the documentation process conducted by CnR experts.

The model and CIDOC CRM extensions are not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.5.Ontology for degradation phenomena and annotation on 3D reconstructions

In [69] a correlation pipeline is proposed for the integration of the three dimensions of a masonry structure: i) semantic dimension, which refers to concepts used by experts in order to describe conservation state, ii) spatial dimension, which refers to spatialized annotations on 3D representations, iii) morphological dimension, which refers to morphological descriptors (e.g., occlusion, curvature, roughness) of annotated regions of 3D representations.

The pipeline uses an ontology (ODPA-3DR) for recording and integrating multidisciplinary observations of the conservation state of masonry structures, spatialized into a reality-based 3D representation3737 [69].

ODPA-3DR was developed based on Lassila’s method [63] and was implemented using Protégé [70,80]. The development of the ontology had to meet the requirements of semantic digital acquisition of data related to the morphology of a masonry structure (e.g., photography), the annotation of 2D or 3D representations of a masonry structure, as well as the four concepts for describing a masonry structure and its conservation state [70]: i) Material, ii) Building technique, ii) Architectural component and iv) Alteration. For each of these concepts specialized thesauri were built, based on the terms collected from experts and existing glossaries (namely ICOMOS [111] and Architecture: méthode et vocabulaire (from now on ARC) [89]). In order to align the proposed ontology with the wider CH domain, certain classes were mapped to CIDOC CRM [25], while CIDOC CRM compatible models (namely CRMsci [31], CRMdig [29] and CRMinf [30]) were re-used.

Based on the presentation of ODPA-3DR ontology in [70], five main thematic clusters can be identified: i) Argumentation, which represents the scientific justification of the observed damages, materials and techniques of a conservation object, ii) Information, which represents documents and digital files about a conservation object, iii) Region, which represents a specific area on a conservation object, iv) Description Concepts, which represents material(s), building technique(s), architectural components and degradation phenomena of an area under study, v) Digital Acquisition, which represents the digitization of a conservation object. Table 5 depicts the correspondence between the CnR aspects (see Section 2) and the thematic clusters covered by ODPA-3DR.

ODPA-3DR is deployed in a system for reality-based 3D semantic annotations of masonry structures’ conservation states [69,70]. The basic functionality of the system is as follows: the user inputs a set of digital images of a physical object, and marks 2D regions on these images. Based on the marks, the system extracts spatial relationships (by 2D-to-3D-to-2D projection), as well as morphological features (by morphological analysis). Then, the user annotates the marked regions with description concepts of different thematic layers. Finally, the user indicates references and resources that justify the annotation. The system is not available online.

In order to be populated for testing purposes, ODPA-3DR has been mapped to a database holding data generated by experts during digital acquisition and observation of a masonry structure (3D point cloud, scientific imagery, documents, etc.). The first results showed the possibility to calculate the overlapping degree between different annotations (associated with different description concept types) of the same structure (e.g., a wall) [70].

The ontology is not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.6.PARCOURS semantic model

The Patrimoine culturel et Restauration-Conservation: Ontologies pour l’ Usage d’ un Referentiel commun aux differentes Sources de donnees (PARCOURS) project followed an ontology-driven approach in order to address the interoperability problem of CnR data provided by different CH Institutes and develop a common infrastructure for the semantic retrieval of integrated CnR data [84].

In this context, [84] proposed a semantic model of CnR of tangible CH, the PARCOURS semantic model, which aims to integrate CnR data from different sources, in order to enable the querying of data in a unified way.

The development of PARCOURS semantic model started with the definition of a core structure and the main CnR requirements of the ontology [10]. During this process scientists and domain experts of the CnR field were involved. Next, a set of sample data structures and example data related to the CnR processes was mapped to existing CH domain ontologies (EDM [24], ABC [36] and CIDOC CRM [25]). CIDOC CRM was considered as the most appropriate choice for CnR data modelling, and it was used and extended -where necessary- for the representation of the domain of interest. Finally, the developed model integrated a set of thesauri in order to tackle the problem of inconsistency among different CnR terms, at both the syntactic and semantic level. The PARCOURS semantic model uses OWL and RDF technology [83,85].

PARCOURS semantic model adopts a layered ontology architecture: i) a top-level ontology, ii) extensions of the top-level ontology with specialized classes, iii) specialized thesauri [83]. The PARCOURS semantic model reuses CIDOC CRM [25] and its official compatible model CRMsci [31]. Additionally, it introduces CRMcr, an extension of CIDOC CRM and CRMsci, developed in the context of the PARCOURS project [10]. The PARCOURS semantic model consists of i) 93 concepts and 82 relationships of CIDOC CRM ontology, ii) 22 concepts and 24 relationships of CRMsci and iii) 63 new concepts and 27 new relationships. Regarding the use of specialized thesauri, most of them were built during the PARCOURS project. They were managed by the Thesaurus Management System (TMS) Gestion Informatisée de Nomenclatures Collaboratives et Ouvertes3838 (GINCO) and were integrated into the CRMcr model [83].

The classes and relations of the PARCOURS semantic model represent knowledge related to five main thematic clusters [10]: i) Conservation object, which represents the identification, production and physical features of a conservation object, ii) Location and owners, which represents information about the conservation object’s locations and ownership during its lifecycle, iii) Degrading events, which represents alterations of a conservation object, iv) Non degrading events, which represent scientific studies and their results, as well as intervention activities, v) Instruments, which represents the equipment used during the various non degrading events. Table 6 depicts the correspondence between the CnR aspects (see Section 2) and the thematic clusters covered by PARCOURS semantic model thematic clusters.

In the context of the PARCOURS project, a data integration and querying system for the CnR domain was developed based on PARCOURS semantic model, providing search and retrieval services [83,85]. The system has the form of a web portal that allows users to retrieve CnR data from multiple datasets. In order to achieve unified access to multiple datasets, the system follows a mediator approach which tackles restrictions imposed by CH institutions, allowing them to keep managing their repositories autonomously. In particular, all data sources involved in the integration process (namely Research Laboratory for Historical Monuments (LRMH) and French Museum’s Research and Restoration Center (C2RMF)) refer to the proposed model, which is used as the mediator. To process a given query, the system interacts simultaneously with the different databases and retrieves data concerning both the physical characteristics of the queried object (e.g., location, authors) and related events (e.g., type of event, used techniques, measurements, actors). Moreover, the system provides a key-word-querying interface, which allows the user to choose specific keywords from the thesauri linked with the PARCOURS semantic model. The system does not appear to be open for public use [83].

The ontology is not available online. According to the bibliographic research the extension CRMcr has been re-used by [121] for the development of a new ontology within the CnR domain (see Section 3.16).

3.7.Cultural heritage artefact partonomy

The GRAVITATE project3939 developed the GRAVITATE platform which provides tools for reconstructing, annotating, and analyzing 3D models of artefacts, as well as for retrieving information about artefacts [90]. The platform is particularly useful for archaeological and conservation study of CH artefacts that are not easily accessible.

A basic outcome of the GRAVITATE project is the Cultural Heritage Artefact Partonomy (CHAP), a SKOS vocabulary developed to describe the relationships between the parts and the overall context of two archaeological collections: the terracotta statues from the port of Salamis, attributed to the Neo-Cypriote style (ca. 600-500 BC), and the small clay statuettes from the Ayia Irini sanctuary, mostly attributed to the Cypro-Archaic period (700-500 BC).

The main classes of CHAP were defined based on an archaeological corpus of texts (i.e., archaeological publications, catalogues, excavation reports) as well as fundamental archeological knowledge [23]. First, the main hierarchy was developed. Next, the hierarchy was aligned to a semantic scheme suitable for representing knowledge about both artefacts and their digital counterparts. The CHAP model was edited in Protégé software [80].

The CHAP meronomy has been modelled as a SKOS hierarchy4040^,4141 and has been aligned to a semantic scheme formed by a combination of CIDOC CRM [25], CRMdig [29], Common Shape Ontology (CSO) [108] and extra classes defined based on requirements of the GRAVITATE project. The extra classes and object properties were introduced as extensions of CIDOC CRM and CRMdig.

The semantic scheme can be divided in two main conceptualization aspects [23]: i) the Physical artefact, which represents statues and figurines, their features and dimensions and ii) the Digital artefact, which represents the digital counterpart of statues and figurines, the type of its geometric representation, its areas and areas’ dimensions. The CHAP meronomy is included in the first aspect and it is organized in five central concepts all related to statues and figurines [23]: i) Body part, which represents anatomical constituents and related characteristics (e.g., hairstyle), ii) Attire, which represents accessories (e.g., necklace), iii) Decoration, which represents figurative decorations and geometric decorations (e.g., flower), iv) Colour, which represents coloring, and v) Technique, which represents manufacturing techniques. Table 7 depicts the correspondence between the CnR aspects (see Section 2) and the conceptualization aspects covered by the CHAP semantic model.

CHAP is deployed in the knowledge base of the GRAVITATE platform and it is exploited by the tools provided by the platform for analysis and annotation of 3D models of artefacts [23,90]. In particular, the GRAVITATE platform’s tools are i) inspection view, which facilitates the parallel visualization of different 3D models and geometric properties, ii) feature recognition, which provides automatic identification of features on 3D models of artefacts and iii) annotation mode, which allows the annotation of areas on 3D models of artefacts. The user can input qualitative and quantitative data about annotated areas, which are stored in the knowledge base of the platform. Data and metadata about artefacts and artefacts’ 3D models are stored in the knowledge base and can be retrieved using queries.

CHAP has been used for modelling the characteristics and production techniques of statues/figurines in three case studies [23]. CHAP has also been used for semantic annotation (automatic or manual), of 3D reconstructions of objects (whether whole artefacts or fragments of artefacts). The annotation of features observed on objects allows experts to search, retrieve and examine objects in juxtaposition (e.g., based on the morphological analysis of their decoration) in order to validate hypotheses regarding their production or original form (e.g., fragments that belong to the same statue) [23].

The ontology is not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.8.Conservation process model

The Built heritage information modelling and management (BHIMM) project4242 explored building information modelling (BIM) techniques and solutions implemented in the CH domain for management and preservation of historical buildings [1]. In this context, an ontology-based deployment is proposed [1] which exploits Autodesk Revit,4343 a building information modelling (BIM) software.4444 The deployment aims to provide integration of geometrical and non-geometrical information related to conservation of architectural heritage.

For the purposes of the BHIMM project the Conservation Process Model (CPM) was developed, a semantic model that specializes in CnR of historical buildings. CPM has two main objectives: i) to represent knowledge about the related CnR processes and ii) to facilitate integration, mediation, and interchange of heterogeneous CnR data, at both the academic and the professional level [1].

For the development of CPM, the main thematic clusters of built heritage were defined. For the formalization of CPM, existing models were taken into account, namely CIDOC CRM [25], FRBRoo [35] and AR model [45]. CPM incorporates classes from CIDOC CRM since the former covers concepts and relations which are also required for the CPM domain of interest. Specialized terms of the ICOMOS [111] were mapped to CPM classes related to decay phenomena, in order to provide standardized terminology for decay specification. CPM was developed in OWL using the Protégé software [80]. Additionally, several rules were developed in Semantic Web Rule Language4545 (SWRL) in order to compare data resulting from different analyses so as to further validate the results of investigation processes [42].

Classes and relations of CPM are organized in five thematic clusters [42]: i) Artefact, which represents the architectural structure of buildings, ii) Investigation Process, which represents the examination of buildings with non-destructive/destructive methods, iii) Actors, which represents people related to the building’s history or study, iv) Lifecycle 1, which represents the description and analysis of alterations of materials and structure and v) Lifecycle 2, which represents conservation planning, interventions and general managing processes. Table 8 depicts the correspondence between the CnR aspects (see Section 2) and the thematic clusters covered by CPM.

Based on CPM, the authors of [1,101] present BIM Semantic Bridge, an ontology-based module that provides a connection between a BIM-based database and a knowledge base of CnR data. Particularly, BIM Semantic Bridge reconstructs the class hierarchies of the two databases, as well as assigned properties and derived instances. In this way, the representations of the two databases are homogenized, allowing the generation of correspondences between similar classes and data stored in the databases, in order to perform data comparison and transfer. BIM-based software Autodesk Revit has been used for the annotation of specific areas of decay or intervention on building models. The combination of Autodesk Revit and CPM facilitated geometrical representation of CnR data.

CPM has been used to represent and manage CnR information related to the 6th-century San Sabaoratory, in Rome, and its components [1,101]. Furthermore, the SWRL rules that had been formulated based on CPM were tested for information inconsistencies which had been provided by the same or different investigation activities and operators. As mentioned in [101], the achieved correlation, querying and inference making over the knowledge captured by CPM enhances the awareness of experts over any interpretation inconsistencies and proposed solution implications, thereby supporting them to choose the most suitable solution.

The CPM model file is not available online. According to the bibliographic search CPM has been re-used for the representation of conservation management of urban buildings, including their main features together with vulnerability and transformation index, using the Risk Map of Cultural Heritage system of Culture Ministry of Italian Republic (MiBAC) [2]. Additionally, [33] presents the development and use of an ontology which adopts the four thematic clusters of CPM (Artefact, Lifecycle 1, Investigation Process and Actors).

3.9.Built cultural heritage ontology

The authors of [118], present the Built Cultural Heritage ontology (BCHO), a semantic model for the preventive conservation of built CH. BCHO was developed in order to facilitate integration and exchange of heterogeneous information related to built CH.

BCHO was developed using On-To-Knowledge methodology [118], which consists of two processes, i) the Knowledge Meta process, which led to the construction of the ontology and ii) the Knowledge process, which specified activities for populating and testing the ontology [102]. Furthermore, BCHO was verified using OntOlogy Pitfall Scanner4646 a tool that detects anomalies in the ontology, improving representation quality [117]. The ontology was developed in OWL using Protégé [80].

BCHO reuses three external ontologies: i) Geneva CityGML, an ontology of the University of Geneva which is based on CityGML [56], ii) Monument Damage Ontology [11] and iii) Erlagen CRM/OWL, the OWL implementation of the CIDOC CRM top-level CH model [27]. Additionally, BCH ontology introduces a number of extra classes and properties in order to provide a complete semantic framework for management and preventive conservation of built CH [119]. In total, BCHO consists of 148 classes and 102 relations [117].

The basic classes of BCHO are organized in four thematic clusters [119]: i) Analysis, which covers the collection of information about the conservation object, ii) Diagnosis, which covers the assessment of current condition and potential risks of the conservation object, iii) Therapy, which covers preventive or remedial interventions (suggested or implemented), and iv) Control, which covers the assessment of the implemented intervention. Table 9 depicts the correspondence between the CnR aspects (see Section 2) and the thematic clusters covered by BCHO.

So far there has been no deployment of BCHO in a particular SW system or service supporting the CnR process, though the possibility of using BCHO for data integration, 3D representation of buildings, inference making etc. is discussed [117].

In order to be evaluated, BCHO was tested in terms of its ability to represent information regarding the preventive conservation cycle of the San Luis seminary, a historical building in the city of Cuenca, Ecuador [117]. The evaluation showed that BCHO is able to represent the information required in each phase. However, the following needs are highlighted which arose during the evaluation [117]: i) guidelines for reference names, ii) user manual with examples of ontology usage, iii) addition of methodologies for risk assessment, intervention assignments, etc.

BCHO is available online,4747 though the creators mention that the use, change, or distribution of this product is forbidden, since it is still in a testing stage. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.10.Polygnosis thesaurus

In the context of the Politismos-Technologia, New Technologies in the Research, Study, Documentation and Access to the Information for Cultural Heritage Objects and Monuments (POLITEIA) project the Polygnosis educational knowledge web platform (Polygnosis platform) was developed [91]. Polygnosis platform can be used to improve access to information about the state-of-the-art of laser applications and methods on conservation, analysis and diagnosis of CH objects [93].

Polygnosis platform deploys an ontological model which is based on CIDOC CRM [25] and its compatible models, CRMsci [31] and CRMdig [29]. Furthermore, the platform deploys Polygnosis Thesaurus which is not an ontology per se, but has been built based on the ontological model of Polygnosis platform. Polygnosis Thesaurus aims to facilitate the collaboration of interdisciplinary working groups, in order to retrieve information about optical and laser-based techniques for advanced imaging, analysis and diagnosis of CH objects (movable and immovable CH), an aspect that falls within the CnR domain [91].

The development of the Polygnosis Thesaurus commenced with the collection and analysis of scientific sources, terminology and thesauri related to i) conservation and diagnosis of CH objects and ii) laser-based examination techniques [91]. Afterwards, the Polygnosis Thesaurus structure was specified developed in three steps: i) definition of semantic categories and facets of the thesaurus according to the main concepts of the Polygnosis platform model, ii) formulation of terms hierarchies and finally iii) definition of semantic relationships between the terms. Polygnosis Thesaurus was developed with TMS THeMaS.4848

The thesauri and vocabularies which were studied, include i) AAT [47], ii) Network of Research Computer Image SystemS in Europe (NARCISSE) [62], iii) Conservation & Restoration Institutions for Scientific Terminology dedicated to Art Learning Network (CRISTAL) [43] and iv) European Illustrated Glossary of Conservation terms for Wall-Paintings & Architectural Surfaces (EwaGlos) [115]. Polygnosis Thesaurus is structured in four main facets which are organized in hierarchies (IS-A relations). Apart from the organization of IS-A hierarchies, terms are interconnected through additional (non-hierarchical) binary relations. Furthermore, the four facets of Polygnosis have been mapped to CIDOC CRM classes (e.g., Material Objects facet maps to cidoc-crm: E70_Thing) [91]. The mapping was conducted to ensure a common understanding of terms and concepts by the different scientific communities of CH.

The main concepts of the Polygnosis platform model are: i) Method Application, which refers to the application techniques of investigation methods, ii) Example, which refers to measurement events, examination and diagnosis procedures, iii) Technical Examination, which refers to investigation methods, iv) Objects, which refers to the object under investigation and its features, v) Data, which refers to data derived from object investigation, vi) Glossary Terms, which refers to relevant terminology and vii) Publications, which refers to relevant bibliographic documentation. The Polygnosis Thesaurus is included in the Glossary Terms and is organized in four facets [91]: i) Material Objects, which represents things with physical substance that constitute complete units and have a relatively stable form with identifiable boundaries in at least one dimension, ii) Investigation Methods, which represents systematic procedures designed to detect, identify and demonstrate Identifiable Features of Material Objects, iii) Identifiable Features, which represents features that are inextricably linked with Material Objects on which they are found (e.g., construction features, deterioration phenomena etc.), and iv) Data, which represents digital informational material related to the documentation of Material Objects or information objects and other processes of information acquisition and/or production (by recording, by digitization, through study, during intervention etc.). Table 10 depicts the correspondence between the CnR aspects (see Section 2) and the main concepts of the Polygnosis platform model.

Polygnosis Thesaurus supports the educational role of the Polygnosis platform, by providing relevant terms that the user might not have thought of, thereby facilitating the exploration and disambiguation of information [91]. The Polygnosis platform stores and integrates data, collected from scientific examination methodologies and applications that have been conducted in the laboratory of Photonics for Cultural Heritage of Institute of Electronic Structure and Laser – Foundation of Research and Technology Hellas (IESL-FORTH). It captures accumulated knowledge and data regarding diagnostic tools and methodologies, and display cases (the use of tools and application of methodologies to CH objects). In practice, the user (conservator, heritage researcher or practitioner) can select the case that interests him/her either by examination type or evidence type and get descriptive texts regarding the examination type and evidence selected, as well as a list of related cases on different CH objects. Polygnosis platform and thesaurus has been evaluated by users4949 twice [91]: during the workshop ‘Laser SYNTHESIS and POLYGNOSIS; The POLITEIA Documentation System and Knowledge Platform’, and, two years later, during a workshop that was conducted in the Department of Conservation of Antiquities and Works of Arts in TEI of Athens. According to the findings, semantic presentation forms and design were considered a key issue for an effective learning platform, while the whole idea of a system which supports a common language in the field of interest was appreciated.

Both the Polygnosis platform model and thesaurus are not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.11.Heritage resilience against climate events on site

The European project HERACLES (HEritage Resilience Against CLimate Events on Site) aimed to develop an ontology-based platform, which provides a knowledge base for the efficient storage and management of data related to the impact of climate change on immovable CH and the mitigation of potentially harmful effects [48].

An important output of this research was the HERACLES application ontology, a semantic model which covers the preservation of immovable CH. As such, the primary object of the HERACLES ontology is the efficient integration, exchange and retrieval of data related to climate change impact, which are often unstructured, incompatible or in some cases partial.

The HERACLES ontology was developed following a workshop-based approach, while the WebGenesis software was used for development [48]. During the workshop, the participants determined requirements for the conceptual representation according to the methodology of [44]. Next, competency questions (CQs) were formulated in order to specify the area of interest. The participants identified the main subdomains of the ontology. Next, classes and their relationships were defined, while descriptions of the classes were added to facilitate human comprehension of the data model. Other ontologies, related to the domain of interest have been studied and used as reference material for the ontology.

The sources used as reference material for the HERACLES ontology, include: i) SWEET ontologies [96], ii) the Materials Ontology [9], iii) Open Geospatial Consortium (OGC) standards including the Sensor Things Application Programming Interface (from now on ST-API) [65] and the Internet of Things Tasking Capability (from now on ITTC) [51]. HERACLES consists of 109 classes, 204 object properties (102 properties plus their inverse), 49 data properties and 141 individuals [48].

HERACLES core classes are [48,49]: i) Cultural Heritage, which represents monumental art and buildings, ii) Cultural Heritage Properties, which represents characteristics of monumental art and buildings, iii) Cultural Heritage Element, which represents components of monumental art and buildings, iv) Damage and Effect, which represents changes that occurred on a component of monumental art and buildings due to an event, as well as abiotic factors that induce climate events that in turn cause damages, v) Material, which represents materials used in CH management, vi) Action, which represents measurements for monitoring, analyzing or preserving monumental art and buildings, vii) Stakeholders, which represents actors in CH preservation, viii) Risk, which represents the risk of harm of monumental art and buildings and ix) Location, which represents spatial information (e.g., the position of a measurement sensor). Table 11 depicts the correspondence between the CnR aspects (see Section 2) and the core classes of HERACLES.

The HERACLES ontology serves as the backbone of the HERACLES knowledge base: every entry in the knowledge base is an instance of the ontology [49]. HERACLES knowledge base collects and integrates multisource information in order to effectively i) provide complete and up-to-date awareness about the conditions occurring in a CH site and ii) support retrieval and decision making for innovative measurements improving CH resilience. Particularly, HERACLES platform provides input forms, through which data are semantically integrated. The input form contains several text fields (e.g., for textual descriptions), while links to other instances can be created through selecting elements from lists. Additionally, an online endpoint is provided to facilitate instance creation/deletion. Regarding presentation of data, for each entry the system provides images and quick links to useful related information (e.g., damages, reports, sensor data). This endpoint is also used by the HERACLES mobile application, which allows reporting of damages on site, by delivering information such as location coordinates and description, as well as pictures, video footage etc. to the HERACLES knowledge base, in order to be presented to the back-end user.

Both the HERACLES platform and the HERACLES ontology have been evaluated in the context of four use-cases [49]. The use-cases included: i) Minoan Palace of Knossos in Heraklion, Crete, ii) Venetian Sea Fortress of Koules in Crete, iii) Consoli Palace in Gubbio, Italy and iv) the town walls in Gubbio, Italy. These test sites represent key case-studies for the impact of climate change on European CH assets. Using the HERACLES ontology made possible the semantic integration of collected data, while the platform effectively supported the retrieval of data required by experts in order to prepare reports for monuments condition state and environmental conditions [49].

The ontology is available online.5050 The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.12.Heritage building core ontology

The COST action Innovation in Intelligent Management of Heritage Buildings5151 (COST-TD1406) aims to create a European network in order to promote synergies between different specialists and stakeholders of the Heritage Building (HB) domain, as well as to achieve a unified common understanding and operation within the domain.

In the context of COST-TD1406, an ontology for the Heritage Building (HB) domain, namely the HB Core Ontology (HBCO), has been developed [105,106]. The ontology aims to support HB experts to efficiently collect and use data, produced by the different specialists of the domain in the context of the various HB lifecycle activities (including analysis, documentation, preventive conservation, restoration, economic aspects of HB, use and management).

HBCO was developed using METHONTOLOGY methodology [41], while it was encoded in OWL, using Protégé [80].

The following terminologies and ontologies were reused for the development of HBCO: buildingSMART Data Dictionary (buildingSMART) [17], FOAF Vocabulary Specification (FOAF) [15], ISA Programme Location Core Vocabulary (Location) [40], ISA Programme Person Core Vocabulary (Person) [54], Time Ontology in OWL (Time) [28] and CIDOC CRM [25]. Additionally, HBCO introduces a number of extra classes and properties in order to represent HB related data [104]. The ontology consists of 87 classes, 43 object properties and 18 data properties [104].

The main subgroups of the HBCO concepts are [104]: i) Location, which refers to building location, ii) Time, which refers to the building’s history and related events, iii) Geometry, which refers to the building’s geometry, expressed using some coordinate reference system, v) Management, which refers to activities related to the management of the building, v) FOAF related concepts, which refer to actors related to the building and their relations and vi) CH related concepts, which refer to the description of the building in terms of materials, construction etc. and activities related to building preservation. Table 12 depicts the correspondence between the CnR aspects (see Section 2) and the main concepts’ subgroups of HBCO.

HBCO was populated with HB data about 12 building projects [105,106]. Furthermore, [104] theoretically describes a platform for storage and management of HB data.

The ontology is not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.13.Conservation reasoning ontology

The work presented in [78] proposes the Conservation Reasoning ontology (CORE), a semantic model which was developed with the purpose of facilitating integration and querying of data related to conservation of byzantine icons.

The CORE ontology extends certain classes from CIDOC CRM [25], in order to facilitate CnR data modelling. The extension of CIDOC CRM was conducted in a bottom-up manner based on empirical analysis, scientific knowledge and existing CnR vocabularies [78]. Finally, inference rules were formulated in order to facilitate semantic querying [78].

For the development of CORE, a number of CIDOC CRM classes and relations were reused in combination with existing CnR vocabularies (namely AAT [47], CAMEO [72] and AIC wiki [116]). The ontology includes general concepts that refer broadly to CnR of artwork, as well as concepts specific to CnR of byzantine icons. The ontology was developed in OWL using Protégé [78,80].

The main classes of the CORE ontology are [78]: i) Actor, which refers to humans and groups involved in the CnR, ii) Condition State, which refers to the categorization of the condition of a conservation object based on visual examination, iii) Date, which refers to the dates of events (e.g., production year), iv) Dimension, which refers to the measurable sizes of an object or its environment, v) Equipment, which refers to investigation, intervention or production instruments and tools, vi) Event, which refers to processes or activities related to a conservation object or its environment, vii) Information Object, which refers to inscriptions, publications and documents related to conservation objects, viii) Man-Made Object, which refers to conservation objects, object components (e.g., structural layer) or production materials, ix) Physical Feature, which refers to features of a conservation object (e.g., a damage), x) Place, which refers to the environment of the conservation object (interior or exterior), xi) Timespan, which refers to the duration of an event. Table 13 depicts the correspondence between the CnR aspects (see Section 2) and the main classes of CORE.

In order to be evaluated, CORE was populated with selected data from conservation reports, while CQs were formed as SPARQL and SPARQL DL queries. According to the evaluation results [78], modelling CnR data using CORE allowed for the formulation of queries such as “Which are the structural layers that have ever been recorded about an object?” and “Which are the structural layers of an object since the last conservation treatment?”. Such queries can assist the CnR professional during the collection of information about the history and condition of a conservation object.

The ontology is not available online. The model has been re-used for the representation of data related to preventive conservation and sensor data [76].

3.14.Chinese ancient buildings damages ontology

An ontology for the CnR of ancient buildings (CABD) is proposed in [113]. CABD was developed in order to facilitate querying and case-based reasoning of CH information related to ancient Chinese building damages and repairs.

For the development of CABD, information related to damages in ancient Chinese buildings was analyzed, focusing on the process of selecting the appropriate CnR method for damage repair. The mapping of specific repair methods to specific damage cases was implemented in the form of SWRL rules [113]. The ontology was developed in Protégé [80].

The main classes of CABD are [113]: i) Ancient building, which refers to the conservation object, ii) Damage body, which refers to the damaged part of the conservation object, iii) Damage object, which refers to the damage, iv) Damage feature, which refers to quantitative characteristics of the damage (e.g., width), v) Repair craft, which refers to the CnR intervention. Table 14 depicts the correspondence between the CnR aspects (see Section 2) and the main classes of CABD.

To be tested, the proposed model was populated with data describing cracks on the surfaces of buildings of WuDian type, together with the appropriate repair method. Based on these data, the ontology was used for retrieving cases of damages and corresponding repair methods that present similarities with a given case (target case), by formulating SPARQL queries [113].

The ontology is not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.15.Network of modular ontologies for heritage buildings

A framework for a Linked Data-based Heritage BIM is proposed in [13], in order to improve the modelling of structured building data and its exchange over the Web. In this context of the framework a network of modular ontologies (Network of Modular Ontologies for Heritage Buildings – NMOHB) is proposed for obtaining rich and interoperable heritage building descriptions. The development of NMOHB was based on the benchmark requirements for the Linked Data-based Heritage BIM.

NMOHB adopts a layered architecture that incorporates: i) core ontologies, which provide the basic concepts related to construction, ii) specialized taxonomies which extend the core ontologies, and iii) metadata ontologies, which are used for data exchange.

In particular, a number of complementary ontologies were developed in the context of the research (namely Construction Tasks Ontology (CTO), ConTax Ontology (ConTax), Construction Dataset Context Ontology (CDC), Construction Properties (CP), Built Heritage Properties (BHP), Ontology for Geometry Formats (FOG), Geometry Metadata Ontology (GOM), MDCS Damage Atlas Ontology (MDCS-O), MWV Damage Ontology (MWV-D), MWV Tasks (MWV-T) and AAT-Buildings), while the Building Topology Ontology (BOT) [98] and Data Catalogue Vocabulary (DCAT) [5] ontologies were reused. In addition, the following ontologies were developed in collaboration with other projects and were included in NMOHB: Ontology for Managing Geometry (OMG) [112], Ontology for Property Management (OPM) [97] and Damage Topology Ontology (DOT) [46]. The core layer of NMOHB is formed by the BOT, DOT, CTO, OMG and OPM ontologies.

The concepts of the core ontologies of the NMOHB are organized in the following groups [13]: i) Building topology, which refers to the building, its structure and materials, ii) Construction damages, which refers to the observable damages or damaged areas of the building, as well as the inspection of those damages, iii) Geometry descriptions, which refers to 2D and 3D representations of a building and their metadata, iv) Construction tasks, which refers to CnR interventions. Table 15 depicts the correspondence between the CnR aspects (see Section 2) and the groups of concepts of the core ontologies of NMOHB.

To be tested, the developed ontology network was deployed in five example cases, inspired by two built heritage projects in Ghent [13]. The example cases demonstrate the key functionalities of NMOHB and underlying technologies, which are: i) the possibility of linking and using geometry from a variety of common geometry formats, ii) the use of flexible and integrated classification systems, iii) the provision of combined views on previously disparate datasets, iv) the provision of feedback mechanisms for construction damages and tasks based on structured data and v) the provision of metadata about construction datasets to manage the flow of internal and exchanged datasets. As discussed in [13] the outcome of the evaluation phase indicated that most requirements related to the data structure and management workflows (e.g., data exchange) have been addressed successfully by the proposed modular ontology network. However, it is highlighted that more contributions of domain experts are considered necessary to maintain, extend and verify the content of the NMOHB taxonomies. Especially the property taxonomies, damage and task classification are currently not extensive enough for use in real projects.

All the different ontologies of the network (BOT,5252 DCAT,5353 OMG,5454 OPM,5555 DOT,5656 CTO,5757 ConTax,5858 CDC,5959 FOG,6060 GOM,6161 MDCS-O,6262 MWV-D,6363 MWV-T,6464 AAT-Buildings6565) used in the scheme are available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

3.16.CRMBnF

The Data mining and ALGOrithms for predicting the condition of COLlections (DALGOCOL) research project6666 is a collaboration between the French National Library (BnF) and the Heritage Sciences Foundation. The project aims to develop a decision support system for conservation experts for analyzing the conservation history of documents in order to enable reliable predictions about their physical state.

In this context, [121] proposes the Conceptual Reference Model of BnF (CRMBnF). The proposed model aims to i) represent the conservation history of a document as a sequence of events (conservation trajectories) providing a unified vocabulary for the events and ii) be used for the pairwise comparison of events in two conservation trajectories overcoming terminological heterogeneity of events.

CRMBnF was developed in close collaboration with domain experts at the BnF and is encoded in RDF/S 2 and OWL. For developing the ontology three external ontologies were re-used [121]: CIDOC CRM [25], CRMsci [31] and CRMcr [10].

CRMBnF includes two main classes [121]: i) Conservation Process, which represent CnR interventions, ii) Degradation, which represent deterioration events that occur on a conservation object. Table 16 depicts the correspondence between the CnR aspects (see Section 2) and the main classes of CRMBnF.

CRMBnF has been used for the comparison of the events between conservation trajectories, a process which serves as a base to build an adequate predictive model for the decision support system [121]. The comparison is performed by computing a similarity score using the ontology, considering the relative position of the concepts corresponding to the names of the events in the ontology.

Three experiments were conducted, using real conservation and communication datasets from the BnF. The first two experiments aimed to assess the effectiveness of using the ontology for finding matching events and for evaluating the similarity of trajectories. The third experiment aimed to show the effectiveness of using the ontology and the LCESS measure by comparing the computed similarity to a gold standard [121]. According to the results, the proposed ontological approach improves the precision of the matching process [121].

The ontology is not available online. The bibliographic search showed no evidence of the model’s re-use in later projects for the development of ontologies within the CnR domain.

4.1.Content

Obviously, a common requirement is the modelling of the conservation object per se. The majority of the reviewed models specialize in a specific type of conservation objects: in particular, 8 out of the 16 reviewed models specialize in immovable CH, while 5 of them specialize in movable CH (with only 3 models covering tangible CH in general). Those numbers suggest a significant interest in SW technologies exploitation in the CnR of the immovable CH community.

Figure 1 depicts the degree to which the CnR aspects defined in Section 2 are covered by the reviewed models. The materials & technology aspect is covered by the great majority (15 out of 16) of the reviewed models, which is expected since the conservation object in itself, its structure, materials and features are in the center of the initial stage of the CnR process since they are used for establishing the identity and condition of the conservation project (see Section 1). The alteration and investigation aspects are also covered by 15 and 14 out of 16 models respectively, which is similarly expected since the main objective of all the reviewed works is the organization and management of CnR data, and the investigation stage constitutes the primary means of collecting such data. The expert conducts investigation in the first stage of the CnR process but also in later stages for evaluating risk and for identifying the appropriate solutions. The main objective of the investigation is to enable the expert in identifying and recording the attributes of a conservation object, both material (e.g., structural layers) and non- material (e.g., historic value), as well as to deduce current and potential preservation issues that may occur due to inaction or action (e.g., deterioration due to environmental conditions, deterioration due to an intervention). The coverage of the aforementioned aspects by the great majority of the models indicates their importance and necessity, while it constitutes a solid base for data modelling endeavors. On the other hand, while the intervention aspect is of apparent interest for the CnR domain and it also constitutes an important source for acquiring CnR information, it is covered only by 12 models, which is less than expected, especially compared to the high coverage of the investigation aspect.

Fig. 1.

Coverage of CnR aspects.

An interesting observation regarding the content is that different models allow different granularities for categorization of the various concepts. For example, the CPM model represents a building (i.e., the conservation object) using classes of specific building types (EcpmA36_Oratory), while the COSCH^KR model represents a building at a more abstract level, using the class coschkr:CompositeObject. In another example, MDO represents the concept of material using a number of specialized classes (mdo:CementMortar, mdo:SodaGlass, etc.) whereas HERACLES provides only the general class heracles:Material for the same purpose. Different granularities of representation may be handy in different use cases (e.g., we may not be certain about the use of a building in order to categorize it as Oratory) or different reasoning requirements.

4.2.Re-use of existing models

While the scope and context of the reviewed works may differ, there is a common interest for providing interoperability of CnR data. Towards that direction, most of the reviewed models were developed either from scratch and were then mapped/aligned to existing ontologies, or they were built entirely by extending existing ontologies (with the exception of MDO, COSCH^KR, and CABD).

The CIDOC CRM ontology, as well as its compatible models, were largely adopted by the majority of the projects. Additionally, specialized ontologies from other knowledge domains relevant to CnR were adopted for the development of the models. Figure 2 depicts the origin domain of the various ontologies, metadata standards, term lists and thesauri that are re-used by the reviewed models.

Most of the works took into account and adopted ontologies from other fields such as the field of Architecture & Construction (BOT was adopted by NMOHB), the field of Chemistry and Material Science (OreChem was adopted by OPPRA, MO was adopted by HERACLES), the field of Earth Science (SWEET ontologies were adopted by HERACLES) and the field of Digital Media (CSO was adopted by CHAP). Furthermore, thesauri, glossaries and controlled vocabularies, specialized either in CnR or in other related domains, were similarly employed.

Fig. 2.

The origin domain of the ontologies, metadata standards, terms lists and thesauri that are reused by the reviewed models.

4.3.Deployment

All the models included in the current survey have been employed for developing SW systems that offer various domain-specific services. In the course of the survey, we identified a number of services that are common among those systems. In particular:

Figure 3 depicts the degree to which the identified services are provided by the systems built upon the reviewed models. It should be mentioned that 5 out of 16 reviewed models (namely BHO, HBCO, CORE, CABD and CRMBnF) have not been deployed in some system, though they have been used for data modelling, testing SPARQL queries and comparison of annotated data. It should be mentioned that CRMBnF is supposed to be used in a decision-support system, for conservation objects classification based on their history.

Apparently, semantic search is the most popular among provided services, while data integration, visualization and semantic annotation follow. The popularity of these services is expected considering that they intend to support the CnR process (see Section 1) and by extension CnR decision-making.

Decision-making plays a central role in the day-to-day work of the CnR professional [50,64]. In broad terms, decision making in CnR can be thought of as equivalent to the conservation process itself [64]. Conservators participate in decision-making processes by i) contributing their expertise to a broader, multi-disciplinary group of CH experts, and ii) making their own CnR decisions [50]. In this context, data integration and semantic search provide unified access to the required knowledge/information (in order to make an effective intervention recommendation), in this way reducing information retrieval time and improving quality of search results (e.g., information completeness), consequently supporting the different stages of the CnR process, from the problem identification to the potential risks and the selection of an efficient solution. Similarly, visualization contributes to CnR decision-making, at different stages of the CnR process, by providing a more articulate and meaningful documentation as well as correlation of the requested information (e.g., the visualization of extent and severity of an alteration phenomenon gives a thorough view of the conservation object’s condition). Finally, semantic annotation, either text-based or visual (2D/3D), is used alongside semantic search and visualization contributing to the production of context which can be retrieved and displayed.

Fig. 3.

Provision of SW services by systems built upon the reviewed models.

On the other hand, actual recommendation of digitization and analysis methods (as an explicit service), is proposed by only one system (the COSCH^KR platform). And even in that case, the recommendation merely covers a certain step of the CnR decision-making process, that is selecting the appropriate digitization and analysis method for identifying the technology and condition of a conservation object in order to determine intervention requirements.

4.4.Conclusions and further research: Towards an ontology of CnR decision-making

As discussed in the previous section, decision-making constitutes the backbone of the CnR task. Through decision-making the expert transforms various chunks of possibly diverse information relevant to a conservation object, such as scientific information (e.g., material ageing), administrative (e.g., loaning preconditions) or even cultural information (e.g., historical value), into concrete and specific intervention decisions. The current survey showed that all the reviewed models more or less represent knowledge relevant to intervention decision-making, and that they have actually been employed for implementing SW services that support intervention decision-making; although merely in an assistive way (for instance, by offering semantic retrieval of data related to a conservation diagnosis). In other words, the decision-making process per se, i.e., including all the parameters, criteria, intervention options etc. potentially involved, and more importantly their complex interdependence, which often generates restrictions that can dramatically affect the decision-making outcome, has not been modelled as of yet. For example, the MDO, OPPRA, DOC-CULTURE, PARCOURS semantic model, CPM, BCHO, HERACLES, HBCO, CORE, CABD, CRMBnF models, as well as some of the models of NMOHB provide the relations that are necessary to correlate an intervention with i) a conservation object and ii) one or more alteration phenomena. However, as [42] highlights, a simple correlation between conservation interventions and alteration phenomena does not adequately represent the potential complexity of interplay between the various parameters that need to be taken into account in order to come to a valid intervention recommendation.

Drawing on the above, it is strongly suggested that further research should be conducted in order to analyze and conceptualize intervention decision-making at a granularity that will allow a more thorough representation, suitable to drive the implementation of services that will deliver intervention recommendations, as an explicit decision-support service. For example, a semantic model of intervention decision-making knowledge by means of a formal ontology can serve as the basis for the development of decision-support systems, with the objective of recommending specific, case-based intervention options. Eventually, such systems will actively assist the CnR expert i) to better organize their thoughts and determine requirements over a decision-making task, and most importantly ii) to retrieve and assess valid intervention options more quickly and effectively. Furthermore, a formal ontology will enable experts to share decision-making knowledge and exchange data with the wider community in a unified way [12]. Dissemination and exchange of know-how about intervention decision- making among CnR experts is bound to benefit them in multiple ways, for example, in training new professionals or effectively communicating information with clients, stakeholders or other (often interdisciplinary) professional groups [73,74], thus, elevating their work to the next level.