OE Related Work

The paper [DuBo2014] describes the evaluation of the evaluation of GoodOD (Good Ontology Design Guideline) [ScSe2012]. "The effectiveness of the GoodOD guideline was evaluated in arandomized controlled trial in which the performance of ontologydevelopers was compared after specific and unspecific training. As outcome parameters gold-standard based similarity measures and a competency question based approach were adopted". Furthermore the authors claim: The GoodOD guideline had a significant effect over the quality of the ontologies developed".

"Ontology quality evaluation and Requirements (OQuaRE) is a framework for Ontology Quality Evaluation, base on the software product quality ISO 24000:2005 named as SQuaRE".

In this section the main characteristics and goals of the study are discussed with respect to our research:

"Structural: Formal and semantic important ontological properties that are widely used in state-of-the-art evaluation approaches: Some subcharacteristics are:" Cohesion, Redundancy, Formal relations support and Tangledness.

Cohesion: "An ontology has a high cohesion if the classes are strongly related". The criterum for strongly related is not clear. Certainly, we will find lots of classes in large ontologies which might be related weakly or not related at all. The list of Fundamental Object Properties contains object properties for different kind of relations between classes. Classes are certainly stronger related to each other if there exists a direct ◊subClassOf relationship between them. Most of the ontologies or ontology tools have a common root class like Category (GFO), Entity (BFO, DOLCE), Thing (Protege, schema.org) or Class (SUMO, UMO). Then it is very likely that all classes in an ontology are connected via the root class. If classes are connect via object properties than also the length of the path from one class to the other could be taken as measure for cohesion. In a losely connected ontology there might be several islands of class networks which are not connected to each other. But this could have been intended in the design because the class networks could be regarded as independent ontology modules.

"Redundancy:  Capability of the ontology to be informative". In programming, there is the classical time-space tradeoff. An absolute redudant-free ontology is certainly less performant compared to a redundant one. Though, one should from the transparancy point of view try to avoid redundancy as much as possible. On the other side different approaches and patterns can be taken to model the same chunk of knowledge. It depends much on the use cases and the usability which modeling approach to take in the special application case. From the viewpoint of a modeller redundancy may be regarded as a deficite since ontologies may seem to be more complex than necessary or than they are. On the other site information is hidden from the modeller which only materializes by inferencing and entailment.

"Formal relations support: Capability of an ontology to represent relations supported by formal theories different from the formal support for taxonomy." It is not quite for us clear what is meant here. What does mean "formal support for taxonomy"? We assume here that the authors meant the formal treatment of Relational Property Characteristics. I.e. to classifiy Fundamental Object Properties as functional, inverse functional, transitive, symmetric, asymmetric, reflexive and irreflexive. Then, taxonomic relations would be part of the general set of possible object properties which arrange ontological concepts in hierachies like class hierarchies, data and object property hierarchies, acitivity hierarchies etc.

"Tangledness: This measures the distribution of multiple parent categories, so that it is related to the existence of multiple inheritance". As one of the results the authors mention on page 12: "The ontologies developed after the training presented less multiple inheritance and were more stable for changes in their classes, properties and terms."  From our perspective this is questionable since less multiple inheritance will lead to more redundancy: Especially in section PC Evaluation we show that using multiple inheritance for modeling Partitioning Classes can lead to big savings in number of triples.

"Functional Adeqancy: The capability of the ontologies to deployed fulfilling functional requirements. Some subcharacteristics are:" Controlled vocabulary, Consistent search and query, Knowledge acuisition and representation and Knowledge Reuse

"Controlled vocabulary: Capability of the ontology to avoid heterogeneity of terms". With our Foundation Core Glossary we will present such a controlled vocabulary later. For the moment, it is not clear for us what "heterogeneity of terms" could mean. Does it mean "redundancy of terms" or "ambiguent terms"?  In statistics heterogeneity means that populations, samples or results are different and homogeniety means they are the same. For us the most reasonable interpretation is to use a controlled vocabulary for the disambiguation of terms.

"Consistent search and query: The aim is to provide better querying and searching methods based on semantic contexts. With OQL we introduce a powerful query language for Knowledge Graphs. This is extended by MQL which also supports transitive closure. Also we provide different inferencing and entailment functionality. Finally, based on the composition of concepts we provide a Deep Semantic Search (DSS).

"Knowledge acquisition and representation: Capability of the ontology to represent the knowledge acquired (ability to support a knowledge base of ontology individuals)". We show numerous examples how aquired knowledge can be represented. So far, we could not find any chunk of knowledge  which we could not adequately represent. It is not quit clear for us what is meant by "ontology individuals". Has it to be interpreted as "individual ontologies" or as in the way we use it as Particulars? In the latter case the set of ontology individuals would be the set of all knowledge subjecst of the Particulars Layer.

"Knowledge reuse: The degree to whch the content of the ontology ca be used to build other ontologies". The modularity and reusabiltity of our ontologies is supported by Named Graphs which do not have to be disjunct. Thus, it is easy to export or reuse any part of an ontology. Candidates that can serve as a basis for creating other ontologies are primarily formal top level ontologies like BFO, DOLCE and GFO or upper ontologies like schema.org and SUMO. Formal top-level ontologies usually come with an axiom system that precisely describes the semantics of their classes and relationships. The focus of the Upper Ontologies is more on mapping a wide variety of application areas as extensively as possible.

Interestingly, none of the Upper Ontologies are based on any of the Top Level Ontologies. We suspect that this is because none of the top level ontologies has been able to establish itself as a standard so far. In addition, the following reasons could speak against the use of Top Level Ontologies and Upper Ontologies:

• The training is too difficult because it requires significant theoretical and mathematical know-how.
• The training is too time-consuming because it is not appropriate in relation to the size of the task or because it means a considerable effort to find the concepts and use them correctly.

"Compatibility: the capability of two or more software components to exchange information and/or perform their required functions while sharing the same hardware or software environment."

"Replaceability: The degree to which the ontology can be used in place of another specified ontology fo the same purpose in the same environment". From our point of view this is a very difficult issue. In order that ontology A can replace an ontolgy B it would have to be structurually equivalent or an superset of B and it would have to semantically equivalent. Unlike to programming languages for ontologies there is nor definied mechanismis like an interface. Also it will be hard to guarantee that the axioms of A are equivalent to those of B. In praxis, we do not know of any non-trivial pair of ontology which fulfill such a criteria.

"Adaptability : The degree in which the ontology can be adapted for different environments (languages, expressivity levels) without applying actions or means other than those provided for this purpose for the ontology considered". For us it seems to be some kind of recursive definition where the goal is defined somehow fuzzy. Anyhow, we have shown, that every ontology can be adapted and enhanced to meet any requirements. E.g. we have shown how to add additional languages. Also we have shown, that with our methods we support multi-level modeling, rules, axioms and description logic.

"Operability: Effort needed for the application. Individual assessement of such application by stated or implied set of users." A subcharacteristic is : Learnability This is more of a technical than a substantive criterion. In general, the users have little influence on the operation of the ontologies. Therefore, determining when and how an ontology should be usable is more of an organizational measure.

"Learnability: The degree to which the ontology enables users to learn its application". For this purpose we provide powerful visualization support in form of OntoGraphs and Hypertrees. This exposes some kind of visual thinking metaphor to the user which allows him to much faster compared to reading RDF/RDFS or OWL syntax. The latter we think are more suited for computers to read and comprehend than for humans.

"Reliability: Capability of an ontology to maintain its level of performance under stated conditions for a given period of time". As with operability, reliability is more a technical/organizational criterion. In particular, the performance depends very much on the complexity and size of the ontology. As requirements increase, other technologies such as cloud computing may need to be used to support application scalability. This may also not be sufficient if, for example, the power of the ontology languages used exceeds the limits of calculability. Therefore, an important task in ontology development will be to continuously monitor the limits of reliability and operability. In the end, there may even be no choice but to give up certain requirements and functionalities.

"Recoverability: The degree to which the ontology can reestablish a specified level of performance and recover the data directly affected in the case of a failure". This requirement is also more of a technical/organisational one. If one assumes that the underlying data management systems have the typical ACID properties of database systems, then one can basically assume that this also covers recoverability. We show, for example with O4Store, that special redundant storage techniques can improve performance by orders of magnitude, of course at the expense of additional storage space for tables and indexes.

"Availability: The degree to which the ontology or part of it, is operational and available when required for use with different applications". In this sense an ontology can only be operational if it provided on a server which exposes it to the web e.g. with a SPARQL end point or via a restAPI. The ontologies described in this treatise are availabe 24/7 using the O4Store engine. By this criterion, we would not consider ontologies as available which are stored as flat files on any computer.

"Capability of ontologies to be modified for changes in environments, in requirements or in functional specifications. Some subcharateristics are:" Modularity, Reusability,  Analysability, Changeability, Modification Stability and Testability.

"Modularity: The degree to which the ontology is composed of discrete components such that a change to one component has minimal impact on other components". Ontologies and tools for creating ontologies offer practically no concepts or methods for modularization. How can you even create ontology modules? An ontology created in RDF/RDFs can usually be made available at a URI/IRI web address. Of course one could group ontologies or parts of them in "Named Graph". That is, each triple also receives the name of the subgraph to which it belongs. However, this is usually not supported directly by the tools or by RDF/RDFs. In contrast to programming, however, there are no comparable interfaces with ontologies. Likewise, the principle of information hiding in ontologies is not known. In ontologies, information is global, so to speak. This also corresponds to the goals of the World Wide Web.

"Reusability: The degree to which an asset (part of) the ontology can be used in more than one ontology, or in building other assets". Reusability is very much dependent on how general or special the ontology is. The more general like with  Upper Ontologies or Top Level Ontologies the more reusable will the ontology be. The more special on ontology is the less likely it is the it might be of use for other ontologies.

Because of different representations and different foundational ontologies reusability so far is very low. One of the first technical solution approaches is the moduralization with named graphs and views. The problem is that there is no standard for named graphs yet and that they are weakly supported in existing systems. Views on named graphs would allow for the adaption of application. At the moment only scientific prototypes exist.

"Analysability: The degree to which an ontology can be diagnosed for deficiences or causes of failures (inconsistences), or for the parts to be modified to be identified". Sebastian Rudolph in [Rudo2011] defines Knowledge Base Satisfiability and Concept Satisfiability. This allows to decide whether a knowledge base is consistent and gives hints for modeling errors.

"Changeability: The degree to which an ontology enables a specified modification to be implemented. The ease with which an ontology can be modified". In the section morphism we discuss various aspects of changing an ontology. How easily an ontology can be adapted to the specified modifications certainly depends on the set of triples to be modified. For simple changes the number will tend to be small, for larger changes rather high. Furthermore, the larger the ontology already is, the lower the proportional change effort will be. On the other hand, the larger the ontology, the more difficult it will be to figure out where the changes need to be made. The question even arises as to whether one should really use large top ontologies such as CYC if the overall effort for changes is greater than the development of a completely new ontology.

"Modification Stability: The degree to which the modified ontology can avoid unexpected effects from modification of the knowledge (terms, classes, properties, etc.)".The authors do not indicate what such unexpected effects might consist of. This could mean, for example, whether the ontology becomes inconsistent or contains errors after a change. These effects could possibly be uncovered by reasoners. In this sense, however, the modification stability is not a property of the ontology, because it does not itself have an influence on which changes a modeler makes. It will be similarly difficult to determine the impact of ontology changes on the applications using the ontology. Even with database applications, it is very difficult to guarantee so-called data independence. Changing the database schema must always be accompanied by a review of the applications. As soon as the applications access parts of a changed schema, they usually have to be adapted accordingly. There is also no general automatism for this.

"Testability: The degree to which the modified ontology can be validated". From our point of view testability is always given if the ontology fulfills the availability and modularity criteria.

The authors of the study claim that using the guidelines ha a significant effect over the quality of the ontologies developed. The study leaves open how big the ontologies modeled were. From our experience most of the modeling issues arise with non-trivial large or very large ontologies.