HQDM Documentation (Wiki) and Depreciation of EXPRESS

[jamrce]

HQDM (the book by Dr West) is extremely thorough in describing the commitments made and how he used them to produce the final data model. This data model is now what is more commonly referred to as HQDM, that is actually just the final chapter 17 of the book now captured within this repository. There is other supporting material available if you know where to look such as this summary paper on choosing a foundation data model.

Whilst EXPRESS is readable for this data model, tooling support is now limited and it would be nice to resolve some of the syntactic issues Matthew faced when documenting HQDM. For example the various permutations of underscores for particular relationships like member_of. Human readable descriptions were also just more traditional software comments in EXPRESS and as such as not easily queryable / workable from this structure.

To these points, everything that is currently captured within EXPRESS could be migrated to modern practices of semantic web to enable the wider community for getting started with HQDM. Careful consideration should be made for how best to describe the theoretical model working with and building on the serialisation effort already made and still to be finalised. The advantage being that the EXPRESS can be kept as original source material but depreciated without loss of understanding about the data model.

Proposal

The HQDM ontology should use RDF(S) with supporting shape constraints (SHACL) to completely describe the intention of the model. To do this it seems common practice to have 2 main files*:

1. A main definition file capturing:

• The context of the data model and main properties of the ontology (creator, version number, ontology definition, code repository location, data created, license, published, etc...). Examples can be seen here: within pyLODE within this example, as well as the top of this ontology. This will utilise DCAT and Dublin Core as recommended by the UK government.
• Relationships and entities described using rdf(s) with the potential for supporting namespaces when necessary such as foaf:Image for where image diagrams exist or skos:example for where examples are already on the wiki.

2. A supporting shape constraint (SHACL) file to:

• Restrict combining class trees through rdfs:subClassOf further down the tree to support reference data library extensions. An abstract_object can only be a class or a relationship as captured on this diagram / EXPRESS ONEOF.
• Capture cardinality constraints. E.g. association can have any number of participants, however employment can only have 2; 1 employer and 1 employee.

There will be other aspects that could or should be included not yet captured, however as foundation I believe these aspects cover the vast majority of current common practice that HQDM could utilise. The current information in the wiki is all captured above, including the inheritance graph should be possible to be automatically calculated / extractable from the ontology but is naturally the most difficult.

Annex

* 2 main files - Other permutations include 1) the SHACL definitions combined with the main definitions, which whilst correct to the specification, tooling such as pySHACL currently struggles with at the time of writing. Or 2) splitting out the documentation (rdfs:Label and rdfs:Comment) away from the main definition file creating 3 files overall, which again whilst also valid tooling has been known to struggle with only accepting one definition file with docs expected. The above breakdown is seen as the best route forward.

[ClimbingAl]

This looks like a well-thought out suggestion. Definitely merits discussion!! I'm pushed for time right now but here are some significant (in my mind) points to consider:

• While I agree that EXPRESS tooling is almost obsolete (at least for our sort of applications) I am not convinced that Linked Data tooling is sufficient for the task. It may be better in some areas but I think it is less good in others.
• I believe the original intention with MagmaCore was to not tie it to external specifications. By adopting RDFS & SHACL we are doing this. This has implications that are worth discussing. I will add comments on this in due-course (e.g. ENTITY TYPE Thing is (something like) rdfs:Class. However, this involved a logic commitment that is formally incompatible with HQDM. We can agree to ingore this but it may create problems for the future.
• Adopting external metadata techniques, even if they are endorsed by UK government, may also introduce short-cuts and mis-interpretation/use of the model itself. Matthew West's book covers Reference Data management so we should compare it with @jamrce 's suggestions.

[jamrce]

These are good points to be agreed upon or if not at least capture the pro's / con's of each as part of any movement made. In particular, how best to articulate both the conceptual model as well as where they could aid information management systems. The second not being the intention of this initial effort, but could be explored if interest by anyone.

Addressing each of your points in turn:

• With regards to the tooling support, querying RDF can be done in a variety of programming languages and tooling ecosystems. Specifications for both RDFS and SHACL can be found in RDF for being queried against. Chiefly this is about expressing the conceptual model of HQDM that tools can read easily for understanding than EXPRESS can be done today.
• This proposal is not to force or mandate any of these specifications on implementations such as MagmaCore, this is focused on how best to articulate the conceptual model so implementations can have the best understanding about the model available and then choose how little or much to adopt themselves to achieve interoperability. The traditional use of SHACL for validating data could be useful, however the focus is on a queryable structure for generating implementations. This also allows comparisons to other models to understand what structural differences are in place.
• I agree that most "metadata" seen today are most commonly undocumented short-hands for what is expected in today's data practices. "All models are wrong, but some are useful". It is clear when metadata on top level entities like thing are used they are a shorthand's that may need more granularity to achieve high quality data, in particular for aspects such as provenance. The drive is to start bridging that gap between the structured understanding and rigour with acknowledgement of short-hands so information can be better evaluated to assess its quality; a middle ground to articulate the HQDM reference material without some of that metadata foundation being in place yet. Its worth noting that when reflecting on other 4D models that short-hands / pragmatic choices that Matthew made within HQDM can be seen, especially compared to his work on ISO 15926-2 that could be further rationalised against where he either reduced the amount of relationships or removed entities altogether such as 15926:cause_of_event and 15926:TemporalBounding respectively.