[FR] Establish FOCUS metadata-specification guidelines

[ijurica]

Problem Statement

Lack of formal guidelines for structuring and writing the FOCUS Metadata Specification (especially the normative requirements of individual elements) increases the risk of inconsistencies, ambiguities, and misalignment across implementations. This can result in varied interpretations, incomplete metadata, or reduced usability for downstream systems and users.

To address this issue, the following is suggested:

• Establish clear and consistent guidelines for structuring and maintaining the FOCUS Metadata Specification, with an emphasis on the normative requirements of individual elements.
• Review existing FOCUS Metadata element specifications and align them with the newly established guidelines.
• Identify inconsistencies, redundant or missing elements, and areas requiring clarification.
• Update the existing FOCUS Metadata Specification where possible, and document broader issues for future iterations.

Note: Assuming this effort includes adding glossary entries for terms like "FOCUS Metadata" and "FOCUS Metadata Schema", it's worth reviewing where these terms already appear in the specification (e.g., in the ChargeDescription column normative requirements) and updating those references to point to the newly added glossary definitions.

Use Cases

As a FOCUS Specification Maintainer
I need clear guidelines for writing metadata element specifications with consistent normative requirements
So that new metadata elements are documented uniformly and reduce ambiguity for implementers

As a FOCUS Data Generator
I need unambiguous metadata specifications with clear normative requirements
So that I can implement FOCUS-compliant metadata without interpretation uncertainty or requiring clarification requests

As a FinOps Tooling Vendor building conformance validators
I need consistently structured metadata specifications with testable requirements
So that I can build automated conformance checks that validate metadata across all FOCUS datasets

As a Data Engineer consuming FOCUS datasets
I need reliable metadata that follows consistent patterns across providers
So that I can build schema-aware ETL workflows without provider-specific workarounds

As a FOCUS Working Group member reviewing contributions
I need established guidelines to evaluate proposed metadata changes
So that I can assess whether contributions meet quality standards and maintain specification consistency

Acceptance Criteria

Guidelines Establishment:

• FOCUS Metadata Specification guidelines are documented with clear structure and requirements
• Guidelines define how to write normative requirements for metadata elements
• Guidelines specify required vs. optional metadata element properties
• Guidelines establish naming conventions and terminology standards
• Guidelines address how metadata elements relate to columns, attributes, and datasets

Existing Metadata Review:

• All existing FOCUS Metadata elements are reviewed against established guidelines
• Inconsistencies, ambiguities, and gaps are identified and documented
• Redundant or missing elements are catalogued
• Prioritization framework exists for addressing identified issues

Specification Alignment:

• Existing metadata specifications are updated to align with guidelines where feasible
• Issues requiring broader discussion are captured as follow-up feature requests
• Glossary entries for "FOCUS Metadata" and "FOCUS Metadata Schema" are added
• References to metadata terms throughout specification are updated to point to glossary

Conformance Foundation:

• Metadata guidelines support definition of static conformance requirements
• Metadata specifications are structured to enable automated conformance testing
• Clear distinction between normative (must/should) and informative content exists

Documentation & Governance:

• Guidelines are accessible to specification contributors and reviewers
• Process for maintaining guidelines over time is established
• Examples of well-structured metadata specifications are provided

Supported Feature Alignment

Current State Assessment

[ ] Enhances existing feature: Schema Metadata

• Current Gap: Feature describes schema metadata purpose but lacks formal specification guidelines, leading to inconsistent metadata documentation and implementation ambiguity
• Needed Capability: Specification maintainers and data generators need consistent, testable metadata specifications to enable conformance validation and reduce implementation uncertainty

[ ] Enables specification governance (meta-capability)

• Note: This feature primarily benefits specification development rather than end-user FinOps capabilities. However, improved metadata specification quality indirectly benefits all practitioners through more consistent implementations and better tooling support.

Enhancement to "Schema Metadata" Feature

Feature Name

Schema Metadata

Description (Enhanced for 1.4)

FOCUS' schema metadata supports communication of important attributes about the data, facilitating notifications about changing structure and database table creation between data generator and consumer. This includes column names, data types, and any other relevant information about the data schema. It also includes information as to the version of FOCUS and Data Generator versioning that the data uses.

Enhanced in 1.4: The FOCUS Metadata Specification now follows formal guidelines ensuring consistency, clarity, and testability across all metadata elements. These guidelines establish normative requirements structure, naming conventions, and documentation patterns that enable automated conformance validation and reduce implementation ambiguity. All existing metadata elements have been reviewed and aligned with these guidelines, providing a foundation for static conformance requirements and tooling.

Applicable Metadata

• Schema
  • Column Definition
  • Metadata Element Guidelines (add - establishes specification structure)
  • Conformance Requirements (add - enables validation)

Supporting Documentation

• Metadata Specification Guidelines
• Glossary entries for metadata terminology
• Examples of conformant metadata specifications

Introduced (Version)

1.1 (baseline), 1.4 (guidelines and alignment)

Desired Outcome / Practitioner Impact

Desired Outcome

• Established FOCUS Metadata Specification Guidelines, supporting both the refinement of existing FOCUS Metadata content and enabling consistent, high-quality contributions going forward.

• Identification and resolution of issues in the existing FOCUS Metadata Specification

  • Note: Broader or more complex issues that cannot be addressed immediately will be captured as follow-up Feature Requests.

• Support for preparation of Static Conformance Requirements and conformance tooling

  • Similar to the refinement of FOCUS Columns and Attributes, refining the Metadata Specification is a prerequisite for defining Static Conformance Requirements, which are required for the implementation of conformance tooling (see pending PR).

- Applied to existing FOCUS metadata
- Once the guidelines exist, we need to implement them for existing FOCUS metadata.

Not in scope

The guidelines we have in place today are not normative (enforceable) as they live outside of the specification. We will need to stand up a new feature request to evaluate which of our guidelines will be refit to be enforced through traditional normative requirement guidelines.

Type of Request

Refinement of an existing FOCUS attribute

Organizations Requesting or Supporting This Feature

• Neos
• FinOps Foundation
• Domo

FinOps Scope Alignment (Optional)

• Public Cloud – e.g., AWS, Azure, GCP, OCI
• Software-as-a-Service (SaaS) – e.g., Salesforce, Snowflake
• Data Center – on-prem compute and infrastructure
• Licensing – subscription or usage-based licensing (under development)
• AI – cost and usage for AI models and platforms (under development)
• Custom – internal tooling, specialized infra (under development)

Impacted Parties

• FinOps Practitioner – end users who analyze or act on the data
• FOCUS Data Generator – providers generating aligned output
• Vendor Supporting FOCUS – tools or platforms using the spec
• Other (please explain in comments)

Level of Ambiguity (1–5)

2

Supporting Documentation

Supports the following Feature Requests:

• FR [FR] Define Requirements Model (RM) for data generators #1054
• FR [FR] Define conformance criteria for FOCUS data generators #1049

Working Group access only

• 1060 Metadata Guidelines Ideation Doc
• 1060 Metadata guidelines

Proposed Solution / Approach

• Create and document clear FOCUS Metadata Specification guidelines.
• Apply those guidelines in a review of existing FOCUS Metadata Specification.
• Flag and address inconsistencies, ambiguities, and gaps.
• Capture broader improvements in follow-up Feature Requests where needed.

Note: Assuming this effort includes adding glossary entries for terms like "FOCUS Metadata" and "FOCUS Metadata Schema", it's worth reviewing where these terms already appear in the specification (e.g., in the ChargeDescription column normative requirements) and updating those references to point to the newly added glossary definitions.

[shawnalpay]

Lots to do here, but thankfully we have a good precedent set on how to do this based on the work in 1.2 per PRs:

• WI #557: Normative Requirements Refinement - Part 1 #787
• WI #557: Normative Requirements Refinement - Part 2 #798