Model YAML structure

[gregcube]

I'd like to discuss the YAML model structure. Currently, it follows the JSON format, but represented in YAML. For example, using the Amber model, it looks like this:

framework:
  name: 'Model Openness Framework'
  version: '1.0'
  date: '2024-12-15'
release:
  name: Amber
  version: 7B
  type: 'Language Model'
  architecture: 'Transformer (Decoder-only)'
  origin: Amber
  producer: LLM360
  components:
    - 
      name: 'Model architecture'
      description: "Well commented code for the model's architecture"
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Data preprocessing code'
      description: 'Code for data cleansing, normalization, and augmentation'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Training code'
      description: 'Code used for training the model'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Inference code'
      description: 'Code used for running the model to make predictions'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Evaluation code'
      description: 'Code used for evaluating the model'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Supporting libraries and tools'
      description: "Libraries and tools used in the model's development"
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Model parameters (Final)'
      description: 'Trained model parameters, weights and biases'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: 'Model parameters (Intermediate)'
      description: 'Trained model parameters, weights and biases'
      location: ''
      license_name: Apache-2.0
      license_path: ''
    - 
      name: Datasets
      description: 'Training, validation and testing datasets used for the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Evaluation data'
      description: 'Data used for evaluating the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Model metadata'
      description: 'Any model metadata including training configuration and optimizer states'
      location: ''
      license_name: 'Component not included'
      license_path: ''
    - 
      name: 'Sample model outputs'
      description: 'Examples of outputs generated by the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Model card'
      description: 'Model details including performance metrics, intended use, and limitations'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Data card'
      description: 'Documentation for datasets including source, characteristics, and preprocessing details'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Technical report'
      description: 'Technical report detailing capabilities and usage instructions for the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Research paper'
      description: 'Research paper detailing the development and capabilities of the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''
    - 
      name: 'Evaluation results'
      description: 'The results from evaluating the model'
      location: ''
      license_name: 'License not specified'
      license_path: ''

Questions:

1. Do we need the framework key and its key-value pairs?
2. Is the description key-value pair for each component necessary? It's included in the JSON format, but we may not need it in YAML.
3. I'm thinking perhaps we should rename location to component_path for each component?

Let me know your thoughts.

[lehors]

I wasn't involved in the development of this format so @matthew-d-white @ccosborne should chime in but here is my take on this:

1. Do we need the `framework` key and its key-value pairs?

My understanding is that it is meant to define which version of the MOF the file is based on. I think a single version identifier could do but it's always good to have some form of identifier no matter what so I wouldn't just drop the whole thing.

2. Is the `description` key-value pair for each component necessary? It's included in the JSON format, but we may not need it in YAML.

This is meant to allow providing a description for non standard components, but such components are exceptional so I think we could make the description optional so that for all of the standard ones we can skip it.

3. I'm thinking perhaps we should rename `location` to `component_path` for each component?

I agree that this would be more consistent with license_path.

[matthew-d-white]

1. Yes it is needed.
2. Yes it is needed. These are descriptive fields for freeform text.
3. seems logical to me. Any changes to the MOF.JSON format require that the version number be incremented.

[matthew-d-white]

I would also suggest keeping the key names in the file even when the value is undefined: as here: -> https://github.com/isitopenai/MOF/blob/main/MOF.json