I think we should cut down on the amount of info collected per model

[lehors]

The Evaluate and Edit model forms prompt for a lot of information that is not actually pertinent to the MOF. Namely:

• Description
• Version/Parameters
• Type
• Architecture
• Treatment
• Base model

In practice all this information can be found on GitHub and/or HuggingFace and to be entered in the MOT requires copy/pasting info from one site to another even though they are of actual no use to the MOF evaluation.

Because the 'Description' wasn't saved in the model YAML file we are now hiding that field (see issue #173) but I think we should get rid of the above fields as well and only keep the following ones:

• Name
• Org
• Git repo
• Hugging Face repo

This is enough to fully identify a model and we want users to focus on entering the important data: what components are included and under what license.

To be clear: thanks to an earlier change, none of the fields I suggest we drop are required - users can just skip them - but I think they distract from what's important. We should make it as easy as possible for people to use the MOT and entering more info than we need goes against that.

Thoughts?

[gregcube]

I agree with you here. Less is better. We should only capture what we need to evaluate a model; link to the github/ hf repo if the user wants to see more.

[matthew-d-white]

Less data collection will be easier but I would say description is a required field. A description of the model is necessary, but if you can pull it from HF or GH, all the better.

The "Base Model" field you cannot pull from anywhere or decipher without the model producer explicitly providing it. This is for any fine-tuned model that is built off of another. The value in knowing this is that the "Base Model" may be something like Llama 4 under the restrictive Llama 4 community license and the fine tune may be released under Apache 2.0. This cannot be done, there is no way to relicense under a permissive license when using a restrictive license that requires it to be preserved. So without this knowledge, your fine tuned model may qualify as an "Open Model" in the MOF when really it is still bound by the terms of its original license which would not qualify it.

[lehors]

Less data collection will be easier but I would say description is a required field. A description of the model is necessary, but if you can pull it from HF or GH, all the better.

Thanks for your input @matthew-d-white but the description field isn't used anywhere in our system and in fact isn't accounted for in the configuration file definition in the MOF (which might have been an oversight but that's how it is). Why do you think it is necessary though? We make no use of it whatsoever and when we have a link to HF or GH users can get to the description in one click.
We need to be pragmatic. NO ONE is submitting their data at this point. What's there is basically only what we entered ourselves. We need to lower the burden of entering data to the minimum.

The "Base Model" field you cannot pull from anywhere or decipher without the model producer explicitly providing it.

In the few cases I've looked into myself, the model producer always said that in their description. When we have to enter this into the MOT it requires digging for that information which is a pain to do.

This is for any fine-tuned model that is built off of another. The value in knowing this is that the "Base Model" may be something like Llama 4 under the restrictive Llama 4 community license and the fine tune may be released under Apache 2.0. This cannot be done, there is no way to relicense under a permissive license when using a restrictive license that requires it to be preserved. So without this knowledge, your fine tuned model may qualify as an "Open Model" in the MOF when really it is still bound by the terms of its original license which would not qualify it.

I totally understand that. But implementing this would require a lot more than we are currently capable of. We'd somehow need to figure out what license the base model is under to start with.

I hope I don't come across as too negative. My goal is to try to make it easier for people to use the MOT which is mostly ignored for now. I don't think it makes sense for us to raise the bar, we are just making it less likely anyone will bother using it.

[matthew-d-white]

I am supportive of anything that reduces the lift and gets more accurate data into the MOT.