Skip to content

Add Common Issues table to databricks-mlflow-evaluation skill#256

Closed
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:fix/mlflow-eval-common-issues
Closed

Add Common Issues table to databricks-mlflow-evaluation skill#256
CheeYuTan wants to merge 1 commit intodatabricks-solutions:mainfrom
CheeYuTan:fix/mlflow-eval-common-issues

Conversation

@CheeYuTan
Copy link
Contributor

Summary

  • Adds a 10-entry Common Issues troubleshooting table to the databricks-mlflow-evaluation skill
  • Covers genai.evaluate() vs evaluate(), predict_fn signatures, dataset format, scorer issues, MemAlign alignment, and GEPA optimization requirements

Test proof

Tested MLflow evaluation patterns locally and against live workspace e2-demo-field-eng:

Test Result
mlflow.genai import PASS
MLflow version >= 3.x PASS — mlflow version: 3.7.0
Correct dataset format (nested inputs) PASS — verified {"inputs": {"query": "..."}} structure
@scorer decorator available PASS — mlflow.genai.scorer importable
Built-in scorers importable (Correctness, Safety, Guidelines) PASS — all 3 import successfully
TEST: MLflow genai import check
  PASS

TEST: MLflow version check (>= 3.x)
  PASS
  → mlflow version: 3.7.0

TEST: Scorer decorator available
  PASS
  → True

TEST: Built-in scorers importable
  PASS
  → [True, True, True]

All 5/5 MLflow evaluation tests passed.

10-entry troubleshooting table covering genai.evaluate() vs evaluate(),
predict_fn signatures, dataset format, scorer issues, MemAlign alignment,
and GEPA optimization requirements.
@calreynolds
Copy link
Collaborator

Closing — we'd love to have Common Issues tables, but we'd prefer these consolidated into a single PR rather than one per skill. Feel free to resubmit as a single combined PR if you're up for it!

@calreynolds calreynolds closed this Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants