Feature genie skills upgrade issue#83

[BlitzBricksterYY-db]

Add serialized space support in Genie API methods

Enhance the create_or_update_genie function to accept a serialized_space parameter for creating or updating Genie spaces using a full serialized configuration. Introduce new methods in AgentBricksManager for exporting and importing Genie spaces with serialized payloads, allowing for full configuration management and cross-wksp migration of Genie spaces. Update documentation to reflect these changes and provide usage examples.

See below screenshots for tested migration successful in 6 steps leveraging MCP tools and skills in this update.

Recording for leverage new skills to migrate Genie spaces from prod wksp to dev wksp
https://github.com/user-attachments/assets/430730d3-775a-4d3e-8a64-baee217302ed

[BlitzBricksterYY-db]

1. significantly overhauled two .py for supporting Genie Space management with structured json output.
2. The updated skill documents in the commit is more than just prettify and organized, it actually solves a bug during my integration test:

• (fixed) space was exported from prod, remapped catalog name in structured json output and then imported into dev wksp successfully, but "description" of space after migration is empty as shown in Genie UI.
• (fixed) by detailed example showing how to setup two MCP servers according to prod + dev profile specified in ~/.databrickscfg, Genie skills are able to switch wksp and migration across wksp smoothly!

[calreynolds]

Awesome! Thanks Yang! Here's what I got back in LLMing on top. Everything else looks great 👍

Typo in SKILL.md — line 471 has a double comma: title=exported["title"],,

create_or_update_genie indentation bug — when serialized_space is provided, the response dict and track_resource block are inside the else branch (non-serialized path). The serialized path never returns a response dict — it falls through to the except block or returns nothing. This is a real bug.

serialized_space version inconsistency — the docstring in genie_update_with_serialized_space says "version":1 in the example, but the skill docs say version 2 is current. Per the API docs, version 2 is correct for new spaces.

Docs are verbose — spaces.md adds ~185 lines and SKILL.md adds ~100 lines. The same 3-step workflow (export → remap catalog → import) is explained 3 times across the two files. Could be trimmed by ~40% without losing information.

[calreynolds]

per comment!

[BlitzBricksterYY-db]

Awesome! Thanks Yang! Here's what I got back in LLMing on top. Everything else looks great 👍

Typo in SKILL.md — line 471 has a double comma: title=exported["title"],,

create_or_update_genie indentation bug — when serialized_space is provided, the response dict and track_resource block are inside the else branch (non-serialized path). The serialized path never returns a response dict — it falls through to the except block or returns nothing. This is a real bug.

serialized_space version inconsistency — the docstring in genie_update_with_serialized_space says "version":1 in the example, but the skill docs say version 2 is current. Per the API docs, version 2 is correct for new spaces.

Docs are verbose — spaces.md adds ~185 lines and SKILL.md adds ~100 lines. The same 3-step workflow (export → remap catalog → import) is explained 3 times across the two files. Could be trimmed by ~40% without losing information.

Every feedback points have been tackled in the new commit of this PR!
I tested again the Genies migration task across wksp using new skills, smooth!

[MigQ2]

Great PR @BlitzBricksterYY-db !!

When I tried to create the serialized_space using an agent it really struggled with the format it has to use as I did not find detailed or proper documentation on this serialization format anywhere.

In #172 I tried my best to add instructions in the SKILL to teach the agent what's the expected format for the serialized_space (it especially struggled with join_specs)

Would be great if this is taken into account in this PR, as someone from Databricks definitely can provide more detail on the expected format for seralized_space maybe based on some internal documentation that can be more detailed than what's publicly available online

[calreynolds]

@BlitzBricksterYY-db one more piece of feedback: can we consolidate the import and export into a single mcp tool? we're motivated to keep the # of tools down as low as possible to prevent agent bloat -- then I think when that's cleared + the linting, we'd be good to merge!

[BlitzBricksterYY-db]

Great PR @BlitzBricksterYY-db !!

When I tried to create the serialized_space using an agent it really struggled with the format it has to use as I did not find detailed or proper documentation on this serialization format anywhere.

In #172 I tried my best to add instructions in the SKILL to teach the agent what's the expected format for the serialized_space (it especially struggled with join_specs)

Would be great if this is taken into account in this PR, as someone from Databricks definitely can provide more detail on the expected format for seralized_space maybe based on some internal documentation that can be more detailed than what's publicly available online

@MigQ2 Helllo, Mig! Thanks for sharing, I agree it is important to figure out the right format of serialized_space. I will research this and probably get this done in a new small PR!

[BlitzBricksterYY-db]

@calreynolds

Totally get it! I want to confirm the intention before proceeding.

My originally thinking is export_genie skill could also be used somewhere by the user, e.g., for fetching the Genie spaces' metadata and combine with other things for a VS index or a retrieval tool so an Agent to use, and is thus not necessarily immediately imported back as a Genie space.

Import_genie on the other side, could take a modified serialized_space, e.g., user updated serialized_space programmatically (like those joins, sql expressions).

Keep them separate could leave some imaginable operations flexible in the middle, so users can not limited to migration as IS.

However, I am willing to proceed with the merge to one skill of "migrate_genie" if you think it is something more preferred.

Let me know! Many Thanks!~