Skip to content

Feat: Generative UI prototype [DO NOT MERGE]#2018

Draft
tomkis wants to merge 7 commits intomainfrom
feat/generative-ui
Draft

Feat: Generative UI prototype [DO NOT MERGE]#2018
tomkis wants to merge 7 commits intomainfrom
feat/generative-ui

Conversation

@tomkis
Copy link
Collaborator

@tomkis tomkis commented Feb 5, 2026

Summary

Linked Issues

Documentation

  • No Docs Needed:

If this PR adds new feature or changes existing. Make sure documentation is adjusted accordingly. If the docs is not needed, please explain why.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @tomkis, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a foundational feature for dynamic user interface generation within the AgentStack ecosystem. It includes a new prototype agent that can generate UI elements on the fly, along with comprehensive updates to both the Python and TypeScript SDKs to support this new "Generative Interface Extension." The AgentStack UI has also been enhanced to interpret and render these dynamically generated UI specifications, paving the way for more interactive and flexible agent-user interactions.

Highlights

  • New Generative UI Agent: Introduces a new Python agent (generative-interface) designed to demonstrate dynamic UI generation capabilities, acting as a financial advisor assistant.
  • Python SDK Extension: Adds GenerativeInterfaceExtensionServer and related models to the agentstack-sdk-py, enabling Python agents to interact with generative UI services.
  • TypeScript SDK Extension: Implements generativeInterfaceExtension and associated schemas/types in agentstack-sdk-ts for client-side handling of generative UI requests and responses.
  • Dynamic UI Rendering in AgentStack UI: Integrates @json-render/react and new components to the agentstack-ui to render dynamically generated UI specifications from agents, enhancing interactive experiences.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • agents/generative-interface/Dockerfile
    • New Dockerfile for the generative interface agent, setting up a Python 3.13 environment with uv for dependency management.
  • agents/generative-interface/pyproject.toml
    • Project configuration for the generative interface agent, specifying dependencies like agentstack-sdk, openinference-instrumentation-beeai, pydantic-settings, and uvicorn.
  • agents/generative-interface/src/generative_interface/init.py
    • Initializer for the generative interface Python package.
  • agents/generative-interface/src/generative_interface/agent.py
    • Core logic for the generative interface agent, including an example financial advisor, LLM interaction, and parsing of UI specifications from LLM responses.
  • apps/agentstack-sdk-py/src/agentstack_sdk/a2a/extensions/services/init.py
    • Imports the new generative_interface module into the Python SDK's services extensions.
  • apps/agentstack-sdk-py/src/agentstack_sdk/a2a/extensions/services/generative_interface.py
    • Defines Python models and server-side logic for the GenerativeInterfaceExtension, including GenerativeInterfaceSpec and GenerativeInterfaceResponse.
  • apps/agentstack-sdk-ts/src/client/a2a/extensions/index.ts
    • Exports the new generative-interface service extension for the TypeScript SDK.
  • apps/agentstack-sdk-ts/src/client/a2a/extensions/services/generative-interface/index.ts
    • Defines the TypeScript client-side extension for the generative interface, including its URI and schema getters.
  • apps/agentstack-sdk-ts/src/client/a2a/extensions/services/generative-interface/schemas.ts
    • Introduces Zod schemas for validating UIElement, GenerativeInterfaceSpec, GenerativeInterfaceResponse, and related demands/fulfillments.
  • apps/agentstack-sdk-ts/src/client/a2a/extensions/services/generative-interface/types.ts
    • Defines TypeScript types derived from the Zod schemas for the generative interface.
  • apps/agentstack-sdk-ts/src/client/core/extensions/resolve-user-metadata.ts
    • Modifies user metadata resolution to include generativeInterfaceResponse when available.
  • apps/agentstack-sdk-ts/src/client/core/extensions/types.ts
    • Extends core types to support GenerativeInterface demands, fulfillments, responses, specs, and a new GenerativeInterfaceRequired task status update type.
  • apps/agentstack-sdk-ts/src/client/core/handle-agent-card.ts
    • Updates agent card handling to extract and fulfill generativeInterfaceExtension demands.
  • apps/agentstack-sdk-ts/src/client/core/handle-task-status-update.ts
    • Adds logic to process GenerativeInterfaceRequired task status updates, using the new generativeInterfaceUiExtension.
  • apps/agentstack-ui/package.json
    • Adds @json-render/core and @json-render/react as new dependencies for UI rendering capabilities.
  • apps/agentstack-ui/src/modules/generative-interface/GenerativeInterfaceRenderer.tsx
    • New React component responsible for rendering dynamic UI based on GenerativeInterfaceSpec using the @json-render/react library.
  • apps/agentstack-ui/src/modules/generative-interface/MessageGenerativeInterface.tsx
    • Component to display the rendered generative interface within agent messages.
  • apps/agentstack-ui/src/modules/generative-interface/components/Button.tsx
    • A basic Carbon Design System button component for use within the generative UI.
  • apps/agentstack-ui/src/modules/generative-interface/components/index.ts
    • Exports the generative interface UI components.
  • apps/agentstack-ui/src/modules/generative-interface/index.ts
    • Exports the main generative interface modules for the UI.
  • apps/agentstack-ui/src/modules/messages/types.ts
    • Updates message types to include GenerativeInterfaceSpec and a new UIGenerativeInterfacePart for displaying generative UI.
  • apps/agentstack-ui/src/modules/messages/utils.ts
    • Adds a utility function getMessageGenerativeInterface to extract generative UI parts from messages.
  • apps/agentstack-ui/src/modules/runs/chat/ChatAgentMessage.tsx
    • Integrates MessageGenerativeInterface to render dynamic UI within the chat interface.
  • apps/agentstack-ui/src/modules/runs/contexts/agent-demands/agent-demands-context.ts
    • Updates the FulfillmentsContext to include generativeInterfaceResponse.
  • apps/agentstack-ui/src/modules/runs/contexts/agent-demands/build-fulfillments.ts
    • Implements the generativeInterface fulfillment, providing the UI catalog prompt to the agent.
  • apps/agentstack-ui/src/modules/runs/contexts/agent-run/AgentRunProvider.tsx
    • Enhances the AgentRunProvider to handle GenerativeInterfaceResponse in agent chat calls and to process GenerativeInterfaceRequired task status updates. It also introduces submitGenerativeInterface for user interaction.
  • apps/agentstack-ui/src/modules/runs/contexts/agent-run/agent-run-context.ts
    • Adds GenerativeInterfaceResponse and submitGenerativeInterface to the agent run context.
  • pnpm-lock.yaml
    • Updated to reflect the new @json-render/core and @json-render/react dependencies.
Activity
  • The pull request was created to introduce a new feature.
  • No human activity (comments, reviews, etc.) has been recorded yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a prototype for a generative UI feature. It adds a new generative-interface agent that uses an LLM to dynamically generate UI components based on a catalog. The changes include the agent's implementation, its Dockerfile, and necessary extensions to both the Python and TypeScript SDKs. Additionally, it adds new React components to the agentstack-ui to render the JSON-based UI specification. My review focuses on cleaning up placeholder code, removing debug artifacts, and addressing an incomplete implementation of user interactions in the new UI components.

Comment on lines 99 to 105
onAction: (a) => {
const setData = setDataRef.current;
const data = dataRef.current;
if (setData) {
// executeAction(a.name, a.params, setData, data);
}
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The UI interaction handling seems incomplete. The onAction handler in buildRegistry has a commented-out call to executeAction, and there's no mechanism to propagate user actions to the MessageGenerativeInterface component to trigger submitGenerativeInterface. This results in a UI that is rendered but not interactive.

To fix this, you could pass an action handler function (e.g., onAction) as a prop to GenerativeInterfaceRenderer and buildRegistry, and then call it from the component's onAction handler.

Comment on lines +107 to +108
except json.JSONDecodeError:
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The try...except json.JSONDecodeError: continue block silently ignores any lines from the LLM output that are not valid JSON. While this makes the parsing robust, it can make debugging difficult if the LLM consistently produces malformed output. Consider adding a log statement within the except block to record these errors for easier debugging.

Suggested change
except json.JSONDecodeError:
continue
except json.JSONDecodeError:
# Consider logging this error for easier debugging of LLM output.
continue

Comment on lines +51 to +59
class FoobarParams(BaseModel):
foobar: str

class GenerativeInterfaceExtensionSpec(BaseExtensionSpec[FoobarParams]):
URI: str = "https://a2a-extensions.agentstack.beeai.dev/services/generative_interface/v1"

@classmethod
def demand(cls) -> Self:
return cls(params=FoobarParams(foobar="xxx"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The FoobarParams class and its use in GenerativeInterfaceExtensionSpec.demand appear to be placeholder code. The parameter foobar is hardcoded to "xxx" and doesn't seem to be used elsewhere. The corresponding TypeScript schema generativeInterfaceDemandsSchema defines an empty object for the demands. To improve clarity and remove prototype remnants, consider removing FoobarParams and simplifying the demand method if no parameters are actually needed.

Suggested change
class FoobarParams(BaseModel):
foobar: str
class GenerativeInterfaceExtensionSpec(BaseExtensionSpec[FoobarParams]):
URI: str = "https://a2a-extensions.agentstack.beeai.dev/services/generative_interface/v1"
@classmethod
def demand(cls) -> Self:
return cls(params=FoobarParams(foobar="xxx"))
class GenerativeInterfaceDemandParams(BaseModel):
pass
class GenerativeInterfaceExtensionSpec(BaseExtensionSpec[GenerativeInterfaceDemandParams]):
URI: str = "https://a2a-extensions.agentstack.beeai.dev/services/generative_interface/v1"
@classmethod
def demand(cls) -> Self:
return cls(params=GenerativeInterfaceDemandParams())

fulfilledMetadata = fulfillFormDemand(fulfilledMetadata, await fulfillments.form(formDemands));
}

console.log(generativeInterfaceDemands);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This console.log statement appears to be for debugging purposes and should be removed before this code is finalized.

Comment on lines 1 to 28
/**
* Copyright 2025 © BeeAI a Series of LF Projects, LLC
* SPDX-License-Identifier: Apache-2.0
*/

'use client';

import { Button as CarbonButton } from '@carbon/react';
import type { ComponentProps, ComponentType } from 'react';

export interface ButtonProps {
id: string;
label: string;
kind?: 'primary' | 'secondary' | 'tertiary';
onInteraction?: (componentId: string, eventType: string, payload?: Record<string, unknown>) => void;
}

export const Button: ComponentType<ButtonProps> = ({ id, label, kind = 'primary', onInteraction }) => {
return (
<CarbonButton
kind={kind as ComponentProps<typeof CarbonButton>['kind']}
size="md"
onClick={() => onInteraction?.(id, 'click')}
>
{label}
</CarbonButton>
);
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This file, along with the components directory it resides in, seems to contain an alternative or unused implementation for UI components. The GenerativeInterfaceRenderer.tsx component defines its own components, including a Button, and does not use this implementation. To avoid confusion and code duplication, consider removing this dead code.

Signed-off-by: Tomas Weiss <tomas.weiss2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant