How an AI Agent Builds Android UI It Has Never Seen: A First Look at a2ui

AI agents can now reason about a task, call tools, and hold a conversation. What they still cannot do well is show you a native screen. An agent that books a flight can describe the options in text, but it cannot hand your Android app a real date picker, a seat map, and a confirm button, because it has no idea what components your app has. a2ui (Agent to User Interface), a new AndroidX module the Android team is still building, is an attempt to close that gap. It defines a protocol and a data layer that let an agent, running on a server, build and update native Android UI on a client it has never seen before, by reading machine readable descriptions of the components that client offers.

Before going further, one point needs to be stated plainly. a2ui is not a released library, and it is not a stable API. It is not even a public experimental one. It is work in progress, developed in the open by the Android team, and large parts of it are still being written: the Compose renderer is a placeholder, and the message orchestration layer does not exist yet. Nothing here ships in an artifact you can depend on today, and the code will keep changing. Everything below is read from that in progress source, which is what makes it worth learning now. The protocol and the data model are already there, so you can see the concept of the system before the implementation settles.

In this article, you'll dive deep into the idea behind agent driven UI, how a2ui lets components describe themselves to an agent with JSON Schema, how the surface protocol streams UI and data from server to client, how data binding connects the two, how the schema validator guards the boundary, and exactly what is implemented today versus still planned.

The fundamental problem: an agent that can reason but cannot draw

Put an agent in front of a real task and a gap appears immediately. The agent can work out what the user needs (a form with three fields, a list of results, a confirmation dialog), but it has no way to place that into your app as native UI. Two obvious workarounds both fail. The agent can return text or markdown, which throws away every native control and gesture your app already has. Or you can predefine every screen the agent might ever need and let it pick one by name, which defeats the point of having a flexible agent at all.

What is missing is a shared contract. The agent needs a machine readable description of the UI building blocks a specific client offers, detailed enough that the agent can choose the right ones and fill them in correctly, without the app and the agent being built together. That description has to be something a language model can read and reason about, and something the client can validate against. a2ui's answer to both requirements is JSON Schema, and the first thing to understand is how a component turns itself into one.

Components that describe themselves to the agent

In a2ui, a component is not a fixed widget the agent must already know. It is a definition that advertises itself. The engine models this with a small interface:

public interface A2uiCoreComponentDefinition {
    /** Component name that would be provided to the AI agent. */
    public val name: String
    /** Describes the component behavior so the AI agent knows when to use it. */
    public val description: String
    public val propertySchema: A2uiSchema
}

The comments matter as much as the types. The name and description are written for a language model to read, not for a compiler, and the propertySchema describes the exact shape of the properties this component accepts. A catalog collects these definitions along with a set of callable functions and an optional theme schema:

public interface A2uiCoreCatalog {
    public val id: String
    public val components: List<A2uiCoreComponentDefinition>
    public val functions: List<A2uiFunction>
    public val themeSchema: A2uiSchema?
    public fun getComponent(name: String): A2uiCoreComponentDefinition?
    public fun getFunction(name: String): A2uiFunction?
}

Functions are described for the agent in the same way. A function definition carries a name, a natural language description of its behavior, an argument schema, and a return type drawn from the JSON compatible types an agent understands:

public interface A2uiFunctionDefinition {
    /** Function name that would be provided to the AI agent. */
    public val name: String
    /** Describes the function behavior so the AI agent knows when to use it. */
    public val description: String
    public val argumentSchema: A2uiSchema
    public val returnType: A2uiFunctionReturnType
}

This is the design decision at the center of a2ui. The catalog is a description of a client's capabilities aimed squarely at a language model. The agent does not need to be compiled against your app. It reads the names, the descriptions, and the schemas, decides which components and functions fit the task, and emits UI that conforms to them. The client stays in control of what exists, and the agent stays in control of what to show. Everything hinges on the schema being precise, so that is the next layer.

JSON Schema as the shared language

a2ui models JSON Schema directly in Kotlin. A2uiSchema is a sealed type whose whole job is to serialize outward into a JSON Schema document:

public sealed class A2uiSchema {
    public abstract val description: String?
    public fun toJsonSchema(): String = toJsonElement().toString()
    internal abstract fun toJsonElement(): JsonElement
}

Notice the direction. There is a toJsonSchema() that produces a schema string, and no matching parser that reads one back in. That fits the purpose: the schema exists to be advertised to the agent and to validate incoming payloads, not to be round tripped. Each leaf models one JSON Schema construct. A string schema emits {"type":"string"}, an object schema emits properties, required keys, and additional property rules:

public class A2uiObjectSchema(
    public val properties: Map<String, A2uiSchema> = emptyMap(),
    public val required: Set<String> = emptySet(),
    public val isAdditionalPropertiesAllowed: Boolean = true,
    public val additionalPropertiesSchema: A2uiSchema? = null,
    public override val description: String? = null,
) : A2uiSchema() {
    init { require(required.all { properties.containsKey(it) }) }
    // toJsonElement writes type, properties, required, additionalProperties, description
}

The full set covers the constructs you would expect: A2uiStringSchema, A2uiNumberSchema, A2uiBooleanSchema, A2uiArraySchema, A2uiObjectSchema, A2uiEnumSchema, A2uiConstSchema, A2uiRefSchema, and the combinators A2uiAnyOfSchema, A2uiOneOfSchema, and A2uiAllOfSchema. There is also an A2uiAnySchema that emits an empty schema {} for unstructured payloads that intentionally bypass strict validation, and an A2uiCompositeSchema base for named, reusable schemas that serialize as a $ref into a $defs section rather than inlining.

On top of these primitives, a2ui builds its own vocabulary in a commontypes package, and this is where the UI specific concepts live. A data binding is a schema for { "path": string } that points into the data model. A child list is a oneOf of either a static array of component ids or a template object that binds a component to a data model array for generating a dynamic list. A dynamic value, A2uiDynamicValueSchema, is a oneOf of a literal, a data binding, or a function call. An action is a oneOf of a server event or a client function call. So the schema layer is not only how components are described, it is how the entire structure of an a2ui interface (its bindings, its lists, and its interactions) is expressed in terms an agent can produce and a client can check.

The surface protocol: how a server drives a client

With the vocabulary established, the runtime protocol is straightforward. The unit of UI is a surface, identified by a surfaceId, and a server drives it by sending messages. The server to client side is a sealed interface. Two of its four messages create a surface and populate it:

public sealed interface A2uiServerToClientMessage {
    public val surfaceId: String
}

public class A2uiCreateSurfaceMessage(
    override val surfaceId: String,
    public val catalogId: String,
    public val theme: Map<String, Any?> = emptyMap(),
    public val shouldSendDataModel: Boolean = false,
) : A2uiServerToClientMessage

public class A2uiUpdateComponentsMessage(
    override val surfaceId: String,
    public val components: List<A2uiComponentPayload>,
) : A2uiServerToClientMessage

The other two write into the surface's data model and tear the surface down:

public class A2uiUpdateDataModelMessage(
    override val surfaceId: String,
    public val path: String = "/",
    public val value: Any? = null,
) : A2uiServerToClientMessage

public class A2uiDeleteSurfaceMessage(override val surfaceId: String) : A2uiServerToClientMessage

The four messages map onto a surface lifecycle. A2uiCreateSurfaceMessage opens a surface bound to a catalog. A2uiUpdateComponentsMessage inserts or replaces components. A2uiUpdateDataModelMessage writes a value at a path in the data model, where a null value means delete. A2uiDeleteSurfaceMessage tears the surface down. Each component on the wire is deliberately minimal:

public class A2uiComponentPayload(
    public val id: String,
    public val type: String,
    public val properties: Map<String, Any?>,
)

An id, a type such as "button" or "text", and an untyped properties map. There are no separate fields for children or bindings, because those live inside properties and are given meaning by the schema layer from the previous section. The client talks back with its own sealed set:

public sealed interface A2uiClientToServerMessage

public class A2uiUserAction(
    public val type: String,          // "click", "swipe", ...
    public val surfaceId: String,
    public val componentId: String,
    public val timestamp: Long,
    public val context: Map<String, Any?> = emptyMap(),
) : A2uiClientToServerMessage

public class A2uiClientError(
    public val code: String,          // "VALIDATION_FAILED", "RUNTIME_ERROR"
    public val surfaceId: String,
    public val message: String,
    public val context: Map<String, Any?> = emptyMap(),
) : A2uiClientToServerMessage

The error path is designed for the agent to read. The code and message are, per the source KDoc, reported back so that the remote agent can recognize the category of failure and diagnose it. An agent that emits an invalid payload gets a structured VALIDATION_FAILED back, with the offending path in the context, rather than a silent failure. The whole protocol is a loop between an agent that proposes UI and a client that renders it, validates it, and reports what the user did.

Data binding: components that react to a streaming data model

The reason the protocol separates components from data is performance and clarity. The server sends component templates once, then streams data as deltas. A component property does not have to hold a literal value; it can hold a binding into a data model that the server updates independently. Paths into that model are JSON Pointer strings, wrapped at runtime by A2uiDataPath:

public class A2uiDataPath(public val path: String) {
    public val normalizedPath: String = normalize(path)
    public val segments: List<String> = parse(normalizedPath)
    public val isAbsolute: Boolean
        get() = path.isEmpty() || path.startsWith("/")
}

It follows RFC 6901 with a few documented conveniences, so that "" and "/" both mean the root and a single trailing slash is stripped, to match the behavior other a2ui implementations expect. When the server sends an A2uiUpdateDataModelMessage for /settings/volume, any component property bound to that path updates, without the server resending the component. This is what A2uiDynamicValueSchema encodes: a property can be a literal, a data binding, or a function call, and the client resolves it against the current data model. Dynamic lists work the same way, with a child template bound to a data model array so that adding an item to the array adds a child to the UI. The component tree describes structure, and the data model carries state, and the two are joined by paths.

The engine's runtime model

On the client, the engine turns these messages into observable domain models. A single surface is an A2uiCoreSurfaceModel. It owns the data model and the component registry for that surface, and it exposes the actions a user can trigger:

public class A2uiCoreSurfaceModel(
    public val id: String,
    public val catalog: A2uiCoreCatalog,
    public val dataModel: A2uiCoreDataModel,
    public val componentRegistry: A2uiCoreComponentRegistry,
    private val onDispatchAction: (A2uiUserAction) -> Unit,
    private val onDispatchError: (A2uiClientError) -> Unit,
    // theme, shouldSendDataModel, timeProvider ...
) {
    internal fun updateDataModel(path: A2uiDataPath, value: Any?) = dataModel.update(path, value)
    internal fun updateComponents(payloads: List<A2uiComponentPayload>) =
        componentRegistry.update(payloads)

    public fun dispatchAction(componentId: String, actionDefinition: Map<String, Any?>) { /* builds A2uiUserAction */ }
    public fun dispatchError(componentId: String, exception: A2uiException) { /* reports + emits A2uiClientError */ }
}

All active surfaces are held by an A2uiCoreSurfaceGroupModel, which is the single observability seam for a host UI framework. It exposes the surfaces as a StateFlow, so a renderer can collect it and react:

public class A2uiCoreSurfaceGroupModel internal constructor() {
    private val _activeSurfaces = MutableStateFlow<List<A2uiCoreSurfaceModel>>(emptyList())
    /** Exposes the currently active surfaces to the host UI framework. */
    public val activeSurfaces: StateFlow<List<A2uiCoreSurfaceModel>> = _activeSurfaces.asStateFlow()
}

The data model and the component registry are not concrete classes. They are interfaces. The component registry's KDoc names the design intent for both: host frameworks back them with native reactive state, for example a SnapshotStateMap in Compose. The data model is the matching storage interface for the JSON data tree:

public interface A2uiCoreDataModel {
    public fun update(path: A2uiDataPath, value: Any?)
    public operator fun get(path: A2uiDataPath): Any?
    public fun dispose()
}

This is a deliberate seam. The engine holds the protocol logic and the surface structure, and it delegates storage and reactivity to whatever platform renders it. In Compose, an implementation would write updates into snapshot state so that reads inside composables recompose automatically. The engine never mentions Compose, which keeps the protocol layer independent of any one UI toolkit.

Validation at the boundary

Because the payloads come from an agent, they cannot be trusted to be well formed. The engine's A2uiCoreSchemaValidator walks a value against an A2uiSchema and throws on the first mismatch, tracking a JSON path breadcrumb as it goes:

internal object A2uiCoreSchemaValidator {
    internal fun validateSchema(payload: Any?, schema: A2uiSchema): Boolean {
        validateSchemaInternal(payload, schema, "$")
        return true
    }
}

It dispatches on the schema type, validating objects against their required and typed properties, arrays against their item schema, and the combinators with the right cardinality, so oneOf requires exactly one match, anyOf at least one, and allOf all of them. When something does not match, it throws an A2uiValidationException carrying the path, such as $.name for a wrong property type or $.list[1] for a bad array element. That exception maps directly onto the A2uiClientError with code VALIDATION_FAILED that the client sends back, so a schema violation becomes a structured message the agent can act on. The validator is the most complete algorithm in the engine, and it is what will let a client refuse corrupt UI before it reaches the screen.

What is built today, and what is still scaffolding

Here is the honest state of the module, because it changes how you should read everything above. a2ui is four Gradle modules at very different stages.

• a2ui-model is the most mature. The protocol messages, A2uiComponentPayload, A2uiDataPath, the exception types, and the full A2uiSchema hierarchy with its commontypes vocabulary are all implemented and tested. This is the real foundation.
• a2ui-engine is partial. The surface models and the schema validator are implemented and tested. But the data model, the component registry, the catalog, and the component definition are interfaces with no concrete implementations in the tree, no components ship in any built in catalog, and there is no orchestration layer. Nothing yet parses an A2uiServerToClientMessage and drives the models from it. The mutating methods are internal and the group model's constructor is internal, and nothing in the module constructs them.
• a2ui-core contains no Kotlin source at all. It is a declared but empty module. Despite the A2uiCore prefix, the A2uiCore classes you have seen throughout this article live in a2ui-engine, not here.
• a2ui-compose is a single placeholder class. The Compose renderer, the piece that would turn a surface and its registry into composables, does not exist yet.

There are also telling gaps. The model has no @Serializable annotations and no JSON codec, so the wire parsing that would turn a real network message into these classes is not written here. An internal A2uiHandleActionMessage refers to a SurfaceActor queue that is not implemented. The integration test app is an empty Activity. So the intended flow (a message arrives, a surface is created or updated, the data model changes, a Compose renderer recomposes) is visible in the design but not yet wired end to end. What exists today is a carefully modeled protocol, a set of domain models, and a validator, which is the skeleton you would build first if you were designing this system from the wire format up.

What you can do with it today

Because the renderer and the orchestration are not written, you cannot yet drop a2ui into an app and let an agent draw screens. It helps to separate what is public from what is not. The surface mutators such as updateComponents and updateDataModel, and the A2uiCoreSchemaValidator, are internal, so the code that drives them today is the engine's own test module, which can reach an internal API. Those tests are the clearest picture of the intended runtime, and they read like this:

val surface = A2uiCoreSurfaceModel(
    id = "surface-1",
    catalog = myCatalog,
    dataModel = myDataModel,        // an A2uiCoreDataModel, e.g. a map or snapshot state backed store
    componentRegistry = myRegistry, // an A2uiCoreComponentRegistry
    onDispatchAction = { action -> sendToServer(action) },
    onDispatchError = { error -> sendToServer(error) },
)

surface.updateComponents(listOf(A2uiComponentPayload("btn-1", "button", mapOf("text" to "Click Me"))))
surface.updateDataModel(A2uiDataPath("/settings/volume"), 10)
surface.dispatchAction("btn-1", mapOf("type" to "click", "context" to mapOf("x" to 100, "y" to 200)))

The validator is exercised the same way from those tests, and it is the piece most worth understanding, because it is what a real client will lean on to reject bad payloads:

val schema = A2uiObjectSchema(
    properties = mapOf("name" to A2uiStringSchema(), "age" to A2uiNumberSchema()),
    required = setOf("name"),
)
A2uiCoreSchemaValidator.validateSchema(mapOf("name" to "Alice", "age" to 30), schema)  // true
// mapOf("name" to 123) throws A2uiValidationException with path "$.name"

From your own module, the fully public layer is a2ui-model. You can build the protocol types, and you can describe a component's contract as a schema and emit it as JSON Schema for an agent to read with toJsonSchema(). In other words, today a2ui is usable as a protocol and schema modeling library, and as a preview of the runtime a renderer will sit on top of. If you are experimenting with agent driven UI, that is enough to prototype the server side contract and the schema vocabulary now, and to be ready for the engine and the renderer when they land.

Conclusion

In this article, you've explored a2ui, a new AndroidX module for agent driven UI that the Android team is still building. Its central idea is that a client describes its components to a language model with JSON Schema, so an agent can build native UI it was never compiled against, choosing components by name and description and filling them in against a precise schema. The protocol is a surface based loop: a server creates a surface, streams components and data model deltas, and receives user actions and structured errors back, with data binding through JSON Pointer paths joining a component tree to a separate, streaming data model. The engine turns those messages into observable surface models and validates every payload against its schema before it reaches the screen.

As always, happy coding!

— Jaewoong (skydoves)