7 min read1 hour ago

–

A Case for Semantic Layers That Machines Can Actually Understand

This essay argues from the perspective of AI agents — not human developers. The distinction matters more than you might think.

For the past decade, UI generation has been the flagship promise of developer productivity. From code templates to visual builders to declarative schemas, we’ve optimized relentlessly for one goal: helping humans build interfaces faster.

But something fundamental has shifted.

The goal is no longer just generating UI. The goal is enabling AI agents to understand, operate, and evolve applications autonomously. This subtle shift rewrites the architecture of modern software — and reveals why UI generation, however sophisticated, is no longer sufficient.

The Blind Spot: Semantic for Whom?

Modern UI frameworks are deeply semantic — for humans.

When a React developer sees <UserRegistrationForm />, they immediately grasp its purpose. The component name conveys intent. The props suggest behavior. The structure implies workflow.

But what does an AI agent see?

A black box.

The agent cannot peer inside the component to discover which fields exist, what validation rules apply, which fields depend on others, or what side effects trigger on submission. The component’s semantics are encapsulated — which is a virtue for maintainability but a barrier for machine operability.

Consider a cascading select pattern:

<CategorySelect onChange={handleCategoryChange} /><SubcategorySelect   options={subcategories}   disabled={!selectedCategory} />

A human developer reads this and understands: “Select a category first, then subcategory options load.” But an AI agent examining this code must infer the dependency from implementation details scattered across state management, effect hooks, and conditional rendering logic.

The semantics exist — but they’re implicit, distributed, and locked inside human-readable abstractions.

Why AI Agents Fail: It’s Not Model Accuracy

When an AI agent “mis-clicks” or “hallucinates form behavior,” we often blame model limitations. But I’ve come to believe the real problem lies elsewhere:

AI fails because applications do not expose their internal semantics in machine-readable form.

Think about what an agent cannot learn from observing a UI:

• Business constraints: “Freight shipments require minimum 50kg weight”
• Field dependencies: “Subcategory options depend on category selection”
• Conditional visibility: “Shipping fields hidden for digital products”
• Validation rules: “SKU must match pattern ^[A-Z0-9-]+$“
• Computed values: “Final price = base price × (1 — discount rate)”
• Workflow state: “Draft → Active → Archived progression”
• Side effects: “Changing category clears subcategory selection”

These rules live in useEffect hooks, validation libraries, service layers, backend logic, and sometimes just tribal knowledge. When an AI agent can only perceive the rendered surface, it must guess the system underneath.

And any system that forces an agent to guess will eventually break.

The Limits of Visual Understanding

Modern multimodal models can analyze screenshots with impressive accuracy. They identify buttons, read labels, and describe layouts. Some argue this makes semantic layers unnecessary — if AI can “see” the UI, why encode meaning separately?

This argument confuses recognition with operation.

Yes, an LLM can look at a form screenshot and describe what it sees. But can it reliably predict:

• Which button becomes disabled when a checkbox is unchecked?
• What API call fires when this dropdown changes?
• Why this field suddenly appeared after selecting “Physical Product”?
• What happens to field B when field A’s value crosses a threshold?

In my experience building automated testing systems, the pattern is consistent: simple forms work well; complex forms with interdependencies fail unpredictably. The agent can read the UI but cannot understand the system.

Screenshot comprehension is inference. Semantic access is knowledge.

What Agents Actually Need

Instead of pixels and DOM nodes, AI agents need direct access to:

Human Artifact Agent Requirement Button Action with preconditions and effects Input field Semantic field with type, constraints, dependencies Form State graph with transitions and invariants Validation error Logical rule that can be evaluated deterministically Screen View that projects an underlying model

The key insight: UI is one possible rendering of a deeper structure. That deeper structure is what agents need to operate on.

This doesn’t mean UI is useless. It means UI is insufficient as the primary interface for machine operation.

A Concrete Example: Declarative View Schemas

To move from philosophy to practice, consider how we might express a product creation form not as UI code, but as a semantic schema:

// Entity: The domain truth (what exists)field.enum('fulfillmentTypeCode', 'Shipping Type', fulfillmentTypes)  .required('Select a shipping type')  .build()field.number('shippingWeight', 'Weight (kg)')  .min(0)  .max(2000)  .build()// View: Presentation + Interaction (how it behaves)viewField.numberInput('shippingWeight', 'shippingWeight')  .dependsOn('productTypeCode', 'fulfillmentTypeCode')  .reaction(    on.change()      .when(        and(          fieldEquals('fulfillmentTypeCode', 'FREIGHT'),          lessThan($.state('shippingWeight'), 50)        )      )      .do(actions.setValue('shippingWeight', 50))  )  .build()

This schema explicitly declares what would otherwise be implicit:

• Dependency: shippingWeight depends on productTypeCode and fulfillmentTypeCode
• Business rule: Freight shipments enforce minimum 50kg weight
• Trigger: Rule applies on value change
• Effect: Automatic value correction

An AI agent reading this schema doesn’t need to reverse-engineer behavior from rendered output. The semantics are exposed, not encapsulated.

The same underlying model can generate multiple projections — a create form, an edit form, an admin view, a mobile interface — while the semantic truth remains constant and queryable.

Why Existing Technologies Aren’t Enough

A reasonable objection: “Don’t OpenAPI, GraphQL, and JSON Schema already provide semantic structure?”

Yes, partially. But each addresses only a fragment:

Technology What It Describes What It Doesn’t OpenAPI API shape and contracts UI state, field dependencies, workflows GraphQL Data relationships and queries Presentation logic, validation UX, conditional rendering JSON Schema Data validation rules View binding, reactive behaviors, side effects MCP Tool contracts and capabilities Field-level dependencies, form state machines

None of these provide a unified model spanning data, view, logic, workflow, dependencies, and actions.

What’s needed is not another fragment but an integration layer — a semantic runtime where all these concerns are expressed in a single, coherent, machine-readable structure.

The Architecture Shift

Traditional applications are built as layered fossils:

UI Components    ↓State Management    ↓Validation Libraries    ↓API Clients    ↓Backend Services    ↓Database Logic

Meaning is fractured across layers. Reconstructing intent requires excavating each stratum and inferring connections.

The alternative architecture centers on a semantic core:

┌─────────────────────┐         │   Semantic Model    │         │  (Entity + View +   │         │   Rules + Workflow) │         └──────────┬──────────┘                    │    ┌───────────────┼───────────────┐    │               │               │    ▼               ▼               ▼Human UI       AI Agent API    Automation Tools(React)         (MCP/API)       (Playwright)

The semantic model becomes the source of truth. UI is one projection. Agent APIs are another. Testing tools access the same underlying structure through different interfaces.

Implications for AI-Native Software

If we accept that AI agents need semantic access, several implications follow:

1. Schema-first development becomes essential

Instead of building UI and inferring schemas, we define schemas and generate UI. The schema is the product; the interface is a view.

2. Business rules must be declarative and exposed

Rules buried in imperative code are invisible to agents. Rules expressed as declarative constraints are queryable, testable, and interpretable.

3. Dependencies become first-class citizens

Field A depends on Field B. This relationship should be explicit in the model, not implicit in effect hooks.

4. Validation moves from runtime checks to static contracts

Agents should know before attempting an action whether it will succeed, not discover failures through trial and error.

5. Workflows become state graphs

Instead of ad-hoc status fields, workflows are explicit graphs with defined states, transitions, guards, and effects.

The Deeper Principle

Here is the philosophical core:

UI solves the problem of human-computer interaction. Semantic layers solve the problem of machine-computer interaction.

These are different problems requiring different solutions.

For humans, visual affordances — buttons, inputs, layouts — are essential. We need to see the interface to operate it.

For agents, visual affordances are noise. They need to understand the interface to operate it. Understanding requires access to meaning, not pixels.

As AI capabilities accelerate, the ability to generate beautiful, functional UI becomes commoditized. Any sufficiently advanced model can produce React components. The scarce resource is not UI generation but semantic clarity — the explicit, machine-readable expression of what an application means and how it behaves.

What Needs to Be Built

Moving from philosophy to action, the AI era demands:

• Declarative entity schemas that define domain concepts with types, constraints, and relationships
• Declarative view schemas that bind UI to entities with explicit dependencies and reactions
• Rule engines that evaluate business logic deterministically and transparently
• Workflow runtimes that expose state graphs as queryable structures
• Introspection APIs that let agents discover available actions, preconditions, and effects
• Semantic tooling that generates UI, tests, documentation, and agent interfaces from unified models

This is not about replacing UI frameworks. It’s about building the layer beneath them — the semantic foundation that makes applications interpretable to any consumer, human or machine.

Conclusion

UI generation was the endgame for human-centric development. It automated the tedious parts of building interfaces for people.

But AI agents are not people. They don’t need buttons and dropdowns. They need structured access to meaning — the rules, dependencies, workflows, and constraints that govern application behavior.

The systems we build today should anticipate the agents we will collaborate with tomorrow. That means investing not just in better UI generation, but in semantic layers that expose the meaning beneath the surface.

UI is how we present applications. Semantics are how we understand them.

For AI-native software, understanding precedes presentation.

This essay reflects a personal philosophy on the future of software architecture. The ideas draw from practical experience building declarative schema systems and observing how AI agents interact with — and fail to interact with — traditional applications.