LLMOps for Persistent Agents: Close the Loop Between Traces and Work

A persistent agent has a different operating problem from a chatbot that answers one question and stops. It carries state across tasks, reacts to new events, and may affect systems long after the original conversation. LLMOps must therefore connect evaluation to the work the agent actually performs.

Money Forward, Tokyo AI, and Databricks explored these issues at The Agent Revolution: Building Tomorrow’s AI on 22 October 2025. The talks linked context engineering, evaluation and governance, and agents designed to participate in continuing business workflows.

Context is an operational resource

Darren Loke’s account of building AgentForward puts context engineering at the center of reliability. The question is what information the agent needs for the next step, and how to provide it without turning every task into an unbounded accumulation of text.

His discussion separates short-term context from longer-lived memory and describes specialized agents with focused responsibilities. A document validator, for example, can perform a defined check without inheriting the whole history of a broader business conversation.

This has a practical implication for debugging. If an agent gives the wrong answer, the team should be able to reconstruct which instructions, retrieved information, and remembered facts were available. Without that view, it is difficult to distinguish missing context from incorrect reasoning or an unsuitable tool response.

Turn production traces into evaluation cases

Shuji Oya, Solutions Architect at Databricks, connects development evaluation with production traces and feedback from domain experts. His loop moves from testing to deployment, observation, revised evaluation sets, and further improvement.

The value of a trace is not just that it records a sequence. It can preserve a failure in a form the team can reproduce and test again. A reviewer’s correction then becomes a candidate evaluation case with an expected result and relevant context.

Oya also discusses gateway-level governance, including access, rate limits, and usage tracking. These controls answer different questions from output evaluation. Knowing that a response is useful does not establish that the request was authorized; knowing that access was permitted does not establish that the answer was correct.

The maintained MLflow tracing documentation provides current implementation guidance. The enduring design principle is to join evidence about the agent’s behavior with evidence about the application’s operating conditions.

A persistent agent needs more than persistent text

Eduardo Gonzalez, Co-founder and CEO of Xpress AI, examines agents that continue working through events rather than depending entirely on a person sending another message. His talk includes hierarchical context and an order workflow connecting information retrieval with a business application.

Persisting a conversation is only part of that challenge. The system also needs a representation of pending work, completed actions, and the event that should trigger the next step. Otherwise, a long-lived agent can remain available without reliably advancing the task.

That distinction becomes visible when a workflow pauses. Waiting for approval, waiting for data, and recovering from a failed integration are different states. They should lead to different behavior and different information for the person responsible.

Foundry’s perspective: operate the whole purchase-request lifecycle

Consider an illustrative agent that prepares purchase requests from approved demand information. A PoC can retrieve data and fill a form. A continuing service must also handle an approval arriving tomorrow, a supplier record changing, or a failed submission that may have completed despite a timeout.

We would represent those conditions as explicit workflow states. A request has an owner, source data, approval status, and a record of any external action. The agent can assist within that structure, but a new conversation should not erase the state of the business transaction.

Production traces then have a clear purpose. If a duplicate request appears, the team can examine whether it came from a repeated trigger, a misunderstood tool response, or missing checks around a retry. The fix can become a focused regression case, rather than a broad instruction to “be more careful.”

A first release might draft requests and manage the review queue while people retain submission authority. The team can measure unresolved work, incorrect state transitions, duplicate actions, and the time reviewers spend correcting the output. Those measures complement model quality and infrastructure cost.

The next release should be judged against the same workflow evidence. A more capable model may reduce interpretation errors while introducing a different tool-use pattern. Keeping the operating record and evaluation set connected makes that change assessable.

For Foundry Labs’ production AI systems, this is the useful meaning of an improvement loop: each deployment teaches the team something specific, and the next deployment is tested against what it learned.

Sources and further reading

This article draws on the TAI AMA #13 - LLM Ops on Databricks programme and the talks identified above. Tokyo AI (TAI) connects researchers, engineers, and industry; explore Foundry Labs’ community work for more context.

Talk materials from Darren Loke, Shuji Oya, and Eduardo Gonzalez are available upon request. Please use our contact form and include the event title and speaker name.

Speaker profiles

Darren Loke. Technical Lead and Solution Architect at Money Forward. His talk introduces AgentForward. LinkedIn.

Shuji Oya. Solutions Architect at Databricks. His talk covers operations for agents and LLM applications. LinkedIn.

Eduardo Gonzalez. Co-founder and CEO of Xpress AI, working on persistent autonomous agents. LinkedIn.

All articles