Beyond Model Accuracy: Three Layers of Reliable Enterprise AI

A convincing AI demonstration leaves several production questions unanswered. Who owns a harmful decision? Did an agent follow the required process? Will people understand when to rely on its output and when to intervene?

The talks at Tokyo AI's Engineering Reliable AI at Scale event offer three complementary ways to approach those questions: establish accountability, evaluate the complete workflow, and design for human behavior. Taken together, they suggest a practical way to examine readiness for enterprise deployment.

Event: 16 June 2026, Shibuya, Tokyo. Organized by Tokyo AI (TAI), with hosts Ilya Kulyatin and Andrew Sharifikia. The program brought together Alireza "Andrew" Sharifikia and Raphael Aubel from Corpy & Co., and Tomomi Tanaka, founder of Behavioral AI Lab. Event details and speaker profiles.

The production approach developed below is Foundry Labs' perspective on putting these ideas into practice.

1. Make governance an operating responsibility

In his talk From PoC to Production, Alireza "Andrew" Sharifikia treats AI quality as a broader problem than model performance. A successful test of individual software components does not establish that the deployed system will behave acceptably in the situations that matter to the business.

Sharifikia uses QA4AI's five quality axes to widen the review: data integrity, model robustness, system quality, process agility, and customer expectations. The QA4AI consortium's guidance provides the underlying framework. It invites teams to examine the data and operating environment, their ability to update the system, and the expectations of the people using it.

Sharifikia's proposed starting point is concrete: maintain an inventory of deployed AI systems, with an owner, purpose, and risk assessment for each. Match the level of oversight to the consequences of failure. His phased roadmap then moves from data readiness and a defined operating domain to integration testing and organizational governance.

For an enterprise team, our reading is that production readiness should have a named owner and a visible body of evidence. A release decision becomes easier to examine when the team can explain which conditions it has tested, which limitations remain, and who is responsible for responding when behavior changes.

2. Test what the agent did, including the steps between input and output

Raphael Aubel, an R&D Engineer at Corpy & Co., focuses on the execution path an agent takes. He describes this as a trajectory: the sequence of observations, decisions, tool calls, and state changes that leads to an outcome.

That distinction matters when an agent can act on business systems. A tool may execute successfully while the decision to call it is wrong. A final response may sound reasonable even though the agent skipped a required approval. Testing the output alone can therefore miss a failure in the process that produced it.

Aubel illustrates this with a local retail customer-service experiment covering 33 tasks, six models, and three repetitions, for 594 runs. He presents the work as an illustration, not an official benchmark. It should not be read as a general ranking of model providers.

One example involves a refund request from an account subject to a risk hold. The scenario's policy requires transfer to a human operator before money moves. In the failure shown, the agent initiates the refund without that transfer. The important question is whether the required sequence was respected, even if the refund operation itself could execute.

Aubel then extends the example to multiple agents: an incorrect action can change shared state, and another agent can use that state in a later decision. This is a reason to inspect the information and permissions passed between agents. It is not evidence that adding another agent automatically fixes the first one's mistake.

The practical implication is to define both acceptable outcomes and required steps. Repeat the scenarios, inspect the resulting state, and keep enough execution history to explain failures. Aubel's closing recommendations emphasize continued evaluation, safeguards, and monitoring in proportion to the risk.

3. Design for the people who will use the system

Tomomi Tanaka, founder of Behavioral AI Lab, adds a behavioral perspective in her talk Why AI Systems Fail After Deployment: The Missing Human Layer. Her framework asks teams to examine four dimensions before launch:

• Trust: Will people understand when the system deserves reliance and when it needs checking?
• Incentives: Do the rewards surrounding the workflow encourage the intended behavior?
• Cognitive load: Does the system make a decision easier, or add another burden?
• Adaptation: How might users and other actors change their behavior once the system is available?

Her safety-by-design approach starts with anticipating harm and designing safeguards, then continues through monitoring and adaptation after launch.

Applied to an enterprise workflow, these questions make human review a design problem in its own right. For example, our interpretation is that an escalation step should specify what the reviewer sees, what decision they are expected to make, and whether they have the time and authority to make it. Simply placing a person at the end of a workflow does not answer those questions.

Turn the three perspectives into a production review

At Foundry Labs, the approach we recommend starts with the business process the AI will join. What triggers the work? Who makes the decision? Which systems record it? What happens when something is missing or goes wrong? These questions turn a promising capability into a proposal that engineering and operations can evaluate together.

Consider an invoice-processing assistant. A proof of concept might extract a supplier name, amount, and purchase-order number accurately from a sample of documents. A live process also needs to establish whether the invoice has already been received, whether it matches the order, who can approve an exception, and how the result reaches the accounting system. We would define success around completing that process with an acceptable amount of review and rework. Extraction accuracy is one part of that assessment.

The next decision is how much authority to give the system. For an initial release, the assistant might prepare a draft accounting entry while a named reviewer retains approval. Missing purchase orders, inconsistent amounts, or changed supplier details would go to an exception queue. The permitted actions should be enforced by the surrounding software and access controls, so that a persuasive model response cannot grant itself permission to bypass the process. This is where governance becomes part of the implementation: a business rule maps to a control, a responsible person, and a test.

That test needs to follow the work across system boundaries. We would include an invoice submitted twice, an unreadable field, a late approval, and an accounting API that times out after receiving a request. In the last case, retrying immediately could create a duplicate entry. The test should establish whether the system checks what actually happened before acting again. The acceptance criteria therefore cover the final business record and the path taken to create it, including approvals and recovery from interrupted operations.

Before allowing the assistant to change live records, a useful next step is to run it alongside the existing process and compare its proposed actions with the team's decisions. Disagreements become material for review. Was the AI wrong, was the policy ambiguous, or did the operator know something absent from the system? Where approved real records are used, the pilot needs the same attention to access and data handling as the eventual service. This stage can reveal work that a demonstration conceals, especially the effort required to investigate exceptions.

Human review also needs its own design. An approver should see the original invoice, the matching order, the proposed entry, and the reason the case was escalated. If they must reopen several systems to reconstruct the context, the assistant may be moving work rather than reducing it. We would examine review time, corrections, repeated overrides, and unresolved cases alongside model quality. A high approval rate alone is ambiguous: it might reflect good recommendations, or reviewers clicking through a queue they cannot realistically inspect.

Expansion should follow evidence from that limited release. A team might first automate routine, well-understood cases while keeping unusual suppliers or disputed amounts under manual control. Each increase in autonomy needs a clear owner and an agreed response if the system starts behaving differently. Recovery must address business effects as well as software: reverting a model version does not undo an accounting entry already created. The team needs a way to identify affected records, pause further actions, and route corrections to someone authorized to make them.

This illustrative invoice workflow makes the connection between the three talks concrete: accountability defines who can act, trajectory testing checks how the action happens, and behavioral design determines whether people can oversee it effectively. Together, they give a team a stronger basis for deciding when a PoC is ready to become part of a live business process.

Foundry Labs designs, builds, deploys, and supports production AI systems. Through Tokyo AI, practitioners bring the engineering and organizational sides of deployment into the same conversation.

Request the talk materials

Presentation materials from the talks by Andrew Sharifikia, Raphael Aubel, and Tomomi Tanaka are available upon request. Please use our contact form and include Engineering Reliable AI at Scale, 16 June 2026, together with the speaker name or names.

Sources and further reading

• Tokyo AI event page and announced program.
• Andrew Sharifikia, From PoC to Production: A Practical Roadmap for Enterprise-grade AI Compliance in the Automotive Industry, Tokyo AI talk.
• Raphael Aubel, System Quality for Multi-Agent AI Systems, Tokyo AI talk.
• Tomomi Tanaka, Why AI Systems Fail After Deployment: The Missing Human Layer, Tokyo AI talk.
• QA4AI official guidelines and downloads.

About the speakers

Alireza "Andrew" Sharifikia

Software Engineering Manager, Corpy & Co. Andrew's background spans AI, computer vision, and data science. His work includes Corpy's XAI initiative and collaboration with the TAI community. His talk connects AI quality assurance with the organizational responsibilities of enterprise deployment.

Raphael Aubel

R&D Engineer, Corpy & Co. Raphael works on applied AI from technical exploration through production delivery. His experience includes logistics document automation, vehicle-routing optimization, and industrial inspection, with a focus on evaluation and operational reliability.

Tomomi Tanaka

Founder, Behavioral AI Lab. Tomomi is a behavioral scientist and AI strategist studying the relationship between AI, human behavior, and decision-making. She is also a Senior Behavioral Advisor at BEworks and an adjunct professor at Kindai University.

All articles