The agent's logs show a clean success. The tool was called. A response came back. The agent reported the action complete to the user. The traces look healthy. Nothing alerted.
Three days later, a customer escalation reveals that the action wasn't actually completed — or it was completed against the wrong record, or with the wrong amount, or in the wrong environment, or against a deprecated endpoint that no longer does what the schema implied.
This is the failure mode that makes tool calling structurally different from regular software bugs. In normal code, a function that returns the wrong thing gets caught by tests, type systems, or runtime errors. In agent systems, a tool that returns the wrong thing is often interpreted by the LLM as a successful action, and the agent moves on. The wrongness propagates to users, to downstream systems, to dashboards — until something noticeably breaks.
The root cause is almost always the same: the schema the agent uses to reason about a tool has drifted from what the tool actually does.
Why Tool Calling Fails Quietly Instead of Loudly
When you give an agent a tool with a JSON schema, you've told it three things: what the tool does (the description), what arguments it takes (the parameters), and what to expect back (often only implicitly). The agent reasons about all of this from the schema alone. It does not see the actual implementation. It does not know whether the API behind the schema has changed since the schema was written.
When the schema is correct, this works. When the schema is wrong, the agent confidently calls a tool that does something other than what the schema described — and integrates the result into its reasoning as if it were correct.
Foldington case: A customer support agent had access to an update_customer_record tool. The system had two customer ID formats — a legacy numeric ID and a newer UUID-based ID. Both were technically accepted by the endpoint, but only the UUID-based ID actually wrote to the production customer record. The schema described customer_id as "the unique identifier for the customer" without specifying format.
Over three weeks, roughly 1,400 customer record updates were written to a deprecated table that nothing read from anymore. The cleanup involved data reconciliation, customer apologies, and a conversation about why nobody had caught it.
The Seven Structural Drift Patterns
Drift 1: The "Succeeded" Hallucination
The schema is technically accurate. The tool runs as described. But the response conflates two different states: "the call succeeded" and "the action succeeded."
Common shapes:
- An API returns
{"status": "accepted"}for an async job that may later fail; the agent treats "accepted" as "completed" - A booking API returns
{"reservation_id": "abc123"}even when provisional; the agent treats provisional as confirmed - An update API returns
{"affected_rows": 0}when the record didn't exist; the agent treats this as success because the call didn't error
Wayloft case: A travel SaaS agent booked hotels through partner APIs. A reservation_id was issued at the provisional stage, before the hotel confirmed availability. Final confirmation happened asynchronously, sometimes hours later. About 3% of provisional reservations were later rejected — dozens of stranded customers per week showing up at hotels with no reservation.
Drift 2: Schema Drift — The Documentation That Lies Slowly
Schemas drift from reality the way any documentation drifts from code: written once, updated less often than the thing they describe.
Crestmark case: An agent had been updating CRM records for fourteen months. A routine audit revealed that the schema referenced a last_contact_date field that had been renamed to last_engagement_date ten months earlier. The CRM API silently accepted writes to the old field name and discarded them. The agent had been "updating" last_contact_date for ten months — none of those updates had actually been written to the CRM.
Patterns that work:
- Schemas auto-generated from the live tool, not hand-written and maintained separately
- Continuous schema validation in CI: a smoke test that calls each tool with schema-valid inputs and checks the response
- Versioned schemas with explicit tool version pins
Drift 3: Type Coercion
The schema says amount is a number. The agent generates a number. The tool accepts the number. Internally, the tool converts the float to a fixed-precision decimal. For certain values that can't be exactly represented in IEEE 754 float, the conversion produces rounding errors.
Reachfield case: A B2B payments platform accumulated invoice aggregate discrepancies measured in fractions of a cent, adding up to several hundred dollars over thousands of transactions. The fix: amount was redefined as a string with a specific decimal format. Agents now generated "1234.56" rather than 1234.56. Precision was preserved end-to-end.
This wasn't a code bug. It was a schema that didn't capture the precision requirements of the underlying domain.
Drift 4: Enum and Identifier Drift
Kelmore case: A DevOps agent deployed services using an environment enum with values "dev", "staging", "prod". Six months earlier, a "prod-canary" environment had been added for staged rollouts. Deployments to "prod" now bypassed the canary stage entirely. The agent, working from the schema, deployed to "prod" for any production request — skipping the canary stage that would have caught regressions.
Drift 5: Optional/Required Ambiguity
A currency field marked as optional defaults to USD. An agent serving European vendors omits the field (the schema says it's optional) and the system applies the USD default.
Greenmarsh case: A procurement platform built an agent that placed purchase orders. For European vendors, orders were placed in USD against vendors expecting EUR. Some vendors accepted the USD-quoted amount at a loss; others rejected and required reissuance. The fix: make currency effectively required at the agent layer with explicit logic to derive currency from vendor region.
Drift 6: Error Responses That Don't Match the Schema
Most schemas describe the happy path in detail and error responses in passing. The agent improvises — and improvisation in error handling produces some of the most surprising failure modes.
Patrolskip case: A shipping agent's schema described error responses as "appropriate HTTP error code with description on failure." The agent's prompt told it to "retry transient failures." The agent didn't distinguish 4xx from 5xx and retried both. A 400 for invalid postal code format — a permanent validation failure — was retried 5 times per affected shipment. The carrier flagged the account for unusual API patterns and rate-limited it.
Drift 7: Semantic Failures — Time, Money, Units, Identity
The schema is technically accurate, the tool behaves as described, but the result is wrong because the agent and the tool disagree about what something means.
Frowse case: A calendar agent generated 2025-09-15T15:00:00 for "book at 3 PM tomorrow." The tool interpreted this as UTC. For a user in EST, the meeting appeared at 11 AM their time. Customer escalations followed. The fix: timezone-required or explicit-default in the schema contract.
Multi-Tool Compounding
When agents orchestrate sequences of tool calls with outputs from one feeding into the next, schema mismatches compound. A customer_id from Tool A might use a different namespace than customer_id in Tool B. The agent passes the value confidently. Both tools accept it. Both return successful responses. The downstream behavior is wrong.
Detection and Mitigation
Schema-driven tool tests: Every tool has automated tests that exercise the schema against the live tool on a nightly basis — catching drift within 24 hours of when it happens.
Outcome verification, not response verification: For tools that perform actions, verify the outcome through an independent observation, not through the tool's response.
Schema versioning with hard pinning: Agents reference specific versions of tool schemas. Mismatches fail loudly rather than silently.
Cross-tool semantic tagging: Fields in schemas carry semantic tags checked at agent orchestration time. A customer_id from the CRM is tagged crm:customer_id; the billing tool's customer_id is tagged billing:customer_id. The agent layer enforces conversion at boundaries.
Centralized tool registry: A single source of truth for tool schemas, with ownership tied to the team that owns the underlying tool. Changes to the underlying tool require schema updates in the registry as part of the deploy.
Tool calling is the part of agent systems where the abstraction is leakiest and the failure modes are quietest. Treating it with the same rigor you'd treat a database schema or a wire protocol — because that's what it is — is what separates production agent systems that survive from ones that produce apology emails.
Meritshot's Data Science and Full Stack programs include schema-discipline for tool calling — outcome verification, version pinning, cross-tool semantic tagging — built into production agent projects where failures have real consequences.
