We Benchmarked Bonito Against AWS AgentCore. 387x Cheaper, Zero Hallucinations, and a Four-Cloud Swarm AWS Cannot Run.
We took the same agent swarm, built it on Amazon Bedrock AgentCore and on Bonito, and ran identical tests. Three rounds, one afternoon, every number published.
The short version: their default stack burned 16.4 million tokens and $11.87 answering 15 customer questions, hallucinated a refund policy that does not exist, and has no answer for a provider outage. Ours answered everything for 3 cents, invented nothing, failed over across providers live without dropping a request, and then ran one swarm across four clouds on production for 3.8 cents a request. We benchmarked ourselves against a hyperscaler and won on cost, reliability, honesty, failover, and observability.
Here is the whole thing, receipts included.
Round one: same swarm, both platforms
The use case is the most-deployed enterprise agent workload there is: customer support. Three agents for a car service shop. A front desk answering from a knowledge base, handing off to a service advisor or a billing desk. Ten questions the knowledge base can answer, five traps where the only honest answer is "I don't know."
On Bonito the whole swarm is 65 lines of yaml, deployed with one CLI command. On AgentCore we used Strands, AWS's own agent SDK, so nobody can say we handicapped their side: 87 lines of Python, a Dockerfile, an ECR repository, an ARM image build, an IAM execution role, and 115 lines of Terraform to be allowed to do any of it. AgentCore has no knowledge base primitive, so retrieval is yours to build. We hand-rolled it. We even hit their first bug for them: Bedrock's Llama rejects tool calling in streaming mode, and AWS's own SDK defaults to streaming, so the stock combination returns 500s until you find the flag.
The test: Bonito answered 15 of 15, invented zero facts, and handled all five traps honestly. AgentCore answered 13 of 15, told a customer "yes, we offer financing on repairs" (the shop does not), and quoted the wrong warranty figure with full confidence.
Then we looked at why two of their questions never came back.
16 million tokens, and nothing was watching
The two dead questions had triggered a handoff loop: front desk to billing, billing back, again and again. AWS's SDK with default settings has no effective brake, and every spin re-sends the entire growing transcript. One question racked up more than 70 model calls before our timeout killed it.
Our battery: 52 thousand tokens, three cents, itemized live on the gateway dashboard. Their identical battery: 16.4 million tokens and $11.87, discoverable only by querying CloudWatch afterward and doing the pricing math by hand. That is 387 times the cost for the same work. Not because AWS is expensive. Because nothing was watching, and nothing stopped it.
On Bonito the engine caps tool iterations, spend caps exist as org policy, and every request lands in the dashboard with its cost as it happens. The runaway class of failure is designed out at the platform layer. On AgentCore it is homework for whoever is on call.
Then we killed a provider on purpose
Mid-session we blackholed the swarm's primary inference provider entirely, then asked a pricing question. The gateway detected the failure, hopped to the org's second connected provider, served the correct answer on a different model family in 15.6 seconds, and logged the hop with reasons. The customer saw nothing. We healed the outage and traffic walked back on its own.
There is no AgentCore version of this test. The model id is baked into the container image. Bedrock offers cross-region profiles, and they are good, but when a provider or a model family has a bad day, cross-region is not the axis that saves you. Our customers' production traffic has already ridden out two full provider outages on exactly this mechanism.
Round two: the wheel
A fair objection: the swarm was small, and the loop is fixable. So we escalated. One orchestrator, five specialists with their own knowledge base domains, the textbook employee onboarding wheel. And to be sporting about it, this round we hand-fixed every AgentCore weakness we knew about before testing.
Getting their wheel to parity took five things built by hand, three container rebuilds, and $7.94 of burned tokens along the way: the streaming flag, a tool budget brake (nothing else stops a loop), fresh agent objects per request (the SDK's agents retain conversation history, so a warm shared container leaks one request's context into the next), retrieval injected into prompts (Bedrock's Llama read a tool result containing the verbatim answer and replied that it was not in the knowledge base), and whole-document retrieval because there is no knowledge base primitive at all.
Every one of those five is something Bonito just does. Our wheel deployed from one 87-line yaml with the knowledge bases and the hub-to-spoke connections wired automatically, and worked on the first run.
And because people ask, here is that file. Not a snippet, not pseudocode, the actual deployment artifact for the entire six-agent wheel, lightly trimmed to the orchestrator plus two of the five specialists (the other three are the same shape with their own knowledge base). This is the whole David side of the fight. The Goliath side is 208+ lines across Python, Terraform, and a Dockerfile, plus an ECR pipeline:
version: "1"
name: onboarding-wheel
description: 1 orchestrator + 5 specialists
knowledge_bases:
kb-hr:
description: HR policy
sources: [{path: kb_hr_policy.md}]
kb-it:
description: IT provisioning
sources: [{path: kb_it_provisioning.md}]
# ...kb-payroll, kb-facilities, kb-compliance: same shape
agents:
onboarding-orchestrator:
display_name: Onboarding Orchestrator
model_id: your-orchestrator-model
system_prompt: >
You coordinate five specialists: hr-policy, it-provisioning,
payroll-benefits, facilities, and compliance-training. For each
request, invoke the specialists whose domains are involved, then
synthesize one clear answer with exact figures and deadlines.
Only state facts a specialist returned. If no specialist has the
answer, say the information is not available.
model_config: {temperature: 0.2, max_tokens: 900}
tool_policy:
mode: allowlist
allowed: [search_knowledge_base, invoke_agent]
delegates: [hr-policy, it-provisioning, payroll-benefits,
facilities, compliance-training]
hr-policy:
display_name: HR Policy
model_id: your-specialist-model
system_prompt: >
You are the HR policy specialist. Answer using ONLY your knowledge
base, quote exact figures. If it is not in the knowledge base, say so.
model_config: {temperature: 0.1, max_tokens: 350}
tool_policy: {mode: allowlist, allowed: [search_knowledge_base]}
rag: {knowledge_base: kb-hr}
it-provisioning:
display_name: IT Provisioning
model_id: your-specialist-model
system_prompt: >
You are the IT provisioning specialist. Answer using ONLY your
knowledge base, quote exact models and SLAs. If it is not in the
knowledge base, say so.
model_config: {temperature: 0.1, max_tokens: 350}
tool_policy: {mode: allowlist, allowed: [search_knowledge_base]}
rag: {knowledge_base: kb-it}
# ...payroll-benefits, facilities, compliance-training: same shapeOne `bonito deploy -f bonito.yaml` and the knowledge bases are ingested, the agents exist, and the hub-to-spoke wiring is live. The `model_id` lines are the entire multi-cloud story: in round two those were an open-weights model on one provider, and in round three we pointed the orchestrator at Claude on AWS Bedrock, two specialists at Gemini, two at Groq, and one at Anthropic direct by changing those strings and nothing else.
After all that hand-tuning, their wheel did reach respectable numbers: 6 of 6 requests, 12 of 17 facts against our 13, faster warm latency, about four cents a battery. Which is exactly the point. Once an expert builds everything the platform does not give you, the economics converge. The difference is who builds it, what the mistakes cost on the way, and what you can see. Their observability was an integer we instrumented ourselves. Ours was a trace tree with every model call, tool call, and cost joined per run, out of the box.
Round two also caught a real bug on our side: the first true parallel fan-out exposed a session handling fault in our engine's concurrent delegation path. We fixed it the same hour and shipped it to production. Dogfooding cuts both ways, and we would rather find it in a lab than in your workload.
Round three: production, four providers, one swarm
One objection remained: our arm ran on a laptop. So we reran on production, and used the neutral layer for what it is.
First, latency, the one line AWS had won in round one at 2.7s median. Same knowledge base, same ten questions, one agent per provider through the production gateway: gemini-2.5-flash answered at 2.2s median with 10 of 10 correct. Groq served the single fastest answer of the entire experiment at 1.1 seconds. The latency gap was never infrastructure. It was a routing choice, and on a neutral layer, routing is configuration. AWS cannot make that move; their serving speed is whatever Bedrock gives them.
Then the showpiece: one wheel, cast deliberately across clouds. A Claude orchestrator on AWS Bedrock, HR and facilities specialists on Gemini, IT and compliance on Groq, payroll on Anthropic direct. Four providers, one swarm, one trace. It posted the best score of any arm in any round: 15 of 17 grounded facts, six of six requests, honest on the trap, and the full five-specialist onboarding plan traced end to end in 31.7 seconds for $0.038. No single-cloud platform can draw that picture, structurally, ever.
And one thing happened that nobody scripted. Mid-battery, a few Groq calls blipped, and the gateway quietly served those requests on identical open weights at another provider, at 728ms average, without a single error reaching the test. The failover slide demoed itself, on production, unprompted.
The scorecard
Round one, same swarm both sides: battery cost $0.03 vs $11.87. Tokens 52K vs 16.4M. Answered 15/15 vs 13/15. Invented facts 0 vs 2. Config 65 lines of yaml vs 208+ lines across three languages plus ECR and IAM. Cost visibility live per request vs next-day CloudWatch archaeology. Provider outage: failed over in 15.6s vs model pinned in the image.
Round two, the wheel: one deploy vs five hand-built fixes, three rebuilds, and $7.94 of failed iterations. Trace trees vs a hand-instrumented integer.
Round three, production: latency line flipped (2.2s beats their 2.7s), best accuracy of the whole experiment (15/17), a four-provider swarm at 3.8 cents a request, and failover that fired on its own.
Where they won, and we report it because we keep our receipts honest: warm single-hop latency in round one, zero-to-first-answer for a fluent AWS operator, and IAM-grade identity at enterprise scale. That is the whole list.
What we would tell that engineer
Use AgentCore if you are all-in on AWS and writing custom agent code you want hosted in managed microVMs. But the platform hands you meters, not answers. Retrieval is your problem. Loop control is your problem. Cost visibility is next-day archaeology across twelve separate billing meters. And the model layer is welded to one vendor in a year when model retirements and provider outages are monthly events.
We built Bonito so the platform does that work: bounded loops, live per-request cost, spend caps, trace trees, and failover that treats every provider, including AWS, as one of eight. Then we tested it against the biggest cloud on earth and published every number.
The fine print, because we read ours: single run per round, the arms ran different 70B-class open models after a catalog quirk broke our same-weights plan, and the runaway loop is fixable by an expert with explicit iteration caps. Our claim is about what the platforms do by default, because defaults are what runs at 2am.
Run the experiment yourself. Ours is 65 lines of yaml.
