Auriko

In a previous life, I traded options as a quant trader. When I started building with AI agents, I needed to switch models quickly across inference providers. A trader’s OCD for finding the lowest price kept pushing me to figure out which provider was cheapest. That sent us down the rabbit hole of comparing inference costs. We realized cost is not just the headline input/output token price. A huge part of our spend came from cache pricing, cache-hit efficiency, and routing choices. We ended up building a system to optimize all of that. And we turned it into auriko.ai.

Congrats! A trading desk for LLM calls is a framing I haven’t seen before and it clicks immediately, model costs do behave like a market. My question: when Auriko routes a call to a cheaper model to save money, how do I protect quality? Can I set a floor per task type? Saving 40% on inference means nothing if my customer-facing outputs get worse and I find out from a complaint.

<p>quant background makes sense for this, arbitrage is fundamentally about finding mispriced spreads and providers pricing caching differently is exactly that. the tension I'd want to understand: prompt caching usually rewards staying on the same provider for a session so the cache stays warm, but a router optimizing per-request could bounce a session across providers chasing the best price each time and never let any single cache warm up. does the routing engine account for cache-state as its own signal, like "this provider already has a warm cache for this context, don't move away from it even if a competitor is nominally cheaper this instant"</p>

<p>Love the “trading desk for LLM calls” framing. Cost optimization across providers is becoming a real pain point as AI apps scale.</p><p>How do you balance cost savings with output quality and latency, especially for production workloads?</p>