flo2

flo2 sits between your application and the model providers you already use, exposing all of them through a single OpenAI- and Anthropic-compatible API key. It's a gateway, router and proxy: you attach your own provider keys, flo2 adds nothing to their rates, and each request is forwarded straight to the provider you chose.

From that one key you can reach models across OpenAI, Anthropic, Groq, Cerebras, DeepInfra and Gemini. Smart routing decides where each call goes — a pinned default, a restricted shortlist, or your whole set of models. Fallback chains hold an ordered, reorderable list, so a request that keeps failing on one model rolls over to the next automatically. Racing sends a call to several models at once and returns whichever replies first, and A/B testing runs a new model in shadow against live traffic while a judge model scores which answer is better.

Every call is logged with its token counts, throughput and computed cost, broken down by model and provider, and optional response caching returns repeat calls under a TTL you set. By default flo2 keeps only metadata — prompts and responses aren't stored unless you turn that on. It's aimed at teams running LLM features in production, especially high-volume, infrastructure-heavy workloads where cost, reliability and data handling matter. flo2 is free during its Beta.