AI Agent Infrastructure
The One does not simply use AI Agents in production. Before using AI agents in production, we need to understand that LLMs are token prediction machines and by nature are non-deterministic . No matter how good you specs are, AI will drop packages and make mistakes. Lack of determinism is just one aspect we need to keep in mind. We also need to keep in mind that it's very easy to jailbreak the models . Adding a chatbot directly to customers has dangers and not only in a security sense, but also for misuse and potentially legal problems. Even if that is all somehow managed and risk is minimized with proper guarantees, one still does not just use agents in production. 20-15 years ago, we would not just deploy APIs to production; we would use an API Gateway. Considering agents and LLMs, we need the same: an AI gateway infrastructure. What happens if your API provider (Anthropic, Google, or OpenAI, for instance) is down? Is your business down?