An AI gateway is a proxy between agents and the services they call, so every request routed through it passes one chokepoint. An LLM gateway fronts model providers. An MCP gateway fronts tool servers. An API gateway fronts internal and external APIs.
A chokepoint is a good place for rules that apply to every request the same way:
- Authentication and authorization, so only known clients reach a service
- Rate limits and quotas, so one runaway agent can't burn through a budget
- Central policy, written once and applied to every call that passes
- Logging, so you keep one record of who called what
Picture the security desk in an office lobby. It checks badges, counts visitors and keeps the sign-in sheet. Any team running agents at scale has good reasons to want one.