Open-source AI-native API gateway for routing, protecting and caching LLM/agent traffic, with a paid managed cloud.
What it does
Higress is a cloud-native, AI-native API gateway for managing LLM APIs and AI agents. It provides a unified proxy across 100+ models with protocol conversion and model-level fallback, plus AI-specific features such as semantic caching, token quota control and content safety. It is offered as free open source (a CNCF sandbox project) and as a fully managed commercial cloud edition.
How to use: Higress can be used to develop AI Agents and manage LLM APIs. It provides features like multi-model switching, content security, semantic caching, API key balancing, token management, traffic graying, and cost auditing. It also supports converting existing RESTful APIs to MCP Servers.