Higress
An open-source AI gateway providing production-grade capabilities for Agent development and LLM API management, unifying access to models, tools, and MCP servers.
- Pricing:
- Free
Overview
Higress.AI is a cloud-native open-source AI gateway and a CNCF Sandbox project, dedicated to production-grade capabilities for Agent development, LLM API management, and MCP tool invocation. It offers a free open-source version and a cloud Serverless commercial edition, unifying model APIs, tools, and MCP servers through one entry to help enterprises build secure, controllable, observable AI infrastructure.
As an AI gateway, Higress.AI unifies governance of three AI traffic types: model APIs, MCP tools, and Kubernetes inference workloads. Its core modules include the LLM Gateway (model API governance) for unified protocol conversion and token governance; the MCP Gateway (tool-call governance) for MCP proxy and protocol conversion; and the Model Gateway (inference workload governance) supporting Gateway API Inference Extension, InferencePool, and Endpoint Picker. It also includes built-in security, traffic governance, and observability for stable production AI apps.
At the feature level, Higress.AI supports unified protocol access and model-level fallback for 100+ major LLMs for flexible backend scheduling. It offers application protection, with privacy-preserving input and content-filtering output. It supports exact and semantic caching, caching similar or repeated inference results to save tokens and cut latency. It also provides token governance, tracking consumer quotas and limiting overuse, with data for token usage analytics.
Higress.AI serves scenarios including digital workers, financial-data MCP services, and enterprise AI gateway upgrades, validated by leading companies. Developers can start the local open-source version with one Docker command and validate core capabilities in 5 minutes; enterprises can choose the Alibaba Cloud AI Gateway Serverless edition, scaling elastically by call volume with fully managed, ops-free operation.
Core capabilities
- Open source and CNCF Sandbox
- a CNCF Sandbox project with free open-source and cloud commercial editions and an active community.
- Multi-model unified proxy
- unified protocol conversion for 100+ LLMs with model-level fallback for flexible scheduling.
- MCP gateway
- unified MCP tool-call governance with proxy and protocol conversion, exposing services as MCP servers quickly.
- Inference workload governance
- Gateway API Inference Extension and InferencePool to govern Kubernetes inference workloads at the gateway.
- Application security
- privacy-preserving input and filtered output for safe LLM interactions.
- AI cache
- exact and semantic cache reuse of inference contexts to save tokens and reduce latency.
- Token governance
- track token usage, manage consumer quotas and overuse limits, and feed usage analytics.
- Observability and traffic governance
- production-grade stability and ops support for AI apps.
Use cases
- Unified multi-model access and protocol conversion to reduce scheduling complexity
- MCP-based tool invocation and orchestration to build Agent apps fast
- AI app data protection and output content filtering
- Inference result caching to cut token cost and latency
- Token usage tracking and per-user quota governance
- Unified traffic governance for Kubernetes inference workloads
- Quickly turn professional APIs such as financial data into MCP services
Who it's for
AI app and Agent developers, LLM platform architects, Enterprise platform engineering teams, MCP server providers and tool developers, Cloud-native infrastructure teams, Technical decision-makers and architecture leads
Traffic analytics
higress.aiRelated tools
Daidaishou HubPaidDDS Hub: a developer-focused AI API gateway with low-latency Hong Kong relay to Anthropic and OpenAI
ProlificFreeProlific enables researchers and AI developers to collect high-quality, representative human data from verified participants in minutes, supporting preference tuning, safety evals, benchmarks, and academic studies with robust participant management.
DeepTranslatePaidA free AI-powered browser extension for instant bilingual translation of web pages, articles, and social media posts in 140+ languages.
CodeWhispererFreeA generative AI–powered coding assistant that accelerates software development across the entire lifecycle with real-time suggestions, agentic automation, security scanning, and deep AWS expertise, now delivered as Amazon Q Developer.