semantic-router
System-level intelligent routing for optimizing mixed multi-model deployment.
- Type
- MCP
- Transport
- http
- Open source
- Yes
- GitHub Stars
- ★ 4.5k
- Source
- mcp-github
- Repository
- github.com/vllm-project/semantic-router
Overview
vLLM Semantic Router is a system-level intelligent routing tool designed to optimize mixed multi-model deployments in cloud, data center, and edge environments. It enables teams to build more efficient, secure, and adaptive model systems through signal-driven approaches. The tool reduces wasted tokens, increases effective output, and maximizes the value of every token. Additionally, it detects jailbreak attempts, sensitive information leaks, and hallucinations, ensuring controllability, trustworthiness, and auditability of agents. Ideal for teams coordinating local, private, and cutting-edge models across different environments.
Capabilities
- ▪Reduce wasted tokens
- ▪Increase effective output
- ▪Maximize token value
- ▪Detect jailbreaks and sensitive information leaks
- ▪Coordinate mixed multi-model deployment
Use cases
Setup
curl -fsSL https://vllm-semantic-router.com/install.sh | bash
This information was compiled by AI from public sources and may contain inaccuracies — please refer to the source.
FAQ
How do I get started with vLLM Semantic Router?
Run the installation script: curl -fsSL https://vllm-semantic-router.com/install.sh | bash
What are the main features of vLLM Semantic Router?
Optimize mixed multi-model deployment to improve efficiency, security, and adaptability.
Related skills
gemini-cli
Gemini CLI is an open-source AI agent that brings Gemini's powerful capabilities directly into the terminal.
sast-skills
Transform your AI coding assistant into a SAST scanner to automatically detect vulnerabilities in code.
recon-skills
Security testing toolkit covering vulnerability detection for CORS, XSS, SQLi, and more.
scientific-agent-skills
Transform any AI agent into a scientific assistant with 147 ready-to-use research skills.
susi_alexa_skill
A skill that enables question-and-answer interactions between Alexa and Susi AI.
claude-context
Provides code search capabilities for Claude Code, turning the entire codebase into context.