LLM Router
On this page
Purpose
LLM Router asks a Judge LLM to choose one connected language model for an input, then returns that model's response and routing details.
When to use it
- Choose between fast and higher-quality models.
- Route cost-sensitive work to a smaller candidate model.
- Compare a small set of models with distinct strengths.
Required setup
Connect one or more candidate language models, a separate Judge LLM, and the input message the router should evaluate.
How to use it
- Connect the candidate language models.
- Connect a Judge LLM.
- Provide the input message.
- Select an optimization preference.
- Connect Output first so routing runs, then use the diagnostic outputs when you need selection details.
Configuration
| Setting | What it controls | Recommended starting point |
|---|---|---|
| Optimization | Tells the judge to favor balanced, quality, speed, or cost. The default is balanced. | Start with balanced. |
| Use OpenRouter Specs | Adds available model details to the judge input and Selected Model Info. It is enabled by default. If lookup or name matching fails, routing continues with model names only. | Leave it enabled unless OpenRouter access is restricted. |
| API Timeout | Limits only the OpenRouter model-catalog request. It does not limit Judge LLM or candidate calls. The default is 10 seconds. | Keep the default unless catalog lookups time out. |
| Fallback to First Model | Calls the first connected model after supported routing errors. It is enabled by default. Malformed or out-of-range judge output still selects the first model when this is off. | Put the intended safety fallback first. |
Expected result
The router returns one non-streaming Message from the selected candidate. A normal run makes two model calls: the judge and the selected candidate. The judge receives up to 1,000 input characters, while the selected candidate receives the full input. If both use Arkus, both calls can deduct credits.
Run Output before reading Selected Model Info or Routing Decision. If either diagnostic runs first, it returns a placeholder. Selected Model Info contains detailed OpenRouter fields only when metadata lookup and model-name matching succeed; otherwise it contains basic name data. Routing Decision is a generated summary, not the judge's reasoning, and may contain technical failure details; do not expose it directly to end users.
Reference details
Keep the candidate set small and meaningfully different. Candidate order matters because the first model handles malformed or out-of-range judge output. Fallback covers only supported routing errors, and there is no second fallback if the first model fails.
Save credits with your own key
LLM Router has no credential field. Configure a saved OpenAI, Anthropic, or Google Credential on each connected Language Model that should use your provider account. Those calls bypass Arkus model credits and may incur provider charges. A deployed flow requires a saved owner Credential. Credential changes apply to subsequent runs; deleting a required Credential prevents the flow from running.
Works well with
Connect Language Model outputs to the candidate list and Judge LLM inputs. Use Chat Input or Prompt Template for the routed text, then send Output to Chat Output. Provider and credential settings remain on the connected Language Model nodes.