82 lines
3.3 KiB
Markdown
82 lines
3.3 KiB
Markdown
# Custom OpenAI-Compatible Providers in Hermes
|
|
|
|
When a provider isn't built-in but offers an OpenAI-compatible `/v1` endpoint, add it manually to `config.yaml`.
|
|
|
|
## Steps
|
|
|
|
1. **Find the base URL and API key env var** — usually `/v1` at the provider's domain.
|
|
|
|
2. **List available models:**
|
|
```bash
|
|
curl -s "https://<provider>/v1/models" \
|
|
-H "Authorization: Bearer $API_KEY" \
|
|
--max-time 15 | python3 -m json.tool
|
|
```
|
|
Note model `id`, `input_modalities`, `context_length`, `supported_features`.
|
|
|
|
3. **Add provider to config.yaml via Python** (don't hand-edit YAML — indentation errors break everything):
|
|
```python
|
|
import yaml, json, os
|
|
|
|
config_path = os.path.expanduser("~/.hermes/config.yaml")
|
|
with open(config_path) as f:
|
|
config = yaml.safe_load(f)
|
|
|
|
# Read API key from .env
|
|
api_key = ""
|
|
with open(os.path.expanduser("~/.hermes/.env")) as f:
|
|
for line in f:
|
|
if line.startswith("YOUR_KEY_PREFIX="):
|
|
api_key = line.strip().split("=", 1)[1]
|
|
break
|
|
|
|
config.setdefault("providers", {})["your-provider"] = {
|
|
"api_key": api_key,
|
|
"base_url": "https://provider.example.com/v1",
|
|
"available_models_json": json.dumps([
|
|
{"id": "model-id", "name": "Display Name"},
|
|
]),
|
|
"model": "default-model-id",
|
|
"model_display_name": "Default Display Name"
|
|
}
|
|
|
|
with open(config_path, "w") as f:
|
|
yaml.dump(config, f, default_flow_style=False, allow_unicode=True, sort_keys=False)
|
|
```
|
|
|
|
4. **Verify with a test call:**
|
|
```bash
|
|
curl -s "https://provider.example.com/v1/chat/completions" \
|
|
-H "Authorization: Bearer $API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"model-id","messages":[{"role":"user","content":"hi"}],"max_tokens":100}'
|
|
```
|
|
|
|
5. **Use the model:** `hermes -m your-provider/model-id` or via `hermes model` picker.
|
|
|
|
## Provider Config Fields
|
|
|
|
| Field | Required | Notes |
|
|
|-------|----------|-------|
|
|
| `api_key` | Yes | Actual key value, not env var reference |
|
|
| `base_url` | Yes | Must end with `/v1` (or `/v1/`) |
|
|
| `available_models_json` | Yes | JSON string of `[{id, name}]` array |
|
|
| `model` | No | Default model ID |
|
|
| `model_display_name` | No | Human-readable default model name |
|
|
| `api_mode` | No | Only set if non-standard (e.g. `anthropic-messages` for MiniMax). Omit for OpenAI-compatible. |
|
|
| `protocol` | No | Usually leave as `''` |
|
|
|
|
## Pitfalls
|
|
|
|
- **Don't hand-edit YAML** — use Python `yaml.safe_load` + `yaml.dump` to avoid indentation corruption.
|
|
- **`api_key` must be the actual value**, not `$ENV_VAR` — Hermes doesn't resolve env vars inside provider config (only in `.env`).
|
|
- **No `api_mode` needed for OpenAI-compatible** — only set this for providers with custom protocols (Anthropic Messages, etc.).
|
|
- **`reasoning` field in responses** — some providers (SenseNova, DeepSeek) return a `reasoning` field in the message object. Hermes handles this natively for reasoning-capable models.
|
|
- **Model discovery** — always call `/v1/models` first; don't guess model IDs from documentation (they change).
|
|
|
|
## Known Custom Providers
|
|
|
|
| Provider | Base URL | Key Env Var | Models |
|
|
|----------|----------|-------------|--------|
|
|
| SenseNova | `https://token.sensenova.cn/v1` | `SN_API_KEY` | sensenova-6.7-flash-lite, deepseek-v4-flash, sensenova-u1-fast |
|