12. Advanced (1): LLM Providers & Models

Choose an access route first, then a model that route actually provides. A vendor's API, account login, Coding Plan and aggregator routes are separate configurations. Their model IDs, credentials, quotas and context windows need not match.

Model lists and available controls depend on the current settings, model picker and actual availability.

Screenshots in this chapter show the native App's Chinese interface with the default light theme, including navigation, the provider list and the configuration area. Click an image to view it at full size.

What you want Where to start
Try a model without a third-party key Log in to omicOS and select the built-in free cloud model
Use your API or Coding Plan Select the matching route in Settings · LLM providers and save its key
Use a third-party account login Complete authorization on the matching OAuth card and test it
Automatically switch between free quotas Configure your own keys, then select Free models · Auto
Connect a private gateway or local server Add a custom provider with its endpoint, API format and model IDs

For a headless deployment, use the environment-variable examples below.

12.1 Provider Selection Priority

For a regular chat request, Core resolves the provider in this order:

  1. The provider explicitly specified in the chat request (the current conversation's choice).
  2. OMICOS_LLM_PROVIDER.
  3. OMICOS_PROVIDER (legacy alias).
  4. The prefix in provider/model, or a model name starting with deepseek.
  5. An available key, in this order: DEEPSEEK_API_KEY, MINIMAX_API_KEY, OPENAI_API_KEY.

No match returns no model provider configured. The mock provider is disabled and cannot substitute for a real connection.

The UI's default selection is not the same as low-level inference. The workbench can offer omicos-cloud for a fresh selection; direct API calls and headless deployments should explicitly specify provider and model. When omicos-free is selected, Core first resolves an actual route from the configured pool members. That virtual card is not a model endpoint.

12.2 Model Selection Priority

  1. The model explicitly specified in the chat request.
  2. OMICOS_LLM_MODEL.
  3. OMICOS_MODEL (legacy alias).
  4. A runtime default.

The regular fallback is deepseek-v4-flash for DeepSeek and gpt-4o-mini otherwise. The built-in cloud and free pool have their own defaults or routing rules. A generic fallback is not guaranteed to work for every provider; select a model from that route's current list.

A matching routing prefix, such as deepseek/ in deepseek/deepseek-v4-flash, can be stripped. Aggregator IDs such as openai/... or deepseek/..., and NVIDIA namespaces, may instead be part of the full wire ID. Do not manually remove everything before the slash.

For routes marked as retired, current clients hide or block those choices and Core rejects the corresponding requests. The same model name on another route has an independent lifecycle. You can still read an old conversation; select an available model before sending again. If a catalog entry requires a newer Core, update the Core that owns the conversation, not just the web UI.

12.3 Supported Providers

Route Examples Configuration
Built-in free cloud omicos-cloud Requires an omicOS login, not a third-party API key
Free-quota auto-switch pool omicos-free Routes only across configured, enabled members; excludes the built-in cloud
API / Coding Plan DeepSeek, Qwen, OpenAI, Moonshot/Kimi, MiniMax, Coding Plan routes and more Use the key issued for that exact route and region
Anthropic anthropic Anthropic Messages, with the matching API key or connected OAuth
Third-party account OAuth Codex, Antigravity, Anthropic, xAI Grok Authorize on the corresponding card; that route determines access and quota
Custom provider Add provider in settings Specify the endpoint, API format, full model ID and context window

Google: Antigravity and Enterprise Gemini CLI

Personal Google accounts use Antigravity OAuth in the current settings. Existing enterprise Gemini CLI / Cloud Code Assist credentials retain a separate import and refresh path; their status displays Gemini CLI (enterprise). Do not confuse the two login flows.

  1. Open the Antigravity OAuth card in settings and start login.
  2. Authorize in the browser. If prompted for a code, paste the authorization code from the callback page into omicOS. It is a credential: do not include it in chats, screenshots or support reports.
  3. Check Token, Project and Runtime. Missing project / Project required means execution is not ready; the presence of a token alone is insufficient.
  4. Test the connection, then send a short message in the actual conversation.

If compatible CLI credentials already exist on this machine, use the card's import option. The internal provider ID remains gemini-cli for compatibility. Seeing that ID in a saved configuration or API response does not mean the personal Gemini CLI flow is still in use, and does not require manual renaming.

OAuth "connected" describes authorization state. Account access, quota, region or network restrictions may still block a model request. API and OAuth routes may also use different model IDs.

Two Different Free Routes

  • Built-in free cloud uses your omicOS login and a shared upstream allowance. Its daily allowance resets on UTC, and peak traffic can cause rate limits or temporary unavailability. When the UI reports that it is unavailable, retry later or select an already-configured API / OAuth route.
  • Free models · Auto uses your own vendor accounts and keys. Save each key, enable "Include in auto-switch", check pool readiness, then select the auto-switch route in the conversation.
  • A provider shown in both its regular group and "Free models" shares the same provider and key; the latter is a filtered model view. Domestic and international regional routes may instead require different accounts and keys.

Auto-switch tries another enabled member only before the first token and on a hard quota-exhaustion error. It does not hand arbitrary network failures or partially generated replies to another vendor, and it never changes a provider that you explicitly pinned.

App settings: free-pool candidates, selection and readiness

Check both selection and readiness in the pool manager. "需要 Key" means that member still needs credentials. Models, counts and the current route shown here describe this client's state, not a fixed configuration or a guarantee of free quota. This is an older-version illustration: Hugging Face is no longer a free auto-switch pool member, and free accounts have no included monthly credits, so you must purchase credits or use a subscription that includes them (see official pricing).

12.4 Where the API Key Comes From

In the App's Settings · LLM providers, choose an API route, enter its key, select a model, then save and test:

App settings: OpenAI key, model and save/test controls

sk-proj-... is an empty input's placeholder, not a real key. Control availability depends on the current configuration; the model shown is an interface example, not a successful test result.

API keys for regular catalog providers and legacy custom_* templates are resolved in this order:

  1. Saved auth.json values: check $OMICOS_LOCAL_HOME/auth.json, $OMICOS_RUNTIME_HOME/auth.json, ~/.omicos/auth.json, then legacy <current directory>/.omicos/auth.json; use the first file containing the required non-empty field.
  2. The corresponding environment variable. The catalog can specify its name; otherwise it normally uses the uppercase provider ID with hyphens replaced by underscores, such as ALIBABA_CODING_PLAN_API_KEY.

Saved settings now take precedence over stale process environment values. Exporting a key does not replace an existing saved value. Update it in settings, or explicitly clear the old saved value before using the environment fallback. OAuth credentials and the built-in cloud login token use different resolution paths.

The built-in anthropic route is an exception: connected Anthropic OAuth takes precedence; without it, ANTHROPIC_API_KEY from the environment is checked before a saved key. Do not apply that route's precedence to custom_anthropic.

Ollama uses the local placeholder ollama when no key is configured. Keys from the custom-provider manager use separate storage fields; do not derive an environment-variable name from the display name. Never commit auth.json or print keys as a diagnostic step.

Headless example, assuming this provider has no saved key:

export DEEPSEEK_API_KEY=sk-your-key
export OMICOS_LLM_PROVIDER=deepseek
export OMICOS_LLM_MODEL=deepseek-v4-flash
omicos serve --no-browser

Leave the service running, then connect the workbench or use the CLI from another terminal to send a short message. A listening port does not validate the key or model; actual request failures appear in the conversation.

12.5 Custom Endpoints

A regular catalog provider uses <PROVIDER>_API_BASE first, then the catalog's api_base. A missing endpoint is an error, not an invitation to silently use another vendor.

For new configurations, use Add provider in settings. Enter a name, endpoint, key and model IDs, and select the server's actual API format.

API Format Interface
OpenAI · Chat Completions /chat/completions
OpenAI · Responses /responses
Anthropic · Messages /messages

App Add provider: three supported API formats

The API format menu is expanded for an existing custom provider; personal configuration fields are masked. The options show the actual interface, not proof of a successful test. Check the service's supported protocol before filling in your own configuration. "OpenAI-compatible" alone is not enough to pick a format.

Enter the service's API root, such as https://api.example.com/v1, rather than a full /chat/completions, /responses or /messages request URL. "OpenAI-compatible" does not guarantee support for both Chat Completions and Responses. Save, test, and verify a conversation using that configuration.

Legacy custom_openai / custom_anthropic templates remain supported, but are not the same IDs as newly added providers:

  • custom_openai uses Chat Completions. CUSTOM_OPENAI_API_BASE defaults to http://127.0.0.1:8000/v1; its key can fall back to OPENAI_API_KEY.
  • custom_anthropic uses Messages. CUSTOM_ANTHROPIC_API_BASE is required; its key can fall back to ANTHROPIC_API_KEY.
  • Both templates resolve their endpoint and key from saved values, then environment variables. For a Responses-only gateway, use the custom-provider manager and select that API format explicitly.

Example for a local Chat Completions service; replace the model ID with one it actually serves:

export OMICOS_LLM_PROVIDER=custom_openai
export CUSTOM_OPENAI_API_BASE=http://127.0.0.1:8000/v1
export CUSTOM_OPENAI_API_KEY=dummy  # only for local services that do not validate keys
export OMICOS_LLM_MODEL=your-served-model-id
omicos serve --no-browser

12.6 The Cloud Model Catalog

Core exposes the built-in catalog through /api/models to settings and the picker. The local cache defaults to ~/.omicos/cloud-models/models.json. Check capabilities for the exact provider and model; a context window reported by one route is not automatically valid for another.

Variable Purpose
OMICOS_MODELS_OFFLINE=1 Disable cloud catalog sync and use the local cache
OMICOS_MODELS_CLOUD_URL Override the catalog fetch address
OMICOS_MODELS_CACHE_DIR Override the cache directory

For providers supporting online listings, "Fetch all models" in settings only discovers the list, still subject to credentials, route and availability filtering; "Fetch and validate available models" sends a short dialogue request for each model and may consume that route's quota, while the free pool's "Refresh" also updates validation results for enabled, configured members. Catalog visibility, a passing connection test and a working conversation are separate checks; passing validation only shows that this account can complete basic dialogue now, not full tool or vision support or a guarantee of permanent free access.

List provider IDs returned by the connected local Core without reading keys; change the port if needed:

curl -fsS http://127.0.0.1:5055/api/models \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); print("\n".join(p["id"] for p in d.get("providers", [])))'

Do not rely on a fixed provider count or ordering. Offline caches can be stale. If refreshing still leaves new controls unavailable, check the version of the Core that owns the conversation.

12.7 Vision Models

Settings · Vision can follow the main model or use a separate image-capable model. Built-in API choices are filtered by vision capability and lifecycle; OAuth and custom routes still need an image test to confirm support. Appearing in the list alone is not proof. A strong reasoning model does not necessarily accept images, and "let the model see the image" cannot add a capability it lacks.

Headless deployments can use OMICOS_VISION_MODEL, OMICOS_VISION_BASE_URL and OMICOS_VISION_API_KEY; see the environment-variable reference. Test with a small non-sensitive image. A successful text request does not validate the vision path.

12.8 Reasoning Levels and Context Windows

The picker displays reasoning controls from the current provider + model capability configuration. Reasoning may be an on/off toggle or several effort levels. Models marked as always-reasoning have no off option; models with no configurable capability show no corresponding control. Automatic detection for custom models is only a candidate hint; verify the server's protocol and test it. Do not copy low, high or none from another model's configuration.

Processing speed is separate from reasoning effort. It is selectable only when that model provides the control; "fast" passes the corresponding service tier. The vendor determines its availability and billing.

A custom model's context window guides budget estimation; it does not increase the server's actual limit. Enter the token count supported by that route, and confirm and save any catalog suggestion. After sending, inspect the conversation's context breakdown to check the effective budget and compact when appropriate.

12.9 Which Machine Owns Remote Credentials

Check the conversation's execution location and the cloud-traffic route in Settings · Connections:

  • Remote direct access: the remote runtime makes model requests, and the picker filters providers against its configuration. A key saved locally does not automatically exist remotely.
  • Via-local relay: model requests use the local machine's credentials and network while analysis runs remotely. This supports compute nodes without outbound access; the remote runtime not holding your local OAuth credentials is normal.

Do not copy an entire key file to a server just to make a model appear. For independent remote access, configure that remote explicitly or use the product's explicit credential-sync option. See Remote / SSH / HPC.

12.10 Verify the Actual Configuration

  1. Save, and confirm the route, region, full model ID and API format.
  2. Test access. For OAuth, check execution readiness rather than only "connected".
  3. Send a short message in the target conversation. Verify the actual execution location; test tool or vision paths separately when needed.
Symptom Check First
No selectable models Saved key / OAuth, hidden choices, remote ownership and catalog minimum-Core requirements
Login succeeds but requests are rejected Account access to the route and model, region, quota and the requesting machine's network
Test succeeds but chat fails Conversation provider / model, API format, gateway namespaces and tool capability
Missing key, or an exported key does not replace the old value Saved settings and credential ownership; do not print the key
Old model is unavailable or rejected Refresh the catalog and choose an available model, rather than bypass lifecycle checks
Free cloud unavailable or quota exhausted Distinguish rate limits, temporary unavailability and exhausted quota; follow the UI or select configured self-provided access

Include provider ID, model ID, Core version, execution location and the full error text in a support report. Remove authorization codes, tokens, keys, authentication headers and credential-bearing URLs.

results matching ""

    No results matching ""