docs: Added documentation for LLMProviders and named instances

This commit is contained in:
2026-07-09 19:35:18 +05:30
parent 6626adf38d
commit cc9d0006e5
10 changed files with 195 additions and 13 deletions

View File

@@ -0,0 +1,97 @@
---
title: LLM Providers & Configuration
description: Details of the LLM provider instances, task routing, and self-bootstrapping setup in Omnia.
sidebar:
order: 5
---
In Omnia, all non-player character behaviors, action validation, intent decoding, and time step logic are simulated using Large Language Models (LLMs). The LLM subsystem is built around **polymorphism, key instance management, and task provider routing**.
## Core Interfaces
All LLM providers implement the common `ILLMProvider` interface defined in `packages/llm/src/llm.ts`:
```typescript
export interface ILLMProvider {
providerName: string;
generateStructuredResponse<T extends z.ZodTypeAny>(
request: LLMRequest<T>,
): Promise<LLMResponse<z.infer<T>>>;
lastCalls?: LLMCallRecord[];
}
```
The codebase provides two primary implementations:
1. **`GeminiProvider`:** The production provider utilizing Google's Gemini Models via the `@langchain/google-genai` SDK.
2. **`MockLLMProvider`:** A stateless, pre-programmed mock provider used for fast, deterministic unit testing and local integration tests.
---
## LLM Provider Instances
To support multiple different API keys, key rotation, and model variation, Omnia utilizes a **Provider Instance model** rather than relying on static configuration:
```typescript
export interface LLMProviderInstance {
id: string;
name: string;
providerName: string;
apiKey: string;
isActive: boolean;
modelName?: string;
}
```
Users can register multiple provider instances in the **Configuration Page** under the GUI. Each instance is given:
* A friendly, human-readable name (e.g., `"Gemini Production Key"`, `"Experimental Gemini Pro"`).
* A provider type (e.g., `google-genai`, `mock`).
* An API key credential.
* A custom target model name (e.g., `gemini-2.5-flash` or `gemini-2.5-pro`).
* An **Active** status flag (one key is marked as globally active).
Configurations are stored globally in `data/settings.db` (separated from specific simulation run databases like `data/sim-*.db` to keep key storage and audit logs isolated).
---
## Task Provider Routing
During a simulation run, the engine executes four distinct LLM operations. To optimize costs, latency, or model accuracy, you can route each of these tasks to different LLM provider instances:
| Task Name | Key ID | Description | Default Model |
| :--- | :--- | :--- | :--- |
| **Actor Prose Generation** | `actor-prose` | Generates roleplay and narrative behavioral prose for Non-Player Characters. | `gemini-2.5-flash` |
| **LLM Validator** | `llm-validator` | Arbitrates and validates proposed actions against the world state rules and constraints. | `gemini-2.5-flash` |
| **Intent Decoder** | `intent-decoder` | Parses and splits free-text actions/prose into structured intent sequences. | `gemini-2.5-flash` |
| **TimeDelta Generator** | `timedelta` | Calculates the duration of character actions to advance the game clock. | `gemini-2.5-flash` |
If no specific provider instance is mapped to a task, the task automatically routes to the globally marked **Active** provider instance.
---
## Automatic Bootstrapping (Environment Key Fallback)
To maintain backwards-compatibility and support headless runs, live evaluation suites, and automated unit tests without requiring database pre-configuration, the config manager supports **self-bootstrapping**:
1. When the provider manager queries the active key instance, if `data/settings.db` contains **0 registered keys**, it checks the process environment for `GOOGLE_API_KEY`.
2. If `process.env.GOOGLE_API_KEY` is present, it automatically creates, saves, and activates a default provider instance (`Default (Env)`) in `settings.db`.
3. If database write locks occur (e.g., during high-concurrency Vitest test suites), the system seamlessly returns a temporary in-memory `LLMProviderInstance` to keep execution fluent and error-free.
---
## Developer Guide: Managing Mappings
Configuration settings are managed through `ProviderManager` static methods:
```typescript
// Query the active provider configuration
const activeConfig = ProviderManager.getActive();
// List all registered provider instances
const allConfigs = ProviderManager.list();
// Retrieve task-specific mappings
const mappings = ProviderManager.getMappings(); // e.g., { "actor-prose": "provider-123" }
// Map a task to a provider instance
ProviderManager.setMapping("actor-prose", "provider-123");
```