mirror of https://github.com/sortedcord/omnia.git synced 2026-07-22 03:52:48 +05:30

Files

Aditya Gupta 264a5ea0fe feat(gui): Duplicate existing model instances (#32 )

(fixes Allow Instance Duplication
Fixes #32)

2026-07-20 11:41:28 +05:30

src

feat(gui): Duplicate existing model instances (#32 )

2026-07-20 11:41:28 +05:30

tests

chore: fix linting issues

2026-07-19 22:18:42 +05:30

package.json

refactor!(llm): Remove bootstrapper in favor of setup-provider

2026-07-16 22:21:59 +05:30

README.md

refactor!(llm): Remove bootstrapper in favor of setup-provider

2026-07-16 22:21:59 +05:30

tsconfig.json

feat: Implement config system

2026-07-06 08:02:54 +05:30

README.md

@omnia/llm

LLM abstraction layer providing pluggable, database-backed provider instances for generative and embedding tasks.

Architecture Overview

The system is built around four layers:

Registry — each provider class self-registers its metadata (id, envVar, capabilities, default model, etc.) via static {} blocks; PROVIDER_REGISTRY is derived from these registrations at runtime — there is no hand-maintained provider list
Provider Manager — SQLite-backed CRUD for persisted provider instances, with env-var bootstrap driven by the registry
Provider Factory — buildLLMProvider(inst) / buildEmbeddingProvider(inst) resolve a stored instance to a live provider class via the registry
Interfaces — contracts that all providers implement

graph TD
    subgraph Self-Registering Providers
        GP["GeminiProvider"]
        ORP["OpenRouterProvider"]
        MP["MockLLMProvider"]
        GEP["GeminiEmbeddingProvider"]
        MEP["MockEmbeddingProvider"]
    end

    subgraph Registry
        PR["ProviderRegistry\n(derived, not authored)"]
    end

    subgraph Storage
        PM["ProviderManager\n(db.ts + bootstrap.ts + row-mapper.ts)"]
        DB[("settings.db")]
    end

    subgraph Factory
        PF["buildLLMProvider()\nbuildEmbeddingProvider()"]
    end

    GP -->|static block| PR
    ORP -->|static block| PR
    MP -->|static block| PR
    GEP -->|static block| PR
    MEP -->|static block| PR

    PM -->|reads/writes| DB
    PM -->|bootstrap from| PR

    PF -->|looks up| PR
    PF -->|instantiates| GP
    PF -->|instantiates| ORP

Core Interfaces

Defined in llm.ts:

`ILLMProvider`

The primary contract for generative (text-to-structured-data) providers.

Member	Type	Description
`providerName`	`string`	Human-readable provider label
`maxContext`	`number?`	Maximum context window in tokens
`generateStructuredResponse<T>(request)`	`Promise<LLMResponse<T>>`	Sends a prompt + Zod schema → returns parsed, typed output
`lastCalls`	`LLMCallRecord[]?`	Audit trail of recent calls (prompts + usage)

`IEmbeddingProvider`

Contract for text-to-vector embedding providers.

Member	Type	Description
`providerName`	`string`	Human-readable provider label
`embed(text)`	`Promise<number[]>`	Returns a dense vector embedding of the input text

`LLMRequest<T>`

Input to generateStructuredResponse:

{
  systemPrompt: string;    // System-level instructions
  userContext:   string;    // User/task-specific context
  schema:        T;         // Zod schema — output is validated against this
  temperature?:  number;    // Sampling temperature (optional)
}

`LLMResponse<T>`

Output from generateStructuredResponse:

{
  success: boolean;
  data?:   T;               // Parsed, schema-validated output
  error?:  string;          // Error message on failure
  usage?:  {
    inputTokens:          number;
    outputTokens:         number;
    totalTokens:          number;
    modelName?:           string;
    providerInstanceName?: string;
    maxContext?:           number;
  };
}

`ModelProviderInstance`

The persisted configuration record for a single provider instance:

{
  id:           string;                     // Unique ID ("provider-<timestamp>")
  name:         string;                     // User-facing name ("Gemini (Env)")
  providerName: string;                     // Provider type key ("google-genai" | "openrouter" | "mock")
  apiKey:       string;                     // API key
  isActive:     boolean;                    // Whether this is the active instance for its type
  modelName?:   string;                     // Specific model to use
  type:         "generative" | "embedding"; // Instance category
  maxContext?:  number;                     // Context window limit
}

`ModelProviderMeta`

Static metadata for each available provider type (used by the UI's provider picker):

Member	Type	Description
`id`	`string`	`"google-genai"` \| `"openrouter"` \| `"ollama"` \| `"mock"`
`displayName`	`string`	Human-readable name
`description`	`string`	Human-readable description
`defaultModel`	`string`	Default generative model
`defaultEmbeddingModel`	`string`	Default embedding model

Provider metadata is self-declared by each provider class in a static {} block and collected into PROVIDER_REGISTRY (derived, not authored). The getAvailableProviders() function and AVAILABLE_PROVIDERS helper in llm.ts read from the registry at call time.

Provider Manager

ProviderManager is a static class that provides full CRUD over provider instances, backed by a SQLite database (data/settings.db at the workspace root). Internally split across:

db.ts — memoized DB handle + schema migrations (PRAGMA user_version)
bin/setup-provider.ts — CLI tool to set up provider instances in the database
row-mapper.ts — mapRow() (written once, used everywhere)
provider-manager.ts — thin CRUD: list, create, delete, setActive, update, getActive, getMappings, setMapping

Storage

The database is auto-created on first access. The table schema:

CREATE TABLE IF NOT EXISTS provider_instances (
  id           TEXT PRIMARY KEY,
  name         TEXT NOT NULL,
  providerName TEXT NOT NULL,
  apiKey       TEXT NOT NULL,
  isActive     INTEGER NOT NULL DEFAULT 0,
  modelName    TEXT,
  type         TEXT NOT NULL DEFAULT 'generative',
  maxContext   INTEGER
);

A second table stores per-task provider overrides:

CREATE TABLE IF NOT EXISTS provider_mappings (
  task                TEXT PRIMARY KEY,
  providerInstanceId  TEXT NOT NULL
);

API

Method	Signature	Description
`list()`	`→ ModelProviderInstance[]`	Returns all saved instances
`create(name, providerName, apiKey, modelName?, type?, maxContext?)`	`→ ModelProviderInstance`	Creates a new instance. Auto-activates if it's the first of its type
`delete(id)`	`→ void`	Removes an instance. If it was active, auto-promotes the next instance of the same type
`setActive(id)`	`→ void`	Deactivates all instances of the same type, then activates the target
`update(id, name, providerName, apiKey?, modelName?, type?, maxContext?)`	`→ void`	Updates an existing instance. If `apiKey` is empty/omitted, the existing key is preserved
`getActive(type?)`	`→ ModelProviderInstance \| null`	Returns the currently active instance for the given type (`"generative"` by default)
`getMappings()`	`→ Record<string, string>`	Returns all task → providerInstanceId mappings
`setMapping(task, providerInstanceId)`	`→ void`	Sets or removes (if `providerInstanceId` is empty) a task-specific mapping

Active Instance Invariants

Only one active instance per type — setActive() deactivates all sibling instances before activating the target.
Auto-promotion on delete — if the deleted instance was active, the first remaining instance of the same type is promoted.
Auto-activation on create — if no active instance exists for the type, the new instance is automatically activated.

Manual Seeding via CLI

Rather than automatically bootstrapping from environment variables at runtime, which adds runtime complexity, you can quickly seed the database using the CLI setup tool:

Seeding All Environment-Variable Providers

pnpm setup-provider --all

This command auto-detects and inserts provider instances into data/settings.db for any registered providers whose corresponding environment variables (such as GOOGLE_API_KEY, OPENAI_API_KEY, etc.) are defined.

Creating a Specific Provider Instance

pnpm setup-provider --provider google-genai --key YOUR_API_KEY [--name "My Gemini"] [--model gemini-2.5-flash] [--type generative] [--max-context 32768] [--endpoint url]

Credential Resolution Cascade

When initializing a provider (e.g. new GeminiProvider()):

Explicit Credentials — If apiKey, modelName, etc. are passed directly to the constructor, they are used.
Active DB Instance — If not explicitly passed, it looks up the active DB instance via ProviderManager.getActive(). If found and matches the provider, its API key and configuration are used.
Environment Fallback — If there is no matching active DB instance, it resolves the key directly from the corresponding environment variable (e.g., GOOGLE_API_KEY) via resolveCredentials.