Every existing agentic coding tool — Claude Code, Hermes, Cursor, etc. — is built around the assumption that the model behind it is large, cloud-hosted, and can absorb a 15–20k token system prompt without blinking. That assumption breaks completely on consumer hardware. A 4GB VRAM laptop running Gemma 4B doesn't have room for both the model weights and a novel's worth of instructions in every request.

ren is an agentic runtime designed from the ground up for small models running locally — 4B-class models on low-end GPUs or even CPU-only laptops. The constraint isn't a limitation to work around; it's the design center.

The core insight: most of the cost in agentic systems isn't the task, it's re-deciding how to do the task every single time. A model doesn't need to reason from scratch about "how do I check if a URL is reachable" every time a user asks — it needs to reason about it once, then remember.

The one-sentence pitch

ren minimizes LLM calls by learning from its own successful executions, converting repeated tasks into deterministic, zero-inference scripts — so the small model is only invoked for genuinely novel work.

2. The Problem

Constraint	Reality on target hardware
VRAM	4GB typical (integrated / entry discrete GPU)
System RAM	4–5GB available for inference workloads
Model class	~4B parameters (Gemma 4B, Phi, Qwen 4B-class)
Context budget	Every token of system prompt is a token not available for task context

Existing tools spend 15–20k tokens on system prompt before the user has typed a single word. On a small model with a small context window, this is not a minor inefficiency — it can be the majority of the usable context. It also means slower time-to-first-token on hardware that is already generation-speed constrained.

ren's answer: keep the system prompt minimal (under ~1.5k tokens), and instead of trying to make the model smarter, make the system smarter about when to invoke the model at all.

3. Core Design Principles

Minimize LLM calls, not just tokens per call. The biggest lever isn't prompt compression — it's not calling the LLM at all when a deterministic path already exists.
Small system prompt, always. No matter how the system grows, the per-request system prompt stays lean. Complexity lives in the architecture, not in prose fed to the model.
Self-improvement is a byproduct of execution, not a separate training loop. ren doesn't fine-tune anything. It learns by converting its own successful tool-use traces into reusable, generalized scripts.
Deterministic execution is always preferred over inference. If a task can be reduced to a script, it should be — scripts are faster, cheaper, auditable, and don't hallucinate.
The model is a fallback, not the default. The steady-state behavior of a mature ren installation should be: most tasks resolve via recipe match, and the LLM is invoked only for genuinely novel requests.
Everything the system learns is inspectable. Traces and recipes are plain JSON/scripts on disk — a user (or the model itself) can read exactly what ren has learned to do and why.

4. High-Level Architecture

                         ┌─────────────────┐
                         │   User Input     │
                         └────────┬─────────┘
                                  │
                                  ▼
                       ┌──────────────────────┐
                       │   Intent Embedder     │  ← tiny model, CPU, ms-scale
                       └──────────┬────────────┘
                                  │
                                  ▼
                       ┌──────────────────────┐
                       │    Recipe Matcher     │
                       │  (cosine similarity   │
                       │   vs. stored triggers)│
                       └─────┬──────────┬──────┘
                     match ≥ threshold  │  no confident match
                              │         │
                              ▼         ▼
                  ┌────────────────┐  ┌──────────────────────┐
                  │ Recipe Runner  │  │      Agent Loop        │
                  │ (zero LLM call)│  │  (LLM + tool calling)  │
                  └───────┬────────┘  └──────────┬──────────────┘
                          │                       │
                          ▼                       ▼
                     Result to user          Result to user
                                                   │
                                                   ▼
                                        ┌──────────────────────┐
                                        │    Trace Recorder      │
                                        │ (saves full run to     │
                                        │  traces/*.json)        │
                                        └──────────┬──────────────┘
                                                   │
                                        success?   │
                                                   ▼
                                        ┌──────────────────────┐
                                        │  Recipe Extractor      │
                                        │ (one LLM call:         │
                                        │  generalize trace →    │
                                        │  parameterized script) │
                                        └──────────┬──────────────┘
                                                   │
                                                   ▼
                                        ┌──────────────────────┐
                                        │    Recipe Store        │
                                        │  recipes/*.json        │
                                        └──────────────────────┘

The system has two loops:

The fast loop (top path): user input → embed → match → execute script. No LLM involved. This is the steady state ren converges toward.
The slow loop (bottom path): user input → LLM agent with tools → trace recorded → if successful, extracted into a new recipe that joins the fast loop's search space.

Over time, the ratio of fast-loop to slow-loop invocations should increase for any given user's workflow, since recurring tasks get captured.

5. Component Deep-Dive

5.1 Agent Loop (`main.rs`)

The core LLM-driven execution engine. Talks to any OpenAI-compatible completions endpoint (llama.cpp server, Ollama, LM Studio, or a cloud API — the client doesn't care).

Contract:

1. Send messages → LLM
2. Push the assistant turn into history (always — even if it only contains tool_calls)
3a. If tool_calls present: execute each, push tool_result messages, go to 1.
3b. If no tool_calls: content is the final answer, terminate.

Key implementation details:

ChatMessage is a single unified struct across all four roles (system, user, assistant, tool), with role-specific fields marked optional and skipped on serialization when absent. This keeps the wire format clean while letting the whole conversation live in one Vec<ChatMessage>.
ToolCall / FunctionCall derive both Serialize and Deserialize — the model emits them, and they must be re-serialized into the next request's message history. Also Clone, since the response is consumed by value while the loop needs to retain it in history.
The assistant turn is always pushed before tool result messages. Most backends validate that every tool_call_id on a tool-role message matches an id on the immediately preceding assistant turn's tool_calls. Wrong order is a 400 error.
content on the response is Option<String> — models frequently return null content when the turn is tool-calls-only. A bare String field would panic on deserialization.
Tool output truncation (MAX_TOOL_OUTPUT = 2000 chars). A single verbose curl response can consume the entire remaining context budget on a small model. Truncation is a hard ceiling per tool call, tunable per-tool in the future.
AgentOutput struct — run_agent always returns Ok(AgentOutput), even on non-convergence (hitting MAX_ITERATIONS). The outer Result::Err is reserved strictly for unrecoverable failures (network down, malformed JSON from the server). This guarantees the message history is always available for trace recording, success or failure.

Current tools implemented:

run_curl — HTTP GET/POST to a URL
run_shell — arbitrary shell command execution

These are intentionally general-purpose placeholders. The long-term tool model is closer to MCP servers / pluggable scripts than a fixed hardcoded set.

5.2 Trace Recorder (`trace.rs`)

Every agent run — successful or not — is serialized to traces/<unix_ms>_<slug>.json.

Why record failures too: a failed trace is diagnostic material. It's not (yet) fed into recipe extraction, but it's the raw material for a future "error reflection" pass — recognizing recurring failure patterns and adjusting tool descriptions or the system prompt.

Trace schema:

{
  "id": 1718123456789,
  "task": "check if wttr.in/delhi is accessible...",
  "status": "success",
  "iterations": 2,
  "duration_ms": 1840,
  "tools_invoked": ["run_curl"],
  "messages": [ /* full ChatMessage history */ ],
  "final_answer": "wttr.in/Delhi is accessible. Currently 38°C.",
  "error": null
}

tools_invoked is derived automatically by walking the message list for assistant turns containing tool_calls, deduplicated in first-seen order. This lets downstream consumers (the extractor) know a trace's tool dependencies without re-parsing the full conversation.
id doubles as both a unique identifier and a natural sort key (Unix ms timestamp).
The full messages array is preserved verbatim — this is what makes recipe extraction possible. Nothing is summarized or lost at record time.

5.3 Recipe Extractor (`recipe.rs`)

Takes a successful Trace and makes one, focused LLM call to generalize it into a reusable recipe.

Why this has to be a separate, single-purpose call rather than something the agent does inline: generalization requires stepping back from the specific instance ("Delhi", "wttr.in") to the abstract pattern ("check URL reachability, parameterized by URL"). Asking the agent to do this mid-task would pollute the tool-execution context and cost tokens on every run, not just successful new-pattern runs.

Process:

render_trace() converts the raw ChatMessage list into readable plain text — [SYSTEM], [USER], [ASSISTANT — tool calls], [TOOL RESULT] sections separated by ---. This is what the extraction LLM actually reads; it's optimized for the model's comprehension, not wire efficiency.
A single prompt instructs the model to:
- Identify the high-level goal, not the specific instance
- Replace every concrete value (URLs, filenames, search terms) with $UPPERCASE_PARAMETER placeholders
- Write a script implementing the generalized task
- Produce a one-sentence trigger description plus 2–3 trigger_examples (alternate phrasings) — this is what will later be embedded for semantic matching
The call uses structured output (response_format: json_schema, strict mode) to constrain the model to a fixed shape. This matters a lot for small models — without a schema, a 4B model is much more likely to add prose commentary around the JSON or drift from the expected keys.
strip_fences() defensively handles models that wrap JSON in markdown fences despite instructions not to — common failure mode below the ~7B range.

Recipe schema:

{
  "id": 1718123999000,
  "trigger": "Check whether a given URL is reachable via HTTP",
  "trigger_examples": [
    "is this site up",
    "check if example.com is accessible",
    "ping this endpoint for me"
  ],
  "parameters": [
    { "name": "URL", "description": "The full URL to check" }
  ],
  "script": "#!/bin/bash\nset -euo pipefail\n: \"${URL:?URL is required}\"\ncurl -s -o /dev/null -w '%{http_code}' \"$URL\"",
  "script_type": "shell",
  "tools_required": ["run_curl"],
  "source_trace_id": 1718123456789,
  "created_at": 1718123999000,
  "run_count": 0,
  "success_count": 0
}

Saved to recipes/<id>_<slugified-trigger>.json.

run_count / success_count are pre-wired into the schema for the recipe runner to update — this gives a future confidence-decay mechanism (a recipe that starts failing repeatedly should get deprioritized or flagged for re-extraction) without needing a schema migration later.

5.4 Script Language Decision

Language	Verdict	Reasoning
Rust	❌ Excluded	Requires compilation — reintroduces the exact latency the recipe system exists to eliminate.
Shell	✅ Default (~80% of recipes)	Zero interpreter startup beyond the OS shell, universally available, native parameter substitution via env vars, and matches what the agent's own tools (`run_curl`, `run_shell`) already produce. Small models have seen enormous volumes of bash and generate it reliably for this class of task.
Python	✅ Fallback (~20% of recipes)	Used when a recipe needs JSON parsing, structured data manipulation, or conditional logic that would be unreadable in bash.

Both interpreters read parameters identically — as environment variables, not string-interpolated arguments. This sidesteps injection risk entirely and keeps the runner interface uniform across script types:

fn execute_recipe(recipe: &Recipe, params: &HashMap<String, String>) -> Output {
    let mut cmd = match recipe.script_type.as_str() {
        "python" => Command::new("python3"),
        _        => Command::new("sh"),
    };
    let script_file = write_temp_script(&recipe.script, &recipe.script_type);
    cmd.arg(&script_file).envs(params);
    cmd.output()
}

Generated shell scripts are expected to start with a safety header:

#!/bin/bash
set -euo pipefail
: "${URL:?URL is required}"

set -euo pipefail ensures failures propagate instead of silently continuing; the : parameter-check pattern gives a clear error if a required env var wasn't supplied.

6. Planned Components (Not Yet Built)

6.1 Semantic Matcher

Embeds incoming user input and compares it against the trigger + trigger_examples of every stored recipe via cosine similarity.

Open design questions:

Embedding model: something CPU-fast and small — all-MiniLM-L6-v2 (~22MB) is the leading candidate. Needs to run in well under 100ms to not undermine the "fast loop" premise.
Vector store: given the target hardware profile, a lightweight embedded option makes more sense than a server — likely a flat in-memory index rebuilt on startup (recipe counts are expected to be in the hundreds/low-thousands, not millions) rather than a full vector DB. lancedb or a hand-rolled flat cosine scan are both plausible.
Threshold tuning is the highest-risk part of the whole system. A false-positive match — running the wrong recipe with the wrong parameters — is worse than falling back to the LLM. The matcher should be tuned for high precision over high recall, and:
- Recipes involving destructive operations (file deletion, overwrites, network mutations) should require either a higher confidence threshold or explicit user confirmation before running, regardless of match score.
- Ambiguous matches (multiple recipes above threshold, or a score in a "gray zone") should fall back to the LLM rather than guessing.

6.2 Recipe Runner

Executes a matched recipe: extracts required parameters from the user's phrasing (likely via a lightweight extraction step, not necessarily the main LLM — could be regex/slot-filling for simple cases, small-model call for complex ones), writes the script to a temp file, executes with envs(), captures output, updates run_count / success_count on the recipe file.

Open question: how does parameter extraction happen without invoking the main LLM? Options being considered:

A much smaller/cheaper model dedicated to slot-filling
Structured regex/pattern matching keyed off the parameter descriptions
A hybrid: try pattern matching first, fall back to a cheap LLM call only if extraction confidence is low

This matters because if parameter extraction always requires an LLM call, the "zero-inference execution" claim weakens — it becomes "one call instead of many" rather than "zero calls." Worth deciding early since it affects the recipe schema (parameters may need extraction hints, e.g. regex patterns, alongside their descriptions).

6.3 Failure Reflection

Failed traces are currently recorded but unused. The planned second consumer of the trace store: periodically (or on-demand) analyze failure patterns to identify:

Tools that are frequently misused (suggesting a description needs improvement)
Recurring task types the agent struggles with (suggesting a manual recipe should be authored, or the system prompt needs a nudge)

This is explicitly lower priority than the success path — the core value proposition depends on the success → recipe pipeline working well first.

6.4 Config

Currently hardcoded (endpoint URL, model name, MAX_ITERATIONS, MAX_TOOL_OUTPUT). Needs to move to a ren.toml before the codebase grows further — flagged as technical debt, not urgent.

7. Technology Choices & Rationale

Choice	Rationale
Rust for the runtime	Near-zero process overhead — critical when 4GB RAM is shared between the OS, the model weights, and the runtime itself. Easy FFI into `llama.cpp` if/when local inference moves in-process. Deterministic recipe execution with no GC pauses or interpreter startup cost.
OpenAI-compatible API surface	Works unmodified against `llama.cpp` server, Ollama, LM Studio, or a real cloud endpoint. The model location (local or cloud) is a deployment detail, not an architectural one.
JSON Schema structured outputs for extraction	Small models are meaningfully less reliable at free-form JSON generation. Constraining the output shape at the API level (rather than hoping the prompt is followed) removes an entire failure class.
Plain JSON files for traces/recipes (no database, for now)	Fully inspectable and diffable by a human or the model itself. No schema migration tooling needed yet at expected scale (hundreds of recipes). Revisit if/when recipe counts grow past what a flat directory scan can handle at matcher startup time.
Environment variables for recipe parameters	Uniform interface across shell and Python, avoids string-interpolation injection risk entirely, and matches how both languages naturally expect external input.

8. What "Done" Looks Like (Steady State)

A mature ren installation, after some period of use on a given machine/workflow, should exhibit:

The large majority of routine requests resolve via the fast loop (recipe match → script execution), with sub-100ms latency and zero token generation.
The LLM is invoked primarily for genuinely novel requests, complex multi-step reasoning, or tasks that don't decompose cleanly into a deterministic script.
The recipe store is a legible, growing library that a user can audit, edit, or delete from directly — it's not a black box.
System prompt size stays flat regardless of how many recipes exist, since recipes are matched via embedding similarity, not stuffed into context.

9. Current Implementation Status

Component	Status
Agentic loop (multi-turn tool calling)	✅ Built
Trace recorder	✅ Built
Recipe extractor (trace → generalized script)	✅ Built
Recipe schema + save/load	✅ Built
Semantic matcher	⏳ Not started
Recipe runner (execution + param extraction)	⏳ Not started
Failure reflection	⏳ Not started
Config file (`ren.toml`)	⏳ Not started
Tool plugin system (beyond `run_curl` / `run_shell`)	⏳ Not started

10. Open Questions Tracker

These are unresolved decisions flagged during design, kept here so they don't get lost:

How is parameter extraction done at recipe-run time without reintroducing a full LLM call per invocation?
What embedding model and similarity threshold produce acceptably low false-positive rates for recipe matching?
Should destructive recipes require explicit user confirmation on every run, or only above a certain risk classification — and who assigns that classification (the extractor, a fixed tool allowlist, or the user)?
At what recipe-store size does a flat directory scan become a real bottleneck, and what's the migration path if so?
Should failed traces eventually feed an automated system-prompt/tool-description tuning loop, or stay purely diagnostic for manual review?

README.md Unescape Escape

ren

1. Vision