Reusable runtime // hexagonal core + streaming loop

Live

rony-llm-agent

Core LLM agent library in Go: provider-agnostic client interface, agent loop with iteration control, retrieval-augmented memory backed by ChromaDB, tool registry with a filesystem sandbox.

GoOpenAI APIllama.cppChromaDBMCPAgent loop
Source ↗
  • OutputAgent runtime
  • RuntimeGo 1.26+
  • DesignPorts + adapters
Agent runtimeReady
rony-llm-agent mark

Compose a provider, persona, memory, tools, and sandbox into a predictable agent core.

Overview

rony-llm-agent is the reusable core behind products such as rony-chat-bot and rony-harness. It owns the generic agent behavior — provider abstraction, orchestration, memory, personas, tools, and sandboxing — while each product chooses its UI, deployment, and configuration.

It is intentionally not a CLI, HTTP server, or opinionated product. Consumers use the public Go interfaces and compose only the adapters they need.

Rony describing the reusable agent runtime

Execution model

The agent loop is streaming-first and explicit about its limits. A run adds the user input to history, asks the configured LLM for a response, executes any requested tools, appends their results, and continues until the model finishes or a guardrail stops the run.

  • iter.Seq2 exposes generated chunks without callback plumbing.
  • MaxIters, token budgets, timeouts, and stop conditions are configuration rather than hidden constants.
  • A mock LLM client makes every iteration deterministic in unit tests without spending real tokens.
  • The response reports content, tool calls, iteration count, duration, and token usage for downstream observability.

Rony explaining the filesystem sandbox used by the agent

Architecture

The library follows ports-and-adapters boundaries so products do not couple their domain logic to a provider SDK or storage implementation:

Package Responsibility
pkg/agent Iterative loop, streaming, termination, approvals, and sub-agent orchestration.
pkg/llm Provider-neutral client interface, message types, capabilities, and streaming chunks.
pkg/rag Retrieval-augmented memory, embeddings, semantic search, and vector-store boundaries.
pkg/persona System prompt assembly, YAML personas, and AGENTS.md discovery.
pkg/tools JSON Schema tools, permissions, registry, results, and execution contracts.
pkg/config YAML loading and hierarchical configuration precedence.

Ports define the behavior the core needs; adapters implement OpenAI, Anthropic, Ollama, llama.cpp, ChromaDB, and embedding backends around those contracts. Swapping an adapter does not change the agent loop.

Safety boundaries

  • Filesystem sandbox — tool paths are validated against the consumer’s project root with os.Root, keeping reads and writes inside the allowed workspace.
  • Permission-aware tools — tools carry Allow, Ask, or Deny policies so a product can require approval before consequential operations.
  • Bounded execution — iteration, token, output-size, timeout, and cancellation limits prevent one model response from consuming an unbounded run.
  • Deterministic tests — mock providers and embedding clients let the security and termination rules be tested without external services.
  • Persona isolation — the consuming project’s AGENTS.md contributes its own instructions, so the same runtime can serve different products without sharing product-specific behavior.

The library provides the safety primitives; the consuming application still owns its approval UX and deployment permissions.

Product adapters

The same runtime is used in different contexts:

Consumer What it adds
rony-chat-bot HTTP/SSE transport, persistent conversations, portfolio retrieval, and a drop-in web widget.
Rony Harness Terminal UI, local/cloud model selection, real software tools, approvals, and persistent sessions.

Rony Harness is a concrete consumer of the runtime: the same core handles model selection, tool execution, approvals, and the resulting files while the TUI owns the interaction surface.

Rony Harness selecting model scripts for an agent session

Rony Harness showing tool calls and generated files

This separation keeps the core small and reusable while allowing each product to expose the right interaction model for its audience.

Files of note

  • pkg/agent/ — loop orchestration, streaming, iteration limits, and sub-agents.
  • pkg/llm/ — public provider interface and model-neutral request/response types.
  • pkg/rag/ — memory and retrieval abstractions.
  • pkg/persona/ — persona loading and AGENTS.md discovery.
  • pkg/tools/ — registry, permissions, tool results, and sandbox integration.
  • docs/architecture.md — ports, adapters, core interfaces, and the execution algorithm.