Knowledge Base Guide

LLM-compiled personal knowledge base system. Raw documents get ingested, compiled by an LLM into structured markdown wiki entries with summaries, backlinks, cross-references, and semantic search.

Inspired by Karpathy's vision of knowledge bases where LLMs serve as the compiler — transforming unstructured documents into a structured, interconnected wiki.

Architecture

Raw Documents  →  LLM Compiler  →  Wiki Entries  →  Search / Q&A / Generate
  (markdown,       (extracts         (structured       (semantic search,
   text, URLs)      concepts,         articles with     backlinks,
                    compiles)         [[wiki-links]])   reports)

Three usage modes:

Quick Start — Plugin Mode

The simplest way to add knowledge base capabilities to an agent:

agent = Nous.Agent.new("openai:gpt-4", plugins: [Nous.Plugins.KnowledgeBase])

# Configuration lives in the run dependencies, not on the agent struct
deps = %{
  kb_config: %{
    store: Nous.KnowledgeBase.Store.ETS,
    kb_id: "my_kb"
  }
}

# Ingest a document
{:ok, r1} =
  Nous.run(
    agent,
    """
    Ingest this article:

    # Understanding GenServers
    GenServers are the workhorse of OTP. They provide a client-server abstraction
    where the server runs as a separate process...
    """,
    deps: deps
  )

# Query the knowledge base
{:ok, r2} =
  Nous.run(agent, "What do we know about GenServers?", deps: deps, context: r1.context)

# Generate a report
{:ok, r3} =
  Nous.run(agent, "Generate a summary report of all OTP concepts",
    deps: deps,
    context: r2.context
  )

With Semantic Search

Add an embedding provider for vector-based search:

agent = Nous.Agent.new("openai:gpt-4", plugins: [Nous.Plugins.KnowledgeBase])

deps = %{
  kb_config: %{
    store: Nous.KnowledgeBase.Store.ETS,
    kb_id: "my_kb",
    embedding: Nous.Memory.Embedding.OpenAI,
    embedding_opts: %{api_key: System.get_env("OPENAI_API_KEY")}
  }
}

Composing with Memory

The KB plugin works alongside Nous.Plugins.Memory:

agent =
  Nous.Agent.new("openai:gpt-4",
    plugins: [Nous.Plugins.Memory, Nous.Plugins.KnowledgeBase]
  )

deps = %{
  memory_config: %{store: Nous.Memory.Store.ETS},
  kb_config: %{store: Nous.KnowledgeBase.Store.ETS, kb_id: "my_kb"}
}

Memory stores personal/episodic information; the knowledge base stores compiled reference material.

Agent Behaviour Mode

For a KB-specialized agent with additional reasoning tools:

agent =
  Nous.Agent.new("openai:gpt-4",
    behaviour_module: Nous.Agents.KnowledgeBaseAgent,
    plugins: [Nous.Plugins.KnowledgeBase]
  )

deps = %{kb_config: %{store: Nous.KnowledgeBase.Store.ETS, kb_id: "my_kb"}}

The KnowledgeBaseAgent adds 4 reasoning tools on top of the 9 standard KB tools:

Workflow Mode

For batch operations without an interactive agent:

Batch Ingest

config = %{
  store: Nous.KnowledgeBase.Store.ETS,
  kb_id: "my_kb"
}

{:ok, state} = Nous.KnowledgeBase.ingest(
  [
    %{title: "GenServer Guide", content: "...", doc_type: :markdown},
    %{title: "Supervisor Patterns", content: "...", doc_type: :markdown}
  ],
  kb_config: config
)

Health Check

{:ok, state} = Nous.KnowledgeBase.health_check(kb_config: config)
report = state.data.health_report
# => %HealthReport{issues: [...], coverage_score: 0.85, ...}

Incremental Update

{:ok, state} = Nous.KnowledgeBase.incremental_update(
  [%{title: "Updated Article", content: "..."}],
  kb_config: config
)

Output Generation

generate/2 requires both a :kb_config and a :topic; the output type is :report, :summary, or :slides:

{:ok, state} = Nous.KnowledgeBase.generate(
  :summary,
  kb_config: config,
  topic: "OTP concepts"
)

summary = state.node_results["generate_output"]

Data Model

Documents

Raw source material before compilation:

%Nous.KnowledgeBase.Document{
  id: "doc_abc123",
  title: "Understanding GenServers",
  content: "...",
  doc_type: :markdown,        # :markdown | :text | :url | :pdf | :html
  status: :compiled,          # :pending | :processing | :compiled | :failed
  checksum: "sha256:...",
  compiled_entry_ids: ["entry_1", "entry_2"],
  kb_id: "my_kb"
}

Entries

Compiled wiki articles — the core unit of the knowledge base:

%Nous.KnowledgeBase.Entry{
  id: "entry_1",
  title: "GenServer",
  slug: "genserver",
  content: "GenServer is the core [[otp]] abstraction for...",
  summary: "Client-server abstraction for stateful processes",
  entry_type: :concept,       # :article | :concept | :summary | :index | :glossary
  concepts: ["genserver", "otp", "processes"],
  tags: ["elixir", "otp"],
  confidence: 0.95,
  source_doc_ids: ["doc_abc123"],
  kb_id: "my_kb"
}

Links

Typed directional connections between entries, from from_entry_id to to_entry_id:

%Nous.KnowledgeBase.Link{
  id: "link_1",
  from_entry_id: "entry_1",
  to_entry_id: "entry_2",
  link_type: :cross_reference,  # :backlink | :cross_reference | :concept | :see_also | :parent_child
  label: "GenServer implements OTP behaviour",
  weight: 1.0,
  kb_id: "my_kb"
}

Health Reports

Audit results from health checks:

%Nous.KnowledgeBase.HealthReport{
  total_entries: 42,
  total_links: 128,
  total_documents: 15,
  coverage_score: 0.85,
  freshness_score: 0.92,
  coherence_score: 0.78,
  issues: [
    %{type: :orphan, entry_id: "entry_5", severity: :medium,
      description: "No incoming links", suggested_action: "Add links from related entries"}
  ]
}

Tools Reference

The plugin provides 9 tools to agents:

Store Backends

ETS (built-in)

Zero-dependency in-memory backend. Suitable for development, testing, and single-node deployments:

%{store: Nous.KnowledgeBase.Store.ETS}

Features: Jaro-distance text search, optional vector search with embeddings, scoped by kb_id.

Custom Backends

Implement the Nous.KnowledgeBase.Store behaviour (19 callbacks, of which search_entries/3, related_entries/3 and link_counts_by_source/1 are optional) for custom backends:

defmodule MyApp.KBStore.Postgres do
  @behaviour Nous.KnowledgeBase.Store

  @impl true
  def init(opts), do: ...

  @impl true
  def store_entry(state, entry), do: ...

  # ... the remaining callbacks
end

Configuration Reference

Store configuration is passed via deps[:kb_config] (plugin mode) or the :kb_config option (workflow mode):

:compiler_model is not part of kb_config — it is a top-level option on the Nous.KnowledgeBase workflow functions (ingest/2, incremental_update/2, health_check/1, generate/2) and defaults to "openai:gpt-4o-mini":

Nous.KnowledgeBase.ingest(documents, kb_config: config, compiler_model: "openai:gpt-4o")