Back to Blog
April 7, 2026 at 08:39 AM

Beyond RAG: Why Andrej Karpathy is Building an "LLM Wiki"

Hitesh Agja
AI Knowledge ManagementLLM WikiAndrej KarpathyBeyond RAGObsidian AI Workflow
Beyond RAG: Why Andrej Karpathy is Building an "LLM Wiki"

Why the Future of AI is "Compiled," Not Just "Retrieved"

In the world of AI, we’ve been obsessed with RAG (Retrieval-Augmented Generation). You feed an AI a bunch of PDFs, ask a question, and it "retrieves" snippets to give you an answer.

But Andrej Karpathy recently pointed out a major flaw: RAG is stateless. It doesn’t "learn" your business. Every time you ask a question, it starts from scratch. Karpathy’s solution? The LLM Wiki—a system where the AI doesn't just read your files; it compiles them into a permanent, interlinked knowledge base.

1. The Core Philosophy

Karpathy views this through a developer's lens:

"Obsidian is the IDE, the LLM is the programmer, and the Wiki is the codebase."

Instead of a messy folder of 500 documents, you have a clean, structured Wiki. When a new document arrives, the AI "refactors" your knowledge—updating old pages, creating new links, and building a map of your business logic.

2. The Three-Layer Architecture

To build this, you need three distinct layers:

LayerComponentResponsibility
1. Raw Sources/raw folderImmutable. Meeting transcripts, emails, and PDFs. The AI reads these but never edits them.
2. The Wiki/wiki folderLLM-Owned. Markdown files generated by the AI. Includes concept pages and project timelines.
3. The SchemaRULES.mdThe Brain. Instructions telling the AI how to format notes and when to update existing pages.

3. Corporate Use Case: Project Management

Imagine you are managing a complex product launch. Usually, info is scattered across Slack, Jira, and endless PDFs.

The Ingest Loop

You drop a "Quarterly Sync Transcript" into the /raw folder. Instead of just indexing it, the AI Agent:

  1. Updates the Project_Timeline.md with new deadlines.
  2. Cross-links a new "Risk Factor" to the Legal_Compliance.md page.
  3. Creates a new bio page for a newly mentioned stakeholder.

The "Linting" Process

Once a week, you ask the AI to "lint" the wiki. It scans for contradictions.

  • Example: "The meeting on Tuesday says the launch is in June, but the Project Brief says July. Which is it?" The AI flags this conflict for a human to resolve.

4. Scaling Up: The Large Dataset Challenge

While the "Wiki-only" approach is perfect for individual projects or small teams, massive corporations face a "Context Window" limit. You cannot feed 10,000 wiki pages into a single prompt.

The Hybrid Solution: For massive datasets, you implement a Tiered Knowledge Strategy:

  • The Wiki (Hot Storage): High-level summaries, core project logic, and interlinked "Index" pages stay in the Wiki. This is what the AI reads first.
  • Vector RAG (Cold Storage): Thousands of granular, old, or highly technical documents stay in a traditional vector database.

When you ask a complex question, the AI uses the Wiki to understand the "Map" of the project and then uses RAG to pull specific "Details" from the deep archives. This prevents "Context Fatigue" while maintaining the compounding intelligence of the Wiki.

5. Practical Implementation (The Free Route)

Step 1: The Workspace

  1. Download Obsidian: A local-first Markdown editor. Create a new "Vault."
  2. Set up Folders: Create 01_raw, 02_wiki, and 03_rules.

Step 2: The Agent (The "Programmer")

You need an AI tool that can edit local files.

  • Windsurf or Aider: These are excellent free/open-source tools that can read and write files in your folder.
  • Google AI Studio: Use the Gemini 1.5 Flash API (Free Tier) with a tool like Cline in VS Code to act as your librarian.

Step 3: The System Prompt

Paste this into your Agent’s instructions:

"You are a Corporate Librarian. Your job is to maintain the /wiki folder based on files in /raw. Use [[WikiLinks]] for all entities. When a new file is added, update relevant pages. Maintain an index.md as the entry point."

6. Why This Wins

  • Audit Trail: Unlike a black-box chatbot, you can see the Markdown files. You know exactly where the AI got its info.
  • Ownership: You own the files. If the AI provider changes their terms, you still have your perfectly organized Obsidian vault on your hard drive.
  • Compounding Value: Every document you add makes the entire Wiki more accurate, rather than just answering a one-off question.