Doorman11991/smallcode: AI coding agent optimized for small LLMs. 87% benchmark with 4B-active model.

AI coding agent optimized for small LLMs. 87% benchmark with 4B-active model. - Doorman11991/smallcode

Summary

SmallCode is a terminal-native AI coding agent written in JavaScript, designed for local LLMs in the 8B–35B parameter range rather than frontier cloud models. It compensates for smaller-model limitations through context budgeting, patch-first editing, a forgiving multi-format tool-call parser, and TODO-driven task decomposition. The project is MIT-licensed and runs entirely offline against any OpenAI-compatible local server.

What it is useful for

SmallCode targets the gap between general-purpose tools like OpenCode, which assume Claude- or GPT-5-class models with 128k+ context, and small open-weight models like Qwen3-8B that run on consumer hardware. The README frames it as a coding assistant that never has to call a cloud API.

Concrete scenarios drawn from the README:

• Working offline with LM Studio, llama.cpp, or Ollama as the backend
• Multi-step refactors where the model needs to remember prior edits without exceeding its context window — tool results are capped at 4k chars, mid-turn eviction drops old results, and semantic compression summarizes history
• Patching existing code rather than rewriting whole files, which the README notes is more reliable for small models that truncate or drift
• Decomposed planning where a TODO file drives atomic, validated steps
• Optional escalation to Claude, OpenAI, DeepSeek, or OpenRouter when a local model hard-fails (opt-in via API key)
• A persistent shell session so cd, env vars, and shell variables survive across bash calls

The README documents limits: models under 4B struggle with multi-step tool use, and the adaptations add little value above 35B. If better-sqlite3 cannot compile on a non-LTS Node version, SmallCode falls back to a JSON-based memory store rather than failing outright.

How engineers can use it

The README documents two install paths. The standard one is via npm:

npm install -g smallcode
cd my-project
smallcode

Prebuilt tarballs for Windows, macOS, and Linux that bundle Node.js are also available, installed with a one-line curl | bash or iwr | iex script that drops files under ~/.smallcode.

For a fresh checkout, the README shows:

npm install
cat > .env <<'EOF'
SMALLCODE_MODEL=your-local-model-name
SMALLCODE_BASE_URL=http://localhost:1234/v1
EOF
node bin/smallcode.js

Requirements are Node.js 18+ (LTS 20.x or 22.x recommended for prebuilt SQLite binaries), Python 3 plus Git for the RAG scraper, and a running local LLM server. If the fullscreen TUI renders poorly, start with node bin/smallcode.js --classic. For RAG indexing, the README documents npm run rag:index with an optional --preset broad flag and a .smallcode/rag/repos.json config file. Full LM Studio and llama.cpp setup, UI walkthrough, and RAG configuration are in docs/rag-harness.md.