The open-source repos of quirq-ai619 folders111 articles24 agents13 repositories
A field guide in five chapters · Edition of 3 October 2026
How the folders on your machine become a search engine and an encyclopedia, and how to add to both.
Sources become JSON indexes. Two readers, a search engine and an encyclopedia, turn those records into pages you can browse.
Folders on disk flow into the indexer, which writes one file, data/index.json. Two readers, search and Innerpedia, read that file and show it to you.
pnpm index into data/index.json; the server reads that one file twice, as search results and as Innerpedia pages, and you read both.Innernet is a personal internet. Its web starts with your folders and can include public GitHub repositories. Its search engine finds them the way you find pages, and its encyclopedia, Innerpedia, gives every project an article and every other folder a stub. Your local data stays here.
For local folders, pnpm index walks the configured roots, reads a handful of small files in each folder, and writes data/index.json through a temporary file and a rename, so the server never sees half of one. The server checks the file's modification time on every request and reloads it when it changes: no restart, no file watcher.
A folder tree drawn as rings of depth from 0 to 6. Pruned folders such as node_modules are struck through, and only a few files in each folder are read.
The crawler starts at each root and goes down, one folder at a time, sorting subfolders by name. A folder gets a page at every depth from the root at 0 to 6; the limit is inclusive, so there are seven levels of pages. A folder at the limit that still has folders inside keeps a tally of what lies below (8 of them do), so it is never called empty.
| Depth | Pages |
|---|---|
| 0 | 1 |
| 1 | 13 |
| 2 | 101 |
| 3 | 139 |
| 4 | 175 |
| 5 | 132 |
| 6 | 58 |
It never follows a symlink, skips any folder holding pyvenv.cfg (a Python virtualenv by any name), and prunes names that are build output, caches or dependencies. Pruned names that do not start with a dot, and .git, are listed on the parent's page as left out. A dot folder belongs to an agent (.claude, .codex, .cursor, .xo) and is read like any other, as the agents' half of the index; only the dot folders a tool generates (.next, .turbo, .venv) and those that hold credentials (.ssh, .aws) vanish without a word.
const PRUNE = new Set([ "node_modules", "bower_components", "jspm_packages", "vendor", ".git", ".hg", ".svn", ".next", ".nuxt", ".svelte-kit", ".turbo", ".vercel", ".cache", ".parcel-cache", ".output", "dist", "build", "out", "coverage", "target", "DerivedData", "Pods", "__pycache__", "venv", ".venv", "site-packages", ".mypy_cache", ".pytest_cache", ".ruff_cache", ".gradle", ".dart_tool", ".idea", ".expo", "storybook-static",]);In each folder it opens three kinds of file at most: the README (its first 14,000 bytes), CLAUDE.md or else AGENTS.md (the first prose paragraph of its first 20,000 bytes), and one manifest (package.json, then pyproject.toml, Cargo.toml, go.mod, requirements.txt). A folder with its own .git also gets five git commands, ten seconds each at most. An agent's own folder also has its instruction and memory files read (CLAUDE.md, AGENTS.md, SOUL.md, rules, memory notes), and its sessions counted by name, size and date. Every other file is only counted: its size, its dates and its extension, which becomes a language.
A folder's ingredients mapped to the parts of an article: the README to the lead and Overview, package.json to the infobox and Technology, .git to History, CLAUDE.md to the agent line, and the files to the languages bar.
.git the History, CLAUDE.md a line for the agents, and the files themselves the languages bar.Every folder becomes one record in the index. Each gets a kind, decided by the first rule that matches, and becomes either an article or a stub. Repositories, projects, document folders of four files or more, folders with agent notes and the roots are articles; everything else is a stub, a short page that asks politely for a README.
| Kind | What makes it, first match wins | Articles | Stubs |
|---|---|---|---|
| project | A manifest, pubspec.yaml, or a README of 25 words or more. Agent notes promote any non-repo to this. | 26 | 0 |
| repo | Has its own .git, a folder or a file. | 13 | 0 |
| docs | Two or more files, at least 60% documents. An article from four files. | 47 | 18 |
| code | Named like source (src, lib, app, components...) or at least half code. | 0 | 288 |
| folder | None of the above. | 1 | 152 |
| assets | Three or more files, at least 60% media; or media alone and no subfolders. | 0 | 50 |
Then the records are joined up. Each page learns its parent and children and the nearest enclosing project (its partOf, set on 598 pages). Articles are filed into categories, 43 of them, and given up to eight See also links by how much they share: frameworks, dependencies, categories, words in their names. Folders that share a name are told apart, which chapter two explains. Last, normalizeIndex blacks out anything that looks like a credential, sets the text in house style, and rolls dates up the tree, before the file is written.
Kind and article: scripts/build-index.ts:1092 to scripts/build-index.ts:1100. The write: scripts/build-index.ts:1305.
{
"slug": "innernet","kind": "repo","isArticle": true,"depth": 1,"languages": ["TypeScript 146", "JavaScript 47", "Markdown 13"],"frameworks": ["Next.js", "React", "Tailwind CSS"],"totalFiles": 334,"markers": ["git", "package.json", "CLAUDE.md", "AGENTS.md"],"summary": "Innernet turns the folders on your machine into a small, connected web. Find a project by…","categories": ["TypeScript projects", "Next.js", "React", +4],"related": ["quirq_ai", "quitter", +6]…}Innernet turns the folders on your machine into a small, connected web. Find a project by its name, language, framework or README text, then open its Innerpedia article to see what it does, how it is organised and how it has …
innernet is a TypeScript Next.js application at the top level of github.com/quirq-ai. It was created in October 2026 and has not been touched since. It holds 334 files across 58 folders, 98 MB in all. Its Git history runs to 14 commits by two authors, all in October 2026. It keeps instructions for coding agents in CLAUDE.md and AGENTS.md. Cursor keeps a folder of its own here, .cursor.
Read the articleThe first reader is a search engine. On the first search after the file changes, the server builds a full-text index of every page with MiniSearch, which takes a quarter of a second or so; after that a search takes a few milliseconds. It answers in the manners of the web: “About 36 results (0.004 seconds)”, blue links that turn violet once visited, a knowledge panel for a clear winner, and a box that suggests pages as you type. Chapter three is about using it well.
The second reader is Innerpedia, which writes each record up the way Wikipedia would: a lead in encyclopedia voice, an infobox, contents, hatnotes for namesakes, categories at the foot. Every kind of page is served by one route.
| Page | Example | Holds |
|---|---|---|
| Main page | /wiki | Featured article, In the news, Did you know, On this day, categories. |
| Article | /wiki/innernet | Projects, repositories, document collections and the roots. |
| Stub | /wiki/rules | Every other folder: a lead, its contents, a request for a README. |
| Disambiguation | /wiki/docs | Folders that share a name, grouped by the project they sit in. |
| Category | /wiki/Category:Next.js | Collections, languages, frameworks, years, kinds, maintenance, parts. |
| Special | /wiki/Special:Statistics | Random, AllPages, Categories and Statistics. |
A site on your internet is a folder. Give one a README, a manifest, a history or a note for the agents, and Innerpedia writes it an article.
Every folder the crawler reaches already has a page: a stub, which says what the folder holds and asks for a README. Any one of a few small files turns it into an article with a lead, an infobox and sections. Which ones, and what each adds, is easiest to see by trying.
Choose what the folder holds, and the figure answers with the indexer's own rules: its kind, whether it earns an article, the lead Innerpedia would write, where its summary comes from, which rows the infobox shows, which sections appear, how it is filed and how search sees it.
~/Programming/side-projects/weather-station/
weather-station: an article, kind project, at /wiki/weather-station.
/wiki/weather-station
weather-station is a TypeScript Next.js application in the side-projects folder. It was created in October 2026. It holds five files, 4.6 KB in all.
A small weather station for the balcony. It reads a sensor once a minute, keeps a week of readings in a file, and draws them as one calm line.
kind:project is:article is:project lang:typescript lang:json lang:markdown fw:next fw:react in:side-projects. Ranking prior ×2.05, and ×4 when its name is the query.Rules mirrored from scripts/build-index.ts:1092 (kind), scripts/build-index.ts:1100 (article or stub), scripts/build-index.ts:1113 and lib/normalize.ts:56 (summary), scripts/build-index.ts:1142 (names), scripts/build-index.ts:1208 (categories), components/wiki/article/lead.ts:257 (the lead) and lib/search.ts:184 (prior). The recipe assumes a folder with no subfolders, in plain folders that are not a project and hold fewer than three articles, touched today, and namesakes that keep the names they have now.
Five steps from left to right: a folder, a README, a manifest, a git history and pnpm index, ending in a new Innerpedia article.
Anywhere under a root, at most 6 folders down. Nothing shows yet: the index is a snapshot, taken when you ask for one.
mkdir -p ~/Programming/weather-stationTwenty-five words make a project, and an article. The first paragraph with forty letters or more becomes the summary: the lead's second paragraph, the snippet under the search result, the knowledge panel. The Overview section renders the whole README.
cd ~/Programming/weather-stationprintf '# weather-station\n\nA small weather station for the balcony. It reads a sensor once a minute, keeps a week of readings in a file, and draws them as one calm line.\n' > README.mdDependency names become frameworks: next files it under Next.js and calls it a Next.js application, react adds React. The infobox gains a Package group, Technology lists scripts and dependencies, and the description stands in when the README has no summary. Every folder inside now belongs to it.
pnpm initnpm pkg set description="Reads a balcony sensor and charts the week" dependencies.next="^16.0.0" dependencies.react="^19.0.0"A repository: a round sigil, a History section with its commits, a place in the Repositories tab and under kind:repo, and a quarter more weight in the ranking.
git init && git add -A && git commit -m "First light"A minute or so. Until the run ends, /wiki/weather-station says “Nothing here, yet”; the running server picks up the new file on the next request, and the article is there.
cd innernetpnpm indexAny one of steps two to four is enough. So is a CLAUDE.md that opens with a paragraph of prose, forty letters or more, or four files side by side when at least three in five of them are documents.
Some names are never read. The crawler skips a dot folder a tool generates, such as .next or .venv, a pruned name such as dist, build or vendor, a name ending .app, a symlink, and a folder holding pyvenv.cfg. Every other dot folder, .claude or .xo, is an agent's, and gets an agent's page. A name containing cred, secret or private, or ending in key or keys, still gets a page, but its README, notes and manifest are never opened.
scripts/build-index.ts:43 and scripts/build-index.ts:51.
A root is a folder the crawl starts from. The roots and the depth live in innernet.config.json, at the top of this project. Add a path to the list and index again; or name the roots for one run with INNERNET_ROOTS, which replaces the list rather than adding to it.
{ "roots": ["~/Programming"], "maxDepth": 6}INNERNET_ROOTS=~/Programming,~/Documents/notes pnpm indexEach root becomes an article at depth 0, typed Root folder, and always the primary topic for its name. Its own name is not part of any path below it, so in: with the root's name does not list what the root holds, only what sits under some other folder of that name; search for what is inside instead. The footers list every root. The local index lives at data/index.json in this project, so a trial index written elsewhere with INNERNET_OUT does not change what the app serves. Remote takes a list of public GitHub repositories from any account, and each list has its own data/github-<hash>.json snapshot, made by its first sync.
Folder names become slugs. A shared name gives one primary topic the bare slug, the others get qualifiers in parentheses such as src (linear-clone), and a disambiguation page lists them all.
A folder's address is its name, with spaces turned into underscores: /wiki/innernet. Names are compared without regard to case, and when several folders share one (as 140 names do here), Innerpedia tells them apart the way Wikipedia does.
If one of them is clearly the main one, it is the primary topic and keeps the bare name: a root, or else the shallowest article, provided no other article sits at its depth and every other namesake lies deeper (23 names have one). Each of the rest takes the name of the folder above it in parentheses, growing to two folders when that is taken, and the crawl order decides who gets the shorter one.
16 folders are named docs. The shallowest is an article with every other deeper, so it keeps the bare name and opens with a hatnote:
For the 15 other folders named docs, see docs (disambiguation).
Some folders named fonts sit under folders that share a name too. The one the crawl reached first took the short qualifier; the next had to add a second folder.
8 folders are named src and none stands out, so /wiki/src is the disambiguation page itself, grouped by project.
and 4 more
You can help Innerpedia. These are the largest pages still written from the folder alone. A README of twenty-five words gives each a summary and an Overview, turns a stub into an article, and takes an article out of Articles lacking a README (48).
Largest first, by files inside. Folders within another project are left out; that project's README speaks for them.
The box takes plain words, the way the web taught us. A few small operators, and a little knowledge of the ranking, make it a precise instrument.
Press / anywhere to reach the search box. It suggests up to seven pages as you type, by name, title, summary and framework; arrow keys pick one, Enter searches everything. Results come ten to a page, in five overlapping tabs: All, Projects (articles that are not document collections), Repositories, Documents and Folders (every stub).
An operator is a word with a colon. It narrows the results to pages that pass its test, and it can stand alone: kind:repo on its own lists every repository. Mix them with words and with each other.
| Operator | Finds | Try |
|---|---|---|
| lang: | Pages whose top three languages, counted over everything inside, include it. ts, js, py, rs, sol, rb and sh stand for the full names. | |
| in: | Pages with a folder above them, below the root, whose name contains it. The page's own folder never counts, so this lists what is inside. | |
| kind: | Exactly one of repo, project, docs, assets, code or folder. Nothing else matches, plurals included. | |
| fw: | A framework whose name, without dots or spaces, contains it, so both examples are the same search, and fw:react also finds React Native. | |
| is: | Articles only, or stubs only. Any other value filters nothing. |
Operator names may be in any case; values are read in lower case. If one is given twice, the last wins, and there is no negation: agent -in:x is the same search as agent in:x. Operators filter after the words are scored, so they never change the order, only what is left. With no words at all, every page that passes is listed in the order of its prior, which the next section explains.
lib/search.ts:136 and lib/search.ts:151.
A query is split into tokens, looked up in the inverted index, weighted by field, multiplied by a prior for each page, and laid out as results with a knowledge panel. Operators filter the list.
A query is lowercased and split at spaces and punctuation, so linear_clone, LINEAR-CLONE and clone linear are the same search. Every word must match. Words of two letters or more match as prefixes, and words of five or more forgive typos: one slip in five to seven letters, two in eight to twelve. When fewer than five pages match every word, pages that match some of them follow below.
Each hit is then weighed by where the words were found. A match in a folder's name counts ten times one in its README.
That score is multiplied by a prior, a sense of how much each page is likely to matter before any word is read: articles over stubs, repositories a little higher, source folders lower, shallow over deep, the recently touched over the long untouched. A folder whose name is the query counts four times over, one whose name begins with it 1.6 times. Last, results that matched every word, and articles whose name holds a rare word of the query, sit above those that matched only some.
function prior(p: Page): number { let w = p.isArticle ? 1.8 : 1; if (p.kind === "repo") w *= 1.25; if (p.kind === "code" && !p.isArticle) w *= 0.55; if (p.partOf) w *= 0.8; // A word shared with a hundred .xo folders should not bury the project it names. if (p.realm === "agent") w *= p.kind === "agent" ? 0.7 : 0.5; w *= 1 / (1 + 0.05 * p.depth); if (p.modified) { const days = (Date.now() - Date.parse(p.modified)) / 86_400_000; w *= 1 + 0.25 * Math.exp(-days / 60); }When a word of four letters or more appears nowhere in the index, the engine looks for the nearest word in an article's name, one slip away for short words and two for long ones, a swap of neighbours counting as one. It offers the fix only when the fixed query finds at least twice as many pages that match every word. With no results and no fix, it suggests the nearest article name instead.
The knowledge panel, the card beside the results, appears for the top result when it is an article and either its name is the query, give or take spaces, dashes and underscores, or it scores well clear of the next. It shows only on the first page of All, and a spelling suggestion hides it.
Under the results, related searches offer up to eight refinements by framework, language or folder, each kept only when it would narrow the list without emptying it. They are the quickest way to learn the operators by accident.
lib/search.ts:413. Knowledge panel: lib/search.ts:265. Try any of these from a terminal with pnpm tsx --conditions=react-server scripts/try-search.ts "linear clone".
Innernet is small enough to hold in your head: one crawler, one file, one search engine and an encyclopedia drawn from it.
It is a Next.js 16 app in TypeScript, served by the same machine it describes. Pages are Server Components that read the index directly; the browser receives HTML and a handful of small client components, each doing something only a browser can. The setup is three commands:
pnpm installpnpm indexpnpm devThe project's folders and how data flows between them: scripts write the index, lib reads and searches it, app routes call lib, components render, and proxy.ts stands in front.
scripts/ writes the index, lib/ reads and searches it, app/ asks lib/ and hands the answer to components/, and proxy.ts stands at the door.The crawler: walks the roots, reads, classifies, names, links and writes the index.
Public GitHub indexing for the demo or local Remote source, with separate output and clone caches.
The search core from a terminal.
Headless screenshots of the running app, with the culprits of any sideways scroll.
The index contract: Page, IndexMeta, SiteIndex.
Markdown to text, first paragraphs, house style, redaction. Shared with the indexer.
Brings any index up to today's rules, at write and at load.
Loads the index, reloads it when it changes, resolves every /wiki slug.
Saved Local and Remote choices, configuration and index paths.
Validation of the editable GitHub account and optional repository list.
The Sources page's storage locations and file-manager and editor actions.
MiniSearch, operators, the prior, snippets, did you mean, suggestions.
wikiHref, categoryHref, searchHref, vscodeHref.
The history's folders: one per session, one JSON Lines file per app.
The database: PGlite on this machine, Neon on the demo, the same SQL for both.
Home: the wordmark, the box, recently touched, then this guide.
History: every session, newest first, merged across apps.
Results, tabs, the knowledge panel.
Every Innerpedia page but the Main page, through resolveSlug.
Suggestions for the search box, as you type.
Writes the history: to this machine's folders, or on a demo with a database, to Neon for 30 days. Same origin only.
Local source inspection, selection, sync and named locations. Same origin only.
Connects, syncs, tests and disconnects the remote database (lib/db/remote-sync.ts). Same origin only.
Store now, from the history page: the index and the history into this machine's database.
The tokens, both themes, the aurora, prose.
The front page's pieces.
Results, chips, tabs, the panel, pagination.
Articles, stubs, categories, the Main page, specials.
This guide, its plates and its recipe, bound in field-guide.tsx.
Every folder's identity: its own logo, or a sigil drawn from its slug.
Refuses any request not addressed to localhost.
The Content-Security-Policy and the other headers.
The roots and the depth.
How it should look and sound.
How to work on it.
The indexer imports only lib/text.ts, lib/normalize.ts and lib/types.ts, so it runs outside Next. lib/data.ts and lib/search.ts import server-only: they never reach the browser. File names open on GitHub.
The changes people make most, with the exact places to make them. The excerpts are read from the code as this page is served, so they are never out of date.
Say, owner: for the author of a repository.
lib/search.ts, add the key to ParsedQuery.filters (:32) and OPS (:134), to the operator regex in parseQuery() and to its copy in correction(), and write its test in matchesFilters() (:151).components/search/query-tools.ts, add it to OP_RE and give it a label in OP_LABEL. That is a Record, so typecheck fails until you do.EXAMPLES in components/home/example-queries.tsx and to README.md and DESIGN.md.const OPS = ["lang", "in", "kind", "fw", "is"] as const; export function parseQuery(raw: string): ParsedQuery { const filters: ParsedQuery["filters"] = {}; const text = raw .replace(/\b(lang|in|kind|fw|is):("([^"]+)"|\S+)/gi, (_, op: string, v: string, quoted?: string) => { const key = op.toLowerCase() as (typeof OPS)[number]; filters[key] = (quoted ?? v).toLowerCase();const OP_RE = /\b(lang|in|kind|fw|is):("([^"]+)"|\S+)/gi; export const OP_LABEL: Record<OpKey, string> = { lang: "Language", in: "Inside", kind: "Kind", fw: "Framework", is: "Is",};Say, Special:Oldest, the articles nobody has touched in years.
/wiki/Special:Name already reaches SpecialView.app/wiki/[slug]/page.tsx before anything renders.SpecialView (components/wiki/special-view.tsx:21; branches test key, the name lowercased without spaces or underscores) and an entry in SPECIALS. Build it like StatisticsView, with PageTitle prefix="Special: ", and keep heavy work in insights.ts behind once().LINKS in components/wiki/wiki-shell.tsx and AREAS in components/wiki/main/browse.tsx.export function SpecialView({ name }: { name: string }) { const key = name.toLowerCase().replace(/[\s_]/g, ""); if (key === "allpages") return <AllPages />; if (key === "statistics") return <StatisticsView />; if (key === "categories") return <Categories />; return <Unknown name={name} />;}Say, TanStack Query, from @tanstack/react-query.
["package-name", "Display Name"] to FRAMEWORKS in scripts/build-index.ts:73. It matches a dependency or devDependency name exactly.[/\bname\b/i, "Display Name"] to PY_FRAMEWORKS, a few lines further down scripts/build-index.ts:89. It matches the dependency names joined together.NOUNS in components/wiki/article/lead.ts and to APP_FRAMEWORKS in components/search/query-tools.ts.fw:, the search weight, See also, the infobox and the result chips all follow.const FRAMEWORKS: [string, string][] = [ ["next", "Next.js"], ["react", "React"], ["vue", "Vue"], ["svelte", "Svelte"], ["@sveltejs/kit", "SvelteKit"], ["astro", "Astro"], ["vite", "Vite"], ["three", "Three.js"], ["@react-three/fiber", "React Three Fiber"], ["tailwindcss", "Tailwind CSS"],Say, Releases, from a folder's tags.
Page in lib/types.ts, fill it in crawl() in scripts/build-index.ts, and backfill it in normalizeIndex() when it can be worked out without the disk, so older indexes still load.components/wiki/article/<name>.tsx exporting hasX(page) and X({ page }), patterned on technology.tsx, with Sub from parts.tsx for labelled blocks.components/wiki/article-view.tsx, add the id to SECTION_IDS so a README heading cannot claim it, push a contents item, and render <Section id title> where it belongs. Stubs have their own view, stub-view.tsx.const SECTION_IDS = ["top", "overview", "instructions", "sessions", "structure", "technology", "history", "see-also", "external-links", "contents"];if (hasStructure) items.push({ id: "structure", label: "Structure" });if (hasTechnology(page)) items.push({ id: "technology", label: "Technology" });if (git) items.push({ id: "history", label: "History" });if (page.related.length) items.push({ id: "see-also", label: "See also" });items.push({ id: "external-links", label: "External links" });export function hasTechnology(p: Page): boolean { return p.languages.length > 0 || p.frameworks.length > 0 || !!p.manifest;}A loop of five stations: edit, typecheck, crawl, screenshot, pull request, and back to edit.
There is no lint, test or format tooling. Edit, let hot reload show you the page, then run the check for the part you touched. The dev server must run from this folder, because it reads the index relative to where it starts.
pnpm -s typecheckA clean exit in about a second.
pnpm tsx --conditions=react-server scripts/try-search.ts "linear clone"Results, the knowledge-panel pick, did you mean and suggestions.
scripts/shot.sh "/wiki/innernet" out.png 1440 2400 darkA settled screenshot, and the elements that scroll the page sideways, if any.
INNERNET_ROOTS=~/some/small/dir INNERNET_OUT=/tmp/innernet-test.json pnpm indexA test index; the real one is left alone.
INNERNET_DB_DIR=/tmp/innernet-db pnpm db:storeThe index and the history stored in a scratch copy; then pnpm db:status with the same folder.
rg -n "[\x{2013}\x{2014}]" -g '!node_modules' -g '!.next' -g '!data' .No matches: no em or en dashes anywhere.
try-search needs --conditions=react-server because the search core imports server-only. shot.sh talks to port 3470 unless SHOT_BASE says otherwise; its last arguments are the height and the theme, SHOT_SCROLL captures the window scrolled to an element, and SHOT_EVAL runs a snippet (a click) in the page first.
Pages are Server Components. The browser asks only this app: the search box asks /api/suggest, the history posts each page to /api/activity (on a demo with a database, its history page also reads and clears the visitor's own sessions there), and the local Sources page uses /api/sources, /api/sources/sync, /api/sources/open and /api/storage. These routes inspect or save choices, refresh the selected sources, open named locations on this machine, and connect or sync the remote database. Remote sync downloads only the public repositories listed; the index and history leave this machine only while a remote database is connected. A new client component needs a reason only the browser can satisfy.
text, normalize, types, links, format and lang-colors in lib/ stay free of server-only and of Next, so the indexer and the browser can use them.
Colours live in app/globals.css, in four places: :root, the dark media query, [data-theme="dark"] and @theme inline. Components use the utilities, never hex. Check both themes.
Slugs hold spaces, parentheses and commas. wikiHref, categoryHref, searchHref and vscodeHref encode them for you.
Snippets and leads are arrays of segments rendered as React nodes; READMEs go through react-markdown with skipHtml. Nothing from the index goes near dangerouslySetInnerHTML.
Encyclopedia voice in article prose. Plain, warm, a little wry. No exclamation marks, and no em or en dashes anywhere: commas, colons, middots or full stops.
Read from CONTRIBUTING.md:238 as the page is served. Ticks last as long as the page.
Innernet is small enough to hold in your head: a crawler, local JSON indexes, one search engine and an encyclopedia drawn from them. This page is how to work on it. For how it behaves (the crawl rules, ranking, operators, every kind of Innerpedia page), read the field guide on the home page at /#guide. For how it should look and sound, read DESIGN.md.
pnpm install
pnpm index # crawl ~/Programming into data/index.json (about a minute)
pnpm dev # http://localhost:3470, bound to 127.0.0.1 only
pnpm dev or pnpm build needs the network once, to fetch the fonts. After
that next/font serves them locally and the browser never leaves localhost./sources) beside Guide. Under Input, choose Local,
Remote, or both. Remote takes public GitHub repository links from any account, one per
line (https://github.com/owner/name or owner/name), up to 50; there is no
whole-account mode. Save, then Sync now. The choices persist in
data/sources.json; Sync now refreshes the selected sources and logs its start and
outcome in the current activity session, as saving is logged. pnpm index remains the
local-only command.
For command-line rebuilds, the server notices the new file on the next request; there
is nothing to restart.Edit, let hot reload show you the page, then check the part you touched. There is no lint, test or format tooling; these are the checks.
| What you changed | Check | Expect |
|---|---|---|
| Anything | pnpm -s typecheck | a clean exit, about a second |
| Search, ranking, operators | pnpm tsx --conditions=react-server scripts/try-search.ts "linear clone" | results, the knowledge-panel pick, did-you-mean and suggestions, printed |
| Anything you can see | scripts/shot.sh "/wiki/innernet" out.png 1440 2400 dark | a settled screenshot, and the names of any elements that scroll the page sideways |
| The indexer | INNERNET_ROOTS=~/some/small/dir INNERNET_OUT=/tmp/innernet-test.json pnpm index | a test index, with your real one left alone |
| The database | INNERNET_DB_DIR=/tmp/innernet-db pnpm db:store, then pnpm db:status with the same folder | the index and history stored in a scratch copy, the real one left alone |
| The history | with the dev server stopped, INNERNET_HISTORY_DIR=/tmp/h INNERNET_DB_DIR=/tmp/hdb pnpm dev; open a few pages, append a line to /tmp/h/<session>/notes.jsonl and reload /activity | the line shows, once, however often the page is read; the real history left alone |
| Sources | visit /sources; check Local, Remote and both; add repository links from two accounts, save, reload, sync; read Generated data; connect, sync and disconnect the remote with a scratch INNERNET_HOME | choices persist, each repository list has its own snapshot, generated paths are relative, a switch to remote asks to confirm and copies the index and history, and the page scrolls as a document |
| Copy | rg -n "[\x{2013}\x{2014}]" -g '!node_modules' -g '!.next' -g '!data' . | no matches |
A few things worth knowing about these tools:
try-search needs --conditions=react-server, because lib/search.ts and
lib/data.ts import server-only. Any terminal script that imports them does too.shot.sh talks to http://localhost:3470 unless SHOT_BASE names another server.
Its last two arguments are the height and light or dark (it sets
prefers-color-scheme). A narrow width gives the narrow layout but not a phone user
agent. Pass a tall height to see a whole page, or SHOT_SCROLL=#guide (an element or
a number of pixels) to capture the window scrolled there, as the home page's hand-off
to the field guide needs. SHOT_EVAL runs a snippet in the settled page first, such as
a click that opens Sources' connect notice.INNERNET_OUT. Inspect that file with
node, or point INNERNET_ROOTS at a small folder and let it write the real one when
you are happy to rebuild. Terminal scripts can search it, though: lib/data.ts reads
data/index.json under the working directory, so write the test index to
/tmp/t/data/index.json and run try-search from /tmp/t, with this folder's
node_modules/.bin/tsx --conditions=react-server and this folder's scripts/try-search.ts.INNERNET_MAX_DEPTH=3 overrides the depth in innernet.config.json for one run, as
INNERNET_ROOTS does the roots..env.neon.local:
set -a; . ./.env.neon.local; set +a; INNERNET_DEMO=1 pnpm build, then the same
prefix for pnpm exec next start -p 3491 -H 127.0.0.1 or pnpm db:status --demo.
Never copy it into .env or .env.local, which Next.js would load. A build that sees
it keeps no Turbopack cache (next.config.ts), since that cache saves the build's
whole environment. Test visits on the demo use ids of the demo's shape; clear them with
the history page's Clear button, or DELETE /api/activity, before you finish.scripts/crawl.mjs
would be a welcome first pull request.Popups never scroll. Popups, modals, dialogs and popovers are only for brief
information or small, focused confirmations that fit without scrolling, including on
phones and at increased text size. If content needs scrolling, put it on a normal
page. Do not clip, hide or shrink content to fit an overlay. Source choices and
storage locations belong on /sources. Check narrow and wide screens when changing
an overlay; the app-wide rule is in
.cursor/rules/non-scrolling-popups.mdc.
Server first. Pages are Server Components that read lib/data.ts and
lib/search.ts. The client components are few and each has a reason only the browser
can satisfy: the search box, the home hero search, the theme toggle and the contents
rail; the field guide's copy button, folder recipe, reveal motion and progress ruler;
the Innerpedia globe and the reveal of the boxes below it; and the history's recorder,
the header's back and forward buttons, the local Sources page controls, and the history
page's two views of this browser. The search box asks /api/suggest, the recorder posts
each page visited to /api/activity. Sources posts to /api/sources to inspect or
save Local/Remote choices and the repository list, /api/sources/sync to rebuild the
saved selection, /api/sources/open to reveal or edit named locations, and
/api/storage to read, test, connect, sync or disconnect the remote database. All include the current session ID
and require the same origin on localhost. Local sync runs the pnpm index crawler into
data/index.json; Remote runs the anonymous public GitHub crawler with
INNERNET_REMOTE_CACHE=.github-cache, INNERNET_GITHUB_REPOSITORIES (a JSON list of
owner/name) and an INNERNET_REMOTE_OUT of data/github-<hash>.json, one per list.
A list never serves another list's snapshot, and the bundled demo index never stands in
locally. Both selected means an in-memory combination of the indexes. This machine's
PGlite is always the database in use; a remote one connected on Sources (lib/storage.ts,
~/.innernet/remote.json) is kept in step with it both ways by lib/db/remote-sync.ts,
which only adds to the remote and deletes there only what was deleted here
(lib/db/tombstones.ts). scripts/try-remote-sync.ts runs two machines and a remote as
PGlite folders, with no network. The demo's database is refused as a remote.
Saving choices records a sources event; sync records sync events with started,
completed or failed status. Concurrent syncs and source changes during a sync return
409. Sources is hidden on the demo and its routes return 404 there. Generated data shows
relative paths with copy and open actions, plus edit for this tab's history file; the
folder settings open from Input. Generated pages come from JSON,
not individual HTML files. On the demo the activity post is made only when the demo has a database, and its history
page then reads and clears the visitor's own sessions on the same route (a POST of
{"read": [ids]} and a DELETE of {"sessions": [ids]}: ids never go in an address);
without one the demo keeps history in localStorage and sends nothing. The history
page's Store now is a plain form posting to /api/db/store, local only. Keep it that way: no other browser fetches, no external browser requests, no new client
components without a reason only the browser can satisfy.
Shared code stays shared. The indexer imports lib/text.ts, lib/normalize.ts and
lib/types.ts, and lib/links.ts, lib/format.ts and lib/lang-colors.ts are meant to
be importable from anywhere. Keep all six free of server-only and of Next imports.
Tokens, not values. Colours live in app/globals.css. A new token goes in four
places: :root, the dark @media block, :root[data-theme="dark"], and @theme inline. Components use the utilities (bg-surface, text-muted, border-line), never
hex. Check both themes.
Links through lib/links.ts. wikiHref, categoryHref, searchHref and
vscodeHref handle encoding, underscores and empty parameters for you.
Index text is never HTML. Snippets and leads are segment arrays rendered as React
nodes; READMEs go through react-markdown with skipHtml. Nothing from the index goes
near dangerouslySetInnerHTML.
Privacy is a feature. The indexer reads READMEs, CLAUDE.md or AGENTS.md, manifests,
git metadata and a project's own logo image (found by name), and nothing else. If you need a new kind of input, read names rather
than contents where you can, and run anything textual through redactSecrets and
cleanLine in lib/text.ts. The database keeps to the same line: this machine's copy
stays in PGlite on this machine (lib/db/neon.ts refuses local mode outright), and the
demo writes to Neon only the index and what lib/db/demo-history.ts lists. Anything new
a demo keeps about its visitors must be anonymous, capped, gone within 30 days and
clearable by the visitor, and README.md must say so.
House style. Encyclopedia voice in article prose ("is a", "was created in"). Plain, warm, a little wry. No exclamation marks. No em or en dashes anywhere, in copy, comments or code: use commas, colons, middots or full stops.
lib/search.ts, add the key to ParsedQuery.filters and OPS, to the operator
regex in parseQuery(), to the copy of that regex in correction(), and its test to
matchesFilters().components/search/query-tools.ts, add it to OP_RE and give it a label in
OP_LABEL. Typecheck will remind you about the label.EXAMPLES in
components/home/example-queries.tsx and to the operator lists in README.md and
DESIGN.md.Values arrive lowercased and may be quoted (in:"two words"). If you add lang:
aliases, add them to both LANG_ALIASES copies (lib/search.ts and
components/search/query-tools.ts).
/wiki/Special:Name reaches SpecialView.app/wiki/[slug]/page.tsx
before rendering.SpecialView in
components/wiki/special-view.tsx (branches test key, the name lowercased with
spaces and underscores removed), add it to SPECIALS in the same file, and build the
view like StatisticsView with PageTitle prefix="Special: ". Put heavy computation
in components/wiki/main/insights.ts behind once().LINKS in components/wiki/wiki-shell.tsx, as
wikiHref("Special:Name"), and from AREAS in components/wiki/main/browse.tsx,
as slug: "Special:Name".["package-name", "Display Name"] to FRAMEWORKS in
scripts/build-index.ts. It matches a dependency or devDependency name exactly.[/\bname\b/i, "Display Name"] to PY_FRAMEWORKS.NOUNS in
components/wiki/article/lead.ts (most specific first) and to APP_FRAMEWORKS in
components/search/query-tools.ts.fw: operator, search weight, See also, the infobox and
the result chips all follow.Page in lib/types.ts, fill it in
crawl() in scripts/build-index.ts, and backfill it in normalizeIndex() if it can
be worked out without the disk (older indexes still load).components/wiki/article/<name>.tsx exporting hasX(page) and X({ page }),
patterned on technology.tsx, with Sub from parts.tsx for labelled blocks.components/wiki/article-view.tsx, add the section id to SECTION_IDS (so a
README heading cannot claim it), push a contents item in order, and render
<Section id title> where it belongs. A section shows only when it has something to
say.components/wiki/stub-view.tsx.pnpm -s typecheck passes.shot.sh names the culprit if there is one).node, before a full
pnpm index.Innernet is built by quirq infra (qq). Its manifest, infra/repo.toml,
pins the Node toolchain and declares one node-app target; read or change it only with
qqsync from quirq-ai/sync, never by hand. The
qq-innernet-* workflows are generated by
quirq-ai/infra-config; change them there, not
here. Every pull request into main needs the innernet-presubmit check, which runs the
install, pnpm build and pnpm typecheck (for now with Node 24 and pnpm 10 on the runner,
until CI moves to the pinned toolchain). Each push to main runs the same steps as
innernet-postsubmit.
Dependency build scripts pnpm may run are listed under allowBuilds in
pnpm-workspace.yaml. pnpm 11 refuses an install with one that is not listed, so a new
dependency with a build script needs an entry there.
data/index.json (with
INNERNET_DB=off, or the stored copy serves instead).lib/links.ts, index text as React nodes.data/ and .next/ left out of the commit.Innernet reads very little, keeps it on this machine, blacks out anything that looks like a key, and answers to no one but you.
A search engine over your own folders sees a great deal that was never meant to be published. Innernet is built so that the index holds as little as it needs, and so that the little it holds never leaves the room.
This machine drawn as a walled enclosure holding the index, search and Innerpedia. One gate, 127.0.0.1, is the only way in; arrows toward the cloud, telemetry and third parties stop at the wall and are struck through. A dial reads 0 bytes sent.
CLAUDE.md, else AGENTS.md: its first paragraph of prose.package.json, pyproject.toml, Cargo.toml, go.mod or requirements.txt.logo, icon, mark, favicon), within 64 KB for SVG and 96 KB for an image..claude, .codex, .xo...), its instructions and memory: CLAUDE.md, AGENTS.md, SOUL.md, its rules and its memory notes, the first 8,000 characters of each, ten files at most..env: not even counted, only noticed when one is .git. Dot folders a tool generates (.next, .turbo, .venv) are skipped, and those that hold credentials (.ssh, .aws, .clerk) are never entered.[redacted]: private keys, passwords in URLs, Stripe, Anthropic, OpenAI, GitHub, Slack, AWS and Google keys, JWTs, Bearer tokens, and long random KEY=value pairs..env, *.pem, id_rsa and anything with secret, token or password in it are never listed.Redaction runs twice: when the index is written, and again whenever the server loads it, so an index written by an older version is cleaned by the rules of today. Agent notes that held a credential are dropped altogether. These three lines were run through the same function as this page was served:
DATABASE_URL=postgres://admin:s3cret-horse-battery@db.internal:5432/appDATABASE_URL=postgres://admin:[redacted]@db.internal:5432/appcurl -H "Authorization: Bearer 9fQ2xLr7Tz4mWk8sPa1v" https://api.example.testcurl -H "Authorization: Bearer [redacted]" https://api.example.testGITHUB_TOKEN=ghp_A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8GITHUB_TOKEN=ghp_[redacted]pnpm dev and pnpm start listen on 127.0.0.1, so no other machine can reach the server. A page on the web can still try a trick called DNS rebinding, pointing a name of its own at 127.0.0.1 and asking from inside your browser. It cannot change the name it asks by, though, and proxy.ts answers anything not addressed to localhost with a 403, on every path, before a single file is read:
const LOCAL = /^(localhost|127\.0\.0\.1|\[::1\]|[\w-]+\.localhost)(:\d+)?$/i; export function proxy(request: NextRequest) { if (DEMO || LOCAL.test(request.headers.get("host") ?? "")) return NextResponse.next(); return new NextResponse("Innernet only answers to localhost.", { status: 403, headers: { "Content-Type": "text/plain; charset=utf-8" } });}In the browser, a Content-Security-Policy holds every page to its own origin; under next dev it also lets the hot-reload websocket through. The fonts are served by the app itself, the pages ask search engines to look away, and browser requests stay on this app: the search box asks /api/suggest for suggestions as you type, the history posts the page you opened to /api/activity, and the local Sources page uses /api/sources to inspect or save choices, /api/sources/sync to refresh them, /api/sources/open to open a named location, and /api/storage to connect or sync the remote database. All require this app's own origin on localhost. The server downloads the public repositories you list, anonymously, when Remote is synced; a sync uploads nothing. The public demo accepts its own visitors' history only when it keeps a database; the next section says what it keeps. Sources is unavailable on the demo.
const frameAncestors = `frame-ancestors ${demoBuild ? "'self' https://www.quirq.dev" : "'none'"}`;const csp = [ "default-src 'self'", `script-src 'self' 'unsafe-inline'${dev ? " 'unsafe-eval'" : ""}`, "style-src 'self' 'unsafe-inline'", "img-src 'self' data:", "font-src 'self'", `connect-src 'self'${dev ? " ws: wss:" : ""}`, frameAncestors, "base-uri 'self'", "form-action 'self'",].join("; ");One caveat: next dev records request URLs, search queries included, in .next/dev/trace. It is gitignored and recreated; delete it whenever you like.
Innernet keeps a copy of its index and its history in a small Postgres database. On your machine it is PGlite, Postgres compiled to WebAssembly, running inside the server on a folder at ~/.innernet/db (folder mode 700). Nothing listens on a port, and local mode refuses the demo's database whatever the environment says, before a byte is sent:
export async function openNeon(): Promise<Db> { if (!DEMO) throw new Error("local mode never connects to the demo's database: this machine's data stays out of it");The history folders stay the record. After each page you open, and whenever the history page is read, the files that changed are read into the database, only the new bytes of each. A session folder you delete, or a line you take out of a file, is forgotten there too. Only when the history folder itself is lost or replaced (the database notes which folder it read) does it keep the sessions that did not come with the new one, so pnpm db:load can write them back. Set INNERNET_DB=off and Innernet runs on its files alone; delete ~/.innernet/db to forget the copy.
Sources lists everything the input generates, each with its path relative to the app or your home folder; this tab's row opens innernet.jsonl for editing, then reload History. Choosing GitHub repositories changes the content you browse; it moves nothing. The copy leaves this machine only if you connect a remote database, your own Neon, which asks you to confirm first: from then on the local index (folder paths, README text, agent instructions) and the history are kept in step with it, and your other machines' history comes down, until you disconnect. Its URL stays in ~/.innernet/remote.json, and the demo's database is refused as a remote.
The public demo is different, and it is the one place anything you do is written to a server. When it has a database (Neon Postgres, in US East), it keeps the pages and searches its visitors open for 30 days: per event, the kind of event, the page's path (none of the rest of its address but the search's own), its title or the search words, whether the back and forward buttons took you there, and the server's time, under a hash of the id the visitor's tab made, which gives back neither the id nor the time in it. Never an IP address, a user agent, a cookie or any other header. Nothing lists sessions, ids travel only in request bodies, so only a browser holding a session's id can read it back, and the Clear button on the history page deletes every one that browser sent. Each session holds at most 400 events and the whole demo stores at most 10,000 a day. A fork without a database keeps every visitor's history in their own browser, as the demo did before.
The demo's rules: lib/db/demo-history.ts:38, its route app/api/activity/route.ts:73. This machine's: lib/db/ingest.ts:245.
Everything in this guide, on your machine. Hand this prompt to an AI assistant and it will clone the repository, read the README and the contributing guide, install with pnpm, index the folders you choose and open your own Innernet at localhost:3470.
Each opens the assistant in a new tab with the prompt below already written.
Help me run Innernet on my own computer. Innernet turns the folders on a machine into a private search engine and an encyclopedia of its projects. The code is at https://github.com/quirq-ai/innernet. Clone it, and read README.md and CONTRIBUTING.md before you change anything. Check that I have Node.js 20.9 or newer and pnpm, then install with pnpm install. Ask me which folders to index, set them as the roots in innernet.config.json, and run pnpm index. Then start the app with pnpm dev and open http://localhost:3470. If you cannot run commands yourself, walk me through each step. Everything stays on my machine: the index is a local file, and the server answers only to localhost.
claude "Help me run Innernet on my own computer. Innernet turns the folders on a machine into a private search engine and an encyclopedia of its projects. The code is at https://github.com/quirq-ai/innernet. Clone it, and read README.md and CONTRIBUTING.md before you change anything. Check that I have Node.js 20.9 or newer and pnpm, then install with pnpm install. Ask me which folders to index, set them as the roots in innernet.config.json, and run pnpm index. Then start the app with pnpm dev and open http://localhost:3470. If you cannot run commands yourself, walk me through each step. Everything stays on my machine: the index is a local file, and the server answers only to localhost."codex "Help me run Innernet on my own computer. Innernet turns the folders on a machine into a private search engine and an encyclopedia of its projects. The code is at https://github.com/quirq-ai/innernet. Clone it, and read README.md and CONTRIBUTING.md before you change anything. Check that I have Node.js 20.9 or newer and pnpm, then install with pnpm install. Ask me which folders to index, set them as the roots in innernet.config.json, and run pnpm index. Then start the app with pnpm dev and open http://localhost:3470. If you cannot run commands yourself, walk me through each step. Everything stays on my machine: the index is a local file, and the server answers only to localhost."Run either in the folder that should hold the clone. Both start an agent with the prompt as its first message.