bun-src/scripts/build/CLAUDE.md

TypeScript Build System

This directory generates build.ninja. The scripts describe the build; ninja performs it.

Goals

Idempotent. Run bun run build over and over — that's it. Same inputs produce the same build.ninja, ninja sees nothing changed, exits immediately. No state drift between runs, no "did I clean first?", no sticky flags from last time. The mechanisms:

• Configure always runs. Every bun run build reconfigures before spawning ninja — no separate first-time configure step, no cached options that persist across runs. Config is profile + overrides evaluated fresh each time. Fast enough to not matter (~200ms — we haven't tried to make it faster yet).
• writeIfChanged() — preserves mtimes on unchanged content, so the always-configure cost is near-zero for ninja: if build.ninja didn't change, ninja doesn't restat the graph
• restat = 1 on fetch/codegen/dep rules — prunes downstream rebuilds when outputs don't actually change
• Self-rebuilding build.ninja — the regen generator rule re-runs configure when running ninja directly and a build script changed

Explicit over implicit. Every decision traceable to a line of code you can grep for. No hidden defaults, no order-dependent global state, no "it works because something else happened to set this."

• One flat Config struct. All derived booleans computed once in resolveConfig(). No if(ENABLE_X) depending on if(CI) depending on if(RELEASE) — the chain resolves here, the result is a plain value.
• One flat table per flag category. Each flag has a when predicate and a desc. To find why -fno-unwind-tables is set, grep for it in flags.ts. Related flags live adjacent so coupling is visible.
• Explicit rule registration — see "registerXxxRules vs emitXxx" below.
• One concern per file — see Module inventory.

Minimal cross-platform diffs. Platform-specific logic is abstracted once, consumed everywhere. Config derives cfg.exeSuffix/cfg.objSuffix/cfg.libPrefix/cfg.libSuffix so callers write lib${name}${cfg.libSuffix} not if windows ".lib" else ".a". Flag tables use when: c => c.darwin predicates — one table entry, not a new branch in N files. shell.ts/stream.ts/tools.ts/compile.ts absorb the remaining cmd.exe-vs-sh, .exe suffix, and clang-vs-clang-cl differences. Where a branch is unavoidable (Windows resources, Darwin dsymutil, Linux setarch), it lives in one function and returns empty on other platforms.

Deps in our graph by default; native build systems when needed. BuildSpec variants:

• direct — list the dep's sources explicitly; each becomes a first-class cc/cxx edge in our graph and the .os go straight into bun's link. The default for the C/C++ deps (zlib, zstd, boringssl, libarchive, mimalloc, …). Skips a sub-process configure entirely and lets LTO see across the dep boundary.
• nested-cmake — invoke the dep's own cmake configure + build as ninja edges. For deps whose build is too entangled to list by hand. Flags forwarded via -DCMAKE_C_FLAGS; cmake's own dependency tracking handles incrementality inside.
• cargo — invoke cargo build (lolhtml). Cargo's incremental build is reliable; restat = 1 keeps our downstream no-ops fast.
• prebuilt — skip build entirely, download compiled .a/.lib (WebKit, nodejs-headers).

The dep pool (depth 4) throttles concurrent nested cmake/cargo sub-builds so they don't oversubscribe cores.

Self-obsoleting workarounds — see "Adding a workaround" below.

Configure time vs build time

Configure time is Phase 1 below — resolve tools, compute flags, glob sources, write build.ninja and constant manifests, validate, pre-create output dirs. Build time is everything ninja does: turning source files into outputs.

The smell: if configure code calls spawnSync to compile something, or compares mtimes with statSync, it's doing ninja's job. Make it a build edge — n.rule() + n.build(). Size doesn't matter; a 1-file compile is still a build edge.

Legitimate spawnSync at configure time: tool detection (clang --version), git revision, xcrun --show-sdk-path. These probe the environment; they don't produce build artifacts.

Ninja primer

A rule is a command template. A build edge instantiates a rule with specific files. The graph is just edges; rules are deduplication. Two real examples from our generated build/<profile>/build.ninja:

rule cc
  command = clang $cflags -MMD -MT $out -MF $out.d -c $in -o $out
  depfile = $out.d
  deps = gcc

build obj/src/foo.c.o: cc ../../src/foo.c | deps/zstd/libzstd.a || codegen/generated.h
  cflags = -O2 -I...

cc is defined via n.rule("cc", {...}) in compile.ts; the edge is emitted by cc(n, cfg, src, opts) per source. The | starts implicit inputs, || starts order-only.

rule dep_fetch
  command = bun fetch-cli.ts dep $name $repo $commit $dest ...
  restat = 1
  pool = dep

build ../../vendor/zstd/.ref: dep_fetch | ../../scripts/build/fetch-cli.ts
  name = zstd
  repo = facebook/zstd
  commit = abc123...

restat = 1: if fetch was a no-op (.ref unchanged), prune everything downstream. pool = dep throttles to 4 concurrent fetches.

All rules and edges are written to build/<profile>/build.ninja by n.write() at the end of configure. compile_commands.json (for clangd/LSP) is written alongside it.

Edge dependency types:

• explicit inputs ($in) — listed on the build line, passed to the command
• implicit inputs (| foo) — tracked for rebuild but not in $in. Use for the PCH, dep lib outputs (invalidation signal for their headers), or a per-file generated header this source is known to read
• order-only inputs (|| stamp) — must exist before this edge runs, but mtime doesn't trigger rebuild. Use for bulk codegen headers: "must be generated first, but the compiler's .d depfile will track which ones I actually read"

restat = 1 — after the command runs, re-stat outputs; if mtime didn't change, prune downstream. Critical for idempotent steps (fetch no-op, codegen unchanged).

depfile — compiler writes foo.o.d listing every #included header. Ninja reads it on the next build to know which headers this .o depends on. Codegen headers are order-only for this reason: they're declared outputs with restat, the depfile gives exact per-file header deps on build 2+, and order-only just ensures they exist for build 1. Dep outputs (lib*.a) are a different story — PCH, cc, and no-PCH cxx use them as implicit deps, because local sub-builds (e.g. WebKit) rewrite forwarding headers as undeclared side effects and order-only would lag one build behind (see Gotchas).

Iterating on the build system

bun scripts/build.ts --configure-only       # regenerate build.ninja, don't run ninja
bunx tsc --noEmit -p scripts/build/tsconfig.json   # typecheck
grep "yourtarget\|yourrule" build/debug/build.ninja  # inspect generated output
ninja -C build/debug -t query <target>      # why does <target> rebuild?
ninja -C build/debug -t deps <target>       # what headers does foo.o depend on?
ninja -C build/debug <target>               # build a specific target (e.g. tinycc, bun-rust)

The generated build.ninja is the ground truth. If an edge isn't doing what you expect, read it there first.

CLI arg parsing

bun scripts/build.ts [build-flags] [exec-args...]. The cutoff: first arg that isn't a recognized build/ninja flag ends build-flag parsing — it and everything after go to the built binary.

Arg shape	Goes to
-j<N>, -k<N>, -l<N>, -v	ninja
--configure-only, --help	build.ts
--<known-field>=<val> or --<known-field> <val>	build.ts (profile/target/config overrides)
--	ends parsing — rest to runtime unconditionally
--<unknown-field>=<val>	errors (typo detection)
Anything else	runtime, and everything after too

Build flags must come before exec args. bun bd --asan=off test foo.ts works; bun bd test --asan=off foo.ts sends --asan=off to bun-debug. Use -- when a runtime flag collides with a build flag: bun bd -- --target=browser script.ts.

--target=<name> builds a specific ninja target instead of the full binary. Every dep gets phonies: <name> (full build), clone-<name> (fetch only), configure-<name> (cmake deps). Also bun, check, bun-rust. List all: ninja -C build/debug -t targets.

Common tasks

Add a compiler flag — one entry in the right table in flags.ts:

{ flag: "-fno-foo", when: c => c.linux && c.release, desc: "why this flag" },

Tables: cpuTargetFlags (-march/-mcpu/-mtune — also forwarded to local WebKit via computeCpuTargetFlags()), globalFlags (bun + all deps), bunOnlyFlags (just bun), linkFlags, stripFlags. Use lang: "cxx" to restrict to C++.

Bump a dependency — edit the commit in scripts/build/deps/<name>.ts. See deps/README.md for adding/removing deps.

Iterate on a dependency from a local checkout — bun bd --local-deps=mimalloc=~/code/mimalloc … builds that dep from the clone instead of the pinned tarball (no fetch, no patches; edits rebuild incrementally). Any github-archive dep the graph compiles (not lolhtml — cargo reads that via Cargo.toml); details in deps/README.md.

Add a codegen step — add a function in codegen.ts following the shape of emitErrorCode (simple) or emitCppBind (needs file-list input). Call it from emitCodegen() and add outputs to the right CodegenOutputs group (rustInputs if the Rust build reads it (the include!d generated .rs files) — cppSources if it's a .cpp to compile, cppAll if it's a header).

Add a Config field — add to Config interface and PartialConfig in config.ts, resolve in resolveConfig(). If it needs a CLI flag, build.ts's arg parser already handles --anyfield=value generically.

Add a profile — one entry in profiles.ts. Copy debug or release-asan.

Build flow: bun run build → binary

Phase 0 — Entry (scripts/build.ts)

1. Windows: re-exec inside VS dev shell if VSINSTALLDIR unset (provides PATH/INCLUDE/LIB for nested cmake).
2. Parse CLI: --profile=<name>, --<field>=<value> overrides, --target=<ninja-target>, -j/-v/-k passthrough, bare positionals = exec args for built binary.
3. Resolve PartialConfig from profile + overrides (or --config-file for ninja's self-reconfigure).

Phase 1 — Configure (configure.ts::configure)

1. resolveToolchain() — find clang/ar/lld/strip/cmake/cargo/bun/esbuild. Version-checked where it matters; paths stored on Toolchain.
2. resolveConfig(partial, toolchain) — produce the flat Config. Detect host, derive all target booleans, compute paths, read package.json version + git sha.
3. validateBunConfig(cfg) + checkWorkarounds(cfg) — fail early with clear errors.
   • generateCargoConfig(cfg) — write the repo-root .cargo/config.toml (git-ignored) with the per-target linker = from the discovered cfg.hostCxx. Advisory only for bun bd (the ninja cargo edge sets the linker via env); it's there for cargo build/cargo check/rust-analyzer run directly.
4. globAllSources() — one filesystem snapshot of all .cpp/.c/.rs/codegen-input globs.
5. new Ninja({buildDir}) + registerAllRules(n, cfg) — register every rule template.
6. emitGeneratorRule(n, cfg, partial) — persist configure.json, emit regen rule so editing any build script triggers reconfigure.
7. emitBun(n, cfg, sources) — assemble the build graph (see Phase 2).
8. n.default([...]) + n.write() — set default targets, write build.ninja + compile_commands.json.
9. mkdirAll(...) — pre-create all object output dirs.

Phase 2 — emitBun (bun.ts::emitBun)

For mode: "full" (the normal case):

1. Codegen — emitCodegen(n, cfg, sources) emits ~20 generation steps (bindgen, .classes.ts → C++, bundled modules, LUTs). Returns grouped outputs.
2. Rust — emitRust(n, cfg, {...}) emits cargo build -p bun_bin → libbun_rust.a (after resolving just lolhtml, its path dep). Codegen and cargo are emitted before the deps on purpose. Scheduling: with no .ninja_log (every CI build) ninja weighs each edge as 1 and runs the longest remaining chain first, ties in emission order — so cargo ties with cc → link in full mode and wins on emission order, but in archive-link mode cc → ar → link outranks it and cargo would start only after every compile had been dispatched (~50s into a CI build). The compile pool in compile.ts (depth = core count, below ninja's default -j of cores+2) is what actually guarantees cargo a slot the moment it is ready.
3. Deps — loop allDeps, call resolveDep(n, cfg, dep). Each emits fetch → configure → build (nested-cmake), or fetch → cargo, or fetch → direct cc+ar, or prebuilt download. Collects lib paths, include dirs, outputs.
4. Flags — computeFlags(cfg) evaluates flag tables → cflags/cxxflags/defines/ldflags/stripflags.
5. PCH — compile root-pch.h → PCH (skipped in CI full mode).
6. Compile — loop sources, cxx()/cc() per file.
7. Link — emitShims(n, cfg) for platform workaround dylibs, then link(n, cfg, exeName, objects, {libs, flags}).
8. Post-link — strip (release only), dsymutil (darwin release only).
9. Smoke test — <exe> --revision catches load-time failures.

Split CI modes: rust-only (lolhtml+codegen+cargo → libbun_rust.a), cpp-only (deps+codegen+compile → archive), link-only (download artifacts → link), rust-and-link (cargo + poll build-cpp + download archive → link). The pipeline's build-bun step uses archive-link (ci-build profile): the full graph on one agent, linking from the same archive cpp-only produces, with the archive, libbun_rust.a and dep libs uploaded from ninja edges as soon as each exists.

Phase 3 — Execute

• CI: collapsible log groups, spawn ninja with spawnWithAnnotations (parses compiler errors into Buildkite annotations), upload/download artifacts.
• Local: spawn ninja with FD 3 dup'd to stderr — stream.ts-wrapped commands write to FD 3, bypassing ninja's per-job output buffering so dep/cargo build progress streams live. If positionals given, exec the built binary with them.

Module inventory

File	Owns
build.ts (parent dir)	CLI entry — parse args, call configure, spawn ninja, optionally exec
configure.ts	configure() — toolchain → config → build.ninja
config.ts	Config/PartialConfig/Toolchain/Host types, resolveConfig()
profiles.ts	Named PartialConfig presets + getProfile()
tools.ts	Tool discovery: findTool(), resolveLlvmToolchain(), version parsing
flags.ts	Flat flag tables, computeFlags(), computeDepFlags(), computeCpuTargetFlags()
ninja.ts	Ninja class — the build-file writer
rules.ts	registerAllRules() — calls each module's registerXxxRules()
compile.ts	cc/cxx/pch/link/ar + registerCompileRules()
unified.ts	WebKit-style unified-source bundling, generateUnifiedSources()
source.ts	Dependency types, resolveDep(), fetch/configure/build emission
codegen.ts	Code generation steps, emitCodegen(), CodegenOutputs
rust.ts	cargo build step, emitRust(), rustLibPath(), cross-compile matrix
cargo-config.ts	Generates the git-ignored .cargo/config.toml (per-target linker from cfg.hostCxx)
bun.ts	emitBun() — assembles deps+codegen+rust+compile+link
shims.ts	Platform/toolchain workaround dylibs, emitShims()
workarounds.ts	Self-obsoleting workaround registry, checkWorkarounds()
macos-sdk.ts	macOS SDK resolution/download for darwin cross-compiles — resolveMacosSdkPath(), ensureMacosSdk()
features-json.ts	Host-side features.json for cross lanes — parsePackedFeaturesList(), crossFeaturesJson()
depVersionsHeader.ts	Generates bun_dependency_versions.h for process.versions
buildOptionsRs.ts	Generates build_options.rs (bun_core::build_options) from Config
jsonByteClass.ts	Generates json_byte_class.{h,rs} — the JSON byte classification shared by the SIMD kernel and the Rust scalar indexer
xmlByteClass.ts	Generates xml_byte_class.{h,rs} — the XML byte classification shared by the SIMD kernels and the Rust scalar indexer
stream.ts	Subprocess output wrapper — FD-3 sideband, prefixed line streaming
shell.ts	quote()/slash() — shell escaping for ninja commands
fs.ts	writeIfChanged(), mkdirAll()
error.ts	BuildError with hint/file/cause, assert()
download.ts	downloadWithRetry(), archive extraction
winsysroot.ts	Windows MSVC CRT + SDK sysroot (xwin): validates, adds case aliases, CI fetch
fetch-cli.ts	Build-time CLI ninja invokes for downloads
ci.ts	CI integration — annotations, artifacts, log groups
clean.ts	bun run clean preset-based cleanup
glob-sources.ts (parent dir)	Source glob patterns + CLI to print them
deps/*.ts	One Dependency object per vendored dep
deps/index.ts	allDeps array — fetch order + link order
shims/*.c	Platform workaround sources

Key types

Dependency (source.ts) — {name, source, patches?, fetchDeps?, build, provides, enabled?, versionMacro?}. The source/build/provides fields are functions of Config so they vary per-target. Source variants: github-archive, local, in-tree, prebuilt. BuildSpec variants covered in Goals above.

Ninja — Accumulates rules/builds/pools/defaults, emits build.ninja. All paths given absolute; converted to buildDir-relative at write time.

registerXxxRules vs emitXxx

Rules are ninja rule blocks — reusable command templates. Build edges are build statements — input→output instances.

Ninja requires all rules defined before any build references them. Hence:

1. registerXxxRules(n, cfg) — each module registers its rules. Called once via registerAllRules().
2. emitXxx(n, cfg, ...) — each module emits build edges.

Why not auto-register in emit functions? Some rules are shared (dep_configure used by both source.ts and webkit.ts local mode). Explicit registration keeps "which rule lives where" clear.

Gotchas

Dep order in allDeps matters. fetchDeps: ["X"] means X must come first (its .ref stamp node must exist). Link order matters too: static linking resolves left→right, providers after users.

PCH, cc, and no-PCH cxx need implicit dep on depHeaderSignal, not order-only. Local WebKit's sub-build rewrites forwarding headers as an undeclared side effect (only lib*.a are declared outputs). Depfiles record those headers, but ninja stats them before the sub-build runs — order-only lags one build. The lib itself is the invalidation signal. Codegen headers stay order-only: they're declared outputs with restat, so depfile tracking is exact.

isExecutable must check isFile(). X_OK on a directory means traversable — a cmake/ dir in PATH would shadow the real cmake binary.

cmd.exe quoting is partial. shell.ts quote() handles spaces/special chars but NOT %VAR% expansion, ^ escape, &|> redirection. If an arg contains those, switch to powershell.

rm -rf build/ doesn't clear the cache locally. cfg.cacheDir is machine-shared at $BUN_INSTALL/build-cache for non-CI builds (ccache, tarballs, prebuilt WebKit). Everything there is content-addressed or version-stamped, so a stale entry can't be hit — don't reach for bun run clean cache as a debugging step. If a build misbehaves, the bug is in the inputs or the graph, not the cache; nuking it just costs you a cold rebuild. CI keeps <buildDir>/cache so rm -rf build/ is still a full reset there.

Node compatibility

The build system runs under Node 24+ with --experimental-strip-types (or Node 25+ without the flag). CI invokes it this way via process.execPath in .buildkite/ci.mjs.

cfg.jsRuntime holds the shell-ready command prefix for running .ts subprocesses (stream.ts, fetch-cli.ts, the regen rule) — it's process.execPath when bun runs configure, or node --experimental-strip-types when node does. The subprocesses inherit whichever runtime started the build.

TODO — remaining cfg.bun usage (codegen only): For a fully bun-optional build:

• cfg.packageManager — bun install or npm install for the one codegen install step.
• Codegen .ts scripts (~20 ninja rules) — either verify they're node-compatible and switch to cfg.jsRuntime, or bundle them via esbuild first and run the output with plain node.
• cfg.esbuild — already separate.

With those done, cfg.bun disappears.

Adding a workaround

Every temporary fix for a toolchain/OS bug registers a self-obsoleting check so it can't rot silently:

1. Put the artifact under scripts/build/shims/ (or patches/ for source patches).
2. Emit it as a ninja build edge from shims.ts (or the appropriate module).
3. Register an entry in workarounds.ts with an expectedToBeFixed predicate — configure fails with cleanup instructions once the upstream fix ships.

expectedToBeFixed typically checks a tool version against a threshold (e.g. cfg.clangVersion >= "23.0.0"). When you know exactly which release has the fix, use that. When you don't — fix merged upstream but not released yet — pick your best guess for the likely release. The check might trip on a version that turns out not to have the fix; that's okay. The error message tells the dev to bump the threshold, which takes 30 seconds. That's cheaper than leaving the check blank and the workaround living forever because nobody remembered to come back.