fast-fs-hash - v0.0.3

fast-fs-hash

If you ever needed to check whether a set of files changed — to invalidate a cache, skip redundant builds, or trigger incremental CI — fast-fs-hash is for you.

It hashes hundreds of files in milliseconds using xxHash3-128 via a native C++ addon with SIMD acceleration.

xxHash3 is a non-cryptographic hash function — it is not suitable for security purposes, but it is more than enough for cache invalidation, deduplication, and change detection, which is what this library is designed for.

Zero external dependencies. Requires Node.js >= 22.

Installation

npm install fast-fs-hash

Requires Node.js >= 22.

The native addon is prebuilt for common platforms via platform-specific optional dependencies. When you run npm install, npm automatically installs only the package matching your current OS and architecture.

Supported platforms: macOS, Linux (glibc & musl), Windows, FreeBSD — both x64 and arm64.

On x64, optimized variants for AVX2 and AVX-512 are included and selected automatically at load time via native CPUID detection. Set FAST_FS_HASH_ISA=avx2|avx512|baseline to override.

CI note: Some CI configurations disable optional dependencies by default (e.g. npm install --no-optional or --omit=optional). To get the native addon in CI, either allow optional dependencies or install the platform package explicitly:

npm install @fast-fs-hash/fast-fs-hash-node-linux-x64-gnu

FileHashCache — Binary cache invalidation

FileHashCache reads, validates, and writes a compact binary cache file that tracks per-file stat metadata (inode, mtime, ctime, size) and content hashes (xxHash3-128).

On the next run it re-stats every tracked file and compares — files whose stat matches are skipped entirely (no re-read), giving near-instant validation for large file sets.

Why use FileHashCache?

Build systems, code generators, and CI pipelines often produce output that depends on many input files. Recomputing that output on every run is expensive — even when nothing changed.

FileHashCache solves this by persisting a fingerprint of all input files between runs. On the next invocation, it checks whether any input changed in sub-millisecond time (stat-only, no re-reading). If nothing changed, you skip the expensive step entirely.

Common use cases:

Incremental builds: track source files → skip compilation when inputs are unchanged
Generated output caching: store a compiled bundle, generated types, or processed assets alongside the cache — rebuild only when dependencies change
CI artifact caching: validate whether a cached artifact is still fresh before uploading or downloading a new one
Multi-step pipelines: each stage writes its own cache file, checked independently
Standalone hashing & compression: the same fast native xxHash3-128 and LZ4 that power the cache are exported directly — hash buffers, strings, or files, or LZ4-compress data on their own

The cache file also supports user data — opaque binary payloads stored alongside the file hashes. This lets you embed build output manifests, dependency graphs, or configuration snapshots directly in the cache, so a single open() tells you both "did anything change?" and "what was the previous result?" — no separate metadata files needed.

Why it's fast — and safe:

Stat-first validation — a warm check is one stat() per file (O(n)), not a read + hash of every byte (O(Σ file sizes)); only files whose stat changed are re-hashed. Typically 10-100× faster for warm caches.
Fast native XXH3 + LZ4 — SIMD xxHash3-128 (AVX2/AVX-512 auto-dispatch) and LZ4, run on a work-stealing thread pool in a single open → fstat → read pass with overlapped readahead and zero-copy buffers — multiple GB/s per core. Both are exported as standalone APIs too.
Concurrency-safe — an OS-level flock (exclusive to write, shared to read) means no build ever sees a half-written cache, with cancellable, lockTimeoutMs-bounded acquisition and no lockfiles to leak. lock() exposes the same lock as a standalone cross-process mutex.

FileHashCache benchmarks (705 files, ~24 MiB)

Native (C++ addon):

Scenario	Mean	Hz	Files/s	Throughput
no change	0.5 ms (491.2 µs)	2 036 op/s	1 435 194 files/s	—
1 file changed	0.8 ms (765.6 µs)	1 306 op/s	920 810 files/s	—
many files changed	2.3 ms (2 289.4 µs)	437 op/s	307 941 files/s	10.8 GB/s
no existing cache	10.0 ms (9 967.5 µs)	100 op/s	70 730 files/s	2.5 GB/s
overwrite	9.9 ms (9 926.7 µs)	101 op/s	71 020 files/s	2.5 GB/s

Node.js v26.5.0, Vitest 4.x — Apple M4 Max, macOS 25.5.0 (arm64), with anti-virus.

Results vary by hardware, file sizes, and OS cache state.

FileHashCache API

A long-lived cache that tracks file content hashes with exclusive OS-level locking. Create the instance once, then call open() on each build cycle. Configuration (files, version, fingerprint) is set via the constructor, setters, or configure().

Example: Build cache with dynamic file list

The typical usage: the file list is only known after a build step. Open without files (reuses the list from the previous cache on disk), then set the new file list before writing. Use compressedPayloads (LZ4-compressed inside the cache body) or uncompressedPayloads (stored raw, readable without decompression) to store arbitrary build metadata alongside the cache.

import { FileHashCache } from "fast-fs-hash";

const cache = new FileHashCache({
  cachePath: ".cache/build.fsh",
  rootPath: ".",
  version: 1,
});

export async function build() {
  using session = await cache.open();

  if (session.status === "upToDate" && session.compressedPayloads.length > 0) {
    return JSON.parse(session.compressedPayloads[0].toString()); // cached result
  }

  const result = await runBuild();

  cache.configure({ files: result.getSourceFiles().map((f) => f.fileName) });

  await session.write({
    compressedPayloads: [Buffer.from(JSON.stringify(result.output))],
  });

  return result.output;
}

Example: Simple build cache with known files

When the file list is known upfront, pass it to the constructor:

import { FileHashCache } from "fast-fs-hash";
import { globSync } from "node:fs";

const cache = new FileHashCache({
  cachePath: ".cache/build.fsh",
  rootPath: ".",
  files: globSync("src/**/*.ts"),
  version: 1,
});

using session = await cache.open();

if (session.status === "upToDate") {
  console.log("Build cache is fresh — skipping.");
} else {
  console.log("Files changed — rebuilding...");
  await runBuild();
  await session.write();
}

API Reference

Constructor: new FileHashCache({ cachePath, files?, rootPath?, version?, fingerprint?, lockTimeoutMs? })

Cache configuration (mutable between opens):

configure(opts) — set multiple config fields at once: files, rootPath, version, fingerprint, lockTimeoutMs
Setters: cache.files, cache.rootPath, cache.version, cache.fingerprint, cache.lockTimeoutMs
needsOpen — true when config changed since last open, or cache was never opened

Cache properties (read-only):

cachePath — the cache file path
fileCount — number of configured files
busy — true while an async cache operation (open/lock/overwrite) is in flight
activeSession — the currently-open FileHashCacheSession, or null

Cache methods:

open(signal?) — acquires an exclusive lock, reads from disk, validates version/fingerprint, stat-matches entries. Returns a FileHashCacheSession.
lock(signal?) — acquires the exclusive lock without reading or parsing the cache file (no entries, no payloads, no stat loop). Returns a lock-only FileHashCacheSession (status === 'missing', lockOnly === true). Use it when you only need cross-process mutual exclusion on the cache path. Calling write(options) on the returned session writes a fresh cache reusing the already-held lock — equivalent to overwrite() but without a release / re-acquire round-trip. See Lock-only sessions.
overwrite(options?) — writes a brand-new cache without reading the old one. Options: payloadValue0..3, compressedPayloads, uncompressedPayloads, signal, lockTimeoutMs.
invalidate(paths) / invalidateAll() — mark files as dirty for the next open (watch mode).
isLocked() / waitUnlocked(timeout?, signal?) — check or wait for lock.
checkCacheFile() — sync stat check if the cache file on disk changed since last open.

Session properties (read-only, from disk):

status — see Cache status below
lockOnly — true when opened via cache.lock(): the session holds only the OS lock and carries no cache data. Its disk-backed accessors (compressedPayloads, uncompressedPayloads) return empty, needsWrite is false, and resolve() throws. files still reflects the configured file list (status is 'missing')
cache — the parent FileHashCache
needsWrite — true if the session holds the lock and the status indicates changes
configChanged — true if cache config was modified since this session was opened
wouldNeedWrite — true if either files changed on disk or config changed
busy / disposed — async operation state
files, fileCount, version, rootPath
diskVersion — the user version (u32) read from the on-disk header. Differs from version only when status === 'staleVersion'; lets migration code identify which old format to parse.
payloadValue0..3 — four f64 numeric values read from disk
compressedPayloads — array of LZ4-compressed binary Buffer payloads read from disk
uncompressedPayloads — array of raw binary Buffer payloads readable without LZ4 decompression

Cache status

Status	Disk readable?	Entries trustable?	Payloads readable?	Action
`'upToDate'`	Yes	Yes	Yes	None
`'statsDirty'`	Yes	Yes (content unchanged)	Yes	Rewrite cache (stats refreshed)
`'changed'`	Yes	Partially (some files changed)	Yes	Re-hash changed files, write
`'stale'`	Yes (well-formed)	No (fingerprint mismatch)	Yes	Discard entries, optionally migrate payloads, write
`'staleVersion'`	Yes (well-formed)	No (`version` differs)	Yes	As above, plus check `session.diskVersion` for version-aware migration
`'missing'`	No	—	No	Write fresh
`'lockFailed'`	—	—	—	Retry later

'stale' and 'staleVersion' both keep the on-disk buffer readable so callers can migrate compressedPayloads / uncompressedPayloads / payloadValueN across config changes. 'missing' covers genuinely unreadable files (no file, truncated header, bad magic, corrupt body) — never thrown, always reported via this status so callers can recover by writing fresh.

Session methods:

write(options?) — hashes unresolved entries, compresses, writes to disk, releases lock. Can only be called once. Options: payloadValue0..3, compressedPayloads, uncompressedPayloads, signal. On a lock-only session (lockOnly === true) it writes a fresh cache from the supplied options, reusing the already-held lock.
resolve(signal?) — completes stat + hash for ALL files, returns FileHashCacheEntries. Can be called before write(). See below.
close() — releases the lock. Also called automatically by using.

Static methods:

FileHashCache.isLocked(cachePath) — check if locked by another process
FileHashCache.waitUnlocked(cachePath, lockTimeoutMs?, signal?) — wait for unlock

Lock behavior:

Cross-process exclusive lock via flock(2) (POSIX) / LockFileEx (Windows)
Cross-thread safe: per-OFD semantics on POSIX and per-handle on Windows make worker_threads in the same process serialize correctly against each other
Crash-safe: automatically released when the process dies
lockTimeoutMs: -1 = block forever (default), 0 = non-blocking, >0 = timeout ms
When lock fails: status === 'lockFailed'. Calling write() falls back to overwrite().
Cancellable via AbortSignal on open(), overwrite(), and waitUnlocked()

Inspecting per-file changes with `resolve()`

After open(), the session knows the aggregate status but not which specific files changed. Call resolve() to complete stat + hash for every file and get per-file metadata.

Note: resolve() stats and hashes every unresolved file on the thread pool. This has a cost proportional to the number of changed files. Use it only when you need per-file information — for simple "changed → rebuild all" workflows, just check session.status.

using session = await cache.open();

if (session.status !== "upToDate") {
  const entries = await session.resolve();

  for (const entry of entries) {
    if (entry.changed) {
      console.log(
        `Changed: ${entry.path} (${entry.size} bytes, hash: ${entry.contentHashHex})`,
      );
    }
  }

  await session.write();
}

Each FileHashCacheEntry provides:

path — absolute file path
size — file size in bytes
mtimeMs / ctimeMs — modification / change time in ms
changed — true if content differs from the cached version (or is a new file)
contentHash — 16-byte xxHash3-128 as a Buffer (zero-copy view)
contentHashHex — 32-char hex string (lazy, computed on first access)

FileHashCacheEntries supports get(index), find(path), and iteration. The result is cached — subsequent calls to resolve() return the same snapshot.

Lock-only sessions

cache.lock() acquires the cross-process lock without reading or parsing the cache file — no entries, no payloads, no stat loop. Use it when you only need mutual exclusion on the cache path, or to write a fresh cache while holding the lock the whole time (no release / re-acquire between deciding and writing).

using session = await cache.lock();

if (session.status === "lockFailed") {
  // another holder owns the lock — retry later
} else {
  // session.lockOnly === true, session.status === "missing"
  // write a fresh cache reusing the already-held lock:
  await session.write({ payloadValue0: Date.now() });
}

The returned session carries no disk-backed cache data: compressedPayloads and uncompressedPayloads return empty, needsWrite is false, and resolve() throws. files still reflects the configured file list (status is 'missing'). close() (or using) releases the lock as usual.

xxHash128 — Direct hashing

When you don't need a persistent cache file — or you want raw xxHash3-128 digests to compare yourself — use the digest functions directly. FileHashCache uses them under the hood, but they are fully usable on their own.

File hashing benchmarks

large file (~197.3 KB):

Scenario	Mean	Hz	Throughput	Relative
native	0.04 ms (41.6 µs)	24 030 op/s	4.7 GB/s	6.6× faster
Node.js crypto (md5)	0.3 ms (274.3 µs)	3 645 op/s	719 MB/s	baseline

medium file (~49.9 KB):

Scenario	Mean	Hz	Throughput	Relative
native	0.03 ms (27.3 µs)	36 681 op/s	1.8 GB/s	4.0× faster
Node.js crypto (md5)	0.1 ms (109.0 µs)	9 171 op/s	458 MB/s	baseline

small file (~1.0 KB):

Scenario	Mean	Hz	Relative
native	0.03 ms (27.7 µs)	36 107 op/s	2.0× faster
Node.js crypto (md5)	0.06 ms (55.9 µs)	17 896 op/s	baseline

Parallel file hashing (705 files)

Scenario	Mean	Hz	Throughput	Relative
native	10.3 ms (10 274.0 µs)	97 op/s	2.4 GB/s	3.4× faster
Node.js crypto (md5)	35.4 ms (35 431.2 µs)	28 op/s	697 MB/s	baseline

In-memory buffer hashing

64 KB buffer:

Scenario	Mean	Hz	Throughput	Relative
native XXH3-128	0.001 ms (1.4 µs)	711 505 op/s	46.6 GB/s	47.4× faster
Node.js crypto md5	0.07 ms (66.6 µs)	15 013 op/s	984 MB/s	baseline

1 MB buffer:

Scenario	Mean	Hz	Throughput	Relative
native XXH3-128	0.02 ms (21.9 µs)	45 762 op/s	48.0 GB/s	48.4× faster
Node.js crypto md5	1.1 ms (1 056.9 µs)	946 op/s	992 MB/s	baseline

Hash files

import { digestFilesParallel, hashToHex } from "fast-fs-hash";

const digest = await digestFilesParallel([
  "package.json",
  "src/index.ts",
  "src/utils.ts",
]);
console.log("Aggregate:", hashToHex(digest));

Sequential variant (feeds files into a single running hash):

import { digestFilesSequential, hashToHex } from "fast-fs-hash";

const digest = await digestFilesSequential(["package.json", "src/index.ts"]);
console.log(hashToHex(digest));

Hash a single file

import { digestFile, hashToHex } from "fast-fs-hash";

const digest = await digestFile("package.json");
console.log(hashToHex(digest));

Hex string convenience

import { digestFileToHex, digestFilesToHexArray } from "fast-fs-hash";

// Single file → 32-char hex string
const hex = await digestFileToHex("package.json");

// Multiple files in parallel → per-file hex strings
const hexes = await digestFilesToHexArray(["src/a.ts", "src/b.ts"], 8);

Function	Description
`digestFileToHex(path, throwOnError?)`	Hash a file → 32-char hex string. Wrapper around digestFile + hashToHex
`digestFilesToHexArray(paths, concurrency?, throwOnError?)`	Hash files in parallel → per-file hex strings. Default concurrency 8

Hash buffers and strings

import { digestBuffer, digestString } from "fast-fs-hash";

const d1 = digestBuffer(myBuffer);
const d2 = digestString("hello world");
console.log(d2.toString("hex"));

Range variant hashes a slice: digestBufferRange(input, offset, length?).

Zero-allocation `…To` variants

Every digest* function has a …To sibling that writes the 16-byte digest into a caller-provided out buffer (at outOffset, default 0) instead of allocating a fresh Buffer, and returns out. Use these on hot paths to avoid per-call allocation — e.g. packing many digests into one large buffer.

import { digestBufferTo, digestFileTo } from "fast-fs-hash";

const out = Buffer.allocUnsafe(32);
digestBufferTo(a, out, 0); // first 16 bytes
digestBufferTo(b, out, 16); // next 16 bytes
await digestFileTo("package.json", out, 0); // async file variant

Function	Writes into `out` the digest of…
`digestBufferTo(input, out, outOffset?)`	a whole buffer
`digestBufferRangeTo(input, offset, length, out, outOffset?)`	a buffer slice
`digestStringTo(input, out, outOffset?)`	a UTF-8 string
`digestFileTo(path, out, outOffset?, throwOnError?)`	a file (async)
`digestFilesSequentialTo(paths, out, outOffset?, throwOnError?)`	files hashed sequentially (async)
`digestFilesParallelTo(paths, out, outOffset?, concurrency?, …?)`	files hashed in parallel, aggregated (async)

Streaming class

For combining file hashes with extra data (config, environment, etc.):

import { XxHash128Stream } from "fast-fs-hash";

const h = new XxHash128Stream();
h.addString("my-config-v2");
await h.addFiles(["src/index.ts", "src/utils.ts"]);
console.log(h.digest().toString("hex"));

Busy guard: Async methods (addFile, addFiles, addFilesParallel) mark the instance as busy while the native worker thread is processing. During this time, calling any synchronous method or starting another async operation will throw an error. Always await each async call before invoking another method. Use the busy getter to check:

const h = new XxHash128Stream();
const promise = h.addFile("large.bin");
console.log(h.busy); // true — async operation in flight
// h.addString("oops"); // would throw!
await promise;
console.log(h.busy); // false — safe to use again

Seeding: new XxHash128Stream(seedLow?, seedHigh?) seeds the hash with two 32-bit halves (default 0); the current seed is exposed via the seedLow / seedHigh fields.

Full method list:

Method	Description
`addBuffer(input)`	Feed a whole buffer (sync)
`addBufferRange(input, offset, length?)`	Feed a buffer slice (sync)
`addString(input)`	Feed a UTF-8 string (sync)
`addFile(path, throwOnError?)`	Read and feed a file (async)
`addFiles(paths, throwOnError?)`	Read and feed files sequentially (async)
`addFilesParallel(paths, concurrency?, …?)`	Read and feed files in parallel (async)
`digest()`	Finalize → 16-byte `Buffer` (does not consume state)
`digestTo(out, outOffset?)`	Finalize into a caller-provided buffer (zero-alloc), returns `out`
`reset(seedLow?, seedHigh?)`	Reset to empty, optionally reseed — reuse the instance
`clone()`	Deep-copy the current hashing state into a new independent instance
`XxHash128Stream.hash(input)` (static)	One-shot digest of a buffer or string

The class also mirrors the standalone functions as statics (XxHash128Stream.digestBuffer, .digestFile, .digestFilesParallelTo, …). The IXxHash128Functions type describes this shared surface.

LZ4 Block Compression

fast-fs-hash exposes the LZ4 block compression API used internally for the cache file format. Both synchronous and asynchronous (pool-thread) variants are available.

LZ4 block format does not embed the uncompressed size — the caller must store it alongside the compressed data and pass it to the decompression function.

compress 64 KB:

Scenario	Ratio	Mean	Hz	Throughput	Relative
native LZ4	0.7%	0.003 ms (3.0 µs)	334 654 op/s	21.9 GB/s	8.5× faster
Node.js deflate level=1	1.0%	0.03 ms (25.4 µs)	39 311 op/s	2.6 GB/s	baseline

decompress 64 KB:

Scenario	Mean	Hz	Throughput	Relative
native LZ4	0.002 ms (2.1 µs)	473 807 op/s	31.1 GB/s	4.0× faster
Node.js deflate	0.008 ms (8.4 µs)	118 896 op/s	7.8 GB/s	baseline

compress 1 MB:

Scenario	Ratio	Mean	Hz	Throughput	Relative
native LZ4	0.4%	0.03 ms (34.3 µs)	29 124 op/s	30.5 GB/s	10.1× faster
Node.js deflate level=1	0.7%	0.3 ms (346.3 µs)	2 888 op/s	3.0 GB/s	baseline

decompress 1 MB:

Scenario	Mean	Hz	Throughput	Relative
native LZ4	0.03 ms (29.5 µs)	33 951 op/s	35.6 GB/s	3.0× faster
Node.js deflate	0.09 ms (86.9 µs)	11 504 op/s	12.1 GB/s	baseline

import {
  lz4CompressBlock,
  lz4DecompressBlock,
  lz4CompressBound,
} from "fast-fs-hash";

const input = Buffer.from("Hello, LZ4!");
const compressed = lz4CompressBlock(input);
const decompressed = lz4DecompressBlock(compressed, input.length);
console.log(decompressed.toString()); // "Hello, LZ4!"

LZ4 API

Function	Description
`lz4CompressBlock(input, offset?, length?)`	Sync compress → new Buffer
`lz4CompressBlockTo(input, output, outputOffset?, inputOffset?, inputLength?)`	Sync compress into pre-allocated buffer → bytes written
`lz4CompressBlockAsync(input, offset?, length?)`	Async compress on pool thread → Promise<Buffer>
`lz4DecompressBlock(input, uncompressedSize, offset?, length?)`	Sync decompress → new Buffer
`lz4DecompressBlockTo(input, uncompressedSize, output, outputOffset?, inputOffset?, inputLength?)`	Sync decompress into pre-allocated buffer → bytes written
`lz4DecompressBlockAsync(input, uncompressedSize, offset?, length?)`	Async decompress on pool thread → Promise<Buffer>
`lz4CompressBound(inputSize)`	Max compressed size for pre-allocation
`lz4ReadAndCompress(path)`	Read a file and LZ4-compress it on pool thread → `Promise<{data, uncompressedSize}>`
`lz4DecompressAndWrite(compressedData, uncompressedSize, path)`	Decompress and write to file on pool thread (creates dirs) → `Promise<boolean>`

Note: LZ4 block compression supports inputs up to ~1.9 GiB (LZ4_MAX_INPUT_SIZE = 0x7E000000). lz4ReadAndCompress caps the source file at 512 MiB; lz4DecompressAndWrite caps uncompressedSize at 256 MiB.

Read and compress a file

lz4ReadAndCompress reads a file and LZ4-block-compresses it in a single pool-thread operation — no JS-thread I/O, no intermediate Buffer allocation visible to the event loop.

import {
  lz4ReadAndCompress,
  lz4DecompressAndWrite,
  lz4DecompressBlock,
} from "fast-fs-hash";

const { data, uncompressedSize } = await lz4ReadAndCompress("large-file.bin");
console.log(`Compressed ${uncompressedSize} → ${data.length} bytes`);

// Decompress back to a file (creates parent directories if needed)
await lz4DecompressAndWrite(data, uncompressedSize, "restored-file.bin");

// Or decompress to a buffer in memory
const original = lz4DecompressBlock(data, uncompressedSize);

File Comparison

Compare two files for byte-equality asynchronously on a native pool thread. Opens both files, compares sizes via fstat, then reads in lockstep chunks with memcmp. Returns false if either file cannot be opened/read or if sizes differ — never throws.

equal files (~49.9 KB):

Scenario	Mean	Hz	Throughput	Relative
native	0.04 ms (39.9 µs)	25 033 op/s	1.2 GB/s	2.7× faster
Node.js (fs.open + read + compare)	0.1 ms (106.3 µs)	9 406 op/s	469 MB/s	baseline

equal files (~197.3 KB):

Scenario	Mean	Hz	Throughput	Relative
native	0.05 ms (49.3 µs)	20 264 op/s	4.0 GB/s	3.4× faster
Node.js (fs.open + read + compare)	0.2 ms (165.6 µs)	6 038 op/s	1.2 GB/s	baseline

different content, same size (~49.9 KB):

Scenario	Mean	Hz	Throughput	Relative
native	0.04 ms (35.4 µs)	28 229 op/s	1.4 GB/s	2.8× faster
Node.js (fs.open + read + compare)	0.1 ms (100.5 µs)	9 949 op/s	496 MB/s	baseline

different sizes (early exit):

Scenario	Mean	Hz	Relative
native	0.04 ms (37.0 µs)	27 015 op/s	2.3× faster
Node.js (fs.open + read + compare)	0.08 ms (84.3 µs)	11 865 op/s	baseline

import { filesEqual } from "fast-fs-hash";

if (await filesEqual("output.bin", "expected.bin")) {
  console.log("Files are identical");
} else {
  console.log("Files differ (or one doesn't exist)");
}

Function	Description
`filesEqual(pathA, pathB)`	Async byte-equality check on pool thread → `Promise<boolean>`

Find Project Root

Walk the parent chain from a start path and locate project markers in a single pass: .git, package.json, tsconfig.json, and node_modules/. Reports nearest* (first hit walking up) and root* (last hit, bounded by the enclosing .git) for each marker, plus gitRoot and gitSuperRoot for submodule/worktree awareness.

The walk stops at the filesystem root, the user's home directory (or any ancestor of it), an optional stopPath, and a depth cap of 128 (symlink-loop defense). Tolerant of missing paths — if startPath doesn't exist, the walk begins from its longest existing ancestor and missing markers are returned as null rather than thrown.

shallow (3 levels deep):

Scenario	Mean	Hz	Relative
native (sync)	0.05 ms (45.0 µs)	22 206 op/s	5.8× faster
native (async)	0.06 ms (56.3 µs)	17 767 op/s	4.6× faster
Node.js (sync, fs.statSync)	0.1 ms (119.6 µs)	8 364 op/s	2.2× faster
Node.js (async, fs.stat)	0.3 ms (259.3 µs)	3 857 op/s	baseline

deep (12 levels deep):

Scenario	Mean	Hz	Relative
native (sync)	0.09 ms (92.4 µs)	10 823 op/s	7.5× faster
native (async)	0.1 ms (105.0 µs)	9 527 op/s	6.6× faster
Node.js (sync, fs.statSync)	0.4 ms (350.7 µs)	2 852 op/s	2.0× faster
Node.js (async, fs.stat)	0.7 ms (692.9 µs)	1 443 op/s	baseline

missing start path (tolerant fallback):

Scenario	Mean	Hz	Relative
native (sync)	0.05 ms (54.4 µs)	18 393 op/s	2.7× faster
Node.js (sync, fs.statSync)	0.1 ms (144.8 µs)	6 904 op/s	baseline

import { findProjectRoot, findProjectRootSync } from "fast-fs-hash";

// Sync (recommended for startup-time / build-tool use)
const info = findProjectRootSync(import.meta.dirname);
console.log(info.gitRoot, info.rootPackageJson, info.nearestTsconfigJson);

// Async — runs on the native thread pool, useful on cold or networked filesystems
const info2 = await findProjectRoot("/some/deep/file.ts");

// Optional stopPath: halt when the walker reaches this directory (or any ancestor of it)
const info3 = findProjectRootSync(start, "/workspace");

Function	Description
`findProjectRootSync(startPath, stopPath)`	Walk parent chain for project markers (sync) → `ProjectRoot`
`findProjectRoot(startPath, stopPath)`	Walk parent chain on pool thread → `Promise<ProjectRoot>`

The returned ProjectRoot object has these fields (each string | null):

Field	Description
`gitRoot`	Innermost `.git` (dir or file). Matches `git rev-parse --show-toplevel`.
`gitSuperRoot`	Outermost `.git` directory. Non-null only in submodules / nested worktrees.
`nearestPackageJson`	First `package.json` walking up.
`rootPackageJson`	Last `package.json` walking up, bounded by `gitRoot`.
`nearestTsconfigJson`	First `tsconfig.json` walking up.
`rootTsconfigJson`	Last `tsconfig.json` walking up, bounded by `gitRoot`.
`nearestNodeModules`	First `node_modules/` directory walking up (also detects when started inside).
`rootNodeModules`	Last `node_modules/` walking up, bounded by `gitRoot`.

Find Nearest Project Files

A trimmed-down sibling of findProjectRoot that finds only the nearest package.json, tsconfig.json, and node_modules/ and exits the walk as soon as all three are populated. No .git probe, no gitRoot boundary, no root* fields — faster than findProjectRoot when callers don't need them.

The walk stops at the filesystem root, the user's home directory (or any ancestor of it), an optional stopPath, and a depth cap of 128. Tolerant of missing paths — missing fields are returned as null rather than throwing.

shallow (3 levels deep):

Scenario	Mean	Hz	Relative
native (sync)	0.03 ms (31.1 µs)	32 153 op/s	4.0× faster
native (async)	0.04 ms (41.0 µs)	24 404 op/s	3.0× faster
Node.js (sync, fs.statSync)	0.06 ms (58.5 µs)	17 097 op/s	2.1× faster
Node.js (async, fs.stat)	0.1 ms (124.5 µs)	8 033 op/s	baseline

deep (12 levels deep):

Scenario	Mean	Hz	Relative
native (sync)	0.07 ms (70.0 µs)	14 286 op/s	6.6× faster
native (async)	0.08 ms (81.0 µs)	12 340 op/s	5.7× faster
Node.js (sync, fs.statSync)	0.2 ms (227.1 µs)	4 403 op/s	2.0× faster
Node.js (async, fs.stat)	0.5 ms (462.4 µs)	2 163 op/s	baseline

missing start path (tolerant fallback):

Scenario	Mean	Hz	Relative
native (sync)	0.03 ms (34.1 µs)	29 284 op/s	2.2× faster
Node.js (sync, fs.statSync)	0.08 ms (76.5 µs)	13 077 op/s	baseline

import {
  findNearestProjectFiles,
  findNearestProjectFilesSync,
} from "fast-fs-hash";

// Sync (recommended for startup-time / build-tool use)
const info = findNearestProjectFilesSync(import.meta.dirname);
console.log(info.packageJson, info.tsconfigJson, info.nodeModules);

// Async — runs on the native thread pool
const info2 = await findNearestProjectFiles("/some/deep/file.ts");

// Optional stopPath: halt when the walker reaches this directory (or any ancestor of it)
const info3 = findNearestProjectFilesSync(start, "/workspace");

Function	Description
`findNearestProjectFilesSync(startPath, stopPath)`	Walk parent chain for nearest markers (sync) → `NearestProjectFiles`
`findNearestProjectFiles(startPath, stopPath)`	Walk parent chain on pool thread → `Promise<NearestProjectFiles>`

The returned NearestProjectFiles object has these fields (each string | null):

Field	Description
`packageJson`	First `package.json` walking up.
`tsconfigJson`	First `tsconfig.json` walking up.
`nodeModules`	First `node_modules/` directory walking up (also detects when started inside).

Utility Functions

Function	Description
`hashToHex(digest)`	Convert a 16-byte digest to a 32-char hex string
`hashesToHexArray(digests)`	Convert an array of digests to hex strings
`findCommonRootPath(files, baseRoot?, allowedRoot?)`	Longest common parent directory of file paths
`normalizeFilePaths(rootPath, files)`	Resolve, sort, deduplicate paths relative to root
`toRelativePath(rootPath, filePath)`	Single path → clean unix-style relative path (or null)
`threadPoolTrim()`	Wake idle native pool threads so they self-terminate and free memory

Environment Variables

Variable	Default	Description
`FAST_FS_HASH_ISA`	auto-detect	Override SIMD variant: `avx512`, `avx2`, or `baseline` (x64 only)
`FAST_FS_HASH_POOL_IDLE_TIMEOUT_MS`	`15000`	Idle timeout for native pool threads (1–3600000 ms). Threads self-terminate after this duration with no work. They respawn automatically when new work arrives.

Acknowledgements

The native C++ backend uses:

xxHash by Yann Collet — xxHash3-128 hashing (BSD 2-Clause License)
LZ4 by Yann Collet — block compression (BSD 2-Clause License)

See NOTICES.md for full license texts.

Building from source

Prerequisites

Tool	Version	Install
Node.js	>= 22	nodejs.org
npm	>= 9	bundled with Node.js
CMake	>= 3.15	`brew install cmake` / `apt install cmake` / cmake.org
C++20 compiler	Clang 14+ / GCC 12+ / MSVC 2022	Xcode CLT / `build-essential` / Visual Studio

Quick start

git clone --recurse-submodules https://github.com/SalvatorePreviti/fast-fs-hash.git
cd fast-fs-hash
npm install
npm run build:all   # compile C++ addon + TypeScript
npm test            # run tests
npm run bench       # run benchmarks

Note: git clone --recurse-submodules is required to pull deps/xxHash (the xxHash source used by the native addon).

Git submodule (xxHash)

The deps/xxHash/ directory is a git submodule pointing to xxHash v0.8.3.

If you cloned without --recurse-submodules, initialize the submodule manually:

git submodule update --init --recursive

See package.json for the full list of available build scripts.

Release process

main — development branch. CI runs lint, typecheck, tests, and builds native binaries for all platforms on every push and PR.
publish — release branch. Pushing to publish triggers the full CI pipeline. After all builds and tests pass, a dry-run publish verifies all packages. An admin must then manually approve the publish job (via the npm-publish GitHub environment) to publish to npm, create a git tag, and deploy docs.

npm packages are published with provenance attestations via GitHub Actions OIDC — no npm tokens are stored in CI.

fast-fs-hash - v0.0.3

fast-fs-hash

Installation

FileHashCache — Binary cache invalidation

Why use FileHashCache?

FileHashCache benchmarks (705 files, ~24 MiB)

FileHashCache API

Example: Build cache with dynamic file list

Example: Simple build cache with known files

API Reference

Cache status

Inspecting per-file changes with `resolve()`

Lock-only sessions

xxHash128 — Direct hashing

File hashing benchmarks

Parallel file hashing (705 files)

In-memory buffer hashing

Hash files

Hash a single file

Hex string convenience

Hash buffers and strings

Zero-allocation `…To` variants

Streaming class

LZ4 Block Compression

LZ4 API

Read and compress a file

File Comparison

Find Project Root

Find Nearest Project Files

Utility Functions

Environment Variables

Acknowledgements

Building from source

Prerequisites

Quick start

Git submodule (xxHash)

Release process

License

Settings

On This Page

fast-fs-hash - v0.0.3

fast-fs-hash

Installation

FileHashCache — Binary cache invalidation

Why use FileHashCache?

FileHashCache benchmarks (705 files, ~24 MiB)

FileHashCache API

Example: Build cache with dynamic file list

Example: Simple build cache with known files

API Reference

Cache status

Inspecting per-file changes with resolve()

Lock-only sessions

xxHash128 — Direct hashing

File hashing benchmarks

Parallel file hashing (705 files)

In-memory buffer hashing

Hash files

Hash a single file

Hex string convenience

Hash buffers and strings

Zero-allocation …To variants

Streaming class

LZ4 Block Compression

LZ4 API

Read and compress a file

File Comparison

Find Project Root

Find Nearest Project Files

Utility Functions

Environment Variables

Acknowledgements

Building from source

Prerequisites

Quick start

Git submodule (xxHash)

Release process

License

Settings

On This Page

Inspecting per-file changes with `resolve()`

Zero-allocation `…To` variants