# Meshpocalypse Benchmark Results

> **Reproducibility:** Anyone can reproduce these results by running:
> ```bash
> git clone https://github.com/one137th/Meshpocalypse
> cd Meshpocalypse
> bash benchmarks/run.sh
> ```
> Requirements: Docker, Node.js 20+, 4 GB RAM

---

## Environment

| Property | Value |
|----------|-------|
| Date | 2026-03-12T23:06:17.150Z |
| OS | Linux 6.1.0-43-cloud-arm64 |
| Architecture | aarch64 |
| CPU cores | 4 |
| Node.js | v22.22.0 |
| Docker | 29.2.1, build a5c7197 |
| NATS image | nats:2.10-alpine |
| Oxigraph image | ghcr.io/oxigraph/oxigraph:latest |

---

## Results

### 1. NATS Baseline (raw pub/sub, no Meshpocalypse overhead)

✅ Status: ok

| Test | Msgs/sec | P50 latency | P99 latency |
|------|----------|-------------|-------------|
| Pub only (128B) | 3603366 | 0.11us | 0.27us |
| Pub+Sub (128B) | 2020552 | 0.11us | 0.26us |
| Request/Reply | 6064 | 163.48us | 195.42us |


### 2. MeshNode Overhead

✅ Status: ok | Iterations: 9000 | Transport: NullTransport

| Metric | P50 (ms) | P95 (ms) | P99 (ms) | Mean (ms) |
|--------|----------|----------|----------|-----------|
| Raw transport (baseline) | 0.0002 | 0.0002 | 0.0002 | 0.0002 |
| MeshNode.send() | 0.0073 | 0.0144 | 0.0192 | 0.0094 |

**Overhead at P50: 4268.9%**

> _Note: Uses NullTransport — measures serialisation + middleware stack overhead, not network I/O_


### 3. Auction Latency (POST /mesh/auction)

✅ Status: ok | Iterations: 100 per bidder count | Bid window: 50ms

| Bidders | P50 (ms) | P95 (ms) | P99 (ms) | Mean (ms) |
|---------|----------|----------|----------|-----------|
| 2 | 51.57 | 52.56 | 76.05 | 51.88 |
| 5 | 51.62 | 52.48 | 52.78 | 51.57 |
| 10 | 51.55 | 52.13 | 52.45 | 51.44 |

> **How to read this:** The ~51ms figure is almost entirely the configurable bid window (default `bidWindowMs: 50ms`) — the time the orchestrator waits for agents to submit bids before picking a winner. The actual protocol cost (HTTP parse → NATS publish → bid collection → winner selection → response) is ~1–2ms. In production, `bidWindowMs` can be tuned to as low as 5–10ms for latency-sensitive workloads, or raised to 200ms+ to give more agents time to bid. Auction latency scales with the bid window, not with bidder count.


### 4. SPARQL Latency (Oxigraph)

✅ Status: ok | Iterations: 50 per query

| Query | P50 (ms) | P99 (ms) | Mean (ms) |
|-------|----------|----------|-----------|
| Simple SELECT (100 triples) | 0.61 | 2.55 | 0.8 |
| Simple SELECT (1,000 triples) | 0.5 | 3.1 | 0.57 |
| Simple SELECT (10,000 triples) | 0.42 | 0.67 | 0.44 |
| Complex 3-join (10,000 triples) | 0.55 | 1.66 | 0.61 |
| INSERT triple | 0.31 | 1.84 | 0.37 |


### 5. Rate Limiter Overhead (InProcessBackend)

✅ Status: ok | Iterations: 90000

| Mode | P50 (ms) | P95 (ms) | P99 (ms) | Mean (ms) |
|------|----------|----------|----------|-----------|
| DISABLED (baseline) | 0.000132 | 0.000224 | 0.000737 | 0.000183 |
| STRICT (3-scope check) | 0.000606 | 0.000853 | 0.001244 | 0.000671 |

**Overhead at P50: 359.1% (STRICT vs DISABLED)**

> _Note: InProcessBackend — measures in-memory token bucket check overhead_


### 6. E2E Task Lifecycle (submitTask → task.completed)

✅ Status: ok | Iterations: 90 per agent count

| Agents | P50 (ms) | P95 (ms) | P99 (ms) | Mean (ms) |
|--------|----------|----------|----------|-----------|
| 1 | 0.25 | 0.38 | 0.86 | 0.27 |
| 2 | 0.25 | 0.47 | 1.21 | 0.29 |
| 5 | 0.29 | 0.45 | 0.61 | 0.31 |

> _Note: submitTask → task.completed round-trip via NATS pub/sub with mock worker agents_


---

## Summary: NATS Baseline vs Meshpocalypse Overhead

| Component | P50 | P99 | Notes |
|-----------|-----|-----|-------|
| NATS request/reply (nats bench) | 163µs | 195µs | Measured by official nats CLI |
| MeshNode serialisation + middleware | +7µs | +19µs | Protobuf encode + middleware stack |
| **Total MeshNode real-world overhead** | **~4%** | **~10%** | vs NATS request/reply baseline |

> **How to read this:** MeshNode adds approximately 7µs of serialisation and middleware cost on top of a 163µs NATS round-trip. That is ~4% overhead in real-world usage — not the "4,269%" figure visible in scenario 02, which compares against a NullTransport no-op and is not a meaningful real-world baseline.


---

## Methodology

- **NATS baseline** measured with the official `nats bench` CLI tool (scenario 01).
- **Meshpocalypse overhead** measured as the delta above baseline on the same machine (scenario 02).
- All TypeScript scenarios use `performance.now()` for sub-millisecond timing resolution.
- Warmup iterations are excluded from all percentile calculations.
- Scenarios are independent — each writes its own `results/*.json` file.
- Failed scenarios are noted here rather than aborting the run.

### How to interpret P50 / P95 / P99

| Percentile | Meaning |
|------------|---------|
| P50 | Median — half of requests complete faster than this |
| P95 | 95th percentile — only 5% of requests are slower |
| P99 | 99th percentile — only 1% of requests are slower (tail latency) |

### Known variables

- **Machine spec matters** — results will differ on slower hardware.
- **Docker networking** adds ~0.1–0.5 ms vs. native NATS.
- **JVM / OxiGraph warm-up** — first few SPARQL queries may be slower.
- **Shared host load** — run on an idle machine for most reproducible results.

---

## Individual scenario commands

```bash
# Run all
bash benchmarks/run.sh

# Run individually
bash benchmarks/scenarios/01-nats-baseline.sh
npx tsx benchmarks/scenarios/02-meshnode-overhead.ts
npx tsx benchmarks/scenarios/03-auction-latency.ts
npx tsx benchmarks/scenarios/04-sparql-latency.ts
npx tsx benchmarks/scenarios/05-rate-limiter.ts
npx tsx benchmarks/scenarios/06-e2e-task.ts

# Re-generate this file from existing results
npx tsx benchmarks/generate-results.ts
```

---

_Generated by `benchmarks/generate-results.ts` · Meshpocalypse benchmark harness_