Performance Benchmarks¶
The project separates Criterion microbenchmarks, controlled multi-connection tests, and Linux process-level startup benchmarks. Network RTT, market state, and backend throttling must not be confused with local OpenD CPU cost.
Benchmark suites¶
| Scope | Command | Primary signal |
|---|---|---|
| API keys and limits | cargo bench -p futu-auth --bench auth_bench |
hash-index lookup, shared key generations, borrowed limit policy |
| Quote caches | cargo bench -p futu-cache --bench cache_lookup |
static/QOT hits, cache-key construction, narrow ticker reads |
| Frame codec | cargo bench -p futu-codec |
44-byte headers and frame encode/decode |
| Encryption | cargo bench -p futu-net |
AES ECB at representative body sizes |
| SQLite | cargo bench -p futu-gateway-core --bench stock_db_lookup |
150k-row code and warrant-owner lookups |
| Telemetry and WS | cargo bench -p futu-core --bench delay_stats, cargo bench -p futu-server --bench ingress_and_metrics |
delay-counter contention, the exact latency ring, WS binary slicing |
KeyStore::verify uses an immutable hash-indexed generation; it no longer scans linearly with the number of keys. Quote and static caches also have dedicated benchmarks, so historical fixed microsecond figures are not treated as cross-machine SLOs.
Comparable runs¶
A comparison must use the same machine, Rust toolchain, Cargo profile, and fixture. Run at least five independent processes; Criterion samples inside one process do not replace independent runs.
# Save a baseline under the same source/environment identity
cargo bench -p futu-auth --bench auth_bench -- --save-baseline perf-before
# Compare after the product change
cargo bench -p futu-auth --bench auth_bench -- --baseline perf-before
scripts/perf_acceptance.py evaluates machine-readable results fail-closed. Missing cells, fewer than five processes, machine/profile/input mismatches, overlapping confidence intervals, and confirmed regressions cannot become false green results. Overlapping intervals produce INCONCLUSIVE; implementations must not be tuned merely to cross a threshold.
Linux startup and resource boundary¶
scripts/linux_stock_cache_benchmark.py loads an incident-scale stock cache and records listener readiness, CPU, RSS, threads, and file descriptors over five attempts. Callers must explicitly provide both the previous-release and candidate archive, SHA-256, and full source SHA. The script no longer downloads or pins v1.5.2.
The candidate must independently satisfy absolute limits: all five attempts ready, no timeout or unexpected exit, TCP maximum 120 seconds, P95 90 seconds, RSS at most 2 GiB, and post-readiness average CPU at most 50%. A faster relative result cannot override a failed absolute limit.
Interpreting results¶
- Local cache APIs can expose microsecond-or-lower CPU differences and are suitable for allocation and lock-contention work.
- APIs that contact the Futu backend are usually dominated by network RTT; local optimization does not prove lower backend latency.
- Internal candidates use
release-ci(Thin LTO), while local--releaseuses full LTO. Results from different profiles are not directly comparable. - A benchmark pass proves only the named machine/input contract, not five-platform behavior, live quote/trade parity, or release readiness.
Criterion HTML reports are generated under target/criterion/report/index.html; this local output is not committed.