eric/onix

Files

Mayuresh e0d7e3508f feat(benchmarks): auto-generate Interpretation/Recommendation and add -report-only flag

generate_report.go:
- buildInterpretation: derives narrative from p99/p50 tail-latency ratio,
  per-action complexity trend (% increase vs discover baseline), concurrency
  scaling efficiency (GOMAXPROCS=1 vs 16), and cache warm/cold delta
- buildRecommendation: identifies the best throughput/cost GOMAXPROCS level
  from scaling efficiency and adds production sizing guidance

run_benchmarks.sh:
- Add -report-only <dir> flag: re-runs parse_results.go + generate_report.go
  against an existing results directory without rerunning benchmarks

REPORT_TEMPLATE.md:
- Replace manual placeholders with __INTERPRETATION__ and __RECOMMENDATION__
  markers filled by the generator

2026-04-09 22:34:52 +05:30

4.8 KiB

Raw Blame History

beckn-onix Adapter — Benchmark Report

Run: __TIMESTAMP__ Platform: CPU · GOOS/GOARCH · GOMAXPROCS=GOMAXPROCS (default) Adapter version: ONIX_VERSION Beckn Protocol: v2.0.0

Part A — Executive Summary

What Was Tested

The beckn-onix ONIX adapter was benchmarked end-to-end using Go's native testing.B framework and net/http/httptest. Requests flowed through a real compiled adapter — with all production plugins active — against in-process mock servers, isolating adapter-internal latency from network variables.

Pipeline tested (bapTxnCaller): addRoute → sign → validateSchema

Plugins active: router, signer, simplekeymanager, cache (miniredis), schemav2validator

Actions benchmarked: discover, select, init, confirm

Key Results

Metric	Value
Serial p50 latency (discover)	P50_US µs
Serial p95 latency (discover)	P95_US µs
Serial p99 latency (discover)	P99_US µs
Serial mean latency (discover)	MEAN_DISCOVER_US µs
Peak parallel throughput	PEAK_RPS req/s
Cache warm vs cold delta	CACHE_DELTA
Memory per request (discover)	~MEM_DISCOVER_KB KB · ALLOCS_DISCOVER allocs

Interpretation

INTERPRETATION

Recommendation

RECOMMENDATION

Part B — Technical Detail

B0 — Test Environment

Parameter	Value
CPU	CPU (GOARCH)
OS	GOOS/GOARCH
Go package	`github.com/beckn-one/beckn-onix/benchmarks/e2e`
Default GOMAXPROCS	GOMAXPROCS
Benchmark timeout	30 minutes
Serial run duration	10s per benchmark × 3 runs
Parallel sweep duration	30s per GOMAXPROCS level
GOMAXPROCS sweep	1, 2, 4, 8, 16
Redis	miniredis (in-process, no network)
BPP	httptest mock (instant ACK)
Registry	httptest mock (dev key pair)
Schema spec	Beckn v2.0.0 OpenAPI (`beckn.yaml`, local file)

Plugins and steps (bapTxnCaller):

Step	Plugin	Role
1	`router`	Resolves BPP URL from routing config
2	`signer` + `simplekeymanager`	Signs request body (Ed25519/BLAKE-512)
3	`schemav2validator`	Validates Beckn v2.0 API schema

B1 — Latency by Action

Averages from run1.txt (10s, GOMAXPROCS=GOMAXPROCS). Percentile values from percentiles.txt.

Action	Mean (µs)	p50 (µs)	p95 (µs)	p99 (µs)	Allocs/req	Bytes/req
discover (serial)	MEAN_DISCOVER_US	P50_US	P95_US	P99_US	ALLOCS_DISCOVER	BYTES_DISCOVER (~MEM_DISCOVER_KB KB)
select	MEAN_SELECT_US	—	—	—	ALLOCS_SELECT	BYTES_SELECT (~MEM_SELECT_KB KB)
init	MEAN_INIT_US	—	—	—	ALLOCS_INIT	BYTES_INIT (~MEM_INIT_KB KB)
confirm	MEAN_CONFIRM_US	—	—	—	ALLOCS_CONFIRM	BYTES_CONFIRM (~MEM_CONFIRM_KB KB)

B2 — Throughput vs Concurrency

Results from the concurrency sweep (parallel_cpu*.txt, 30s per level).

THROUGHPUT_TABLE

B3 — Cache Impact (Redis warm vs cold)

Results from cache_comparison.txt (10s each, GOMAXPROCS=GOMAXPROCS).

Scenario	Mean (µs)	Allocs/req	Bytes/req
CacheWarm	CACHE_WARM_US	CACHE_WARM_ALLOCS	CACHE_WARM_BYTES
CacheCold	CACHE_COLD_US	CACHE_COLD_ALLOCS	CACHE_COLD_BYTES
Delta	CACHE_DELTA	—	—

B4 — benchstat Statistical Summary (3 Runs)

__BENCHSTAT_SUMMARY__

B5 — Bottleneck Analysis

Populate after reviewing the numbers above and profiling with go tool pprof.

Rank	Plugin / Step	Estimated contribution	Evidence
1
2
3

Profiling commands:

# CPU profile
go test ./benchmarks/e2e/... -bench=BenchmarkBAPCaller_Discover \
  -benchtime=30s -cpuprofile=benchmarks/results/cpu.prof -timeout=5m
go tool pprof -http=:6060 benchmarks/results/cpu.prof

# Memory profile
go test ./benchmarks/e2e/... -bench=BenchmarkBAPCaller_Discover \
  -benchtime=30s -memprofile=benchmarks/results/mem.prof -timeout=5m
go tool pprof -http=:6060 benchmarks/results/mem.prof

# Lock contention (find serialisation under parallel load)
go test ./benchmarks/e2e/... -bench=BenchmarkBAPCaller_Discover_Parallel \
  -benchtime=30s -mutexprofile=benchmarks/results/mutex.prof -timeout=5m
go tool pprof -http=:6060 benchmarks/results/mutex.prof

Generated from run __TIMESTAMP__ · beckn-onix ONIX_VERSION · Beckn Protocol v2.0.0

4.8 KiB Raw Blame History Unescape Escape