Performance Baseline · April 14, 2026
Production APM snapshot — last 24h via Datadog + parser docker logs
CSGO Inventory Query (24h)
Avg 349ms · Peak 656ms · Off-peak 158ms
GET /api/v2/inventory/data · 28,350 requests · 0% error rate
CSGO dominates — largest index, HaloSkins merged
Peak: 15:00–21:00 UTC (5–11 PM CET) · Off-peak: 02:00–08:00 UTC
Peak Hours
656ms
3,926 requests · 15:00–21:00 UTC
Off-Peak Hours
158ms
3,641 requests · 02:00–08:00 UTC
4.2x latency increase during peak — EU evening traffic
Worst: 17:00 UTC (851ms) and 19:00 UTC (826ms)
Group Browse
290ms
Default listing · 3,754 req
Unstacking
371ms
Single group · 10,544 req
With Filters
463ms
type/rarity/etc · 515 req
| Query Type | Off-Peak | Peak | Multiplier |
|---|---|---|---|
| Group Browse | 137ms | 642ms | 4.7x |
| Unstacking | 168ms | 660ms | 3.9x |
Key Observation
Both query types degrade equally under load. The bottleneck is the monolithic inventory_internal_730 index — all queries compete for the same shard resources during peak. Marketplace-aware split targets this directly.
CSGO _internal_ indices are 2–4x slower than non-CSGO
| Index Pattern | Avg Latency | Calls/24h | Purpose |
|---|---|---|---|
inventory_internal_* | 129ms | 39,504 | CSGO item replacement + count |
group_internal_* | 76ms | 8,149 | CSGO group search |
inventory_* (non-CSGO) | 40ms | 16,827 | TF2/RUST/Steam items |
stickers_internal | 33ms | 3,374 | Sticker search |
group_* (non-CSGO) | 19ms | 2,728 | TF2/RUST/Steam groups |
Post-deploy: inventory_internal_730 splits into inventory_730 + inventory_730_haloskins
Source: docker logs · 2 PM2 workers (4 PIDs), every 10 min for 730
| Game | Items | Groups | Fetch | Parse | Bulk Write | Total |
|---|---|---|---|---|---|---|
| CSGO A | ~328k | ~15k | 14s | 14s | ~95s | ~135s |
| CSGO B | ~353k | ~18k | 15s | 14s | ~103s | ~145s |
| RUST | 83k | 2.2k | 0.9s | 1.4s | ~13s | ~17s |
| TF2 | 107k | 5k | 1.3s | 2.0s | ~21s | ~25s |
| Steam | 399k | 0.9k | 1.3s | 6.8s | ~72s | ~81s |
Range across 20+ cycles · ~328k items per write (Worker A)
Throughput: ~328k docs in ~95s = ~3,450 docs/s
Worst times at :27/:37 when RUST/TF2 overlap with 730 bulk write — shard contention
mustNot camelCase silently ignored by OpenSearch
queryCountItemsOnGroup sends "mustNot" instead of "must_not". Reserved-asset filtering is not applied to item counts.
Fixed in marketplace-aware branch (commit 1ca70ac8)
| Metric | Current Baseline | Target |
|---|---|---|
| CSGO 730 avg latency | 349ms | Same or lower |
| CSGO peak latency | 656ms | Lower (split indices) |
| OS inventory_internal avg | 129ms | Replaced by 2 smaller indices |
| OS group_internal avg | 76ms | Replaced by 2 smaller indices |
| Non-CSGO latency | 40–95ms | Unchanged |
| Parser 730 cycle time | ~135–145s | Own lane only; HaloSkins separate |
| Error rate | 0% | Stay at 0% |
Generated April 14, 2026 · Datadog APM + docker logs · tradeit.gg