CROSS-CUTTING INFRA · FOUNDATION TIER · 2026-05-10

OpenSearch Patterns

Cluster topology, rick/morty alias-swap reindex, reserved-asset must_not filter, marketplace fanout — the inventory search engine of tradeit.gg.

1. At-a-glance

2. Purpose & boundaries

3. Process / runtime model

Self-hosted OpenSearch in AWS eu-west-1, ARM/Graviton (m7g). Role-split topology with mTLS. All four client services share an identical dual-client pattern (coord for search, ingest for writes).

Cluster nodes

Client services

mTLS

All four clients decode three base64 env vars at boot to build the TLS context: OPENSEARCH_SSL_CA, OPENSEARCH_SSL_CERT, OPENSEARCH_SSL_KEY — ASCII/UTF-8 PEM strings (opensearch.js:15–19, openSearch.manager.ts:13–15). Self-managed certs — see §11 for the historical expiry incident.

4. Data flow diagrams

4.1 Indexing flow — sinv → parser → alias swap

graph TB
    subgraph "Source"
        S1["Steam Web API"]
        S2["3rd-party marketplaces
(UU, halo, …)"]
    end

    subgraph "Aggregation (Redis)"
        SINV["sinv:{gameId}:compressed
tradeit-inventory-server/sinvbot"]
        CINV["cinv (consumer view)"]
    end

    subgraph "Parser (NestJS)"
        CTRL["GET /index/:gameId
inventory.controller.ts"]
        LOCK["Redis lane lock
15 min TTL"]
        SVC["openSearchService.index()
bulk writes"]
    end

    subgraph "OpenSearch (eu-west-1)"
        ING["Ingest node"]
        DAT["Data nodes ×4"]
        ALIAS["alias inventory_730_trade
→ rick OR morty"]
    end

    S1 --> SINV
    S2 --> SINV
    SINV --> CTRL
    CINV --> CTRL
    CTRL --> LOCK
    LOCK --> SVC
    SVC --> ING
    ING --> DAT
    SVC -->|"POST /_aliases
atomic remove+add"| ALIAS
    ALIAS --> DAT

    classDef src fill:#1e2438,stroke:#a855f7,stroke-width:2px,color:#e9d5ff
    classDef redis fill:#0e2a1e,stroke:#10b981,stroke-width:2px,color:#a7f3d0
    classDef parser fill:#0c2536,stroke:#06b6d4,stroke-width:2px,color:#a5f3fc
    classDef os fill:#2a1f0e,stroke:#f59e0b,stroke-width:2px,color:#fde68a
    class S1,S2 src
    class SINV,CINV,LOCK redis
    class CTRL,SVC parser
    class ING,DAT,ALIAS os

4.2 Query flow — search read path

sequenceDiagram
    participant FE as new-tradeit (Nuxt)
    participant BE as tradeit-backend
    participant CFG as configurations
    participant R as Redis (reserve_assets)
    participant OS as OpenSearch (coord)

    FE->>BE: GET /inventory?gameId=730&filter=…
    BE->>CFG: getValueByKey('enableTradeLocked')
    BE->>CFG: getValueByKey('hideBotsItemsFromInventory')
    BE->>R: getAllReserveAssets(gameId)
    BE->>BE: getEnabledMarketplaceKeys() → [null,'uu','halo',…]
    BE->>BE: aliases = keys.map(buildInventoryAlias)
    BE->>OS: msearch across N aliases
    Note over BE,OS: must_not: terms{assetId: reserved}
+ terms{botIndex: hidden}
+ filters (price, float, stickers, …)
    OS-->>BE: hits + by_item agg buckets
    BE-->>FE: merged + ordered result set

4.3 Rick/Morty alias-swap state machine

graph LR
    subgraph "State A — alias points to RICK"
        A_RICK["inventory_730_rick
(active, queryable)"]
        A_MORTY["inventory_730_morty
(empty / stale)"]
        A_ALIAS{{"alias inventory_730"}}
        A_ALIAS -->|points to| A_RICK
    end

    subgraph "Reindex into MORTY"
        B1["1. ensureIndexExists(morty)"]
        B2["2. setRefreshInterval('-1')"]
        B3["3. bulkWrite chunks"]
        B4["4. refreshIndex(morty)"]
        B5["5. POST /_aliases
remove rick → add morty"]
        B6["6. delete rick"]
        B7["finally: setRefreshInterval('1s')"]
    end

    subgraph "State B — alias points to MORTY"
        C_MORTY["inventory_730_morty
(active)"]
        C_ALIAS{{"alias inventory_730"}}
        C_ALIAS -->|points to| C_MORTY
    end

    A_RICK --> B1 --> B2 --> B3 --> B4 --> B5 --> B6 --> B7
    B7 --> C_MORTY

    classDef ok fill:#0e2a1e,stroke:#10b981,stroke-width:2px,color:#a7f3d0
    classDef warn fill:#2a1f0e,stroke:#f59e0b,stroke-width:2px,color:#fde68a
    classDef hi fill:#1e2438,stroke:#a855f7,stroke-width:2px,color:#e9d5ff
    class A_RICK,C_MORTY ok
    class A_MORTY warn
    class B1,B2,B3,B4,B5,B6,B7 hi

5. Inputs & outputs

Writes (parser → OS)

Reads (backend / tradeit-service → OS)

6. Redis surface

The lane lock uses a Lua EVAL compareAndExpireScript for safe renewal — only the holder can extend (inventory.controller.ts:223). Lock holder spawns a background renewal interval at 7.5 min; releases on completion. For the full Redis keyspace see Redis Architecture.

7. MySQL surface

OS does not query MySQL directly. The dependency is one-directional:

• Parser → MySQL (read): inventory.service.ts reads csgo_collections, item_types, sale_offers to enrich docs before indexing (inventory.service.ts:35–37, 40, 71).
• Backend → MySQL (read): configurationService keys (§10) live in the configurations table.

MySQL is not a fallback for OS — when OS is down, the backend's only fallback is the degraded "own-inventory only" alias path (§11).

8. OpenSearch surface

8.1 Naming contract — @zengamingx/opensearch-indexing-contract v0.4.0

Single source of truth for every alias and index name. Repo: ~/Projects/opensearch-indexing-contract.

alias        = entityType "_" gameId [ "_" marketplaceKey ] [ envSuffix ]
physicalIdx  = alias "_" generation

Production examples

Public API (src/index.ts)

8.2 Document mapping — fields actually queried

_id of inventory docs = assetId; _id of group docs = groupId. Bulk write at openSearch.service.ts:prepareListForIndex sets _id: item.id.

8.3 Query DSL composition

openSearchService.js builds one query body per request, conditionally pushing clauses:

• filter (bool.filter): groupId/groupIds term(s), price range, float range, sticker exists, StatTrak match, trade-lock range, colors, onlyStore
• must_not (bool.must_not): built by mustNotGroup (:243–249) — composes terms{assetId:reservedAssets} (:319, 492, 686, 829, 873, 995), terms{botIndex:hideBotsItemsFromInventory} (:322, 493), exists{saleOfferOwner} when showUserListing=false
• must (bool.must): free-text on name — prefix-search subqueries + match with fuzziness AUTO
• aggregation by_item: terms{field:groupId, size:itemIds.length} with two child filters (store_items, trade_items) each containing a top_hits (size 1, sort by storePrice / priceForTrade asc) — produces "cheapest item per group" rollup (:1034–1047)

8.4 Bulk-write contract (parser)

8.5 Alias-swap mechanism

replaceAliasForIndex(previous, next, alias) at openSearch.service.ts:136+:

if (previous) {
  POST /_aliases
  { actions: [
    { remove: { index: previous, alias } },
    { add:    { index: next,     alias } }
  ]}
} else {
  PUT /{next}/_alias/{alias}
}
// post-swap: cat.aliases — verify alias points at `next`, throw otherwise
// then: DELETE /{previous}

The POST /_aliases is atomic in OpenSearch — the alias never points at two indices nor zero indices. Post-swap verification via cat.aliases (:110) prevents silent alias drift.

8.6 Slowlog thresholds

Source: inventory_template cluster definition (referenced from the reserved-assets-os-field design spec).

9. Async continuations

OpenSearch is involved in no Bull queue and no pub/sub channel in tradeit. The async surface is exclusively HTTP + cron-style external triggers.

Continuations from OS writes: none. After a successful alias swap the parser silently completes; there is no "indexed" event published to Redis or anywhere else.

10. Configuration flags & guards

configurations table keys

Env vars (parser + backend + service + admin-backend)

Backlinks for these keys are registered on the Configuration page (id 65).

11. Failure modes

Linked runbooks: Incident playbook (id 43, 44).

12. Observability

• Datadog: opensearch.* integration metrics (CPU, JVM, query/index latency, circuit breaker). Service-level: opensearch.error.rate, opensearch.slow_queries.count from slowlog ingestion. [OPEN: dashboard URL]
• Slowlog: cluster-level template (inventory_template) — every query >200 ms / index >500 ms hits the warn channel; tail via _cat/indices/{idx}?v.
• Slack alerts: x_content_parse_exception on queryByAssetIds (openSearchService.js:773–786); bulk permanent-failure surfaces from submitBulkChunk (parser).
• Logs: backend createModuleLogger('opensearch') at opensearch.js:7; parser ClassWithLogger mixin for every OS service.
• Cronitor: parser endpoint pings (whoever calls GET /index/:gameId is responsible — see [OPEN]).
• Prometheus: not currently scraping OS — uses Datadog integration instead.

13. Code map

Backend — ~/Projects/tradeit-backend/server/

Parser — ~/Projects/tradeit-inventory-parser/src/

Indexing contract — ~/Projects/opensearch-indexing-contract/src/

Other consumers

14. Open questions

Cross-references

• Foundation tier (Phase-1): Platform Configuration (id 65) · Redis Architecture (id 61)
• Domain spine (Phase-1): Inventory Search & Indexing (id 47) · Trade Lifecycle (id 57)
• Reference impl: tradeit-tradebot-server (id 67) — the deep-dive template
• Adjacent specs: ~/Projects/tradeit-backend/docs/superpowers/specs/2026-05-10-reserved-assets-os-field-design.md — the proposed must_not.terms → reservedUntil field migration

Generated 2026-05-10 · tradeit.gg engineering