Service Deep-Dive · Backend · May 15, 2026
End-to-end: how item prices flow from market data through the pricing engine into MySQL
pricing-manager is a standalone NestJS service that recalculates buy/sell prices for all ~25,000 CS2 skin items in a single synchronous pass. It is the only writer of the 5 canonical price fields (bot_trade_price, player_trade_price, store_price, instant_sell_price, stable_price) in steamtrade.item_prices_raw. Triggered externally via HTTP; Redis-locked to prevent concurrent runs.
item_prices_rawprice_adjusterGET /snapshots/take — point-in-time price snapshotssteam-pricing-scraper)tradeit-buff-scrapper)tradeit-stock-manager writes item_stocks)ec2-54-171-92-22.eu-west-1.compute.amazonaws.com--restart unless-stopped, port 4000 (host) → 3000 (container)npm run start:dev; production container uses Node defaultNODE_ENV !== 'production'zengamingx/pricing-manager (GitHub Packages), built via docker build --build-arg DOTENV_KEY=... --build-arg NODE_ENV=production.env.vault + DOTENV_KEY build arg)
graph LR
subgraph "Upstream scrapers (separate services)"
SPS["steam-pricing-scraper"]
BUFF["tradeit-buff-scrapper"]
CSGOS["csgoskins-pricing"]
end
subgraph "External API (called at run time)"
CSAPI["csgoskins.gg\n/prices"]
end
subgraph "MySQL steamtrade (reads)"
BPS[("buff_prices_scraper")]
SNAP[("item_prices_snapshot")]
TIS[("trade_item_stats")]
SIS[("store_item_stats")]
EOD[("item_eod")]
PG[("pricing_groups")]
CONF[("configurations")]
end
subgraph "MySQL tradeitConnection (reads)"
ISTK[("item_stocks")]
end
subgraph "pricing-manager"
PM["AppService\n.updatePrices()"]
end
subgraph "MySQL steamtrade (writes)"
IPR[("item_prices_raw")]
PA[("price_adjuster")]
end
subgraph "Consumers"
BE["tradeit-backend\ntrade evaluation"]
NT["new-tradeit\nmarketplace display"]
ADM["tradeit-admin\npricing overrides"]
end
SPS --> BPS
BUFF --> BPS
CSAPI --> PM
BPS --> PM
SNAP --> PM
TIS --> PM
SIS --> PM
EOD --> PM
PG --> PM
CONF --> PM
ISTK --> PM
PM --> IPR
PM --> PA
IPR --> BE
IPR --> NT
IPR --> ADM
graph TD
S1["1. READ buff_prices_scraper\n(CS2 appId=730 items only)"]
S2["2. JOIN item_stocks\n(bot/container/reserved/listing stock)"]
S3["3. FETCH csgoskins.gg /prices API\n(market prices by market_hash_name)"]
S4["4. STABLE PRICE\nECDF + quantile (mathjs) + mean of last 30 snapshots"]
S5["5. ASSIGN MARGINS\npricingGroups lookup by stable_price tier (cached 2 min)"]
S6["6. MERGE TRADE STATS\ntrade_item_stats + store_item_stats (if config flag enabled)"]
S7["7. BASE PRICES\nMultiplier enum tiers: Base=1.75, Low=1.725, High=1.775..."]
S8["8. LOAD PREVIOUS PRICES\nitem_prices_raw (prev bot/player/stable)"]
S9["9. LOAD EOD PRICES\nitem_eod avg/median for 7d and 30d max stock"]
S10["10. READ RUNTIME CONFIG\nweeklyInOutRatioWeight + wantedMaxStockLowCapacityRatio"]
S11["11. CALCULATE PER ITEM\ncurrentStock, wantedMaxStock, neededStock, deficit"]
S12["12. ADJUST and WRITE\nspread ratio, protection rails, markup flags\n→ UPSERT item_prices_raw + price_adjuster"]
S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 --> S8 --> S9 --> S10 --> S11 --> S12
classDef read fill:#0e2a1e,stroke:#10b981,stroke-width:2px,color:#a7f3d0
classDef calc fill:#1e2438,stroke:#a855f7,stroke-width:2px,color:#e9d5ff
classDef write fill:#2a1218,stroke:#f43f5e,stroke-width:2px,color:#fecdd3
classDef config fill:#2a1f0e,stroke:#f59e0b,stroke-width:2px,color:#fde68a
class S1,S2,S3,S8,S9 read
class S4,S5,S6,S7,S11 calc
class S12 write
class S10 config
| Direction | Surface | Description |
|---|---|---|
| IN | GET /update-prices | HTTP trigger — starts the full pipeline. Returns {"message":"Prices updated successfully"} after completion. Redis-locked for 3600 s. |
| IN | GET /snapshots/take | Manually triggers a price snapshot. Reads item_prices_raw.live_price → batch-writes to item_prices_snapshot. Also Redis-locked (3600 s). |
| IN | GET /health | NestJS Terminus: pings both DBs, checks disk <75%, heap <150 MB. |
| IN | GET / | Health-check hello — returns HELLO_MESSAGE env var. |
| IN | MySQL steamtrade | Reads: buff_prices_scraper, item_prices_snapshot, trade_item_stats, store_item_stats, item_eod, pricing_groups, configurations, item_prices_raw (prev prices). |
| IN | MySQL tradeitConnection | Reads: item_stocks (bot stock, container stock, reserved, locked, listing). |
| IN | csgoskins.gg /prices | External HTTP API — market prices per market_hash_name across multiple markets (csfloat, buff, skinport etc.). |
| OUT | MySQL steamtrade.item_prices_raw | UPSERT of 5 price types (bot, player, store, instant-sell, stable) + stock fields + tags per item. |
| OUT | MySQL steamtrade.price_adjuster | Writes adjustment percentages (item_price_adjust_percent) and protection diff columns. |
| OUT | MySQL steamtrade.item_prices_snapshot | Batch-written by GET /snapshots/take — one row per item per snapshot. |
Redis is used only for distributed locking and in-process cache. No pub/sub, no persistent key storage for prices. Redis client uses plain SET key 'locked' NX EX ttl.
| Key | TTL | Written by | Read by | Behavior on lock held |
|---|---|---|---|---|
updatePrices |
3600 s | app.service.ts:73 (@LockWithRedis decorator) |
Same decorator on entry | Silent no-op — logs "Another instance is already running" and returns immediately |
takeSnapshot |
3600 s | item-prices-snapshot.service.ts:23 |
Same | Silent no-op |
pricingGroups in-process |
120 s | pricing-groups.service.ts:33 (@CacheReturnValue) |
Same service, subsequent calls within TTL | Cache miss → re-reads DB. Not a Redis key — stored in a JS Map. |
Connection: redis://${TI_REDIS_HOST}:${TI_REDIS_PORT} — no auth in env schema (shared ElastiCache). Lock key naming is plain string — no prefix/namespace. Lock is released in finally block (redis.decorator.ts:34) but a crashed process leaves the key until TTL expiry.
MYSQL_* env vars)| Table | Ops | Key columns / indexes | Notes |
|---|---|---|---|
buff_prices_scraper |
SELECT | item_id, buff_buy, buff_sell, buff_sell_amount, buff_buy_amount, is_doppler |
Filtered to app_id=730 (CS2 only). Written by tradeit-buff-scrapper. |
item_prices_snapshot |
SELECT INSERT | item_id, live_price, created_at — indexes: item_id_created_at_IDX |
Reads last 30 snapshots per item to compute stable price mean. Written by /snapshots/take. |
item_prices_raw |
SELECT UPSERT | item_id (UNQ), bot_trade_price, player_trade_price, store_price, instant_sell_price, stable_price, live_price, wanted_max_stock, needed_stock, current_stock, final_deficit, popularity_score, ... |
The primary price table. Consumers (tradeit-backend, new-tradeit) read from here. Indexes: UNQ_item_id, IDX_updated_at. |
price_adjuster |
UPSERT | item_price_adjust_percent, protection diff columns (wanted_max_stock_spike, wanted_max_stock_in_out_ratio) |
Stores per-item adjustment breakdown. Written by price-adjuster.service.ts:adjustFinalPrices. |
pricing_groups |
SELECT | min_price, group, margin |
Price tier table — maps stable_price ranges to margin groups. Cached in-process for 2 min. |
trade_item_stats |
SELECT | item_id, monthly_in_amount, monthly_out_amount, weekly_in_amount, weekly_out_amount, daily_in_amount, monthly_in_unique_amount, monthly_out_unique_amount, ... |
Bot/trade volume stats used for wanted_max_stock calc. Written by tradeit-backend. Merged with store_item_stats via mergeWith(add). |
store_item_stats |
SELECT | Same schema as trade_item_stats |
Store-channel volume stats. Conditionally included via pricingIncludeStoreItemStats config flag. |
item_eod |
SELECT | item_id, stable_price, bot_trade_price, store_price, instant_sell_price, wanted_max_stock |
End-of-day historical prices. Used to compute 7d/30d median max stock for wanted_max_stock calculation. |
configurations |
SELECT | key, value (varchar), type |
Runtime config flags — see §10. Values always returned as strings; booleans stored as '1'/'0'. |
TI_MYSQL_* env vars)| Table | Ops | Key columns | Notes |
|---|---|---|---|
item_stocks |
SELECT | item_id, bot_stock, container_bot_stock, tradable_container_bot_stock, locked_reserved_items_stock, temp_reserved_items_stock, user_listing_stock |
Written by tradeit-stock-manager. Effective current stock = botStock + containerBotStock + tradableContainerBotStock − lockedReservedItemsStock − userListingStock. |
N/A — pricing-manager has no OpenSearch dependency. Prices are read from MySQL by the services that feed OpenSearch (tradeit-backend, tradeit-inventory-parser).
pricing-manager has no Bull queues, no pub/sub, no cron jobs. It is purely request-driven — both endpoints are triggered by external callers.
| Endpoint | Called by | Cadence | Notes |
|---|---|---|---|
GET /update-prices |
[OPEN: who calls this?] | ~every 10 min (inferred from pricing cycle in cron catalog) | Externally orchestrated. The Redis lock prevents overlap — concurrent calls no-op. |
GET /snapshots/take |
[OPEN: who calls this?] | Unknown cadence | Point-in-time snapshot. Must be called regularly to keep the last-30-snapshots mean meaningful for stable price. |
All flags read from steamtrade.configurations via AdminConfigService.getByKey(). Values are varchars; boolean flags use '1'/'0' and are compared with === '1'.
| Key | Type | Read at | Effect |
|---|---|---|---|
weeklyInOutRatioWeight | float | app.service.ts:106 | Weight for weekly in/out trend in wanted_max_stock calculation. |
wantedMaxStockLowCapacityRatio | float | app.service.ts:107 | Cuts max stock proportionally when bot capacity is low. |
blockDepositByPrevMonthPrice | bool ('1'/'0') | price-adjuster.service.ts:584 | When '1', sets deposit price to 0 for items where price dropped vs prev-month average. |
adjustByLiveToStablePriceRatio | bool | price-adjuster.service.ts:585 | Enables live-to-stable price ratio adjustment pass. |
blockDepositByAvg7DStablePrice | bool | price-adjuster.service.ts:586 | Blocks deposits when live price dips below 7-day stable average threshold. |
liveToStableModifier | float | price-adjuster.service.ts:587 | Scaling factor for the live-to-stable price ratio adjustment. |
avg7DStableBotPriceModifier | float | price-adjuster.service.ts:588 | Scaling factor when adjusting bot price using 7-day average stable price. |
tradePriceMarkup | float | price-adjuster.service.ts:740 | Final markup added to bot trade price. |
storePriceMarkup | float | price-adjuster.service.ts:741 | Markup added to store price. |
instantSellPriceAdjustPercent | float (percent) | price-adjuster.service.ts:742 | Percentage adjustment applied to instant-sell price. Stored as whole number (e.g. 5 = 5%). Divided by 100 at read time. |
decreaseDepositPriceForCheapItems | bool | price-adjuster.service.ts:743 | Reduces deposit price for items below the cheap-item threshold. |
pricingIncludeStoreItemStats | bool | store-item-stats.service.ts:17 | When enabled, merges store-channel volume stats with bot-channel stats for wanted_max_stock calculation. |
| Constant | Value | File:line | Purpose |
|---|---|---|---|
TRADE_MULTIPLIER | 1.75 | price-adjuster.service.ts:27 | Core ratio between tradeit price and weighted market price. |
Multiplier.Base | 1.75 | base-price-calculator.enum.ts:2 | Default Multiplier tier for base bot/player price calculation. |
Multiplier.Low | 1.725 | base-price-calculator.enum.ts:3 | Used for low-margin-group items. |
Multiplier.High | 1.775 | base-price-calculator.enum.ts:4 | |
Multiplier.VeryHigh | 1.81 | base-price-calculator.enum.ts:5 | |
Multiplier.Extreme | 1.84 | base-price-calculator.enum.ts:6 | |
OVERSTOCK_DEFICIT | -1 | price-adjuster.service.ts:26 | Deficit value assigned when item is overstocked (>30 stock & daily incoming ≥ daily max). |
LAST_SNAPSHOT_PRICES_NUM | 30 | stable-price-calculator.service.ts:17 | Number of historical snapshots used for stable price mean. |
TOP_PERCENTILE | 0.70 | stable-price-calculator.service.ts:18 | ECDF quantile upper bound — filters out top 30% of prices. |
UNSURE_PRICE_MULTIPLIER | 1.75 | price-adjuster.service.protections.ts:9 | Fallback multiplier applied to stablePrice when finalBot is zero (Doppler protection). |
TRADEIT_MARKET_SHARE_MAX | 0.50 | item-prices-raw.service.protections.ts:93 | Cap: tradeit cannot target holding > 50% of total market supply for an item. |
TRADEIT_MARKET_SHARE_NORMAL | 0.30 | item-prices-raw.service.protections.ts:94 | For high-value items (>250 USD stable), cap at 30% market share. |
TRADEIT_HIGH_STABLE_PRICE | 25000 (¢) | item-prices-raw.service.protections.ts:95 | $250 stable price — threshold for stricter market-share cap. |
PERCENTAGE_CHANGE_THRESHOLD | 0.03 | price-adjuster.service.ts:28 | Maximum allowed percentage change for monthly price averages. |
MarginGroupThreshold.VeryHigh | 0.95 | base-price-calculator.enum.ts:10 | Margin group tier thresholds. |
Trigger: Node process killed/OOM during a run before the finally block in redis.decorator.ts executes.
Blast radius: All subsequent GET /update-prices calls silently no-op for up to 1 hour (lock TTL). No error surfaced to caller — returns 200 immediately.
Recovery: redis-cli DEL updatePrices on the ElastiCache Redis, or wait for TTL expiry.
Trigger: HTTP error from csgoskins.gg /prices endpoint (rate limit, downtime).
Blast radius: Pipeline aborts mid-run (axios catchError throws). item_prices_raw not updated for this cycle. Redis lock is released in finally so next trigger retries normally.
Recovery: Automatic — next scheduled trigger retries. No circuit breaker implemented.
Trigger: Network interruption or Aurora failover during run.
Blast radius: Pipeline throws, prices stale until next successful run. Docker restart policy (--restart unless-stopped) restarts the container.
Recovery: Container auto-restarts. Health endpoint (/health) pings both connections — useful to identify which DB is down.
Trigger: Run takes longer than the trigger interval. Since the Redis lock is held for the duration and up to 3600 s, overlapping requests are silently dropped.
Blast radius: Prices may lag behind market for items processed late in the pipeline.
Recovery: Monitor run duration via @timer() logs on AppService.updatePrices.
Trigger: Using axios instead of fetch for Doppler items in tradeit-buff-scrapper. This is an upstream scraper issue but worth noting.
Blast radius: Doppler items get no buff data → PriceAdjusterServiceProtections.protectDopplers applies the UNSURE_PRICE_MULTIPLIER (1.75) fallback on stablePrice.
Recovery: Fix is in tradeit-buff-scrapper — Doppler requests MUST use native fetch.
@LogAllFunctions() on AppService logs every method entry/exit. @timer() on key services logs execution time. Container logs via docker logs pricing-manager.GET /health — NestJS Terminus checks: tradeit-database ping, pricing-database ping, disk <75%, heap <150 MB. Returns standard Terminus JSON.app.service.ts logs weeklyInOutRatioWeight and wantedMaxStockLowCapacityRatio values every run (visible in Docker logs). price-adjuster.service.ts logs storePriceMarkup and instantSellPriceAdjustPercent on every run.| Component | File:line | Notes |
|---|---|---|
| HTTP entry — GET /update-prices | src/app.controller.ts:12 | Calls appService.updatePrices() |
| HTTP entry — GET /snapshots/take | src/item-prices-snapshot/item-prices-snapshot.controller.ts:11 | Calls snapshotService.takeSnapshot() |
| HTTP entry — GET /health | src/health/health.controller.ts:25 | NestJS Terminus: 2 DB pings + disk + heap |
| Pipeline orchestrator | src/app.service.ts:75 | AppService.updatePrices() — 12-step pipeline |
| Redis lock decorator | src/app.service.ts:73 | @LockWithRedis('updatePrices', 60 * 60) |
| Lock implementation | src/common/redis/redis.decorator.ts:7 | SET key 'locked' NX EX ttl via LockService |
| Redis client setup | src/common/redis/redis.module.ts:16 | Creates client using TI_REDIS_HOST/TI_REDIS_PORT |
| Step 1 — Buff data fetch | src/buff-prices-scraper/buff-prices-scraper.service.ts:16 | Reads buff_prices_scraper WHERE app_id=730 |
| Step 2 — Stock join | src/item-stocks/item-stocks.service.ts:18 | Reads item_stocks (tradeitConnection) |
| Step 3 — csgoskins fetch | src/csgoskins-prices-fetcher/csgoskins-prices-fetcher.service.ts:21 | HTTP GET /prices via HttpService (axios) |
| Step 4 — Stable price | src/stable-price-calculator/stable-price-calculator.service.ts:27 | ECDF + quantileSeq (mathjs) + mean of last 30 snapshots |
| Snapshot mean lookup | src/item-prices-snapshot/item-prices-snapshot.service.ts:23 | Reads item_prices_snapshot, locked with @LockWithRedis('takeSnapshot', 3600) |
| Step 5 — Margin groups | src/pricing-groups/pricing-groups.service.ts:22 | Finds tier by findLast(lte(minPrice, price)); cached 2 min |
| Step 6 — Item stats merge | src/app.service.ts:211 | Parallel read of trade_item_stats + store_item_stats; merged via mergeWith(add) |
| Step 7 — Base prices | src/base-price-calculator/base-price-calculator.service.ts | Applies Multiplier enum tiers based on marginGroup |
| Multiplier enum | src/base-price-calculator/base-price-calculator.enum.ts:1 | Base=1.75, Low=1.725, High=1.775, VeryHigh=1.81, Extreme=1.84 |
| Step 8 — Previous prices | src/item-prices-raw/item-prices-raw.service.ts | Reads previous item_prices_raw row per item |
| Step 9 — EOD prices | src/item-eod/item-eod.service.ts:49 | getDayAvg and getDayMedian for 7d/30d columns |
| Step 11 — Deficit calc | src/price-adjuster/price-adjuster.service.ts:48 | calcDeficit() — inventory balance algorithm. OVERSTOCK_DEFICIT=-1 |
| Step 12 — Final adjust + write | src/price-adjuster/price-adjuster.service.ts:735 | adjustFinalPrices() — spread ratio, markup flags, store price, instant-sell |
| Doppler protection | src/price-adjuster/price-adjuster.service.protections.ts:18 | Fallback: stablePrice * UNSURE_PRICE_MULTIPLIER (1.75) when finalBot = 0 |
| Market-share protection | src/item-prices-raw/item-prices-raw.service.protections.ts:93 | Caps wanted_max_stock at 50% (30% for high-value items) of total market supply |
| Admin config reader | src/admin-config/admin-config.service.ts:12 | Reads from configurations table; returns null when key absent |
| Module wiring | src/app.module.ts:43 | Two TypeORM connections: default (MYSQL_*) + tradeitConnection (TI_MYSQL_*) |
| Bootstrap | src/main.ts | Listens on port 3000 |
The service has no internal cron. Something external calls GET /update-prices on schedule. Candidates: tradeit-backend cron, a dedicated scheduler service, or a crontab on the same EC2. Owner to confirm.
Same question — the snapshot endpoint is separate from the main pipeline and must be called independently to keep the last-30-snapshots window fresh. Cadence unknown.
No Datadog agent config or Cronitor ping found in the repo. Confirm whether production container has a sidecar agent or if this service is currently unmonitored.
src/popularity-marker/popularity-marker.service.ts:60 — markPopularity() has // Placeholder for actual implementation. The in-memory popularity map is populated (shuffleArray logic) but never persisted. Confirm whether this is WIP or intentionally disabled.
app.service.ts and price-adjuster.service.ts have bare console.log() calls that log config values on every run. These should be converted to this.verbose() or removed to reduce log noise in production.
Generated 2026-05-15 · tradeit.gg engineering · Owner: pricing team