DropWatch
Agent online
Splunk Agentic Ops · Observability

Provable correctness.
Clean signals.

DropWatch ships every flash-drop event to Splunk over HEC, then an LLM agent reads it back through the MCP Server, scores drop health, flags oversell-bot subnets, and monitors its own reasoning.

▸ Open the live dashboardWatch the 2:35 demoRequest access
dropwatch://ops · telemetry: mcp · agent: aiml LIVE

This isn’t a mockup — open the live agent dashboard → running on Cloudflare Workers, pulling telemetry over the Splunk MCP Server (telemetry: mcp).

0
oversold, by construction
75
drop-health, scored live
/24
oversell-bot subnet, caught
01 — How it works

An agent, not a dashboard

The same closed loop runs over any Splunk index. The flash drop is just the showcase payload.

01
InstrumentEvery hot path emits a structured event: claim, hold, expiry, oversell-reject, waitlist, checkout.
02
Ship to SplunkEvents stream to Splunk over the HTTP Event Collector. Splunk is the system of record.
03
ReasonThe agent pulls telemetry back via the MCP Server, reasons with Splunk Hosted Models (OpenAI-compatible fallback), and scores drop health 0–100.
04
ActIt recommends one concrete fix, applies it in one click, and auto-pages on-call — writing the action back to Splunk.
pull → summarize → reason → score → recommend → apply → page
02 — Why the signals are clean

The oversell-proof guarantee

Most stores read stock, then write the order — under a spike, hundreds pass that check at once. ZeroDrop never reads stock: every claim is one atomic conditional write. That’s why an oversell-reject can’t be a bug — only a bot. The signal is clean by construction.

  • ✓ No locks, no read-modify-write races — DynamoDB serializes every writer
  • ✓ 10,000 race for 100 units → exactly 100 win, the rest get an atomic waitlist slot
  • ✓ So a post-sellout reject cluster from one /24 = automated checkout bots, zero false positives
  • ✓ That guarantee is what makes the telemetry trustworthy enough to act on
the entire claim path
UpdateItem {
  Key: { PK: "DROP#aura-1", SK: "META" },
  ConditionExpression:
    "claimed < totalStock AND status = live",
  UpdateExpression: "SET claimed = claimed + 1"
}
// DynamoDB serializes every writer.
// 100 units -> exactly 100 winners. Oversold: 0
03 — Beyond flash drops

One pattern, any telemetry

Works on any Splunk index

Under the drop-specific detectors sits a generic z-score anomaly detector that knows nothing about the flash-drop taxonomy. Point the same agent at any stream and it scores health, ranks anomalies, and pages on-call the same way.

App / serviceserror-rate spikes, latency
ITOpsqueue depth, saturation
NetOpspacket drops, anomalies
Securityabuse + bot clusters
04 — What’s new

Built for Splunk’s latest AI capabilities

Recently shipped, all open source and exercised by the test suite.

Security
OWASP OAT-005 scalping detectionThe oversell-bot cluster is raised as a security finding (OWASP Automated Threats OAT-005) with a confidence score and a block action, reasoned by Splunk’s Foundation-Sec hosted security model. Provable correctness makes it a zero-false-positive bot signal.
Hosted Models
Splunk-native reasoning, with a floorThe agent reasons on Splunk Hosted Models via an OpenAI-compatible endpoint — and never goes dark: reasoning degrades Hosted Models → hosted API → a deterministic rules engine, while the read path falls back MCP → REST → local buffer.
AI monitoring
Agent self-observabilityDropWatch monitors its own agent: LLM tier, latency, token usage, estimated cost, confidence and drift, shipped to Splunk as dropwatch:agent. Parity with Splunk’s AI Agent Monitoring.
MCP
Runnable MCP path + | dropwatch commandThe agent pulls telemetry over the Splunk MCP Server’s run_splunk_search contract end-to-end, and | dropwatch runs the same detection natively in the Splunk search bar.
Detect
Anomaly detection + early warningA baseline z-score detector flags off-pattern behavior on any index, and claim-rate velocity plus p95 write latency warn of a building stampede before it crosses the threshold.
Respond
Alert webhooks + packaged Splunk appHigh-severity findings auto-page Slack or PagerDuty with the agent’s reasoning, and an installable Splunk app runs six detectors natively on a schedule.
05 — Access

Join the waitlist

DropWatch is built in the open. Get early access and product updates.

>
Splunk HECMCP ServerFoundation-SecNext.jsDynamoDBTypeScript
Jerom Tom
Built by
Jerom Tom

Building DropWatch in the open: agentic observability for oversell-proof flash drops.

jeromtom.comGitHub