Ask any analyst how they spot a story turning against a brand and they'll describe a shape: volume jumps well above baseline, and then, instead of the usual snap-back, sentiment stays low hour after hour. A spike that reverts is noise. A spike that holds negative is a story developing.
Notice what kind of knowledge that is. It's a pattern over a sequence of events — this, then this, then at least N of that. Analysts carry these patterns in their heads and apply them by reading dashboards after the fact. The pattern is real expertise, and it's stored in the one place where it can't run continuously: a person.
Stream processing has a tool aimed exactly at this: complex event processing, which in Flink's SQL arrives as `MATCH_RECOGNIZE`. You declare the shape, and the standing query watches every brand's stream for it:
```sql PATTERN (BASE SPIKE NEG{6,}) DEFINE SPIKE AS SPIKE.mentions >= BASE.mentions * 4, NEG AS NEG.avg_sentiment < -0.3 ```
Read it aloud and it's the analyst's sentence: a baseline hour, then a fourfold volume spike, then six or more consecutive hours of sentiment below -0.3. When the shape completes for some brand, out comes one row — brand, when the spike hit, how bad the sentiment held — and that row can go straight to a Slack channel.
Three things change when the pattern moves out of the analyst's head and into a declaration.
It runs at all hours, across every brand at once, which no analyst does. The Tuesday 3am story gets flagged at Tuesday 3am.
It becomes reviewable. "Fourfold spike, six negative hours, threshold -0.3" is on the page, so it can be argued with, tuned, and versioned. Tacit pattern knowledge can't be code-reviewed; declared patterns can.
Best of all, it becomes testable against history. Run the same pattern over last quarter's archived events in batch mode and count what it would have flagged. Forty alerts a quarter means the thresholds are wrong; three alerts, and all three were real stories, means you've captured something. You tune the definition on the past before trusting it with the present — the same discipline as backtesting a trading rule, minus the money. (Standard gloss: none of this is trading or investment advice.)
The honest limit: a declared pattern only catches shapes you've already articulated. The story that develops in a way nobody anticipated slips through, and that's what the human read of the data remains for. CEP doesn't retire the analyst. It takes the patterns the analyst already believes in and puts them on shift work.
What's still open
Which patterns earn a declaration? Spike-then-sustained-negative is the obvious first one. Slow leaks (steady decline with no spike at all) need a different shape, and detecting the pattern nobody has named yet is a research question, one the knowledge-graph side of the house is better positioned to answer than the stream side.
Related
-