Home › Enterprise Intelligence Architecture

Decision Intelligence & Anomaly Detection

Decision support architecture, statistical baselines and isolation forest, the economics of false positives, and closing the loop from signal to decision to recorded outcome.

Last reviewed: 2026-09-04 · Українською

In this area

Layer four of an enterprise intelligence architecture does not produce data — it produces decisions. This module shows why an anomaly dashboard is not a decision support system; how isolation forest actually scores a point and why its contamination parameter is a staffing plan rather than a model hyperparameter; and how to pick an operating threshold in currency rather than in F1. One thesis runs through all three lessons: a detector whose decision outcomes are never recorded has neither labels to learn from nor evidence of its own worth.

From signal to action: the anatomy of a decision support system

E7.1

A dashboard is not a decision support system. A dashboard displays state; a DSS walks a named person from signal to action and records what came of it. The difference is not cosmetic: with no recorded outcome the system never acquires labels, and with no labels no model inside it can learn and no analyst can prove it pays for itself.

The four classical DSS subsystems — and what each means in a modern architecture:

• Data — not table rows but resolved ontology entities: this payment, this counterparty, this network node. An anomaly bound to a row_id rather than to an entity has no addressee and does not survive a duplicate merge.
• Models — detectors, rules, simulators. A model produces a score, never a verdict.
• Knowledge — thresholds, playbooks, access rights, purpose limitation. This is where business policy lives, including the binding constraint of the whole layer: how many alerts the team can actually clear per shift.
• Dialogue — the decision surface: why it fired, what the evidence is, which actions are available, who owns it and by when.

The decision is a first-class object. The minimum record: entity identifier, trigger (which model, which version), score and the threshold in force at fire time, an evidence snapshot, the action chosen, the decider, the timestamp, the reversibility class, and — mandatory — an outcome field written later. That field closes the detect → decide → act → observe loop and turns an operational system into a source of training data. A dismissed alert is also a decision: dismissals are the bulk of your negative labels, and they are the ones most often never written down.

The decision latency budget. Architecture is set not by model capacity but by the time left between the signal and the moment an action still changes anything. Card authorisation: hundreds of milliseconds. Network incident triage: minutes. Credit-limit review: a day. The same detector lives in three different places across those regimes — the synchronous transaction path, a streaming pipeline, an overnight batch. Trying to serve all three from one synchronous call is the most expensive mistake in this layer; the correct move is to precompute heavy aggregates into a feature store and leave only scoring on the latency-critical path.

Label lag. Ground truth is almost never immediate: confirmed fraud arrives as a chargeback 30–60 days later, the consequence of a clinical decision after weeks. Your training set therefore always trails production. Store two timestamps: when the truth occurred and when you learned it. Cut training sets on the second one — cutting on the first quietly trains the model on information that did not exist at scoring time.

-- The decision as a first-class object: the record that closes the loop.
-- EVERY alert that reaches a human lands here, including the dismissed ones.
CREATE TABLE decision (
  decision_id   BIGSERIAL PRIMARY KEY,
  entity_urn    TEXT             NOT NULL,  -- ontology object, NOT a source row id
  detector      TEXT             NOT NULL,  -- 'iforest_payments'
  detector_ver  TEXT             NOT NULL,  -- exact model version that fired
  score         DOUBLE PRECISION NOT NULL,
  threshold     DOUBLE PRECISION NOT NULL,  -- threshold IN FORCE at fire time
  evidence      JSONB            NOT NULL,  -- top features + baseline snapshot
  action        TEXT             NOT NULL
                CHECK (action IN ('block','hold','ignore','escalate')),
  reversible    BOOLEAN          NOT NULL,  -- gates what may auto-execute
  decided_by    TEXT             NOT NULL,  -- analyst id, or 'auto'
  decided_at    TIMESTAMPTZ      NOT NULL DEFAULT now()
);

-- Outcome arrives LATER. Two timestamps, never one:
-- when the truth happened, and when we found out.
CREATE TABLE decision_outcome (
  decision_id   BIGINT      PRIMARY KEY REFERENCES decision(decision_id),
  label         TEXT        NOT NULL CHECK (label IN ('tp','fp','fn','unknown')),
  loss_amount   NUMERIC(14,2),
  occurred_at   TIMESTAMPTZ NOT NULL,
  known_at      TIMESTAMPTZ NOT NULL       -- label lag = known_at - decided_at
);

-- Training sets are cut on known_at, never on occurred_at: cutting on
-- occurred_at leaks labels that did not exist when the score was produced.
CREATE VIEW trainable_decisions AS
SELECT d.*, o.label, o.loss_amount,
       o.known_at - d.decided_at AS label_lag
FROM decision d
JOIN decision_outcome o USING (decision_id)
WHERE o.known_at <= now() - INTERVAL '1 day'
  AND o.label <> 'unknown';

Baselines and isolation forest: what you measure deviation against

E7.2

An anomaly is a violated expectation, not a large number. The first architectural decision is therefore not the algorithm but the comparison base: the global distribution, the individual entity, or a peer group. A €40,000 payment is anomalous for a retail customer and entirely ordinary for a wholesaler; a global 'amount > 10,000' rule is not a detector but a filter that surfaces the same hundred known names every day.

Three classes of anomaly demand three different mechanisms:

• Point — a single observation far from the rest. The working baseline is a robust z-score on median and MAD: 0.6745·(x−med)/MAD, not mean and σ. The reason is mechanical: outliers inflate σ themselves and thereby mask themselves, whereas median and MAD have a 50% breakdown point.
• Contextual — the value is normal in general but not in this context: 300 requests at 3 a.m. on a Sunday. Deseasonalise by (day-of-week × hour) before scoring, or the detector will announce the start of the working week every Monday at 09:00.
• Collective — no single point is an outlier; the sequence or subgraph is: 200 transfers of €900 each (structuring). Point methods cannot see this by construction; you need per-entity windowed aggregates or graph models over resolved entities.

Isolation forest: why it works. Rather than describing 'normal', the algorithm isolates a point with random splits on random features. Anomalies are cut off in few splits, so the average path length E(h(x)) is short; the score s = 2^(−E(h(x))/c(n)) normalises it into (0,1). Practical consequences: ~O(n·log n) cost, no labels required, robustness in high dimension — and one counter-intuitive detail: a 256-point subsample per tree outperforms the full set, because large subsamples amplify swamping.

Contamination is not a hyperparameter, it is a staffing decision. It fixes the fraction of events you declare anomalous, which directly sets the length of the review queue. Leaving the default 0.1 on a stream of 480,000 events per day orders 48,000 alerts from a team that can clear 120. Invert it: take daily capacity, divide by stream volume, and take the corresponding quantile of the score distribution.

Masking and swamping. Several similar anomalies adjacent to one another hide each other (masking); normal points beside a dense anomaly cluster inherit high scores (swamping). Subsampling and per-entity baselines mitigate both — and neither is visible in an aggregate metric, only in case-level review.

# Per-entity baseline + isolation forest, with the alert threshold taken from
# the team's review capacity rather than from a default contamination value.
import numpy as np
from sklearn.ensemble import IsolationForest

REVIEWS_PER_DAY = 120        # analyst capacity -- the real binding constraint
EVENTS_PER_DAY  = 480_000

def robust_z(g):
    med = g.median()
    mad = (g - med).abs().median()
    return 0.6745 * (g - med) / (mad if mad > 0 else 1e-9)   # MAD, not sigma

# Point deviation, measured against the counterparty's OWN history:
df['amt_z'] = df.groupby('counterparty_urn')['amount'].transform(robust_z)

# Contextual deviation: same hour, same weekday, same counterparty.
df['ctx_z'] = (df.groupby(['counterparty_urn', 'dow', 'hour'])['req_count']
                 .transform(robust_z))

feats = df[['amt_z', 'ctx_z', 'velocity_1h', 'new_device', 'geo_jump']].values

# max_samples=256 is the published sweet spot: larger subsamples increase
# swamping and cost, they do not improve recall.
iso = IsolationForest(n_estimators=200, max_samples=256,
                      contamination='auto', random_state=7).fit(feats)

score = -iso.score_samples(feats)        # higher = more anomalous (a RANK,
                                         # not a calibrated probability)

# Threshold = capacity quantile, NOT a guessed contamination rate.
alert_rate = REVIEWS_PER_DAY / EVENTS_PER_DAY        # 0.00025
threshold  = float(np.quantile(score, 1.0 - alert_rate))
df['alert'] = score >= threshold

assert df['alert'].sum() <= REVIEWS_PER_DAY * 1.2, 'review queue will overflow'

The economics of false positives: the wallet picks the threshold, not F1

E7.3

99.9% accuracy on a sparse problem is an empty number. At a fraud prevalence of 0.1%, a detector that always answers 'normal' scores 99.9% accuracy. The only quantity that matters to the review team is the positive predictive value (PPV, i.e. precision), and it is governed by the base rate at least as much as by model quality.

The arithmetic worth doing in your head. One million transactions, 0.1% prevalence → 1,000 fraudulent. A detector with 90% sensitivity and a 1% false-positive rate yields 900 true and 9,990 false alerts. Precision = 900 / 10,890 ≈ 8%: an analyst opens twelve cases to find one. To reach 50% precision at the same sensitivity, FPR must fall to roughly 0.09% — a tenfold improvement. That is why this layer optimises specificity rather than sensitivity, and why any metric measured on a balanced 1:1 test set does not transfer to production at all.

Economics picks the threshold. A threshold sets a price: E[cost](τ) = FN(τ)·L + FP(τ)·C, where L is the average loss from a miss and C the cost of reviewing one alert (analyst minutes plus customer friction). F1 implicitly assumes L = C, which is almost never true: in AML a miss costs thousands and a review costs units; in clinical triage the ratio inverts. The optimum is the minimum of that curve, and it must be recomputed whenever L, C or headcount changes.

The second constraint is capacity. Even a cost-optimal threshold is useless if the queue grows faster than it is cleared: alerts/day ≤ analysts × shifts × 60 / minutes_per_alert. Exceeding it does not mean 'we catch more' — it means time-to-response grows, which is a real loss of recall that no offline metric will show you.

The feedback loop poisons the sample. Labels arrive only from reviewed alerts, i.e. from above the line. A model retrained solely on those grows ever more confident in repeating its own past decisions while the region below the threshold becomes a blind spot. The antidote costs money and is not optional: a permanent random audit sample of 1–2% of sub-threshold events, reviewed despite their low score. It is the only way to measure false negatives and obtain unbiased recall.

A new version ships into shadow first. The challenger scores the live stream without emitting alerts; comparison is precision@k at an equal alert budget, never AUC. Promotion happens only after the label-lag window has elapsed and outcomes for that same period are known.

# Choosing the operating point by expected cost under a capacity constraint,
# not by argmax F1. Scores come from the shadow-scored live stream.
import numpy as np

LOSS_PER_FN  = 850.0     # EUR, measured average uncaught loss
COST_PER_FP  = 7.5       # EUR, 6 analyst-minutes + customer friction
CAPACITY     = 120       # alerts/day the team can actually clear
DAILY_EVENTS = 480_000

def operating_point(scores, labels):
    best = None
    for tau in np.quantile(scores, np.linspace(0.90, 0.99999, 400)):
        flag = scores >= tau
        tp = int((flag & (labels == 1)).sum())
        fp = int((flag & (labels == 0)).sum())
        fn = int((~flag & (labels == 1)).sum())
        alerts_day = float(flag.mean()) * DAILY_EVENTS
        if alerts_day > CAPACITY:        # hard constraint, tested FIRST
            continue                     # an unreviewable alert is not detection
        cost = fn * LOSS_PER_FN + fp * COST_PER_FP
        if best is None or cost < best['cost']:
            best = {'tau': float(tau), 'cost': cost,
                    'precision': tp / max(tp + fp, 1),
                    'recall':    tp / max(tp + fn, 1),
                    'alerts_day': alerts_day}
    return best

# Unbiased recall needs labels from BELOW the line. Keep a permanent random
# audit sample of non-alerted events in the review queue -- forever, not once.
audit_mask = np.random.rand(len(scores)) < 0.01

# NEVER report precision measured on a rebalanced test set: precision is not
# prevalence-invariant, and 1:1 down-sampling inflates it by ~100x here.

Sources this area derives from

1. A Look Toward the Future Decision Suppor
2. A Framework to Assess Intelligent Decisi
3. A conceptualization and general architec
4. A Mobile Emergency Triage Decision Suppo
5. A Comparative Study of Anomaly Detection
6. A Revealing Large Scale Evaluation of Un
7. Graph Anomaly Detection with Graph Neural Networks
8. A Systematic Review on Anomaly Detection
9. A Review of Anomaly based Intrusion Dete
10. A Survey on Anomaly Based Network Intrus
11. A Review of Machine Learning based Anoma

Continue in this track