Understand the analyst queue

Course · Part 2 · Part 2 exercises · Practice notebook

Help Mina understand how many reviews her detector creates, then work out what the rates do not tell her.

Core route. With attacks 20, recall 90%, false-positive rate 5% and capacity 50, compare only total events 1,000 → 2,000. Predict whether workload rises before moving the slider. Read the counts if arithmetic slows you down; explain why they changed. Extra rate edits are optional.

Teaching steps: predict, do, observe, explain

1. Set 1,000 events, 20 attacks, recall 90%, false-positive rate 5%, capacity 50. Predict true alerts and false alerts.

2. Work 0.90×20=18 true alerts and 0.05×980=49 false alerts. Queue is 67; precision uses 18/67, not 18/20.

3. Change only total events to 2,000. Predict the direction first, then read queue 117 and the capacity shortfall.

4. Explain what the totals cannot say about the top 50. Rate edits are optional; independent sliders are not attainable-model promises.

The in-demo step controls move between instructions. They never perform the next experiment for you. Record one observation before moving on.

Try it with a partner

1. Start with 1000 events, 20 attacks, 90% recall, 5% false-positive rate and capacity 50. Predict true alerts and false alerts.
2. Change total events to 2000 while leaving the other values fixed. Read the changed queue and precision.
3. Explain the denominator for one displayed rate. Optional: change that rate. Ask what measurements would be needed to achieve those rates in a real detector.

Explain what you observed

Write: “I changed ___. I predicted ___. I observed ___. The missing evidence is ___.” Swap operator and observer for a second attempt.

This is arithmetic with stipulated rates. Sliders can vary rates independently; real models have measured trade-offs. Whole-event counts are rounded. The queue aggregates do not identify which attacks would survive a top-N cut.

If the experiment is unavailable

At 1000 events, TP=18 and FP=49. Queue=67; precision≈26.9%. At 2000 with 20 attacks, calculate FP=0.05 × 1980 before adding TP.

Explain it to your partner

Question: Why can high recall still leave Mina with mostly false alerts?

Take 30 seconds each: point to the input, name the one change, then cite an observation from your run. A useful explanation: There are many more benign events than attacks. Apply each rate to its own population before calculating queue precision.

Read next: Concept and research connection. The reading is optional; you do not need its mathematics for this exercise.