Part 2 exercises · Detection and its evaluation

All exercises · Part 1 · Part 3

Slides: Parts 1–3 slides

Is the detector useful? About 25 minutes. Work through the exercises in order; the last one is a short scenario that combines them.

What you will practise

• Fill a confusion matrix from a story
• Choose the right denominator for precision, recall and the false-positive rate
• Move a threshold and watch the analyst queue
• Compute precision at a low base rate, and see through “99.9% accurate”
• Fit a detector to analyst capacity
• Read three alerts together and decide what the evidence justifies

Every exercise gives you feedback after you try, and names the slide to look at. You can retry as often as you like, ask for hints, and see the answer after a failed attempt. Your progress is saved only in this browser and is never sent anywhere.