Train a login-alert triage model

Learning path · Part 1 · Part 1 exercises · Workbook

Mina needs a priority score for login alerts. Triage means deciding which leads to review first. A feature is a measured clue; a label is the supplied example answer. Training adjusts parameters; validation checks separate examples without adjusting them. Lower mean squared error (MSE) means closer agreement with those labels. First shape the function by hand; then let the optimiser learn from examples. We use this model in lesson 1 and return to poisoned labels in lesson 4.

Demonstrated by the lecturer on Parts 1–3 slide 33 · repeat at your own pace. Core route (notebook lesson 1): shape a line and make one update. Then choose Clean triage labels · 32 alerts, Neural network · 12 hidden units, Reset, then Train 1,000 steps once. Record validation MSE. Keep this full tab open. Later change only threshold to 0.15 and 0.70, then probe x=1.20. Leave the test sealed; manual hidden-unit edits and test revelation are extensions.

The longer sections below are for reference; you do not need to finish them in one sitting. Follow the core route in the notebook.

Teaching steps: predict, do, observe, explain

1. Identify x (login burst) and y (priority label). Predict which burst should reach review; a score is not an attack probability.

2. Select Straight line. Reset; set w=0, b=0, then b=0.50. Predict the shift. Try w=1, b=0 before training.

3. Compare one prediction with its label. Square the difference. MSE averages this disagreement across training examples.

4. Press One update once. Inspect the weight and error. Reset afterwards so the next run starts at zero updates.

5. Choose clean labels and 12 hidden units. Reset, then Train 1,000 steps once. Record validation MSE; leave the test sealed.

6. Keep the model fixed. Compare thresholds 0.15 and 0.70. Record false alerts, misses and queue size; explain the trade-off.

7. Probe x=1.20. Weights stay fixed: inference. Leave the test sealed in core time; optional revelation locks all later tuning.

The in-demo step controls move between instructions. They never perform the next experiment for you. Record one observation before moving on.

Reading the experiment without JavaScript

A model is an adjustable function. For a line, prediction = weight × input + bias. Training compares predictions with labels, measures error and updates those parameters. A small neural network combines curved functions, allowing more flexible shapes. It still learns only from the examples and objective it is given.

Here x is a normalised login-burst feature: negative values mean below the reference level, positive values above it. The signed priority label follows the rule y = 0.75\tanh(2.5x) + 0.15x. A reference label of at least 0.35 defines a priority alert. The review threshold starts at 0.35 and is adjustable; the priority labels stay fixed. Scores are not attack probabilities or permission to block an account. Training uses noisy sampled labels. Validation and test use distinct deterministic samples of the clean function within −1 to 1. They are different examples from the same task and are not evidence about another real-world population. The slider extends to ±1.5 to expose unsupported extrapolation.

Predict what a line can and cannot fit. Compare it with a curved model. Then ask what happens when examples cover only a small interval or when someone changes selected labels. A low loss on altered labels can mean the optimiser is successfully learning the wrong target.

Before training: change the security decision by hand

1. Select Straight line. Set weight w = 0, bias b = 0: no alert reaches the review threshold. Increase b to 0.50: every alert does. Compare missed reviews and false alerts.
2. Set w = 1, b = 0. The score now depends on the login feature. Change w, then b; explain the changed slope and offset before training.
3. Select the 12-unit network. Choose a hidden unit and adjust input weight w, hidden bias b and output weight v. Output bias c shifts every prediction. The network sums all units, so a single unit's effect depends on the other parameters.
4. Start training from your edited values. A manual edit keeps other parameters and clears optimiser history and step count. Parameter edits are disabled while training and after test revelation.

The false-alert denominator is the number of normal-priority validation labels; the missed-review denominator is the number of priority labels. Both refer to the reference rule at 0.35, not confirmed real attacks. Probe x separately: this uses fixed parameters and never trains the model.

Security question: who can modify labels or deploy changed model weights? An unauthorised bias change can suppress the analyst queue without altering account permissions.

Move the threshold, keep the model fixed

Use 6 · Threshold in the presentation, or the threshold slider on this page. The amber dotted line moves; the learned curve, training steps and MSE stay fixed. Score ≥ threshold means analyst review. Reset restores 0.35. Revealing the held-out test locks threshold selection as well as training and manual weights.

Record TP, FP, FN and TN on validation examples. Precision = TP / (TP + FP); recall = TP / (TP + FN); false-positive rate = FP / (FP + TN). An empty review queue has undefined precision, displayed as N/A. Lowering the threshold cannot reduce recall for fixed scores and labels, but precision need not change monotonically. It may increase analyst workload without finding another priority alert.

Reference classes use the original clean rule at 0.35 even after poisoning. This prevents moving the definition of a correct answer along with the review threshold. The clean teaching problem may separate perfectly; this does not predict real detection quality. In practice, attack labels can be uncertain, delayed or missing, and the proportion of attacks affects the meaning of the queue.

Use lessons 1 and 3 in the practice notebook. Select model, stopping point and threshold on validation data before revealing the held-out test. Test output includes false alerts and missed reviews at the chosen threshold.

Primary reading: scikit-learn's threshold selection and precision–recall. The browser implementation is our own small regressor; it does not use scikit-learn.

Evidence to record

Record the held-out test only after choosing the model and stopping point. Reusing the fixed test to choose later configurations turns it into development evidence; a genuinely final evaluation would need a fresh independent test set.

Why this is function approximation

The unknown relationship is approximated by a parameterised function. Optimisation searches for useful parameter values. Model capacity, examples and the training objective all affect the result. Being able to represent many functions does not guarantee that training finds the right one or that predictions generalise.

Reading: Goodfellow, Bengio and Courville, Deep Learning, machine-learning basics and feedforward networks. The interactive implementation and all data are purpose-built for this workshop.

Next in lesson 2: build a phishing detector. The approval-rule lab belongs to lesson 5.

Explain it to your partner

Question: A login burst produces a higher score after a bias edit. What changed before any training?

Take 30 seconds each: point to the input, name the one change, then cite an observation from your run. A useful explanation: Bias shifts the function. Inference uses its current parameters; it does not fit new ones.

Read next: Concept and research connection. The reading is optional; you do not need its mathematics for this exercise.