Train a file detector and inspect its blind spots

Course · Part 2 · Part 2 exercises · Notebook · Example bank

Self-study after Part 2 · slide 45 (notebook lesson 3). Predict, then compare F1/F2 with entropy alone, then F2/F4 with both features. F3/F6 and the remaining settings are further practice. Entropy measures byte-pattern unpredictability; sensitive behaviour is a second supplied observation. Their security meaning depends on what the representation leaves out.

Teaching steps: predict, do, observe, explain

1. Choose Byte entropy only, then Train file detector. Compare F1/F2: identical entropy cannot distinguish the two labels.

2. Choose Entropy + sensitive behaviour and retrain. Compare F2 and F4 scores before revealing their contexts.

3. Compare F2 and F4; their represented features match. Reveal each context and name the authorisation evidence the model lacks.

4. Save one error and one record you would request. F3/F5/F6 are further practice; no file is executed.

The in-demo step controls move between instructions. They never perform the next experiment for you. Record one observation before moving on.

Predict, try, explain

1. Choose Byte entropy only, then Train file detector. Compare F1 Compressed updater and F2 Packed data thief: their entropy is identical. Predict whether the model can distinguish them.

2. Choose Entropy + sensitive behaviour and train again. Compare the scores for F2 Packed data thief and F4 Authorised migration utility before revealing context.

3. F2 and F4 have identical represented features but different reference labels. Reveal each context and identify the missing authorisation evidence.

Record: I changed ___. I predicted ___. I observed ___. This does not establish ___.

More experiments

After the core exercise, inspect F3 Plain-text stealing script: low entropy does not make its described behaviour harmless. Inspect F6 Quiet scheduled thief: what did the short observation window miss? Work all six files if time permits; explain the harmless archive, the packed thief and the authorised utility. Request one record that could distinguish an ambiguous pair. More features need reliable collection and evaluation.

What this example establishes

The detector uses the existing actual logistic learner. Eight generated training rows and six development cases are purpose-built; no executable is downloaded, parsed or run. Authored context is revealed for investigation, not supplied to the classifier.

Explain it to your partner

Question: The thief and approved migration share both features. Will more training on these two clues separate them?

Take 30 seconds each: point to the input, name the one change, then cite an observation from your run. A useful explanation: F2 and F4 have the same supplied features. Their different authorisation needs additional evidence or a different representation.

Read next: Concept and research connection. The reading is optional; you do not need its mathematics for this exercise.