AI & Cybersecurity

HSLU / UiA · English

When does an AI output justify action?

Two afternoons, six Parts. First AI as a tool for defenders, then the AI system itself as the target. Read each Part beside its slides, then try the ideas yourself.

Start with Part 1 Slides

Course informationSlidesBoth decks in the browser and as PDF. Parts 1–6The six PartsOne page per Part, in slide order, with figures, demos and check questions. InteractiveExercisesPractise each Part and get feedback as you go. Hands-onExperiments and worked examplesSeventeen browser experiments, 36 worked examples and a practice notebook.

The guiding question

Two questions run through both days:

What does the AI output actually establish?
What evidence and controls do we need before acting?

Each Part answers one question, and each example on the way teaches one idea. The opening sets out the guiding question; the summary on slide 73 and the answer on slide 47 of Parts 4–6 return to it.

The six Parts

Day 1 · AI for security

Part 1 · slides 12–33How does a model learn from data?Machine-learning foundations Part 2 · slides 34–48Is the detector useful?Detection and its evaluation Part 3 · slides 49–71Where do evaluations and AI tools mislead?Limits and new capabilities

Day 2 · Security of AI

Part 4 · slides 6–18How do attackers fool a frozen model, or teach a retrained one?Adversarial machine learning Part 5 · slides 19–33What can a model leak, and what happens when it reads untrusted text?Privacy, supply chain and LLM applications Part 6 · slides 34–47Which controls make an AI system defensible, and what evidence shows it?Securing AI systems in practice

The audience is first-year master's students with security foundations and introductory AI.

What each Part covers

Parts 1–3 · Machine learning for threat detection

Part	Slides	Content
Opening	1–11	Lecturer, course map, outcomes; three relationships between AI and security, why ML (start of Part 1)
1 · Machine-learning foundations	12–33	Learning paradigms, rules as learned models, libraries that do the maths, representation, model families, optimisation, generalisation; demo, slide 33: Lab 01
2 · Detection and its evaluation	34–48	Confusion matrix, precision, recall, ROC, base rates, analyst capacity, applications, explanations; demo, slide 42: threshold explorer
3 · Limits and new capabilities	49–71	Evaluation pitfalls, drift, LLMs in the SOC, AI security tools (vendor and open source) and their misuse, 2026 evidence, AI-enabled threats
Close and back matter	72–81	Reading, summary (the last main slide, up during questions), sources, references, Appendix A (discovery is not an operational guarantee) and Appendix B (terms and source labels); at the end of Part 3

Activities: quick checks and predictions on slides 24, 41, 53–55 and 66; the opening question “Is 99.9% accuracy good?” on 48; the control vote on 71.

Parts 4–6 · Security of AI systems

Part	Slides	Content
Opening	1–5	Three retrieval questions, outcomes, route, why AI systems need their own security analysis; at the start of Part 4
4 · Adversarial machine learning	6–18	Attacker descriptions, evasion, robustness claims, poisoning and backdoors; demo, slide 12: frozen phishing detector
5 · Privacy, supply chain and LLM applications	19–33	Privacy attacks, supply chain, prompt injection, EchoLeak, agency, OWASP 2026, defences; demo, slide 32: Lab 06
6 · Securing AI systems in practice	34–47	MITRE ATLAS, threat modelling, hospital group exercise, NIST AI RMF, EU AI Act, retesting, synthesis
Self-study and sources	48–58	Review questions, research directions, reading routes, sources and references; at the end of Part 6

Activities: retrieval questions on slide 2; the attacker description on 7; discussions on 18 and 33; the hospital group exercise on 37–38; the guiding-question vote on 47.

Learning outcomes

By the end of Parts 1–3 you should be able to:

Explain how supervised and unsupervised learning turn security data into a detector, and what the choice of representation implies.
Calculate precision and daily alert load from detection rate, false-positive rate and base rate, and justify an operating threshold.
Critique a published or vendor evaluation of an ML detector against known methodological pitfalls, and propose a sounder evaluation design.
Assess where large language models add value to security operations, and the risks they introduce, including their use by attackers.

By the end of Parts 4–6 you should be able to:

Characterise an attack by the attacker’s goal, knowledge, capability and lifecycle stage.
Explain and evaluate evasion, poisoning, privacy and supply-chain attacks, and test a defence against an adaptive attacker.
Analyse injection and excessive-agency risks in LLM applications and agents, and the authority boundary that contains them.
Justify controls: map threats to MITRE ATLAS or OWASP and controls to NIST AI RMF functions and, where it applies, EU AI Act Article 15.

How to use this site

Start from a Part. Each Part page opens with its question, what you will be able to do and where to find the slides. Its headings give the slide numbers from the slide footers, so you can read it beside the slides. Open slide jumps to that slide.
Read and try. Where a slide builds up step by step, the page shows the final state and lists the same steps with the same numbers. Where a slide runs a real computation, the page embeds a version you can step, run, reset and vary locally. The pages mirror the 81 + 58 slides and their click sequences. Mathematics is typeset with local KaTeX.
Answer before you open. Answers to check questions are collapsed: decide first, then check.
Practise. Each Part ends with its exercises and related experiments. The practice notebook maps its six practice sections to the Parts and records predictions, observations and limits. The 36 worked examples, data modalities and the reading list go further.

Some practice cases use practice colleagues.

During class

Predict, discuss with a partner and explain what an observation does and does not establish. The lecturer demonstrates Lab 01: login-alert learning in Part 1 and Lab 06: an assistant workflow in Part 5. The threshold/base-rate and frozen-detector evasion demonstrations are built into the slides. The other labs are self-study resources.

Use the experiments

No Python installation, account or API key is needed. Numerical learners train locally in your browser on supplied data. Assistant proposals and incident traces are scripted simulations, not live LLM output. Do not enter confidential organisational information.

For self-study, work with a partner: one changes the experiment, the other predicts and explains. Swap roles and reset between comparisons. Record what changed, what happened and what remains unknown. Notes are not saved automatically; use the notebook, a text file or paper.

The teaching computer's localhost address is not accessible from another laptop. Use an address shared by the lecturer for access from your own device.