Part 3 exercises · Limits and new capabilities

All exercises · Part 2 · Part 4

Slides: Parts 1–3 slides

Where do evaluations and AI tools mislead? About 25 minutes. Work through the exercises in order; the last one is a short scenario that combines them.

What you will practise

• Spot a leaky split and choose a fair one
• Name what is wrong with a vendor’s claim
• Show how a shortcut scores 100% in the lab and 50% in real use
• Tell three kinds of change after deployment apart
• Decide whether to act, verify or escalate on an LLM draft
• Choose defences that do not depend on spotting the fake

Every exercise gives you feedback after you try, and names the slide to look at. You can retry as often as you like, ask for hints, and see the answer after a failed attempt. Your progress is saved only in this browser and is never sent anywhere.