How a hunt works

The case packet

Read the briefing first. It tells you:

• The threat intelligence report: the actor, what it targets, how it gets in, and the hypotheses to test first.
• What you're asked: the questions your findings answer.
• What you have: the period to hunt and its captured telemetry, including the days before it, so you can see what normal looked like.
• Telemetry sources: what is collected, where, and what isn't. A source marked "available, not graded" can be searched, but its evidence IDs earn no credit.
• How you're graded and what to expect of the environment.

The rest of the packet is reference material the briefing names: the asset inventory, the account directory, the on-call roster, the change tickets, the network map, the file-share classification, and public threat reporting. The roster and the change tickets are how you tell routine administration from something that isn't.

What a finding is

Each finding answers one question and has up to five parts:

1. What was involved: hosts, accounts and resources (scheduled tasks, shares, external addresses, telemetry sources), each picked from a complete list. One finding asks for movements instead, as source and destination pairs.
2. When: a time window, start and end, in UTC. Enter both or neither. For a single moment, enter a window from one minute before it to one minute after.
3. Your verdict: malicious, benign or inconclusive. The attribution finding uses its own three labels. Inconclusive is a correct answer when the telemetry can't settle the question.
4. Your confidence: 40, 55, 70, 85 or 95%. It answers one question: how likely is it that your verdict is right?
5. Evidence: up to five document IDs from the console that support the finding.

You can save a finding and come back to it. It counts once it has a verdict and a confidence; until then it scores nothing. A finding you delete scores nothing.

Triage items open after certain findings. Each shows two events, and you give each one a verdict and a confidence. Opening an item locks the finding before it, so file that finding first. The two events appear in a fixed order that tells you nothing about them.

How points work

Each finding scores from 0 to 1, from four parts:

A finding that doesn't ask for a part is scored on the others, in the same proportions.

• What was involved. Each correct pick earns its share of the list. Each wrong pick takes credit away, and picking every item on a list scores zero for that list. Each list is scored on its own, and the lists are averaged.
• When. An edge within 5 minutes of the true edge counts as exact. Then your window is compared with the true one: the time they share, divided by the whole time either covers. A blank window scores zero.
• Verdict. The right call earns the full part. Where the answer is malicious or benign, inconclusive earns 40% of it. The opposite call (benign where the answer is malicious, or the reverse) scores zero for the whole finding, however good the rest is. Where the answer is inconclusive, only inconclusive earns credit.
• Evidence. Most findings need one supporting ID; the attribution finding needs two, from two different findings. An ID that exists but doesn't support the finding earns nothing and costs nothing. An ID that doesn't exist gets a warning when you save; if you keep it, each one costs half the evidence part.

Each triage event is scored on its verdict the same way, and an item is the average of its two events.

Your score is the weighted average of every finding and triage item, out of 100. In Hunt: Low Tide each finding counts 1 and each triage item counts half as much.

Passing, medals and calibration

You pass when your score reaches the hunt's pass mark. Medals are awarded on a pass: bronze, silver and gold, at three score lines, and gold also needs your calibration to reach the hunt's bar. In Hunt: Low Tide the pass mark is 84, the medal lines are bronze 84, silver 89 and gold 95, and gold's calibration bar is 0.88. Your result shows your score, whether you passed, and any medal. If your score reached gold but your calibration didn't, it says so.

Calibration measures whether your confidence matches how often you're right. Every verdict you give counts, on a filed finding or a triage event. For each one:

• right: the penalty is (1 − your confidence)²;
• anything else: the penalty is your confidence².

Calibration is 1 minus the average penalty. Here, "right" means exactly the answer: an inconclusive that earned partial credit counts as not right.

Confidence on a wrong call costs the most, but low confidence on a right one costs too. Give the confidence you actually have in the verdict, not in your window or your evidence.

A worked example

A fictional finding about a scheduled task, on a twelve-host network. The answer: hosts HOST-B and HOST-C, account svc.alpha, scheduled task Nightly-Sync, from 14:00 to 14:40, malicious.

The analyst files:

Finding score: 0.35 × 0.83 + 0.20 × 0.67 + 0.25 × 1 + 0.20 × 0.5 = 0.775.

What would change it:

• Dropping the ID that doesn't exist: 0.875.
• Leaving the window blank: 0.64.
• Picking all twelve hosts: hosts score 0, so 0.72.
• Calling it inconclusive: 0.625.
• Calling it benign: 0.

Calibration, over four verdicts: right at 95% (penalty 0.0025), right at 85% (0.0225), wrong at 85% (0.7225), right at 55% (0.2025). The average penalty is 0.2375, so calibration is 0.76. The one confident miss accounts for most of it.

• Next: Low TideYour first case on the Specializations path.