Threat Hunting: Hypotheses, Data & Repeatable Hunts

Your SIEM is quiet and the EDR console is green. Does that mean nobody is inside your network – or only that nobody has done anything your rules were written to catch? Threat hunting answers that question on purpose. In this lesson you learn what hunting is, the hunt loop professionals use and how to write a testable hypothesis. Then you follow one complete hunt from the first idea to a confirmed finding, a new detection and a written record – and a second, shorter hunt on identity data.

• 12lessons, from first hypothesis to a team hunt programme
• 6stages in the hunt loop you learn here
• 2complete hunts worked through in this lesson
• ~20 minreading time for this lesson

What threat hunting is – and what it is not

Threat hunting is the proactive, human-led search for adversary activity that your existing detections have not reported. Proactive: nothing has to fire first – you start from a question, not an alert. Human-led: an analyst forms the question, decides what evidence answers it and judges the result. Tools count; people reason.

Hunting rests on one uncomfortable assumption: assume breach. You do not assume you are compromised, but you accept that a capable adversary could be present without tripping a single rule. Rules cover behaviour someone has already imagined; attackers using valid accounts, built-in admin tools or ordinary web traffic can stay below every threshold. Hunting looks into exactly that space.

An IOC sweep is useful but shallow: hashes and IP addresses are cheap for an adversary to change, while behaviour – how they persist, move and talk to their infrastructure – is not. David Bianco's Pyramid of Pain captures this: the higher you detect (from hashes up to tactics, techniques and procedures), the more evasion costs the adversary. Good hunts aim high.

A hunt that finds nothing can still succeed

Beginners measure a hunt by whether it found an attacker; professionals by what it changed. A well-designed hunt ends in one or more of four outcomes: a finding handed to incident response; a data gap (the evidence you needed was not collected or not searchable); a baseline (you now know what normal looks like for this behaviour); or a new or improved detection so that next time a rule does the work. Only the first is rare; the other three come out of nearly every hunt and make a team steadily harder to surprise.

The hunt loop: six stages from idea to improvement

Published models such as the Sqrrl hunting loop, the TaHiTI methodology from the Dutch financial sector and the PEAK framework (Prepare, Execute, Act with Knowledge) share the same logic. This course uses a six-stage version because each stage maps to a concrete piece of work you can check off:

• 1 Hypothesisa specific behaviour, scope and data source
• 2 Data checkis the evidence collected, parsed and complete?
• 3 Query & stackcount, rank and filter to a short list
• 4 Investigateexplain every outlier or escalate it
• 5 Outcomefinding, gap, baseline – written down
• 6 Improvedetections, logging, next hypothesis

Two rules govern the loop. First, stage 2 is not optional: hunts often fail because the data did not contain what the hunter assumed – a source that stopped forwarding, an empty field, retention shorter than the hunt window. Second, a confirmed threat stops the hunt: likely live attacker activity becomes an incident with its own process. Hand it over and finish the hunt record later.

Three ways a hunt starts

Hunts are hypothesis-driven (an ATT&CK technique such as scheduled-task persistence, T1053.005), intelligence-driven (a report on a group that targets your sector) or analytics-driven (a dashboard shows one workstation contacting a domain no other host uses). In practice they blend: intelligence suggests a behaviour, ATT&CK names it, analytics show where to look first. Lessons 2 and 3 turn each trigger into a repeatable planning step.

Writing a hypothesis you can actually test

"Let's look for bad stuff in the proxy logs" is not a hypothesis: no behaviour, no scope, no end. A testable hypothesis has four parts and fits in one sentence:

"Adversaries may be using [behaviour, ideally an ATT&CK technique] on [scope: systems, users, time range], which would be visible as [observable pattern] in [data source]."

The behaviour keeps you focused and the scope keeps the hunt finite. The observable defines a positive result before you see data, so you cannot talk yourself into "something odd". The data source tells you where to run stage 2 – and if you do not collect it, the hunt has already produced its first outcome: a logging gap.

Under the hood: what a hunter actually computes

Most hunting is not exotic: it is counting things in a smart order and measuring how unusual a value is. Here is the raw material and the two calculations behind the worked hunt.

One normalised proxy event

A web proxy records each outbound request. After your log pipeline parses and normalises it, a single event might look like this (synthetic example):

{
  "time":        "2026-06-29T08:15:02Z",
  "src_host":    "WS-FIN-044",
  "src_ip":      "10.20.14.44",
  "user":        "m.keller",
  "method":      "POST",
  "dest_domain": "cdn-sync.example.net",
  "dest_port":   443,
  "url_path":    "/api/v2/check",
  "status":      200,
  "bytes_out":   412,
  "bytes_in":    96,
  "user_agent":  "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
  "category":    "uncategorized"
}

Every field is a hunting dimension. dest_domain lets you measure how many hosts ever talk to a destination (its prevalence); time gives rhythm; bytes_out versus bytes_in hints at uploading versus checking in. For HTTPS, an explicit proxy logs the host name from the CONNECT request, but paths and user agents exist only with TLS inspection – a stage-2 question, because without it url_path and user_agent may be empty for most events.

Calculation 1: stacking by prevalence

Stacking (also called least frequency of occurrence) means grouping events by a value and counting. Group 30 days of proxy events by dest_domain, count requests and count distinct hosts. Domains used by hundreds of workstations are the company's normal internet. Domains used by one or two hosts and contacted hundreds of times are the interesting long tail: something on that machine wants to talk to that destination a lot, and nobody else does.

Calculation 2: timing regularity

Software on a timer produces near-constant intervals; people clicking links do not. A simple measure is the coefficient of variation (CV) of the gaps between requests: the standard deviation divided by the mean. A CV near 0 means a metronome; a CV above 1 means bursts and pauses. With five gaps of 298, 305, 291, 310 and 296 seconds, the mean is 300 s, the sample standard deviation is about 7.5 s and the CV is 0.025. With five gaps from a person browsing – 12, 340, 45, 1800 and 95 seconds – the mean is about 458 s, the standard deviation about 761 s and the CV about 1.66.

Neither measure proves anything alone – update agents, monitoring consoles and chat clients all poll on timers. That is why you stack first: timing is computed only for destinations that are already rare, so the legitimate pollers left are few enough to explain one by one.

Worked hunt: timer-driven web traffic to rare destinations

The organisation is fictional: a logistics company with about 1,200 Windows workstations, a web proxy for all outbound web traffic and EDR everywhere. All data is synthetic. Queries use a generic pipe-based pseudo-syntax (not tied to any product) and SQL over a table of normalised events.

1. Write the hypothesis and set a timebox. "Adversaries with a foothold on Windows workstations may use web-protocol command and control (ATT&CK T1071.001), visible as low-volume, highly regular requests from one or a few hosts to destinations that the rest of the company never contacts, in the last 30 days of proxy logs." Scope: all workstations; servers excluded for this round. The hunt starts on Wednesday 1 July 2026, so "last 30 days" means June. Timebox: two analyst days. Stop condition: every candidate is explained or escalated.
2. Check the data before trusting it. Count how many distinct workstations appear in the proxy logs per day. If much of the fleet is missing, "rare" only means "unseen".

   # pseudo-query: proxy coverage per day
   from proxy
   | where time > ago(30d) and src_host startswith "WS-"
   | stats hosts = dcount(src_host) by day = bin(time, 1d)
   | sort day asc

   Example output (synthetic):

   day          hosts
   2026-06-01    1094
   2026-06-02    1102
   ...
   2026-06-06      38    # Saturday: only always-on machines
   ...
   2026-06-29    1098
   2026-06-30    1101

   About 1,100 of 1,200 workstations report on a working day. Leave explains most of the gap, but 41 warehouse workstations never appear: their VLAN bypasses the proxy. Outcome number one – a data gap – is logged before the first hunting query.

3. Stack destinations by prevalence. Group 30 days of requests by destination, count requests and distinct hosts, and keep the rare-but-busy ones.
   Pseudo-query SQL

   # generic pipe syntax - adapt names to your SIEM
   from proxy
   | where time > ago(30d) and src_host startswith "WS-"
   | stats requests = count(), hosts = dcount(src_host) by dest_domain
   | where hosts <= 3 and requests >= 200
   | sort hosts asc, requests desc

   -- PostgreSQL / DuckDB style; table proxy_events holds normalised events
   SELECT dest_domain,
          COUNT(*)                 AS requests,
          COUNT(DISTINCT src_host) AS hosts
   FROM   proxy_events
   WHERE  ts >= CURRENT_TIMESTAMP - INTERVAL '30 days'
     AND  src_host LIKE 'WS-%'
   GROUP  BY dest_domain
   HAVING COUNT(DISTINCT src_host) <= 3
      AND COUNT(*) >= 200
   ORDER  BY hosts ASC, requests DESC;

   Example output (synthetic) – 7 of roughly 42,000 distinct destinations survive the filter:

   dest_domain                       requests  hosts
   tiles.weather-widget.example          4320      1
   cdn-sync.example.net                   960      1
   img.newsfeed.example                   230      1
   status.backup-vendor.example         17280      2
   telemetry.cad-plugin.example           640      2
   api.label-printer.example             1210      3
   login.partner-portal.example           410      3

   The thresholds are judgement calls: "three hosts or fewer" defines rare for this fleet, "200 requests or more" removes one-off visits. Record them so the next hunter can reproduce or change them deliberately.

4. Measure timing for the survivors only. For each host and candidate domain over the last seven days, compute the gaps between consecutive requests, ignore gaps longer than an hour (nights, lunch, a laptop lid closed) and calculate the CV.

   -- PostgreSQL / DuckDB style
   WITH seq AS (
     SELECT src_host, dest_domain,
            EXTRACT(EPOCH FROM ts - LAG(ts) OVER (
              PARTITION BY src_host, dest_domain ORDER BY ts)) AS gap_s
     FROM   proxy_events
     WHERE  ts >= CURRENT_TIMESTAMP - INTERVAL '7 days'
       AND  dest_domain IN ('tiles.weather-widget.example', 'cdn-sync.example.net',
                            'img.newsfeed.example', 'status.backup-vendor.example',
                            'telemetry.cad-plugin.example', 'api.label-printer.example',
                            'login.partner-portal.example')
   )
   SELECT src_host, dest_domain,
          COUNT(gap_s)                                               AS gaps,
          ROUND(AVG(gap_s)::numeric, 0)                              AS mean_s,
          ROUND(STDDEV_SAMP(gap_s)::numeric, 0)                      AS sd_s,
          ROUND((STDDEV_SAMP(gap_s) / AVG(gap_s))::numeric, 3)       AS cv
   FROM   seq
   WHERE  gap_s IS NOT NULL AND gap_s < 3600
   GROUP  BY src_host, dest_domain
   HAVING COUNT(gap_s) >= 20
   ORDER  BY cv ASC;

   Example output (synthetic):

   src_host     dest_domain                     gaps  mean_s  sd_s     cv
   WS-IT-02     status.backup-vendor.example    2015     300     1  0.003
   WS-IT-07     status.backup-vendor.example    2015     300     1  0.003
   WS-RCPT-01   tiles.weather-widget.example    1007     600     2  0.003
   WS-FIN-044   cdn-sync.example.net             475     300    11  0.037
   WS-SHP-03    api.label-printer.example        212     184   161  0.875
   WS-ENG-11    telemetry.cad-plugin.example     148     902   411  0.456
   WS-MKT-09    img.newsfeed.example              51     611   939  1.537

   Four rows look like timers (CV well below 0.1). The gap counts add a clue: the backup console and weather widget run around the clock, but WS-FIN-044 shows 475 gaps – exactly 95 per working day for five days – so its timer runs only while someone is logged on.

5. Investigate every timer. Use context, not intuition. IT confirms the backup vendor's console ("refresh every 5 minutes"); facilities confirms the lobby display's weather widget. For WS-FIN-044 you pivot into EDR network-connection data to see which process talks to the domain:

   # pseudo-query: which process opens the connections?
   from edr_network
   | where host == "WS-FIN-044" and remote_domain == "cdn-sync.example.net"
   | stats connections = count(), first = min(time), last = max(time)
           by process_path, process_signer, parent_path

   Example output (synthetic):

   process_path                                          process_signer  parent_path                 connections  first                 last
   C:\Users\m.keller\AppData\Local\SyncHelper\synch.exe  (unsigned)      C:\Windows\explorer.exe            960  2026-06-17T07:58:40Z  2026-06-30T16:41:12Z

   An unsigned executable in a user-writable folder, started by Explorer at logon, present on one machine in the fleet, first seen two weeks ago, posting small requests to an uncategorised domain every five minutes – and nobody recognises it. You do not need certainty: this needs incident response.

6. Hand over, then record the outcomes. You open an incident ticket with the evidence and pause the hunt. Incident response takes over (in this scenario they isolate the host and later confirm an unauthorised implant delivered through a fake invoice). Then you record the remaining outcomes: two benign timers added to a documented allowlist with an owner and a review date; the warehouse VLAN logged as a visibility gap with a request to route it through the proxy; and a detection handover so that a scheduled analytic repeats steps 3 and 4 every week.

What the handover to detection engineering looks like

A query that worked once is not yet a detection: it needs an owner, a schedule, a false-positive profile and a response. Hand it over as a short specification:

Note the schedule: low-volume, high-context analytics often belong in a weekly review queue, not the 24/7 alert queue. Lesson 10 covers when a hunt graduates into a real-time rule.

Second hunt: password spraying in cloud sign-in logs

The method does not depend on proxy logs. Here it is on identity data, which matters most once e-mail and files live in the cloud. Password spraying (ATT&CK T1110.003) means trying one or two common passwords against many accounts, which stays under per-account lockout thresholds.

Hypothesis: "Adversaries may be password-spraying accounts in our cloud identity provider (T1110.003), visible in the last 14 days of sign-in logs as a single source IP address that fails for many distinct accounts with only one or two attempts each." The data check confirms that successful and failed sign-ins are both exported. In this synthetic data set, result code 50126 means "invalid user name or password", as it does in Microsoft Entra ID sign-in logs; other providers use other codes.

Stacking needs two levels this time: first per source and account, then per source. SQL expresses that cleanly:

-- PostgreSQL / DuckDB style; signin_events holds normalised sign-in logs
WITH per_user AS (
  SELECT ip_address, user_principal,
         COUNT(*) AS attempts, MIN(ts) AS first_ts, MAX(ts) AS last_ts
  FROM   signin_events
  WHERE  ts >= CURRENT_TIMESTAMP - INTERVAL '14 days'
    AND  result_code = 50126
  GROUP  BY ip_address, user_principal
)
SELECT ip_address,
       SUM(attempts)  AS failures,
       COUNT(*)       AS users,
       MAX(attempts)  AS max_per_user,
       MIN(first_ts)  AS first_seen,
       MAX(last_ts)   AS last_seen
FROM   per_user
GROUP  BY ip_address
HAVING COUNT(*) >= 20
ORDER  BY users DESC;

Example output (synthetic):

ip_address     failures  users  max_per_user  first_seen           last_seen
198.51.100.23       412    388             2  2026-06-23 02:10:44  2026-06-23 03:55:09
192.0.2.10          263    191             3  2026-06-17 07:02:11  2026-06-30 17:48:40
203.0.113.77         58     29             2  2026-06-25 11:20:03  2026-06-25 11:24:51

Read the rows as stories. 198.51.100.23 failed against 388 accounts in under two hours at night, never more than twice each – the textbook shape of a spray. 192.0.2.10 is the company's own office address (shared behind NAT), with failures spread over two weeks of working hours: ordinary typos. 203.0.113.77 hit 29 accounts in five minutes – until the onboarding calendar shows new hires signing in for the first time.

The decisive question for a spray is never "did it happen?" but "did it work?". Pivot from the suspicious source to successful sign-ins:

# pseudo-query: any success from the spraying source?
from signin
| where ip_address == "198.51.100.23" and result_code == 0
| project time, user_principal, app, client_protocol, mfa_result

time                 user_principal       app    client_protocol  mfa_result
2026-06-23 03:41:17  r.huber@example.com  Mail   IMAP             not required

One account fell through a legacy mail protocol exempt from multi-factor authentication. That is an incident: reset the password, revoke sessions, review access and block legacy authentication. The improvement outcome is a detection counting distinct failed accounts per source in a short window, plus an alert on any success from a source that just sprayed.

What went wrong if you see this instead

Investigating outliers without fooling yourself

Stage 4 is where hunts are won or lost: every item on the short list needs a defensible label. The tree below is deliberately simple, because under time pressure simple rules get followed.

Three habits keep this honest. Ask the owner, not your intuition – their one-sentence explanation goes into the record. Define a positive before you look – otherwise every data set contains something suspicious. Label everything – a skipped outlier returns next month and costs the same time again.

Common mistakes – and better habits

Troubleshooting: when a hunt goes sideways

How experienced hunters work

Experienced hunters rarely credit a better query language. They credit discipline – a rhythm you can copy from day one:

• First 15 minutes – frame. Write down what a positive result looks like, where it lives and which fields you need. If you cannot, refine the hypothesis first.
• Next 30 minutes – verify the data. Coverage per day, field population, retention, time zone – the cheapest time in the whole hunt.
• Then – go from wide to narrow. Aggregate across the whole scope, filter to a short list, and only then open individual events. Each query goes into the hunt journal with its row count and one line of interpretation.
• Explain the top, sample the middle. Every item at the rare end gets a label. A few items from the common end get a spot check – that is how you catch an attacker who hides in plain sight on many machines.

Advanced hunters have their own traps: confirmation bias (over-reading weak signals so two days "pay off"), tuning for tidiness (tightening filters until the outliers you wanted disappear) and single-source verdicts (a finding deserves corroboration from a second source, such as EDR for a proxy hit, before it is called an incident).

Defender's view: make your environment huntable

Hunting is only as good as its telemetry. Both hunts here depended on earlier decisions: a logging proxy for all workstations, EDR recording which process opened a connection, failed cloud sign-ins exported and kept. These investments make every future hunt cheaper.

Two settings illustrate how small these investments can be. On Windows, process-creation auditing and command-line capture are off by default. In a lab machine you can enable both from an elevated prompt (in production you would set the same options by Group Policy):

:: enable auditing of process creation (Security event 4688)
auditpol /set /subcategory:"Process Creation" /success:enable

:: include the full command line in event 4688
reg add "HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\Audit" /v ProcessCreationIncludeCmdLine_Enabled /t REG_DWORD /d 1 /f

:: verify
auditpol /get /subcategory:"Process Creation"

Example output of the verification command:

System audit policy
Category/Subcategory                      Setting
Detailed Tracking
  Process Creation                        Success

Then start notepad.exe, find event 4688 in the Security log and confirm the command-line field is filled. A setting not verified in the data is not coverage.

Check your understanding

You can now…

• Define threat hunting – and tell it apart from triage, incident response, IOC sweeps and detection engineering.
• Write a testable hypothesis – behaviour, scope, observable, data source.
• Stack by prevalence – in pseudo-syntax and in SQL.
• Measure timing regularity – with gaps, mean, standard deviation and CV.
• Hunt for password spraying – with two-level aggregation and the "did it work?" pivot.
• Hand a hunt over to detection engineering – as a specification, not a pasted query.

Glossary