Zero to SOC · Free open lesson series · Lesson 1 of 7

Lesson 1: Build a SIEM From Scratch

FreeNo signupRuns in your browserNo install

Lesson 1 of 7 · Detection arc

Time: ~90–120 minutes, one sitting.

You will be able to:

Build the four-stage pipeline — ingest, parse, detect, alert — in ~30 lines of Python
Parse raw auth.log lines into structured events with a regular expression
Write and tune a threshold rule that flags a brute-force without flagging normal users
Generate your own attack telemetry and watch your detector catch it

Prerequisites: None — no security background assumed. You need a terminal and enough Python to read a for loop. Python used: open() / readlines(), re with named groups, collections.defaultdict, list comprehensions, json.dumps, and — at the very end — datetime.strptime with timedelta.

Lessons 1–6 are the detection arc; Lesson 7 opens the investigation arc.

Lesson	The skill it adds
1. Build a SIEM from scratch	ingest → parse → detect → alert	← you are here
2. Catch the breach	correlate events; baseline what's normal
3. Defeat the counter	pivot the group-by key
4. The patient attacker	stretch the horizon; count occasions
5. The drip that landed	chain detectors — rules that read alerts
6. The login that looked fine	weigh weak signals; sum the risk
7. Follow the thread	investigate: pivot, timeline, blast radius

This is the free intro to the Zero to SOC flagship. It is meant to be the clearest
explanation of how detection actually works that exists anywhere. If it isn't, it isn't
done. The quality of this one lesson is the entire pitch.

What you'll have built by the end

A working SIEM. Yours. Written by you, in about 30 lines of Python.

It reads real log data, finds an SSH brute-force attack hiding in it, and raises an alert — the exact thing a Security Operations Center does, minus a million dollars of vendor tooling. Then you'll generate your own attack and watch your code catch it.

No prior security knowledge assumed. You need to be able to open a terminal and read basic Python. That's it. Everything about security, we build from scratch.

There is one rule in this course: no magic. We never hand you a tool and say "trust it." You build the thing, so you understand the thing.

What is a SOC, actually?

Strip away the jargon. A SOC (Security Operations Center) is a team whose entire job is one sentence: watch what's happening on the computers, and notice when something's wrong.

The "watch what's happening" part is logs — every login, every connection, every file opened gets written down as a line of text somewhere. A busy company produces millions of these lines a day. No human reads them all.

So the "notice when something's wrong" part has to be code. That code is called a SIEM (Security Information and Event Management). Vendors sell SIEMs for enormous sums and wrap them in dashboards, and new analysts come away thinking a SIEM is a magic box.

It isn't. A SIEM is four steps:

  logs in  →  parse  →  rule  →  alert

Ingest — read the raw log lines.
Parse — pull the meaning out of each line (who, what, from where).
Detect — apply a rule that decides what's suspicious.
Alert — raise your hand when the rule matches.

That's the whole product. We're going to build all four. By the end you'll look at a $1M vendor SIEM and recognize every part of it, because you wrote the small version.

The attack we're hunting: SSH brute-force

When you log into a Linux server remotely, you use SSH. Every attempt — success or failure — gets written to a log file, usually /var/log/auth.log.

An attacker who wants in but doesn't have the password does the obvious thing: they guess. Thousands of times. root/123456, root/password, admin/admin, on and on. This is a brute-force attack, and it is one of the most common things on the internet — any server with a public IP sees it within minutes of going online.

It leaves a very specific fingerprint in the log: one source IP, many failed passwords, in a short window. A human scrolling the log would miss it among thousands of normal lines. A five-line rule catches it every time. That gap — between what a human can watch and what code can watch — is the entire reason the SIEM exists.

The lab

Step 0 — Get some data

A real auth.log line from the SSH daemon looks like this:

Jun 22 09:14:02 server sshd[2011]: Accepted password for alice from 192.0.2.15 port 52344 ssh2

Read it like a sentence: on Jun 22 at 09:14:02, on host server, the SSH daemon (sshd) Accepted a password for user alice coming from IP 192.0.2.15. A failed attempt is identical but says Failed instead of Accepted.

Make a file called auth.log with this content. It's mostly normal activity — alice and bob logging in, fumbling a password once like real people do — with one brute-force burst buried in the middle. (The IP addresses here are from ranges reserved for documentation, so they're safe to use as examples.)

auth.log — edit freely, then create it

Jun 22 09:14:02 server sshd[2011]: Accepted password for alice from 192.0.2.15 port 52344 ssh2
Jun 22 09:15:31 server sshd[2014]: Failed password for bob from 198.51.100.23 port 41122 ssh2
Jun 22 09:16:05 server sshd[2015]: Accepted password for bob from 198.51.100.23 port 41122 ssh2
Jun 22 09:18:44 server sshd[2020]: Failed password for alice from 192.0.2.15 port 52401 ssh2
Jun 22 09:19:02 server sshd[2021]: Accepted password for alice from 192.0.2.15 port 52401 ssh2
Jun 22 09:20:01 server sshd[2030]: Failed password for invalid user admin from 203.0.113.77 port 50001 ssh2
Jun 22 09:20:03 server sshd[2031]: Failed password for invalid user root from 203.0.113.77 port 50002 ssh2
Jun 22 09:20:05 server sshd[2032]: Failed password for invalid user test from 203.0.113.77 port 50003 ssh2
Jun 22 09:20:07 server sshd[2033]: Failed password for root from 203.0.113.77 port 50004 ssh2
Jun 22 09:20:09 server sshd[2034]: Failed password for invalid user oracle from 203.0.113.77 port 50005 ssh2
Jun 22 09:20:11 server sshd[2035]: Failed password for invalid user postgres from 203.0.113.77 port 50006 ssh2
Jun 22 09:20:13 server sshd[2036]: Failed password for invalid user git from 203.0.113.77 port 50007 ssh2

Twelve lines, so you can eyeball the answer: 203.0.113.77 is clearly hammering the server. Your code is about to find that without your eyes. On a real day this would be 12 lines hidden in 2 million — same code, same result.

Note the invalid user phrasing on some lines: the attacker is guessing usernames that
don't even exist on the box. That alone is a tell, and a sign of how much signal is
sitting in plain text in these logs.

Step 1 — Ingest: read the lines

The least glamorous step, and the foundation of everything. Get the raw text into the program.

Python — editable, runs in your browser

with open("auth.log") as f:
    lines = f.readlines()

print(f"read {len(lines)} lines")

Your output

Run it. It prints read 12 lines. That's ingest. A vendor calls this a "log collector" or a "forwarder" and charges for it. It's open().

Step 2 — Parse: turn text into meaning

A log line is a string. To reason about it, we need structured fields: the result (Accepted/Failed), the user, the source IP. We pull them out with a regular expression — a pattern that describes the shape of the line.

Don't worry about memorizing regex syntax. Read the pattern as: timestamp, then some host, then sshd[...], then either Accepted or Failed, then password for an optional invalid user, then a username, then from an IP address.

Python — editable, runs in your browser

import re

LINE = re.compile(
    r"^(?P<ts>\w{3}\s+\d+\s[\d:]+)\s+\S+\s+sshd\[\d+\]:\s+"
    r"(?P<result>Accepted|Failed) password for (?:invalid user )?"
    r"(?P<user>\S+) from (?P<ip>\d+\.\d+\.\d+\.\d+)"
)

def parse(line):
    m = LINE.search(line)
    if not m:
        return None                      # not an ssh login line — skip it
    return {
        "ts":     m.group("ts"),
        "result": m.group("result"),
        "user":   m.group("user"),
        "ip":     m.group("ip"),
    }

events = [e for e in (parse(l) for l in lines) if e]

print(events[0])

Your output

Run it. The first event prints as a clean dictionary:

Expected output

{'ts': 'Jun 22 09:14:02', 'result': 'Accepted', 'user': 'alice', 'ip': '192.0.2.15'}

That's parsing. We turned a string a human reads into data a program reasons about. Every SIEM lives or dies on this step — real logs come in hundreds of formats and the parser is where most of the unglamorous work actually is.

Step 3 — Detect: write the rule

Now the part everyone thinks is hard. The fingerprint of our attack was: one IP, many failed passwords. So: count failed attempts per IP, and flag any IP over a threshold.

Python — editable, runs in your browser

from collections import defaultdict

THRESHOLD = 5

def detect(events):
    failures = defaultdict(int)
    for e in events:
        if e["result"] == "Failed":
            failures[e["ip"]] += 1            # tally a failure against its IP

    return [
        {"ip": ip, "failed_attempts": n}
        for ip, n in failures.items()
        if n >= THRESHOLD                      # only IPs over the line
    ]

Your output

That's a detection rule. The entire thing. Vendors call these "analytics" or "correlation searches" and sell them in content packs. Yours is six lines and you understand every one.

Step 4 — Alert: raise your hand

A detection nobody sees is worthless. Emit a structured alert — not just a printed sentence, but a record another system could ingest (a ticketing tool, a dashboard, another script). JSON is the universal format for this.

Python — editable, runs in your browser

import json

for a in detect(events):
    a.update(rule="ssh_bruteforce", severity="high")
    print(json.dumps(a))

Your output

Run the whole thing top to bottom. Output:

Expected output

{"ip": "203.0.113.77", "failed_attempts": 7, "rule": "ssh_bruteforce", "severity": "high"}

Stop and notice what just happened. Your code read the log, understood it, and caught the attacker — 203.0.113.77, 7 failures — while correctly ignoring alice and bob, who each fumbled a password once. No false alarm on the normal users. That discrimination, real attack vs. normal noise, is the whole job.

You just built a SIEM. Here's the map from what you wrote to what the vendors sell:

You wrote	Industry name	What they charge for
`open()` + `readlines()`	Log ingestion / forwarder	Collecting logs at scale
The regex `parse()`	Parser / field extraction	Supporting hundreds of formats
`detect()`	Detection rule / correlation search	Pre-written "content"
`json.dumps()` alert	Alerting / SOAR handoff	Routing alerts to humans & automation

Every expensive SIEM is this, scaled up and wrapped in a UI. You now know the shape of the whole thing because you built the small version. No magic.

Now generate the attack yourself

Reading about an attack is one thing. Creating one and catching it closes the loop — and it's exactly how this course works all the way to the capstone: you generate the telemetry, then you detect it.

Append a fresh brute-force burst from a new IP to your log:

Python — editable, runs in your browser

import random

attacker = "198.51.100.99"
users = ["root", "admin", "test", "oracle", "postgres", "ubuntu", "git"]

with open("auth.log", "a") as f:
    for i in range(20):
        u = random.choice(users)
        f.write(
            f"Jun 22 09:31:{i:02d} server sshd[{3000+i}]: "
            f"Failed password for invalid user {u} "
            f"from {attacker} port {40000+i} ssh2\n"
        )

print("attack telemetry appended")

Your output

Run that, then run your SIEM again. It now catches both attackers:

Expected output

{"ip": "203.0.113.77", "failed_attempts": 7, "rule": "ssh_bruteforce", "severity": "high"}
{"ip": "198.51.100.99", "failed_attempts": 20, "rule": "ssh_bruteforce", "severity": "high"}

You attacked your own system and your own detector caught it. That feedback loop — attack, detect, confirm — is the most powerful way to learn defense, and you'll run it for every technique in the full course.

Make it real: where the simple rule breaks

The demo works. A demo is not a product. Here's what a real analyst immediately pokes at — and where the actual craft begins.

The threshold is a guess. Why 5? Too low and you alert on a user who forgot their password and retried — a false positive, the thing that drowns real SOCs in noise. Too high and a slow, patient attacker slips under it — a false negative, the one that gets you breached. Tuning that number against real traffic is a genuine skill, not a footnote.

"Per IP, ever" is too naive. Our rule counts failures across the whole log with no sense of time. 5 failures over 5 months isn't an attack; 5 failures in 5 seconds is. Real detections work over a sliding time window. We parsed the timestamp in Step 2 but haven't used it — let's fix that:

Python — editable, runs in your browser

from datetime import datetime, timedelta

THRESHOLD, WINDOW = 5, timedelta(seconds=60)

by_ip = defaultdict(list)
for e in events:
    if e["result"] == "Failed":
        t = datetime.strptime(e["ts"], "%b %d %H:%M:%S")
        by_ip[e["ip"]].append(t)

for ip, times in by_ip.items():
    times.sort()
    for i in range(len(times)):
        burst = [t for t in times if times[i] <= t < times[i] + WINDOW]
        if len(burst) >= THRESHOLD:
            print(f"{ip}: {len(burst)} failures within 60s "
                  f"starting {burst[0]:%H:%M:%S}")
            break

Your output

Expected output

203.0.113.77: 7 failures within 60s starting 09:20:01
198.51.100.99: 20 failures within 60s starting 09:31:00

Same attackers, but now the rule has a notion of burst — and a slow attacker spreading guesses across days no longer trips it, while a fast one still does. This is the difference between a toy and a detection you'd actually deploy.

One honest wart while we're here: syslog timestamps carry no year, so strptime quietly files every event under 1900. Doesn't matter in this lab — we only ever compare events inside one log — but a real pipeline has to pin the year itself, or a log crossing New Year's Eve time-travels backwards (and Python 3.13+ warns on year-less formats for exactly this reason).

And we're still only getting started. A real attacker, once they realize they're being counted, will spread the attack across many IPs (distributed brute-force), or slow it to one guess an hour (low-and-slow), or use a stolen valid password so there are no failures at all. Every one of those is a new detection problem with a real answer. Defense is a craft, and you've just done the first rep.

Check yourself

Before moving on — answer from your head, then verify against your code:

The starter log has nine Failed lines in total. What does detect() from Step 3 (THRESHOLD = 5) emit for it — how many alerts, and with what failed_attempts?
Why is THRESHOLD 5 and not 2? What do you trade as you move it in either direction?
Name a specific attack this per-IP counter is structurally blind to — no amount of tuning fixes it.

Answers

One alert: 203.0.113.77 with failed_attempts: 7. The other two failures belong to bob (1) and alice (1) — each far under the threshold, so the rule stays silent about them. That silence is the point: no false alarm on normal users.
At 2, alice or bob retrying a fumbled password would fire the rule — false positives, the noise that drowns real SOCs. Lower the threshold and you buy false positives; raise it and a patient attacker slips under — a false negative. The number is a dial you tune against real traffic, not a truth.
A distributed brute-force: 2 guesses each from 50 different IPs. Same 100 guesses, but no single IP ever reaches 5, so a rule that counts per source IP never fires. The attacker controls the source, so any per-source count can be diluted. (Low-and-slow and a stolen valid password beat it too — each for its own reason.)

What you just did, and what's next

In one sitting you built a working detection pipeline — ingest, parse, detect, alert — generated a real attack, caught it, and saw exactly where naive rules fall apart. That is genuinely the core of a SOC analyst's job, and most people who hold the title have never built it from the ground up the way you just did.

This is where the flagship opens — Act I in miniature. The full Zero to SOC course starts with the build, exactly like this, and takes the same build-from-scratch path the whole way:

Foundations pulled in exactly when you need them — no months of theory before

anything runs.

The mini-SIEM grows into a real detection lab across Windows, Linux, and cloud.
An AI teaching assistant sits beside you while you build — answering in context,

reviewing your detection rules, and throwing fresh attack scenarios at your lab so no two students get the same exercise.

You graduate by building and defending a full detection pipeline against attacks you

generated — a portfolio artifact, not a multiple-choice certificate. Capability is the credential.

If this lesson was clear, that's the whole course: hard things explained plainly, built by your own hands, no magic.

Free to use and share. This intro is open-source on purpose — it's the clearest way we know to show what the flagship is. If it helped, that's the pitch.

← All lessons Next lesson →