A SIEM, meaning security information and event management, collects logs from across an environment, normalizes them so they can be compared, and lets you search and alert on the result.
That description makes it sound like a database with ambition. The value is in the normalization and the correlation: being able to ask one question that spans a firewall, a domain controller, and a cloud audit log at the same time.
What a SIEM Is Actually For
Three jobs. Detection, meaning rules that fire when a pattern appears. Investigation, meaning search across everything when you have a question. And retention, meaning being able to answer questions about last quarter, which matters both for incidents and for audits.
Investigation is the one that gets undervalued during purchasing and turns out to matter most during an incident. When something is happening at two in the morning, the thing you need is the ability to ask arbitrary questions quickly.
The Log Sources That Matter
Authentication logs, from your directory service and identity provider. Almost every intrusion touches identity somewhere.
Endpoint telemetry, ideally from an EDR product, which gives process execution, command lines, parent and child relationships, and network connections per process. This is the highest value source for detection by a wide margin.
DNS query logs, which are small, cheap, and enormously useful for finding command and control, data exfiltration, and shadow IT.
Network logs from firewalls or proxies, cloud control plane audit logs, email security logs, and logs from whatever your crown jewel application is.
Everything else is usually optional and often expensive. Most SIEM cost problems come from ingesting high volume, low value data because it was easy to turn on.
Why Splunk Specifically
Splunk is one of the most widely deployed SIEM platforms in the world, which means learning it transfers directly to a large number of jobs. It is also genuinely good at the investigation use case, which is why it survived a decade of cheaper competitors.
The main alternatives worth knowing by name are Microsoft Sentinel, Elastic Security, IBM QRadar, and Exabeam. The concepts move between all of them, and only the syntax changes.
How Splunk Searching Works
SPL reads as a pipeline. You start with a set of events and pass them through transformations, each one narrowing or reshaping what came before.
The pattern that covers most real work is: pick your data by index and source type, filter to the events you care about, count or group by a field, sort the result, and look at both ends of the list. The top of a list of counts shows you what is common. The bottom shows you what is rare, and rare is usually more interesting in security.
The second pattern is time. Compare this week to last week. A host that has always talked to twelve destinations and today talked to two hundred is worth a look even if none of those destinations are on any threat list.
The Mistakes That Waste the Money
Ingesting everything because storage seemed cheap, then discovering the license or cloud bill is the second largest line item in the security budget.
Turning on every out of the box detection rule without tuning, which produces an unusable alert volume and teaches analysts to ignore the console.
Not normalizing fields, so the same concept has four different names across sources and no search can span them cleanly.
Buying a SIEM with nobody assigned to run it. A SIEM is a program with staff, not a product you install. This is the most common failure and the most expensive.
SIEM and EDR Are Not the Same Thing
People sometimes ask whether a SIEM replaces endpoint detection and response, or the other way round. Neither. An EDR gives deep visibility and control on endpoints. A SIEM gives breadth across everything including systems no agent can be installed on.
You investigate a suspicious process in the EDR. You find out whether the same thing happened on forty other machines, and what your firewall thought about it, in the SIEM. See EDR and XDR explained.
Learning It Without a Job
Splunk offers a free tier with a daily ingest limit that is more than enough to learn on, and there are public sample data sets and boss of the SOC style exercise data available for practice.
The fastest way to get good is to give yourself questions and answer them with searches. Which account failed to log in most often. Which host talked to the most unique destinations. What was the first time this process ever appeared. That habit is what Week 8 of the mentorship builds, and there is more background on our Splunk page.
Learn This at HackRange
Week 8 of the mentorship is your first week in Splunk, and Week 9 turns those searches into a full intrusion investigation.