Complete Guide to Log Management & Digital Forensics
Log Correlation Techniques: Connecting Signals Across Systems
Log correlation connects events from multiple sources to detect threats invisible to single-source analysis. Key techniques include rule-based correlation (if-then detection logic), statistical correlation (anomaly detection against baselines), entity-based correlation (linking events by user, IP, or asset), and temporal correlation (sequencing events across a time window).
Andreas Johansson · Chief Executive Officer
Senior IT management leader with 25 years of experience in Cloud, Security, and Datacenter infrastructure.
Why Single-Source Analysis Fails
Individual log sources tell incomplete stories. A failed authentication attempt in isolation might be a user mistyping their password. But a failed authentication followed by a successful authentication from a different country, followed by mailbox rule creation, followed by email forwarding to an external address — that's account takeover.
Log correlation combines signals from multiple sources to detect complex attack patterns that no single log source would reveal. This is the fundamental value proposition of SIEM and security analytics platforms.
Correlation Techniques
Rule-Based Correlation
The most straightforward technique: define conditions that, when met across multiple log sources, generate a correlated alert.
- Simple threshold rules. "More than 10 failed logins from the same IP in 5 minutes" — aggregates individual login failure events into a brute-force detection.
- Sequence rules. "Failed login → successful login → privilege escalation → data access, all from the same user within 1 hour" — detects account compromise followed by exploitation.
- Negation rules. "VPN connection established without corresponding MFA verification" — detects when expected security steps are missing.
- Cross-source rules. "New firewall allow rule created AND data transfer to that destination exceeds 1 GB within 24 hours" — correlates configuration changes with data exfiltration.
Statistical Correlation
Statistical methods identify anomalies by comparing current behaviour to established baselines:
- Volume anomalies. Alert when log event volumes significantly exceed historical norms. A sudden spike in DNS queries, authentication events, or file access operations may indicate automated malicious activity.
- Time anomalies. Activity outside normal operating hours for a given user, system, or process. A database server that normally generates 100 queries per minute during business hours suddenly generating 10,000 queries at 3 AM warrants investigation.
- Frequency anomalies. Changes in the pattern of events. A user who normally accesses 10 files per day suddenly accessing 1,000 files triggers investigation for potential data exfiltration.
Entity-Based Correlation
Link events across log sources using common entities — users, IP addresses, hostnames, or asset identifiers:
- User-centric correlation. Track all activity associated with a specific user identity across authentication logs, email logs, file access logs, and application logs. This reveals the complete picture of a user's behaviour regardless of which systems they use.
- IP-centric correlation. Correlate all activity from or to a specific IP address: network connections, DNS queries, web requests, authentication attempts, and threat intelligence matches. An IP address that appears in firewall logs AND threat intelligence feeds AND authentication failures paints a clear threat picture.
- Asset-centric correlation. Aggregate all events associated with a specific server or workstation: process execution, network connections, file changes, authentication events, and vulnerability scan results. This provides a complete security view of the asset.
Temporal Correlation
Time-window analysis links events that occur within defined time periods:
- Sliding window. Evaluate events within a rolling time window (e.g., last 5 minutes, last 1 hour). Useful for detecting patterns that unfold over short periods.
- Session correlation. Group events belonging to the same logical session — from authentication to logoff — regardless of time span. This reconstructs complete user sessions for investigation.
- Kill chain correlation. Map events to attack kill chain phases and alert when events covering multiple phases are observed within plausible time windows.
Building Effective Correlation Rules
- Start with known attack patterns. Use MITRE ATT&CK techniques as a framework for designing correlation rules that detect specific attack methods.
- Tune for your environment. Generic correlation rules generate excessive false positives. Customise thresholds, entity whitelists, and time windows to match your organisation's normal patterns.
- Layer correlation with behavioural analysis. Machine learning complements rule-based correlation by detecting anomalies that don't match pre-defined patterns.
How SeqOps fits
SeqOps keeps your findings and alerts from connected cloud accounts and servers in one place, tracks each alert from New to Resolved, and lets you export results as CSV, PDF or Excel or send scheduled PDF reports to your team. It doesn't replace a log management or forensics tool.