Complete Guide to Log Management & Digital Forensics
Log Retention Policies: Balancing Compliance, Cost, and Forensics
Log retention policies should define retention periods by log type and compliance requirement: authentication logs for 1-3 years, cloud audit logs for 1-7 years, firewall logs for 90 days to 1 year, and application logs for 30-90 days. Use tiered storage (hot/warm/cold) to manage costs while meeting forensic and compliance needs.
Andreas Johansson · Chief Executive Officer
Senior IT management leader with 25 years of experience in Cloud, Security, and Datacenter infrastructure.
Why Retention Policies Matter
Without defined retention policies, organisations either retain too little (losing critical evidence and failing compliance) or too much (incurring unnecessary storage costs and creating a larger data breach exposure surface). Effective retention policies balance three competing needs:
- Compliance requirements. Regulatory frameworks specify minimum retention periods for security logs. Failing to retain logs for required periods creates compliance violations.
- Forensic investigation needs. Security incidents may not be discovered for months. Logs must be retained long enough to support retrospective investigation. With average dwell times exceeding 200 days, logs retained for only 30 days are useless for most incident investigations.
- Cost management. Log storage costs accumulate quickly. A mid-market organisation generating 200 GB per day accumulates 73 TB per year at full retention. Tiered storage and selective retention manage costs.
Retention Periods by Compliance Framework
| Framework | Minimum Retention | Typical Implementation |
|---|---|---|
| SOC 2 | 1 year | 1-2 years |
| ISO 27001 | Not specified (demonstrate effectiveness) | 1 year |
| PCI DSS | 90 days immediately available, 1 year total | 1 year hot + archive |
| NIS2 | Not specified (adequate for incident investigation) | 1-2 years |
| GDPR | Not specified (proportionate to purpose) | 1 year minimum |
| HIPAA | 6 years | 6-7 years |
| Financial regulations (SOX, etc.) | 7 years | 7 years |
Retention Periods by Log Type
Not all logs warrant the same retention period:
- Authentication and access logs. 1-3 years. These logs are critical for investigating unauthorised access and insider threats. Long dwell times mean recent authentication evidence is often needed for historical investigation.
- Cloud audit logs (CloudTrail, Activity Log). 1-7 years depending on regulatory requirements. These record every cloud configuration change and API call — essential for compliance evidence and cloud incident investigation.
- Firewall and network logs. 90 days to 1 year. High volume but essential for network-level investigation. Tiered storage with 30 days hot and 12 months cold balances access speed with cost.
- Security tool logs (EDR, IDS/IPS). 6-12 months. Detection and alert data supports incident timeline reconstruction and detection rule validation.
- Application logs. 30-90 days. Unless application logs contain security-relevant data (authentication, authorisation decisions, data access), shorter retention is appropriate.
- DNS query logs. 90 days to 1 year. DNS logs are valuable for investigating command-and-control channels and data exfiltration. Relatively small storage footprint justifies longer retention.
Implementing Tiered Storage
Cost-effective retention requires tiered storage:
- Hot tier (0-30 days). Fully indexed, instantly searchable. Used for active monitoring and real-time investigation. Highest cost per GB but essential for operational security.
- Warm tier (30-180 days). Compressed and indexed. Searchable with slightly higher query latency. Used for ongoing investigations and recent forensic analysis.
- Cold tier (180 days+). Compressed, archived to object storage. Requires rehydration for searching. Used for compliance retention and historical investigations. Lowest cost per GB.
- Immutable storage. Critical for forensic integrity. Write-once storage prevents log modification or deletion, even by administrators. Essential for logs that may be used as legal evidence.
How SeqOps fits
SeqOps keeps your findings and alerts from connected cloud accounts and servers in one place, tracks each alert from New to Resolved, and lets you export results as CSV, PDF or Excel or send scheduled PDF reports to your team. It doesn't replace a log management or forensics tool.