How to Export AWS Logs from Athena as JSON for Investigation

Summary
A tutorial shows how to export filtered CloudTrail logs from S3 with Athena’s UNLOAD command as newline-delimited JSON, then download and review them in LogTotal. It covers permissions, partition filters, and troubleshooting.
Key points
- Athena queries log files stored in S3; it does not store the logs itself.
- Athena’s UNLOAD command can export query results as newline-delimited JSON to an S3 prefix; ordinary query results are saved as CSV.
- The example filters CloudTrail logs by region, date, and a suspicious user identity.
- The guide covers Athena, Glue, S3, and optional KMS permissions, plus workgroup setup.
- Filtering on partition columns limits scanned data; the article warns that unfiltered queries can scan years of logs.
- The command-line workflow waits for the query to succeed, copies the output from S3, and combines files with zcat; scripts should also handle failed queries.
- The tutorial directs users to upload the exported file to LogTotal for sanitization and review but does not explain those steps.
Article Details
- Topic
- Exporting and reviewing log data with Amazon Athena
Vendors
Products
AthenaAthena doesn’t store logs itself. It queries log files in S3, such as CloudTrail, VPC Flow Logs, load balancer and WAF logs, through tables you define.CloudTrailIt queries log files in S3, such as CloudTrail, VPC Flow Logs, load balancer and WAF logs, through tables you define.LogTotalBelow is how to get a slice out, sanitize it, and investigate it for free in LogTotal.S3It queries log files in S3, such as CloudTrail, VPC Flow Logs, load balancer and WAF logs, through tables you define.
