How to Export Amazon Security Lake Logs with Athena
Summary
A tutorial explains how to use Athena UNLOAD to export Amazon Security Lake data as JSON lines to S3, download it, and review it. It covers access requirements, partition filters, and common errors.
Key points
- Amazon Security Lake stores events in Parquet using OCSF; the guide recommends querying them with Athena rather than downloading them directly.
- Athena UNLOAD can export query results as JSON to an S3 prefix, which must be empty; the resulting files can be downloaded and combined into a JSONL file.
- Query access requires SELECT on the tables through Lake Formation, Athena permissions, and write access to an S3 bucket.
- Filtering by eventday and accountid limits the data scanned; the article notes Athena charges by data scanned.
- The guide lists common fixes for missing tables, insufficient permissions, non-empty export prefixes, and compressed output.
- The article suggests uploading the JSONL file to logtotal[.]com for sanitizing and review but provides no further detail on those steps.
Article Details
- Topic
- Exporting Amazon Security Lake logs with Athena UNLOAD and reviewing them in LogTotal
Vendors
Products
Amazon AthenaUse Amazon Athena with an UNLOAD statement and format = ‘JSON’: Athena writes the results as JSON lines to an S3 prefix you choose, and you download that.Amazon Security LakeAmazon Security Lake stores your data in S3 as Parquet files in the OCSF schema, so you don’t download it directly.AWS GlueSecurity Lake tables live in a Glue database named amazon_security_lake_glue_db_<region>, one table per source.AWS Lake FormationAccess is governed by Lake Formation, so you need SELECT on the tables, plus Athena permissions and write access to an S3 bucket for the export.LogTotalBelow is how to get data out, sanitize it, and investigate it for free in LogTotal.S3Amazon Security Lake stores your data in S3 as Parquet files in the OCSF schema, so you don’t download it directly.