How to Export BigQuery Security Logs for Investigation

Summary
A guide shows how to export BigQuery security logs as newline-delimited JSON or query small result sets, with time filters to limit scanned data, required permissions, and common troubleshooting tips.
Key points
- Use BigQuery EXPORT DATA to write query results as newline-delimited JSON files in Cloud Storage.
- For small results, use bq query with JSON output and jq to flatten rows into newline-delimited JSON.
- Filter on timestamp or a partition column to reduce scanned data; use --dry_run to estimate bytes scanned.
- The guide lists required BigQuery job, dataset-read, and bucket-write permissions.
- It covers common export issues, including missing URI wildcards, insufficient bucket permissions, and row limits.
Article Details
- Topic
- Exporting and reviewing security logs stored in BigQuery using EXPORT DATA or bq query, with LogTotal for investigation.
Products
BigQueryWhen security logs sit in BigQuery (from a Cloud Logging sink, a Google Workspace export or your own pipeline), export them with EXPORT DATA.Cloud LoggingWhen security logs sit in BigQuery (from a Cloud Logging sink, a Google Workspace export or your own pipeline), export them with EXPORT DATA.Google Cloud StorageIt writes newline-delimited JSON to a Cloud Storage bucket, and you download that.LogTotalBelow is how to get a slice out, sanitize it, and investigate it for free in LogTotal.
