How to Export BigQuery Security Logs for Investigation

· Original article ↗

Summary

A guide shows how to export BigQuery security logs as newline-delimited JSON or query small result sets, with time filters to limit scanned data, required permissions, and common troubleshooting tips.

Key points

  • Use BigQuery EXPORT DATA to write query results as newline-delimited JSON files in Cloud Storage.
  • For small results, use bq query with JSON output and jq to flatten rows into newline-delimited JSON.
  • Filter on timestamp or a partition column to reduce scanned data; use --dry_run to estimate bytes scanned.
  • The guide lists required BigQuery job, dataset-read, and bucket-write permissions.
  • It covers common export issues, including missing URI wildcards, insufficient bucket permissions, and row limits.

Article Details

Topic
Exporting and reviewing security logs stored in BigQuery using EXPORT DATA or bq query, with LogTotal for investigation.

Products

Tools

Related Articles