How to Export Databricks Audit Logs for Security Review

· Original article ↗

Summary

A tutorial shows how to export Databricks audit logs with Spark SQL, save them as JSON in a Unity Catalog volume, and download them with the Databricks CLI. It also highlights activity to investigate and troubleshooting tips.

Key points

  • Reading system.access.audit requires the system schema to be enabled and a SELECT grant; exporting to a Unity Catalog volume requires write access.
  • A notebook can query audit events by date and action, write newline-delimited JSON to a volume, and download the files with the Databricks CLI.
  • For security review, the article suggests checking unusual logins, token creation, permission or group-membership changes, off-hours configuration changes, and bulk downloads.
  • The SQL Statement Execution API is an alternative, but its rows and separate column-name manifest require an extra conversion step.
  • For workspaces without Unity Catalog, the article says to use audit log delivery to storage and DBFS paths instead.

Article Details

Defense Focus
Export Databricks audit and security-table logs for investigation, then review audit activity for unusual access, token, permission, configuration, or bulk-download behavior.
Detection Methods
  • Review logins and token logins from unusual IP addresses.
  • Look for newly created personal access tokens.
  • Review changes to permissions and group membership.
  • Investigate workspace or cluster configuration changes outside normal hours.
  • Review bulk downloads of notebooks or query results.
Data Sources
  • Databricks system.access.audit audit table
  • Security data stored in Delta tables or other user tables
  • Audit log files delivered to the workspace's own storage
Platforms
  • Databricks
  • LogTotal
Defensive Actions
  • Enable the system schema and obtain SELECT access to read system.access.audit.
  • Filter exports by date to limit the amount of data scanned.
  • Export query results as newline-delimited JSON to a Unity Catalog volume, then download and combine only the part-*.json files.
  • Sanitize the exported JSONL file and review it in LogTotal.
  • For workspaces without Unity Catalog, use audit-log delivery to storage and DBFS paths.

Vendors

Products

Tools

Related Articles