Hive Bundle Logs

Luke Hebert

- 2 minutes read - 285 words

Hive bundle logs capture the execution details of Hive queries running in a cloud data warehouse. When a query misbehaves or a job fails, these logs are usually the first place to look.

Where the Logs Live

In a typical Cloudera Data Warehouse (CDW) or similar managed Hive deployment, bundle logs are written to S3 under the following path:

s3://<s3_bucket_name>/clusters/<environment_id>/system/warehouse/tablespace/external/hive/sys.db/logs/dt=YYYY-MM-DD/

Replace the placeholders with your actual values:

  • <s3_bucket_name>: the S3 bucket provisioned for your environment.
  • <environment_id>: the unique identifier for your CDW cluster or environment.
  • YYYY-MM-DD: the date partition for the logs you want to inspect.

What You Will Find

Each date-partitioned folder contains log files that record:

  • Query submission and completion timestamps.
  • Execution plan details and stage-level metrics.
  • Error messages and stack traces for failed queries.
  • Resource usage (CPU, memory, I/O) per query.

Quick Tips for Troubleshooting

  1. Filter by date first. The dt= partition makes it easy to narrow the search window.
  2. Look for the query ID. If you have a specific query ID from the UI or API, grep the log files for that identifier.
  3. Check the stage metrics. Slow stages often point to data skew, missing indexes, or undersized executors.
  4. Correlate with cluster events. If multiple queries failed at the same time, the issue may be at the cluster or resource level rather than query-specific.

Accessing the Logs

You can browse the path directly in the AWS S3 console, or pull files with the AWS CLI:

aws s3 ls s3://<s3_bucket_name>/clusters/<environment_id>/system/warehouse/tablespace/external/hive/sys.db/logs/dt=2026-10-07/

Once you identify the relevant file, download it and open it in your preferred text editor or log viewer.

Keeping this path bookmarked will save you a few minutes every time you need to dig into a Hive query issue.