Namenode Loglevel

Luke Hebert

- One minute read - 170 words

When you are debugging HDFS state transitions, the default log level on the Namenode does not give you enough detail. Bumping the org.apache.hadoop.hdfs.StateChange logger to DEBUG surfaces the internal state machine events that explain why a block, file, or directory moved from one state to another.

Web UI

  1. Navigate to http://<namenode>:50070/logLevel.
  2. In the Set row, enter the following values:
    • Log: org.apache.hadoop.hdfs.StateChange
    • Level: DEBUG
  3. Click Set to apply the change immediately. No restart is required.

Command line

Run the following on the Namenode host:

hadoop daemonlog -setlevel <namenode>:50070 org.apache.hadoop.hdfs.StateChange DEBUG

Reverting the change

DEBUG logging on the StateChange logger can produce a large volume of output, especially on a busy cluster. Once you have captured the information you need, set the level back to INFO (or your preferred default) using the same UI or command, swapping DEBUG for INFO.

Important

Keep DEBUG enabled only as long as you need it. Sustained DEBUG output on a production Namenode can fill log disks quickly and add measurable overhead to the state machine.