java.lang.Object
net.i2p.router.tasks.RouterWatchdog
All Implemented Interfaces:
Runnable
public class RouterWatchdogextends Object implements Runnable
Router health monitor and automatic recovery system.

This watchdog runs continuously to monitor router health and detect potentially hung or unresponsive states. It checks various system components including job queue processing, client responsiveness, and network connectivity.

Monitoring Checks:

  • Job queue liveliness - detects stuck jobs
  • Client manager responsiveness - ensures clients are alive
  • Network error rates - detects connectivity issues
  • Communication system status - checks network health

Recovery Actions:

  • Logs detailed system status when problems detected
  • Generates thread dumps for debugging hung states
  • May force JVM restart after consecutive failures
  • Configurable via watchdog.haltOnHang property

The watchdog runs every minute and will attempt recovery if problems persist across multiple consecutive checks. This helps prevent router from becoming completely unresponsive and ensures automatic recovery from transient issues.

  • Field Details

    • _log

      private final Log _log
      The log for the watchdog.

    • _context

      private final RouterContext _context
      The router context.

    • _consecutiveErrors

      private int _consecutiveErrors
      Consecutive check errors so far.

    • _isRunning

      private volatile boolean _isRunning
      True while the watchdog runs.

    • _lastDump

      private long _lastDump
      When the status was last dumped.

    • MAX_JOB_RUN_LAG

      private static final long MAX_JOB_RUN_LAG
      Most a job may lag before a hang.
      See Also:

    • MIN_DUMP_INTERVAL

      private static final long MIN_DUMP_INTERVAL
      Least interval between the status dumps.
      See Also:

  • Constructor Details

    • RouterWatchdog

      public RouterWatchdog(RouterContext ctx)
      Create a new router watchdog. The watchdog monitors router health and can force a restart if the router appears to be hung or unresponsive.
      Parameters:
      ctx - the router context for accessing router services and logging

  • Method Details

    • shutdown

      public void shutdown()
      Shutdown the watchdog gracefully. Sets the running flag to false to stop the monitoring loop.

    • verifyJobQueueLiveliness

      public boolean verifyJobQueueLiveliness()
      Verify that the job queue is processing jobs normally. Checks if any job has been running for too long, which could indicate a hung job queue.
      Returns:
      true if job queue appears healthy, false if a job has been running longer than the maximum allowed time

    • verifyClientLiveliness

      public boolean verifyClientLiveliness()
      Verify that client applications are responsive. Delegates to the client manager to check if all client applications are still responding properly.
      Returns:
      true if all clients appear healthy, false otherwise

    • shutdownOnHang

      private boolean shutdownOnHang()
      Whether a hang should restart the router.
      Returns:
      true with enough consecutive errors and a wrapper

    • dumpStatus

      private void dumpStatus()
      Writes the router status to the log.

    • run

      public void run()
      Main watchdog monitoring loop. Runs continuously while the watchdog is active, checking router health every minute and taking action if problems are detected.
      Specified by:
      run in interface Runnable

    • getSleepTimeForLeaseExpiry

      private long getSleepTimeForLeaseExpiry()
      Calculate sleep time based on how close leases are to expiration.
      Returns:
      sleep time in milliseconds

    • monitorRouter

      public void monitorRouter()
      Monitor router health and take action if needed. Checks job queue liveliness, client responsiveness, and network status. If problems are detected, logs status and may force a restart after consecutive failures.