java.lang.Object
net.i2p.util.LogSuppressor
public class LogSuppressorextends Object
Collapses repeated occurrences of the same log message.

Some conditions repeat thousands of times a minute - a peer resetting a connection, a tunnel failing to build - and each occurrence is individually uninteresting while the volume hides the messages that matter. At a few thousand lines a minute the log also stops being readable during an incident and starts costing real disk through rotation.

A caller passes a stable key identifying the condition and logs through Log.warnThrottled(String, String) and friends. The first occurrence is always written; after that, an occurrence is written when either enough repeats have accumulated (DEFAULT_BURST) or enough time has passed (DEFAULT_WINDOW_MS), whichever comes first, and the line states how many were collapsed. A condition that fires once an hour is never hidden; one that fires two thousand times a minute is reduced to a couple of lines a minute that still say how big it was.

Keys are scoped to the Log instance, which LogManager caches per class, so state is effectively per logging class and keys cannot collide across classes. The key table is bounded: entries whose window has long expired are pruned when it grows past MAX_KEYS, so a caller that keys on something unique per occurrence - a peer hash, say - cannot leak.

Instances are thread-safe.

  • Nested Class Summary

    Modifier and Type
    Class
    Description
    static final class 
    LogSuppressor.Decision
    What the caller should do with one occurrence.
    private static final class 
    LogSuppressor.State
    Per-key bookkeeping.

  • Field Summary

    Modifier and Type
    Field
    Description
    private final int
    _burst
    Messages allowed per key before suppression.
    private final ConcurrentHashMap<String, AtomicLong>
    _counts
    Messages seen per key since the last report.
    private final ConcurrentHashMap<String, LogSuppressor.State>
    _reported
    When each key was last reported.
    private final long
    _windowMs
    How long a key stays suppressed, in ms.
    static final int
    DEFAULT_BURST
    Repeats to accumulate before writing again, unless the window expires first.
    static final long
    DEFAULT_WINDOW_MS
    Longest a repeat can stay hidden once the first occurrence has been written.
    (package private) static final int
    MAX_KEYS
    Upper bound on tracked keys.

  • Constructor Summary

    Constructors
    Constructor
    Description
    LogSuppressor()
    Create a suppressor with the default burst and window.
    LogSuppressor(int burst, long windowMs)
    Construct a suppressor with an explicit burst size and window.

  • Method Summary

    Modifier and Type
    Method
    Description
    (package private) static boolean
    isDue(long occurrences, long lastReportCount, long lastReportMs, int burst, long windowMs, long nowMs)
    Decide whether one occurrence of key should be written.
    private void
    prune(long nowMs)
    Drop bookkeeping for keys that can no longer report.
    LogSuppressor.Decision
    record(String key)
    Record one occurrence of key and say whether to write it.
    LogSuppressor.Decision
    record(String key, long nowMs)
    Record one occurrence at an explicit time.
    void
    reset()
    Forget all bookkeeping, so every key starts counting from scratch again.
    int
    size()
    How many distinct keys are being tracked.

    Methods inherited from class Object

    clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DEFAULT_BURST

      public static final int DEFAULT_BURST
      Repeats to accumulate before writing again, unless the window expires first.
      See Also:

    • DEFAULT_WINDOW_MS

      public static final long DEFAULT_WINDOW_MS
      Longest a repeat can stay hidden once the first occurrence has been written.
      See Also:

    • MAX_KEYS

      static final int MAX_KEYS
      Upper bound on tracked keys. Pruned down to the live window past this size, so a caller keying per-occurrence cannot grow the table without bound.
      See Also:

    • _counts

      private final ConcurrentHashMap<String, AtomicLong> _counts
      Messages seen per key since the last report.

    • _reported

      private final ConcurrentHashMap<String, LogSuppressor.State> _reported
      When each key was last reported.

    • _burst

      private final int _burst
      Messages allowed per key before suppression.

    • _windowMs

      private final long _windowMs
      How long a key stays suppressed, in ms.

  • Constructor Details

    • LogSuppressor

      public LogSuppressor()
      Create a suppressor with the default burst and window.

      A suppressor allows _burst messages through before suppressing for _windowMs, so a repeated error is logged without flooding the log.

    • LogSuppressor

      public LogSuppressor(int burst, long windowMs)
      Construct a suppressor with an explicit burst size and window.
      Parameters:
      burst - repeats to accumulate before writing again, forced to at least 1
      windowMs - longest a repeat may stay hidden, in ms, forced to at least 1

  • Method Details

    • isDue

      static boolean isDue(long occurrences, long lastReportCount, long lastReportMs, int burst, long windowMs, long nowMs)
      Decide whether one occurrence of key should be written.

      Pure, so the policy can be pinned by test without a clock or a logger. The first occurrence of a key always reports; afterwards a report is due once the repeats since the last one reach burst, or once windowMs have passed - time as well as count, so a slow trickle stays visible.

      Parameters:
      occurrences - total occurrences of this key including this one
      lastReportCount - occurrence count at the previous report, zero if never
      lastReportMs - wall-clock ms of the previous report, zero if never
      burst - repeats to accumulate before reporting again
      windowMs - longest a repeat may stay hidden
      nowMs - current wall-clock ms
      Returns:
      true if this occurrence should be written

    • record

      public LogSuppressor.Decision record(String key)
      Record one occurrence of key and say whether to write it.
      Parameters:
      key - stable identifier for the condition being repeated
      Returns:
      whether to write, and how many occurrences were collapsed since the last write

    • record

      public LogSuppressor.Decision record(String key, long nowMs)
      Record one occurrence at an explicit time.
      Parameters:
      key - stable identifier for the condition being repeated
      nowMs - current wall-clock ms
      Returns:
      whether to write, and how many occurrences were collapsed since the last write

    • prune

      private void prune(long nowMs)
      Drop bookkeeping for keys that can no longer report.

      Called on every recorded occurrence once the table is over its cap, so a caller that keys per-occurrence settles at the cap instead of growing forever. Cheap because it only runs when the table is already large.

      Parameters:
      nowMs - current wall-clock ms

    • reset

      public void reset()
      Forget all bookkeeping, so every key starts counting from scratch again.

    • size

      public int size()
      How many distinct keys are being tracked.
      Returns:
      current key count