java.lang.Object
net.i2p.stat.Rate

public class Rateextends Object
Simple rate calculator for periodically sampled data points - determining an average value over a period, the number of events in that period, the maximum number of events (using the interval between events), and lifetime data.

If value is always a constant, you should be using Frequency instead.

Step arithmetic. A rate's RRD database is created with an archive step of period/1000, so the archive step index a sample belongs to is exactly archiveStep(timestampMs, period). coalesce() stamps each sample with the step it is actually due for rather than with the current wall clock, so consecutive coalesces land in consecutive steps even when the coalesce timer drifts relative to the period; see coalesceDueMs(long, long). A backlog too large to replay is collapsed into the current step and counted by getCoalesceBacklogCollapses(), because a rate keeps only the current partial period and cannot reconstruct the steps it missed.

What a listener receives. RateSummaryListener.add(double, long, double, long) cannot carry a timestamp — its signature is fixed by implementors outside this package — so a listener that writes to storage derives the sample's step from delivery time, not from the step this class assigned it. The authoritative step stamp therefore travels on the two paths that can use it: the delivery queue, for same-step supersession (see RateSampleDelivery), and the retained-sample ring read back by getRecentSamples(int), which is what a healed listener backfills from.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    One coalesced sample, retained so a healed listener can backfill missed steps.
  • Constructor Summary

    Constructors
    Constructor
    Description
    Rate(long period)
    A rate with period shorter than Router.COALESCE_TIME = 50*1000 has to be manually coalesced before values are fetched from it.
    Rate(Properties props, String prefix, boolean treatAsCurrent)
    Create a new rate and load its state from the properties, taking data from the data points underneath the given prefix.
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addData(long value)
    Accrue the data in current period as an instantaneous event.
    void
    addData(long value, long eventDuration)
    Accrue the data in current period as if the event took the specified amount of time.
    (package private) static long
    archiveStep(long timestampMs, long period)
    The archive step a sample stamped timestampMs belongs to.
    boolean
    Clear the summary listener, but only if it is still the one registered.
    void
    Coalesce the current period's data into the last period.
    (package private) static long
    coalesceDueMs(long lastCoalesce, long period)
    The timestamp a coalesce due after lastCoalesce must stamp its sample with: the next point on this rate's own period grid.
    Computes the averages for this rate.
    computeAverages(RateAverages out, boolean useLifetime)
    Computes the averages and stores them in the provided object.
    boolean
    This is used in GraphGenerator and GraphListener.
    double
    What was the average value across the events in the last period?
    double
    The average value, or the lifetime average if no recent data.
    long
    How many steps coalesce() gave up on because the timer had been starved for more than one whole period.
    long
    Number of coalesce() calls that actually coalesced.
    long
    Number of coalesce() calls that declined to coalesce because the rate period had not yet elapsed.
    long
    When was this rate created?
    long
    In current (partial) period, how many events have occurred?
    long
    In current (partial) period, how much of the time has been spent doing the events?
    double
    In current (partial) period, what is the total value accrued through all events?
    double
    During the extreme period (i.e.
    long
    When the max(totalValue) was achieved, how many events occurred in that period? Note that this is not necessarily the highest event count; that isn't tracked.
    double
    During the extreme period (i.e.
    double
    During the extreme period (i.e.
    long
    When the max(totalValue) was achieved, how much of the time was spent doing the events?
    double
    What was the max total value accrued in any period?
    long
    The instant this rate's period grid has reached, which is the step the next coalesce is due for.
    long
    Wall clock of the last successful coalesce, as distinct from the period grid instant that getLastCoalesceDate() now reports.
    long
    In the last full period, how many events occurred?
    double
    During the last period, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?
    double
    Using the last period's rate, what is the total value that could have been sent if events were constant?
    long
    In the last full period, how much of the time was spent doing the events?
    double
    In the last full period, what was the total value accrued through all events?
    double[]
    getLastValues(int count)
    The last N data points from the underlying storage (RRD).
    double
    What was the average value across the events since the stat was created?
    long
    How many events have occurred over the rate's lifetime?
    double
    During the lifetime of this stat, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?
    long
    How many periods have we already completed?
    long
    How much of the time was spent doing the events over the rate's lifetime?
    double
    What was the total value accrued through all events over the rate's lifetime?
    double
    What was the total value, compared to the total value in the extreme period (i.e.
    double
    How large was the last period's value as compared to the lifetime average value? Warning- returns ratio, not percentage (i.e.
    long
    How large should this rate's cycle be?
    Return the rate stat this rate belongs to, or null.
    Retained coalesced samples, oldest first, for a listener to backfill from.
    Return the current summary listener, or null.
    int
    It doesn't appear that Rates are ever stored in a Set or Map (RateStat stores in an array) so let's make this easy.
    (package private) static boolean
    isBacklogCollapse(long now, long lastCoalesce, long period)
    Whether a coalesce arriving at now is further than one whole period behind its grid, and must therefore collapse its backlog into the current step rather than stamp the step it is nominally due for.
    final void
    load(Properties props, String prefix, boolean treatAsCurrent)
    Load this rate from the properties, taking data from the data points underneath the given prefix.
    void
    The rate stat this rate belongs to.
    (package private) void
    Route coalesced samples through delivery instead of calling the summary listener inline.
    void
    The listener to notify on coalesce.
    void
    store(String prefix, StringBuilder buf)
    Stores the rate data to a string builder.
    void
    store(String prefix, StringBuilder buf, boolean addComments)
    Stores the rate data to a string builder.
     

    Methods inherited from class Object

    clone, getClass, notify, notifyAll, wait, wait, wait
  • Constructor Details

    • Rate

      public Rate(long period) throws IllegalArgumentException
      A rate with period shorter than Router.COALESCE_TIME = 50*1000 has to be manually coalesced before values are fetched from it.
      Parameters:
      period - number of milliseconds in the period this rate deals with, min 1, max Integer.MAX_VALUE
      Throws:
      IllegalArgumentException - if the period is invalid
    • Rate

      public Rate(Properties props, String prefix, boolean treatAsCurrent) throws IllegalArgumentException
      Create a new rate and load its state from the properties, taking data from the data points underneath the given prefix.

      (e.g. prefix = "profile.dbIntroduction.60m", this will load the associated data points such as "profile.dbIntroduction.60m.lifetimeEventCount"). The data can be exported through store(outputStream, "profile.dbIntroduction.60m").

      Parameters:
      props - the properties previously written by store()
      prefix - prefix to the property entries (should NOT end with a period)
      treatAsCurrent - if true, we'll treat the loaded data as if no time has elapsed since it was written out, but if it is false, we'll treat the data with as much freshness (or staleness) as appropriate.
      Throws:
      IllegalArgumentException - if the data was formatted incorrectly
  • Method Details

    • getCoalesceSkips

      public long getCoalesceSkips()
      Number of coalesce() calls that declined to coalesce because the rate period had not yet elapsed.

      One skip per cycle is normal — the coalesce timer does not divide evenly into the rate period, so most periods are visited once before they are due and are skipped. A count that grows much faster than that indicates the rate is not coalescing at all.

      Returns:
      the running skip count since construction
    • getCoalesceCount

      public long getCoalesceCount()
      Number of coalesce() calls that actually coalesced.

      Paired with a storage listener's own written-step count, this is the ledger invariant: the two must agree over the window since that listener attached, and any positive drift is a step that was never recorded.

      Returns:
      the running coalesce count since construction
    • getCoalesceBacklogCollapses

      public long getCoalesceBacklogCollapses()
      How many steps coalesce() gave up on because the timer had been starved for more than one whole period.

      Each one is a whole period of aggregate data that will never reach storage: a rate holds only the current partial period, so the steps it missed cannot be reconstructed afterwards. Zero is the healthy state, and a nonzero value means the coalesce timer was not keeping up with the period — a router-wide condition, since every rate shares that timer.

      Returns:
      the running collapse count since construction
    • getRecentSamples

      public List<Rate.CoalescedSample> getRecentSamples(int max)
      Retained coalesced samples, oldest first, for a listener to backfill from.

      Only the last RECENT_SAMPLE_RING successful coalesces are kept, in a fixed ring: this runs on the coalesce path under two monitors and must not grow the heap. Never null.

      Parameters:
      max - the most samples to return; clamped to the ring size, and non-positive values yield an empty list
      Returns:
      an immutable list of at most max samples, oldest first
    • archiveStep

      static long archiveStep(long timestampMs, long period)
      The archive step a sample stamped timestampMs belongs to.

      A rate's RRD is created with an archive step of period/1000 (see GraphListener.startListening), so the step index is simply timestampMs / period and nothing else needs to know arcStep. Shared with RateSampleDelivery so there is one formula for "which step is this sample for".

      Parameters:
      timestampMs - the sample's step stamp in ms
      period - the rate period in ms, must be positive
      Returns:
      the archive step index
    • coalesceDueMs

      static long coalesceDueMs(long lastCoalesce, long period)
      The timestamp a coalesce due after lastCoalesce must stamp its sample with: the next point on this rate's own period grid.

      Stamping now instead puts a late tick's aggregate into whichever step the current wall clock falls in, not the step whose data it is. With a 50s coalesce timer and a 60s period that is the common case, and the effect is a sample written into a step a later period also writes into. Deriving the stamp from the grid makes consecutive coalesces exactly period apart, hence exactly one step apart.

      Parameters:
      lastCoalesce - the previous coalesce timestamp on this rate's grid
      period - the rate period in ms
      Returns:
      lastCoalesce + period
    • isBacklogCollapse

      static boolean isBacklogCollapse(long now, long lastCoalesce, long period)
      Whether a coalesce arriving at now is further than one whole period behind its grid, and must therefore collapse its backlog into the current step rather than stamp the step it is nominally due for.

      Replaying the missed steps is not an option: a rate retains only the current partial period, so the aggregates for steps already gone were never held anywhere. Writing the backlog into the current step is the only honest choice, and getCoalesceBacklogCollapses() makes the gap visible instead of leaving it as a hole in the graph.

      Parameters:
      now - wall-clock ms of this coalesce visit
      lastCoalesce - the previous coalesce timestamp on this rate's grid
      period - the rate period in ms, must be positive
      Returns:
      true if the backlog is more than one period deep
    • getCurrentTotalValue

      public double getCurrentTotalValue()
      In current (partial) period, what is the total value accrued through all events?
      Returns:
      the current period's total value
    • getCurrentEventCount

      public long getCurrentEventCount()
      In current (partial) period, how many events have occurred?
      Returns:
      the current period's event count
    • getCurrentTotalEventTime

      public long getCurrentTotalEventTime()
      In current (partial) period, how much of the time has been spent doing the events?
      Returns:
      accumulated event time in ms for the current period
    • getLastTotalValue

      public double getLastTotalValue()
      In the last full period, what was the total value accrued through all events?
      Returns:
      the total value accrued in the last full period
    • getLastEventCount

      public long getLastEventCount()
      In the last full period, how many events occurred?
      Returns:
      the last period's event count
    • getLastTotalEventTime

      public long getLastTotalEventTime()
      In the last full period, how much of the time was spent doing the events?
      Returns:
      accumulated event time in ms for the last full period
    • getExtremeTotalValue

      public double getExtremeTotalValue()
      What was the max total value accrued in any period?
      Returns:
      the extreme total value
    • getExtremeEventCount

      public long getExtremeEventCount()
      When the max(totalValue) was achieved, how many events occurred in that period? Note that this is not necessarily the highest event count; that isn't tracked.
      Returns:
      the extreme event count
    • getExtremeTotalEventTime

      public long getExtremeTotalEventTime()
      When the max(totalValue) was achieved, how much of the time was spent doing the events?
      Returns:
      accumulated event time in ms for the extreme period
    • getLifetimeTotalValue

      public double getLifetimeTotalValue()
      What was the total value accrued through all events over the rate's lifetime?
      Returns:
      the lifetime total value
    • getLifetimeEventCount

      public long getLifetimeEventCount()
      How many events have occurred over the rate's lifetime?
      Returns:
      the lifetime event count
    • getLifetimeTotalEventTime

      public long getLifetimeTotalEventTime()
      How much of the time was spent doing the events over the rate's lifetime?
      Returns:
      accumulated event time in ms over the rate's lifetime
    • getLastCoalesceDate

      public long getLastCoalesceDate()
      The instant this rate's period grid has reached, which is the step the next coalesce is due for. Advances by exactly one period per coalesce so the archive steps stay aligned; see getLastCoalesceNow() for the wall clock of the visit itself.
      Returns:
      the current grid instant in ms
    • getCreationDate

      public long getCreationDate()
      When was this rate created?
      Returns:
      the creation timestamp in ms
    • getLastCoalesceNow

      public long getLastCoalesceNow()
      Wall clock of the last successful coalesce, as distinct from the period grid instant that getLastCoalesceDate() now reports.
      Returns:
      the wall clock ms of the last successful coalesce
    • getPeriod

      public long getPeriod()
      How large should this rate's cycle be?
      Returns:
      the coalescing period in ms
    • getRateStat

      public RateStat getRateStat()
      Return the rate stat this rate belongs to, or null.
      Returns:
      the rate stat this rate belongs to, or null
    • setRateStat

      public void setRateStat(RateStat rs)
      The rate stat this rate belongs to.
      Parameters:
      rs - the rate stat this rate belongs to
    • addData

      public void addData(long value)
      Accrue the data in current period as an instantaneous event. If value is always a constant, you should be using Frequency instead. If you always use this call, eventDuration is always zero, and the various get*Saturation*() and get*EventTime() methods will return zero.
      Parameters:
      value - the amount to accrue into the current and lifetime totals
    • addData

      public void addData(long value, long eventDuration)
      Accrue the data in current period as if the event took the specified amount of time. If value is always a constant, you should be using Frequency instead. If eventDuration is nonzero, then the various get*Saturation*() and get*EventTime() methods will also return nonzero.
      There are at least 4 possible strategies for eventDuration:
      
      1) eventDuration is always zero.
           The various get*Saturation*() and get*EventTime() methods will return zero.
      
      2) Each eventDuration is relatively small, and reflects processing time.
           This is probably the original meaning of "saturation", as it allows you
           to track how much time is spent gathering the stats.
           get*EventTime() will be close to 0.
           get*EventSaturation() will return values close to 0,
           get*SaturationLimit() will return adjusted values for the totals.
      
      3) The total of the eventDurations are approximately equal to total elapsed time.
           get*EventTime() will be close to the period.
           get*EventSaturation() will return values close to 1,
           get*SaturationLimit() will return adjusted values for the totals.
      
      4) Each eventDuration is not a duration at all, but someother independent data.
           get*EventTime() may be used to retrieve the data.
           get*EventSaturation() are probably useless.
           get*SaturationLimit() are probably useless.
      
      Parameters:
      value - value to accrue in current period
      eventDuration - how long it took to accrue this data (set to 0 if it was instantaneous)
    • coalesce

      public void coalesce()
      Coalesce the current period's data into the last period. If the measured period is less than the rate period minus slack, this is a no-op.

      The no-op path is counted (getCoalesceSkips()) because it is otherwise invisible: no log, no counter, no exception. A rate stuck skipping stops feeding its summary listener, and therefore its RRD database, while every other health check still passes. That is precisely the failure this count exists to make observable. Slack guards early visits only — it decides whether a period is due yet, and nothing about what the sample is stamped with, so it cannot make a late visit land in the wrong step.

      The sample is stamped with the step it is due for (coalesceDueMs(long, long)) and the grid is advanced to that same instant, so the steps stay exactly one period apart however much the timer drifts. A backlog deeper than one period cannot be replayed and is collapsed into the current step instead, counted by getCoalesceBacklogCollapses().

      Being on the grid makes a rate coalesce once per period even when the timer visits more often than the period elapses, which is the point: an RRD holds one value per step, so the samples that used to land two steps apart left every intervening step empty.

      This runs on the shared coalesce timer while StatManager holds its monitor, so it does no I/O and takes no lock: the counters are updated under this rate's monitor, the ring entry is a fixed array write, and the listener goes to the delivery queue.

    • setSummaryListener

      public void setSummaryListener(RateSummaryListener listener)
      The listener to notify on coalesce.
      Parameters:
      listener - the listener to notify on coalesce, or null to clear
    • setSampleDelivery

      void setSampleDelivery(RateSampleDelivery delivery)
      Route coalesced samples through delivery instead of calling the summary listener inline.

      Owned by StatManager so that one delivery thread serves the whole rate table; passing it in per rate would mean a thread per rate. Pass null to restore inline delivery.

      Parameters:
      delivery - the shared delivery queue, or null for inline delivery
    • clearSummaryListener

      public boolean clearSummaryListener(RateSummaryListener expected)
      Clear the summary listener, but only if it is still the one registered.

      A plain setSummaryListener(RateSummaryListener) clear is not safe here. Listeners outlive the registration that created them — a console torn down after a replacement has already registered its own listener would clear the new one, because it cannot tell the two apart. The survivor keeps its database open, so every health check still reports it as attached and nothing re-arms it: the rate goes permanently unrecorded with nothing logged. Tearing down something you no longer own must be a no-op, which is what the identity test below provides.

      Identity, not equality: two distinct listeners for one rate compare equal (they implement equals by rate), but only one of them is ever the registered instance, so equals would let the wrong one clear.

      Parameters:
      expected - the listener believed to be registered; may be null
      Returns:
      true if the listener was cleared by this call, false if the registration was already absent or held by someone else
    • getSummaryListener

      public RateSummaryListener getSummaryListener()
      Return the current summary listener, or null.
      Returns:
      the current summary listener, or null
    • getLastValues

      public double[] getLastValues(int count)
      The last N data points from the underlying storage (RRD). Delegates to the summary listener if it supports historical data. Returns NaN-padded array if the listener is null or data is unavailable.
      Parameters:
      count - number of data points
      Returns:
      array of length count
    • getAverageValue

      public double getAverageValue()
      What was the average value across the events in the last period?
      Returns:
      the average value
    • getExtremeAverageValue

      public double getExtremeAverageValue()
      During the extreme period (i.e. the period with the highest total value), what was the average value?
      Returns:
      the extreme average value
    • getLifetimeAverageValue

      public double getLifetimeAverageValue()
      What was the average value across the events since the stat was created?
      Returns:
      the lifetime average value
    • getAvgOrLifetimeAvg

      public double getAvgOrLifetimeAvg()
      The average value, or the lifetime average if no recent data.
      Returns:
      the average or lifetime average depending on last event count
    • getLastEventSaturation

      public double getLastEventSaturation()
      During the last period, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?
      Returns:
      ratio, or 0 if event times aren't used
    • getExtremeEventSaturation

      public double getExtremeEventSaturation()
      During the extreme period (i.e. the period with the highest total value), how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?
      Returns:
      ratio, or 0 if the statistic doesn't use event times
    • getLifetimeEventSaturation

      public double getLifetimeEventSaturation()
      During the lifetime of this stat, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?
      Returns:
      ratio, or 0 if event times aren't used
    • getLifetimePeriods

      public long getLifetimePeriods()
      How many periods have we already completed?
      Returns:
      the number of whole periods elapsed since creation
    • getLastSaturationLimit

      public double getLastSaturationLimit()
      Using the last period's rate, what is the total value that could have been sent if events were constant?
      Returns:
      max total value, or 0 if event times aren't used
    • getExtremeSaturationLimit

      public double getExtremeSaturationLimit()
      During the extreme period (i.e. the period with the highest total value), what is the total value that could have been sent if events were constant?
      Returns:
      event total at saturation, or 0 if no event times are measured
    • getPercentageOfExtremeValue

      public double getPercentageOfExtremeValue()
      What was the total value, compared to the total value in the extreme period (i.e. the period with the highest total value), Warning- returns ratio, not percentage (i.e. it is not multiplied by 100 here).
      Returns:
      ratio of last total to extreme total, or 0 if no data
    • getPercentageOfLifetimeValue

      public double getPercentageOfLifetimeValue()
      How large was the last period's value as compared to the lifetime average value? Warning- returns ratio, not percentage (i.e. it is not multiplied by 100 here).
      Returns:
      ratio of last period value to lifetime average period value, or 0 if no data
    • computeAverages

      public RateAverages computeAverages()
      Computes the averages for this rate.
      Returns:
      a thread-local temp object containing computed averages.
    • computeAverages

      public RateAverages computeAverages(RateAverages out, boolean useLifetime)
      Computes the averages and stores them in the provided object.
      Parameters:
      out - where to store the computed averages.
      useLifetime - whether the lifetime average should be used if there are no events.
      Returns:
      the same RateAverages object for chaining
    • store

      public void store(String prefix, StringBuilder buf)
      Stores the rate data to a string builder. Includes comment lines.
      Parameters:
      prefix - prefix to the property entries (should NOT end with a period)
      buf - the builder the properties are appended to
    • store

      public void store(String prefix, StringBuilder buf, boolean addComments)
      Stores the rate data to a string builder.
      Parameters:
      prefix - prefix to the property entries (should NOT end with a period)
      buf - the builder the properties are appended to
      addComments - add comment lines to the output
    • load

      public final void load(Properties props, String prefix, boolean treatAsCurrent) throws IllegalArgumentException
      Load this rate from the properties, taking data from the data points underneath the given prefix.
      Parameters:
      props - the properties previously written by store()
      prefix - prefix to the property entries (should NOT end with a period)
      treatAsCurrent - if true, we'll treat the loaded data as if no time has elapsed since it was written out, but if it is false, we'll treat the data with as much freshness (or staleness) as appropriate.
      Throws:
      IllegalArgumentException - if the data was formatted incorrectly
    • equals

      public boolean equals(Object obj)
      This is used in GraphGenerator and GraphListener. We base it on the stat we are tracking, not the stored data.
      Overrides:
      equals in class Object
    • hashCode

      public int hashCode()
      It doesn't appear that Rates are ever stored in a Set or Map (RateStat stores in an array) so let's make this easy.
      Overrides:
      hashCode in class Object
      Returns:
      whether h code is present
    • toString

      public String toString()
      Overrides:
      toString in class Object