If value is always a constant, you should be using Frequency instead.
Step arithmetic. A rate's RRD database is created with an archive step of period/1000, so the archive step index a sample belongs to is exactly archiveStep(timestampMs, period). coalesce() stamps each sample with the step it is actually due for rather than with the current wall clock, so consecutive coalesces land in consecutive steps even when the coalesce timer drifts relative to the period; see coalesceDueMs(long, long). A backlog too large to replay is collapsed into the current step and counted by getCoalesceBacklogCollapses(), because a rate keeps only the current partial period and cannot reconstruct the steps it missed.
What a listener receives. RateSummaryListener.add(double, long, double, long) cannot carry a timestamp — its signature is fixed by implementors outside this package — so a listener that writes to storage derives the sample's step from delivery time, not from the step this class assigned it. The authoritative step stamp therefore travels on the two paths that can use it: the delivery queue, for same-step supersession (see RateSampleDelivery), and the retained-sample ring read back by getRecentSamples(int), which is what a healed listener backfills from.
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final classOne coalesced sample, retained so a healed listener can backfill missed steps.Constructor Summary
ConstructorsConstructorDescriptionRate(long period) A rate with period shorter than Router.COALESCE_TIME = 50*1000 has to be manually coalesced before values are fetched from it.Rate(Properties props, String prefix, boolean treatAsCurrent) Create a new rate and load its state from the properties, taking data from the data points underneath the given prefix.Method Summary
Modifier and TypeMethodDescriptionvoidaddData(long value) Accrue the data in current period as an instantaneous event.voidaddData(long value, long eventDuration) Accrue the data in current period as if the event took the specified amount of time.(package private) static longarchiveStep(long timestampMs, long period) The archive step a sample stampedtimestampMsbelongs to.booleanclearSummaryListener(RateSummaryListener expected) Clear the summary listener, but only if it is still the one registered.voidcoalesce()Coalesce the current period's data into the last period.(package private) static longcoalesceDueMs(long lastCoalesce, long period) The timestamp a coalesce due afterlastCoalescemust stamp its sample with: the next point on this rate's own period grid.Computes the averages for this rate.computeAverages(RateAverages out, boolean useLifetime) Computes the averages and stores them in the provided object.booleanThis is used in GraphGenerator and GraphListener.doubleWhat was the average value across the events in the last period?doubleThe average value, or the lifetime average if no recent data.longHow many stepscoalesce()gave up on because the timer had been starved for more than one whole period.longNumber ofcoalesce()calls that actually coalesced.longNumber ofcoalesce()calls that declined to coalesce because the rate period had not yet elapsed.longWhen was this rate created?longIn current (partial) period, how many events have occurred?longIn current (partial) period, how much of the time has been spent doing the events?doubleIn current (partial) period, what is the total value accrued through all events?doubleDuring the extreme period (i.e.longWhen the max(totalValue) was achieved, how many events occurred in that period? Note that this is not necessarily the highest event count; that isn't tracked.doubleDuring the extreme period (i.e.doubleDuring the extreme period (i.e.longWhen the max(totalValue) was achieved, how much of the time was spent doing the events?doubleWhat was the max total value accrued in any period?longThe instant this rate's period grid has reached, which is the step the next coalesce is due for.longWall clock of the last successful coalesce, as distinct from the period grid instant thatgetLastCoalesceDate()now reports.longIn the last full period, how many events occurred?doubleDuring the last period, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?doubleUsing the last period's rate, what is the total value that could have been sent if events were constant?longIn the last full period, how much of the time was spent doing the events?doubleIn the last full period, what was the total value accrued through all events?double[]getLastValues(int count) The last N data points from the underlying storage (RRD).doubleWhat was the average value across the events since the stat was created?longHow many events have occurred over the rate's lifetime?doubleDuring the lifetime of this stat, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?longHow many periods have we already completed?longHow much of the time was spent doing the events over the rate's lifetime?doubleWhat was the total value accrued through all events over the rate's lifetime?doubleWhat was the total value, compared to the total value in the extreme period (i.e.doubleHow large was the last period's value as compared to the lifetime average value? Warning- returns ratio, not percentage (i.e.longHow large should this rate's cycle be?Return the rate stat this rate belongs to, or null.getRecentSamples(int max) Retained coalesced samples, oldest first, for a listener to backfill from.Return the current summary listener, or null.inthashCode()It doesn't appear that Rates are ever stored in a Set or Map (RateStat stores in an array) so let's make this easy.(package private) static booleanisBacklogCollapse(long now, long lastCoalesce, long period) Whether a coalesce arriving atnowis further than one whole period behind its grid, and must therefore collapse its backlog into the current step rather than stamp the step it is nominally due for.final voidload(Properties props, String prefix, boolean treatAsCurrent) Load this rate from the properties, taking data from the data points underneath the given prefix.voidsetRateStat(RateStat rs) The rate stat this rate belongs to.(package private) voidsetSampleDelivery(RateSampleDelivery delivery) Route coalesced samples throughdeliveryinstead of calling the summary listener inline.voidsetSummaryListener(RateSummaryListener listener) The listener to notify on coalesce.voidstore(String prefix, StringBuilder buf) Stores the rate data to a string builder.voidstore(String prefix, StringBuilder buf, boolean addComments) Stores the rate data to a string builder.toString()
Constructor Details
Rate
A rate with period shorter than Router.COALESCE_TIME = 50*1000 has to be manually coalesced before values are fetched from it.- Parameters:
period- number of milliseconds in the period this rate deals with, min 1, max Integer.MAX_VALUE- Throws:
IllegalArgumentException- if the period is invalid
Rate
public Rate(Properties props, String prefix, boolean treatAsCurrent) throws IllegalArgumentException Create a new rate and load its state from the properties, taking data from the data points underneath the given prefix.(e.g. prefix = "profile.dbIntroduction.60m", this will load the associated data points such as "profile.dbIntroduction.60m.lifetimeEventCount"). The data can be exported through store(outputStream, "profile.dbIntroduction.60m").
- Parameters:
props- the properties previously written by store()prefix- prefix to the property entries (should NOT end with a period)treatAsCurrent- if true, we'll treat the loaded data as if no time has elapsed since it was written out, but if it is false, we'll treat the data with as much freshness (or staleness) as appropriate.- Throws:
IllegalArgumentException- if the data was formatted incorrectly
Method Details
getCoalesceSkips
public long getCoalesceSkips()Number ofcoalesce()calls that declined to coalesce because the rate period had not yet elapsed.One skip per cycle is normal — the coalesce timer does not divide evenly into the rate period, so most periods are visited once before they are due and are skipped. A count that grows much faster than that indicates the rate is not coalescing at all.
- Returns:
- the running skip count since construction
getCoalesceCount
public long getCoalesceCount()Number ofcoalesce()calls that actually coalesced.Paired with a storage listener's own written-step count, this is the ledger invariant: the two must agree over the window since that listener attached, and any positive drift is a step that was never recorded.
- Returns:
- the running coalesce count since construction
getCoalesceBacklogCollapses
public long getCoalesceBacklogCollapses()How many stepscoalesce()gave up on because the timer had been starved for more than one whole period.Each one is a whole period of aggregate data that will never reach storage: a rate holds only the current partial period, so the steps it missed cannot be reconstructed afterwards. Zero is the healthy state, and a nonzero value means the coalesce timer was not keeping up with the period — a router-wide condition, since every rate shares that timer.
- Returns:
- the running collapse count since construction
getRecentSamples
Retained coalesced samples, oldest first, for a listener to backfill from.Only the last
RECENT_SAMPLE_RINGsuccessful coalesces are kept, in a fixed ring: this runs on the coalesce path under two monitors and must not grow the heap. Never null.- Parameters:
max- the most samples to return; clamped to the ring size, and non-positive values yield an empty list- Returns:
- an immutable list of at most
maxsamples, oldest first
archiveStep
static long archiveStep(long timestampMs, long period) The archive step a sample stampedtimestampMsbelongs to.A rate's RRD is created with an archive step of
period/1000(seeGraphListener.startListening), so the step index is simplytimestampMs / periodand nothing else needs to knowarcStep. Shared withRateSampleDeliveryso there is one formula for "which step is this sample for".- Parameters:
timestampMs- the sample's step stamp in msperiod- the rate period in ms, must be positive- Returns:
- the archive step index
coalesceDueMs
static long coalesceDueMs(long lastCoalesce, long period) The timestamp a coalesce due afterlastCoalescemust stamp its sample with: the next point on this rate's own period grid.Stamping
nowinstead puts a late tick's aggregate into whichever step the current wall clock falls in, not the step whose data it is. With a 50s coalesce timer and a 60s period that is the common case, and the effect is a sample written into a step a later period also writes into. Deriving the stamp from the grid makes consecutive coalesces exactlyperiodapart, hence exactly one step apart.- Parameters:
lastCoalesce- the previous coalesce timestamp on this rate's gridperiod- the rate period in ms- Returns:
lastCoalesce + period
isBacklogCollapse
static boolean isBacklogCollapse(long now, long lastCoalesce, long period) Whether a coalesce arriving atnowis further than one whole period behind its grid, and must therefore collapse its backlog into the current step rather than stamp the step it is nominally due for.Replaying the missed steps is not an option: a rate retains only the current partial period, so the aggregates for steps already gone were never held anywhere. Writing the backlog into the current step is the only honest choice, and
getCoalesceBacklogCollapses()makes the gap visible instead of leaving it as a hole in the graph.- Parameters:
now- wall-clock ms of this coalesce visitlastCoalesce- the previous coalesce timestamp on this rate's gridperiod- the rate period in ms, must be positive- Returns:
- true if the backlog is more than one period deep
getCurrentTotalValue
public double getCurrentTotalValue()In current (partial) period, what is the total value accrued through all events?- Returns:
- the current period's total value
getCurrentEventCount
public long getCurrentEventCount()In current (partial) period, how many events have occurred?- Returns:
- the current period's event count
getCurrentTotalEventTime
public long getCurrentTotalEventTime()In current (partial) period, how much of the time has been spent doing the events?- Returns:
- accumulated event time in ms for the current period
getLastTotalValue
public double getLastTotalValue()In the last full period, what was the total value accrued through all events?- Returns:
- the total value accrued in the last full period
getLastEventCount
public long getLastEventCount()In the last full period, how many events occurred?- Returns:
- the last period's event count
getLastTotalEventTime
public long getLastTotalEventTime()In the last full period, how much of the time was spent doing the events?- Returns:
- accumulated event time in ms for the last full period
getExtremeTotalValue
public double getExtremeTotalValue()What was the max total value accrued in any period?- Returns:
- the extreme total value
getExtremeEventCount
public long getExtremeEventCount()When the max(totalValue) was achieved, how many events occurred in that period? Note that this is not necessarily the highest event count; that isn't tracked.- Returns:
- the extreme event count
getExtremeTotalEventTime
public long getExtremeTotalEventTime()When the max(totalValue) was achieved, how much of the time was spent doing the events?- Returns:
- accumulated event time in ms for the extreme period
getLifetimeTotalValue
public double getLifetimeTotalValue()What was the total value accrued through all events over the rate's lifetime?- Returns:
- the lifetime total value
getLifetimeEventCount
public long getLifetimeEventCount()How many events have occurred over the rate's lifetime?- Returns:
- the lifetime event count
getLifetimeTotalEventTime
public long getLifetimeTotalEventTime()How much of the time was spent doing the events over the rate's lifetime?- Returns:
- accumulated event time in ms over the rate's lifetime
getLastCoalesceDate
public long getLastCoalesceDate()The instant this rate's period grid has reached, which is the step the next coalesce is due for. Advances by exactly one period per coalesce so the archive steps stay aligned; seegetLastCoalesceNow()for the wall clock of the visit itself.- Returns:
- the current grid instant in ms
getCreationDate
public long getCreationDate()When was this rate created?- Returns:
- the creation timestamp in ms
getLastCoalesceNow
public long getLastCoalesceNow()Wall clock of the last successful coalesce, as distinct from the period grid instant thatgetLastCoalesceDate()now reports.- Returns:
- the wall clock ms of the last successful coalesce
getPeriod
public long getPeriod()How large should this rate's cycle be?- Returns:
- the coalescing period in ms
getRateStat
Return the rate stat this rate belongs to, or null.- Returns:
- the rate stat this rate belongs to, or null
setRateStat
The rate stat this rate belongs to.- Parameters:
rs- the rate stat this rate belongs to
addData
public void addData(long value) Accrue the data in current period as an instantaneous event. If value is always a constant, you should be using Frequency instead. If you always use this call, eventDuration is always zero, and the various get*Saturation*() and get*EventTime() methods will return zero.- Parameters:
value- the amount to accrue into the current and lifetime totals
addData
public void addData(long value, long eventDuration) Accrue the data in current period as if the event took the specified amount of time. If value is always a constant, you should be using Frequency instead. If eventDuration is nonzero, then the various get*Saturation*() and get*EventTime() methods will also return nonzero.There are at least 4 possible strategies for eventDuration: 1) eventDuration is always zero. The various get*Saturation*() and get*EventTime() methods will return zero. 2) Each eventDuration is relatively small, and reflects processing time. This is probably the original meaning of "saturation", as it allows you to track how much time is spent gathering the stats. get*EventTime() will be close to 0. get*EventSaturation() will return values close to 0, get*SaturationLimit() will return adjusted values for the totals. 3) The total of the eventDurations are approximately equal to total elapsed time. get*EventTime() will be close to the period. get*EventSaturation() will return values close to 1, get*SaturationLimit() will return adjusted values for the totals. 4) Each eventDuration is not a duration at all, but someother independent data. get*EventTime() may be used to retrieve the data. get*EventSaturation() are probably useless. get*SaturationLimit() are probably useless.- Parameters:
value- value to accrue in current periodeventDuration- how long it took to accrue this data (set to 0 if it was instantaneous)
coalesce
public void coalesce()Coalesce the current period's data into the last period. If the measured period is less than the rate period minus slack, this is a no-op.The no-op path is counted (
getCoalesceSkips()) because it is otherwise invisible: no log, no counter, no exception. A rate stuck skipping stops feeding its summary listener, and therefore its RRD database, while every other health check still passes. That is precisely the failure this count exists to make observable. Slack guards early visits only — it decides whether a period is due yet, and nothing about what the sample is stamped with, so it cannot make a late visit land in the wrong step.The sample is stamped with the step it is due for (
coalesceDueMs(long, long)) and the grid is advanced to that same instant, so the steps stay exactly one period apart however much the timer drifts. A backlog deeper than one period cannot be replayed and is collapsed into the current step instead, counted bygetCoalesceBacklogCollapses().Being on the grid makes a rate coalesce once per period even when the timer visits more often than the period elapses, which is the point: an RRD holds one value per step, so the samples that used to land two steps apart left every intervening step empty.
This runs on the shared coalesce timer while
StatManagerholds its monitor, so it does no I/O and takes no lock: the counters are updated under this rate's monitor, the ring entry is a fixed array write, and the listener goes to the delivery queue.setSummaryListener
The listener to notify on coalesce.- Parameters:
listener- the listener to notify on coalesce, or null to clear
setSampleDelivery
Route coalesced samples throughdeliveryinstead of calling the summary listener inline.Owned by
StatManagerso that one delivery thread serves the whole rate table; passing it in per rate would mean a thread per rate. Pass null to restore inline delivery.- Parameters:
delivery- the shared delivery queue, or null for inline delivery
clearSummaryListener
Clear the summary listener, but only if it is still the one registered.A plain
setSummaryListener(RateSummaryListener)clear is not safe here. Listeners outlive the registration that created them — a console torn down after a replacement has already registered its own listener would clear the new one, because it cannot tell the two apart. The survivor keeps its database open, so every health check still reports it as attached and nothing re-arms it: the rate goes permanently unrecorded with nothing logged. Tearing down something you no longer own must be a no-op, which is what the identity test below provides.Identity, not equality: two distinct listeners for one rate compare equal (they implement
equalsby rate), but only one of them is ever the registered instance, soequalswould let the wrong one clear.- Parameters:
expected- the listener believed to be registered; may be null- Returns:
- true if the listener was cleared by this call, false if the registration was already absent or held by someone else
getSummaryListener
Return the current summary listener, or null.- Returns:
- the current summary listener, or null
getLastValues
public double[] getLastValues(int count) The last N data points from the underlying storage (RRD). Delegates to the summary listener if it supports historical data. Returns NaN-padded array if the listener is null or data is unavailable.- Parameters:
count- number of data points- Returns:
- array of length count
getAverageValue
public double getAverageValue()What was the average value across the events in the last period?- Returns:
- the average value
getExtremeAverageValue
public double getExtremeAverageValue()During the extreme period (i.e. the period with the highest total value), what was the average value?- Returns:
- the extreme average value
getLifetimeAverageValue
public double getLifetimeAverageValue()What was the average value across the events since the stat was created?- Returns:
- the lifetime average value
getAvgOrLifetimeAvg
public double getAvgOrLifetimeAvg()The average value, or the lifetime average if no recent data.- Returns:
- the average or lifetime average depending on last event count
getLastEventSaturation
public double getLastEventSaturation()During the last period, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?- Returns:
- ratio, or 0 if event times aren't used
getExtremeEventSaturation
public double getExtremeEventSaturation()During the extreme period (i.e. the period with the highest total value), how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?- Returns:
- ratio, or 0 if the statistic doesn't use event times
getLifetimeEventSaturation
public double getLifetimeEventSaturation()During the lifetime of this stat, how much of the time was spent actually processing events in proportion to how many events could have occurred if there were no intervals?- Returns:
- ratio, or 0 if event times aren't used
getLifetimePeriods
public long getLifetimePeriods()How many periods have we already completed?- Returns:
- the number of whole periods elapsed since creation
getLastSaturationLimit
public double getLastSaturationLimit()Using the last period's rate, what is the total value that could have been sent if events were constant?- Returns:
- max total value, or 0 if event times aren't used
getExtremeSaturationLimit
public double getExtremeSaturationLimit()During the extreme period (i.e. the period with the highest total value), what is the total value that could have been sent if events were constant?- Returns:
- event total at saturation, or 0 if no event times are measured
getPercentageOfExtremeValue
public double getPercentageOfExtremeValue()What was the total value, compared to the total value in the extreme period (i.e. the period with the highest total value), Warning- returns ratio, not percentage (i.e. it is not multiplied by 100 here).- Returns:
- ratio of last total to extreme total, or 0 if no data
getPercentageOfLifetimeValue
public double getPercentageOfLifetimeValue()How large was the last period's value as compared to the lifetime average value? Warning- returns ratio, not percentage (i.e. it is not multiplied by 100 here).- Returns:
- ratio of last period value to lifetime average period value, or 0 if no data
computeAverages
Computes the averages for this rate.- Returns:
- a thread-local temp object containing computed averages.
computeAverages
Computes the averages and stores them in the provided object.- Parameters:
out- where to store the computed averages.useLifetime- whether the lifetime average should be used if there are no events.- Returns:
- the same RateAverages object for chaining
store
Stores the rate data to a string builder. Includes comment lines.- Parameters:
prefix- prefix to the property entries (should NOT end with a period)buf- the builder the properties are appended to
store
Stores the rate data to a string builder.- Parameters:
prefix- prefix to the property entries (should NOT end with a period)buf- the builder the properties are appended toaddComments- add comment lines to the output
load
public final void load(Properties props, String prefix, boolean treatAsCurrent) throws IllegalArgumentException Load this rate from the properties, taking data from the data points underneath the given prefix.- Parameters:
props- the properties previously written by store()prefix- prefix to the property entries (should NOT end with a period)treatAsCurrent- if true, we'll treat the loaded data as if no time has elapsed since it was written out, but if it is false, we'll treat the data with as much freshness (or staleness) as appropriate.- Throws:
IllegalArgumentException- if the data was formatted incorrectly
equals
hashCode
toString