All Implemented Interfaces:
Service

public class FloodfillNetworkDatabaseFacadeextends KademliaNetworkDatabaseFacade
The Kademlia network database, using floodfill participants as the only storage tier.
  • Field Details

    • CAPABILITY_FLOODFILL

      public static final char CAPABILITY_FLOODFILL
      Floodfill capability flag.
      See Also:
    • CAPABILITY_UNREACHABLE

      public static final char CAPABILITY_UNREACHABLE
      Unreachable capability flag.
      See Also:
    • CAPABILITY_BW12

      public static final char CAPABILITY_BW12
      Bandwidth 12 Kbps capability flag.
      See Also:
    • CAPABILITY_BW32

      public static final char CAPABILITY_BW32
      Bandwidth 32 Kbps capability flag.
      See Also:
    • CAPABILITY_BW64

      public static final char CAPABILITY_BW64
      Bandwidth 64 Kbps capability flag.
      See Also:
    • CAPABILITY_BW128

      public static final char CAPABILITY_BW128
      Bandwidth 128 Kbps capability flag.
      See Also:
    • CAPABILITY_BW256

      public static final char CAPABILITY_BW256
      Bandwidth 256 Kbps capability flag.
      See Also:
    • CAPABILITY_BW512

      public static final char CAPABILITY_BW512
      Bandwidth 512 Kbps capability flag.
      See Also:
    • CAPABILITY_BW_UNLIMITED

      public static final char CAPABILITY_BW_UNLIMITED
      Unlimited bandwidth capability flag.
      See Also:
    • CAPABILITY_CONGESTION_MODERATE

      public static final char CAPABILITY_CONGESTION_MODERATE
      Moderate congestion capability flag.
      See Also:
    • CAPABILITY_CONGESTION_SEVERE

      public static final char CAPABILITY_CONGESTION_SEVERE
      Severe congestion capability flag.
      See Also:
    • CAPABILITY_NO_TUNNELS

      public static final char CAPABILITY_NO_TUNNELS
      No tunnels capability flag.
      See Also:
    • _activeFloodQueries

      private final ConcurrentHashMap<Hash, FloodSearchJob> _activeFloodQueries
      Active flood queries keyed by search hash.
    • _floodfillEnabled

      private volatile boolean _floodfillEnabled
      Whether this router is currently floodfill-enabled.
    • _verifiesInProgress

      private final Set<Hash> _verifiesInProgress
      Hashes whose verification is in progress.
    • _floodThrottler

      private FloodThrottler _floodThrottler
      Throttles excessive flood operations.
    • _lookupThrottler

      private LookupThrottler _lookupThrottler
      Throttles excessive lookup operations.
    • _lookupBanner

      private LookupBanHammer _lookupBanner
      Ban hammer for abusive lookup peers.
    • _ffMonitor

      private final Job _ffMonitor
      Floodfill monitor job for capability management.
    • _probeStalePeerJob

      private final ProbeStalePeerJob _probeStalePeerJob
      Job to probe stale floodfill peers.
    • _contactRefreshJob

      private final ContactDrivenRefreshJob _contactRefreshJob
      Job to refresh RouterInfo on contact.
    • _introducerLookupJob

      private final IntroducerLookupJob _introducerLookupJob
      Job to prefetch introducer RIs of unreachable peers.
    • _searchTimeouts

      Batched search timeout entries keyed by bucket.
    • _timeoutProcessor

      Periodic processor for elapsed search timeouts.
    • MAX_TO_FLOOD

      public static final int MAX_TO_FLOOD
      This is the flood redundancy. Entries are sent to this many other floodfills. Was 7 through release 0.9; 5 for 0.9.1. 4 as of 0.9.2; 3 as of 0.9.9 Raised to 10: wider RI/LS propagation improves lookup success and reduces dead-zone risk from date-based routing key rotation. Bandwidth cost is negligible (~2KB per store × 10 FFs).
      See Also:
    • FLOOD_PRIORITY

      private static final int FLOOD_PRIORITY
      Priority for flood outbound messages.
      See Also:
    • MAX_RI_AGE_BEFORE_REFRESH_MS

      static final long MAX_RI_AGE_BEFORE_REFRESH_MS
      Max age (ms) for a RouterInfo stored in our NetDB before we consider refreshing it. RIs published within this window are treated as fresh.
      See Also:
    • FLOOD_TIMEOUT

      private static final int FLOOD_TIMEOUT
      Timeout for individual flood messages.
      See Also:
    • NEXT_RKEY_RI_ADVANCE_TIME

      static final long NEXT_RKEY_RI_ADVANCE_TIME
      Advance time before midnight to flood to next routing key for RouterInfo.
      See Also:
    • NEXT_RKEY_LS_ADVANCE_TIME

      private static final long NEXT_RKEY_LS_ADVANCE_TIME
      Advance time before midnight to flood to next routing key for LeaseSet.
      See Also:
    • NEXT_FLOOD_QTY

      private static final int NEXT_FLOOD_QTY
      Number of additional peers to flood near routing key rollover.
      See Also:
    • PUBLISH_JOB_DELAY

      private static final int PUBLISH_JOB_DELAY
      Delay before publishing RouterInfo at startup.
      See Also:
    • _lastClientSubDbWarn

      private static volatile long _lastClientSubDbWarn
      Rate-limit Client subDb exploratory drop WARN (10s). @since 0.9.71+.
    • CLIENT_SUBDB_WARN_INTERVAL

      private static final long CLIENT_SUBDB_WARN_INTERVAL
      Interval between the client subDB warnings.
      See Also:
    • PROP_FLOODFILL_PARTICIPANT

      public static final String PROP_FLOODFILL_PARTICIPANT
      Config property for floodfill participation.
      See Also:
    • PROP_FLOODFILL_AT_RESTART

      public static final String PROP_FLOODFILL_AT_RESTART
      Config property recording floodfill status at previous restart.
      See Also:
    • PUBLISH_TIMEOUT

      static final long PUBLISH_TIMEOUT
      This maybe could be shorter than RepublishLeaseSetJob.REPUBLISH_LEASESET_TIMEOUT, because we are sending direct, but unresponsive floodfills may take a while due to timeouts.
      See Also:
    • INITIAL_CONCURRENT

      private static final int INITIAL_CONCURRENT
      Initial number of concurrent floodfill peers for store operations.
      See Also:
    • MIN_ACTIVE_PEERS

      protected static final int MIN_ACTIVE_PEERS
      Minimum number of active peers before dropping router info lookups. NTCP connections drop quickly but SSU takes a while, so this is kept a little higher than 1 or 2.
  • Constructor Details

    • FloodfillNetworkDatabaseFacade

      public FloodfillNetworkDatabaseFacade(RouterContext context)
      Main DB.
      Parameters:
      context - the router context
    • FloodfillNetworkDatabaseFacade

      public FloodfillNetworkDatabaseFacade(RouterContext context, Hash dbid)
      Sub DBs.
      Parameters:
      context - the router context
      dbid - null for main DB
  • Method Details

    • startup

      public void startup()
      Start monitor jobs, throttle, and optional router refresh.
      Specified by:
      startup in interface Service
      Overrides:
      startup in class KademliaNetworkDatabaseFacade
    • createHandlers

      protected void createHandlers()
      Register message handlers for DatabaseLookup and DatabaseStore.
      Overrides:
      createHandlers in class KademliaNetworkDatabaseFacade
    • registerSearchTimeout

      public void registerSearchTimeout(Hash peer, IterativeSearchJob search, long timeoutMs)
      Register a search timeout to be processed in batches. Instead of creating individual IterativeTimeoutJob for each peer, we register here and a single BatchedSearchTimeoutJob processes all at once.
      Parameters:
      peer - the target key whose search timeout is being registered
      search - the search term
      timeoutMs - the absolute time when the timeout expires
    • processElapsedTimeouts

      public void processElapsedTimeouts()
      Process all elapsed search timeouts. Called by BatchedSearchTimeoutJob.
    • startTimeoutProcessor

      public void startTimeoutProcessor()
      Start the batched timeout processor. Called from startup().
    • shutdown

      public void shutdown()
      If we are floodfill, turn it off and tell everybody.
      Specified by:
      shutdown in interface Service
      Overrides:
      shutdown in class KademliaNetworkDatabaseFacade
    • publish

      public void publish(RouterInfo localRouterInfo) throws IllegalArgumentException
      Send our RI to the closest floodfill.
      Overrides:
      publish in class KademliaNetworkDatabaseFacade
      Parameters:
      localRouterInfo - the RouterInfo to publish
      Throws:
      IllegalArgumentException - if the local router info is invalid
    • shouldFloodOnStore

      static boolean shouldFloodOnStore(boolean floodfillEnabled, DatabaseEntry ds)
      Whether sendStore should wide-flood the entry via flood(DatabaseEntry) when this router participates in floodfill.

      Floodfill participants wide-flood both RouterInfos (existing) and LeaseSets so published LSs reach the routing-key-closest peers that lookups query, not only the few peers chosen for the reply-based store job. Narrow store-only delivery leaves keys missing on other floodfills (verify FailNoKey).

      Parameters:
      floodfillEnabled - whether this router is a floodfill participant
      ds - the entry being stored, may be null
      Returns:
      true if flood() should be invoked for this entry
    • storeJobCount

      static int storeJobCount(boolean floodfillEnabled, DatabaseEntry ds, int selectedCount)
      How many FloodfillStoreJob instances sendStore should schedule alongside (or instead of) a wide flood.

      Floodfill + RouterInfo: 0 — wide flood only (existing; no reply-based job). Floodfill + LeaseSet: 1 — wide flood covers propagation; this job supplies reply-based success/failure callbacks and verify accounting. Otherwise: selectedCount staggered multi-peer jobs (non-floodfill path).

      Parameters:
      floodfillEnabled - whether this router is a floodfill participant
      ds - the entry being stored, may be null
      selectedCount - peers chosen by selectFloodfillParticipants
      Returns:
      number of FloodfillStoreJob instances to schedule; never negative
    • sendStore

      void sendStore(Hash key, DatabaseEntry ds, Job onSuccess, Job onFailure, long sendTimeout, Set<Hash> toIgnore)
      Send out a store.
      Specified by:
      sendStore in class KademliaNetworkDatabaseFacade
      Parameters:
      key - the DatabaseEntry hash
      ds - the database entry
      onSuccess - may be null; called on reply-based store success (not used for wide-flood-only RI path)
      onFailure - may be null; called when no floodfill peers are available or reply-based store fails
      sendTimeout - timeout in ms for the reply-based FloodfillStoreJob path
      toIgnore - may be null, if non-null, all attempted and skipped targets will be added as of 0.9.53; passed through to FloodfillStoreJob (unused for wide-flood-only RI path)
    • selectFloodfillParticipants

      private Set<Hash> selectFloodfillParticipants(Set<Hash> toIgnore, int concurrent)
      Selects a set of floodfill participant peers to which data should be sent: random selection among the filtered floodfill participants, preferring peers not queried in the last 60s to spread load, with a fallback top-up so store operations never starve.
      Parameters:
      toIgnore - the set of peer hashes to exclude from selection, may be null
      concurrent - the maximum number of peers to select
      Returns:
      a set of floodfill participant Hashes; never null but possibly empty if no suitable peers found
    • selectStoreParticipants

      static Set<Hash> selectStoreParticipants(List<Hash> shuffled, int concurrent, Set<Hash> recentlyQueried)
      Select up to concurrent store participants, preferring peers not queried in the last 60s to spread load, with a fallback top-up so store operations never starve.
      Parameters:
      shuffled - the shuffled candidate peers (may be empty)
      concurrent - the maximum number of peers to select
      recentlyQueried - hashes of peers queried within the cooldown period
      Returns:
      the selected peers; never null, possibly empty
    • shouldThrottleFlood

      boolean shouldThrottleFlood(Hash key)
      Increments and tests.
      Parameters:
      key - the key being flooded, tested against the per-key flood allowance
      Returns:
      true if the flood should be throttled for this key
    • shouldBanLookup

      boolean shouldBanLookup(Hash from, TunnelId id)
      Check if lookup from a peer should be banned.
      Parameters:
      from - the peer hash
      id - the tunnel ID
      Returns:
      true if the lookup should be banned
    • shouldThrottleLookup

      boolean shouldThrottleLookup(Hash from, TunnelId id)
      Increments and tests.
      Parameters:
      from - the peer hash that sent the lookup
      id - the tunnel ID the lookup arrived on
      Returns:
      true if the lookup should be throttled for this peer/tunnel
    • floodConditional

      public boolean floodConditional(DatabaseEntry ds)
      If we are floodfill AND the key is not throttled, flood it, otherwise don't.
      Parameters:
      ds - the entry to flood, which must be a LeaseSet for the store to accept it
      Returns:
      if we did
    • flood

      public void flood(DatabaseEntry ds)
      Send to a subset of all floodfill peers. We do this to implement Kademlia within the floodfills, i.e. we flood to those closest to the key.
      Parameters:
      ds - the entry to store and flood to the closest floodfill participants
    • shouldFloodTo

      private boolean shouldFloodTo(Hash key, int type, SigType lsSigType, Hash peer, RouterInfo target)
      Determines if we should flood the database store to a peer.
      Parameters:
      key - the database entry hash
      type - database store type
      lsSigType - may be null
      peer - the target peer hash
      target - the peer's RouterInfo, may be null
      Returns:
      true if the store should be flooded to this peer
    • setFloodfillEnabled

      public void setFloodfillEnabled(boolean yes)
      Public, called from console. This wakes up the floodfill monitor, which will rebuild the RI and log in the event log, and call setFloodfillEnabledFromMonitor which really sets it.
      Parameters:
      yes - true to become floodfill, false to stop
    • setFloodfillEnabledFromMonitor

      void setFloodfillEnabledFromMonitor(boolean yes)
      Package private, called from FloodfillMonitorJob. This does not wake up the floodfill monitor.
      Parameters:
      yes - true to become floodfill, false to stop; true also creates the throttler and HFDSMJ stats
    • floodfillEnabled

      public boolean floodfillEnabled()
      Whether this router is currently acting as a floodfill.
      Overrides:
      floodfillEnabled in class NetworkDatabaseFacade
      Returns:
      true if enabled, as set from the config or the floodfill monitor
    • isFloodfill

      public static boolean isFloodfill(RouterInfo peer)
      Checks if a RouterInfo has floodfill capability.
      Parameters:
      peer - may be null, returns false if null
      Returns:
      true if the RouterInfo advertises the 'f' floodfill capability
    • getKnownRouterData

      public List<RouterInfo> getKnownRouterData()
      Return all RouterInfo entries from the local data store.
      Returns:
      list of known RouterInfo objects, never null
    • search

      SearchJob search(Hash key, Job onFindJob, Job onFailedLookupJob, long timeoutMs, boolean isLease)
      Lookup using exploratory tunnels.

      Caller should check negative cache and/or banlist before calling.

      Begin a kademlia style search for the key specified, which can take up to timeoutMs and will fire the appropriate jobs on success or timeout (or if the kademlia search completes without any match).

      Specified by:
      search in class KademliaNetworkDatabaseFacade
      Parameters:
      key - the hash to search for
      onFindJob - job on find
      onFailedLookupJob - job on failure
      timeoutMs - timeout in ms
      isLease - whether searching for lease
      Returns:
      always null; the caller is notified through onFindJob or onFailedLookupJob
    • search

      SearchJob search(Hash key, Job onFindJob, Job onFailedLookupJob, long timeoutMs, boolean isLease, Hash fromLocalDest)
      Lookup using the client's tunnels.

      Caller should check negative cache and/or banlist before calling. Every path either schedules a search or queues onFailedLookupJob — never returns without notifying the caller.

      Specified by:
      search in class KademliaNetworkDatabaseFacade
      Parameters:
      key - the hash to search for
      onFindJob - job on find
      onFailedLookupJob - job on failure
      timeoutMs - timeout in ms
      isLease - whether searching for lease
      fromLocalDest - use these tunnels for the lookup, or null for exploratory
      Returns:
      null always
    • getMainFacade

      private FloodfillNetworkDatabaseFacade getMainFacade()
      Main (router) NetDb facade, or null if not yet available. Client sub-DBs delegate exploratory searches here so callbacks always fire.
      Returns:
      the main FloodfillNetworkDatabaseFacade, or null
    • failSearchCallbacks

      private void failSearchCallbacks(Job onFindJob, Job onFailedLookupJob)
      Queue failure callbacks so a search that cannot start never leaves the caller waiting for a timeout that will never be scheduled. Success and failure jobs may both be non-null (lookupDestination); on failure both roles collapse to the finished/failed job.
      Parameters:
      onFindJob - may be null
      onFailedLookupJob - may be null
    • complete

      void complete(Hash key)
      Must be called by the search job queued by search() on success or failure.
      Parameters:
      key - the key whose pending search is being retired
    • contactHeardFrom

      public void contactHeardFrom(Hash peer)
      Notify that we heard from or about a peer (e.g. received a DatabaseStoreMessage from them). The ContactDrivenRefreshJob will consider refreshing their RouterInfo if it is stale.
      Parameters:
      peer - the peer hash we heard from or about
    • probeStalePeer

      void probeStalePeer(Hash peer, RouterInfo ri)
      Fire a DirectLookupJob for a stale peer, deduplicated via _activeFloodQueries. Caller should already have verified the peer is stale and not recently probed.
      Parameters:
      peer - the stale peer's router hash, which keys the deduplication map
      ri - the peer's RouterInfo, used by the job to verify the entry
    • getFloodfillPeers

      public List<Hash> getFloodfillPeers()
      List of the hashes of currently known floodfill peers.
      Returns:
      list of floodfill peer hashes, not including our own hash; not sorted and not shuffled
    • isVerifyInProgress

      boolean isVerifyInProgress(Hash h)
      Checks if verification is in progress for the specified hash.
      Parameters:
      h - the key whose verifying floodfill is being tested
      Returns:
      true if a FloodfillVerifyStoreJob for this key is still running
    • verifyStarted

      void verifyStarted(Hash h)
      Marks verification as started for the specified hash.
      Parameters:
      h - the key being verified by a FloodfillVerifyStoreJob
    • verifyFinished

      void verifyFinished(Hash h)
      Marks verification as finished for the specified hash.
      Parameters:
      h - the key whose verifying floodfill has just completed
    • lookupBeforeDropping

      protected void lookupBeforeDropping(Hash peer, RouterInfo info)
      Search for a newer router info, drop it from the db if the search fails, unless just started up or have bigger problems.
      Overrides:
      lookupBeforeDropping in class KademliaNetworkDatabaseFacade
      Parameters:
      peer - the router hash to drop after the lookup failure
      info - the router info
    • getTunnelName

      public String getTunnelName(Destination d)
      Look up the tunnel pool nickname for a destination.
      Parameters:
      d - the destination whose inbound, else outbound, pool nickname is wanted
      Returns:
      the configured nickname, or the empty string if neither pool has one