java.lang.Object
net.i2p.router.Blocklist
public class Blocklistextends Object
Manage blocking by IP address, in a manner similar to the Banlist, which blocks by router hash.

We also try to keep the two lists in sync: if a router at a given IP is blocked, we will also banlist it "forever" (until the next reboot).

While the reverse case (blocking the IP of a router banlisted forever) is not automatic, the transports will call add() below to block the IP, which allows the transports to terminate an inbound connection before the router ident handshake.

And the on-disk blocklist can also contain router hashes to be banlisted.

So, this class maintains three separate lists:

1) The list of IP ranges, read in from a file at startup
2) The list of hashes, read in from the same file
3) A list of single IPs, initially empty, added to as needed

Read in the IP blocklist from a file, store it in-memory as efficiently as we can, and perform tests against it as requested.

When queried for a peer that is blocklisted but isn't banlisted, banlist it forever, then go back to the file to get the original entry so we can add the reason to the banlist text.

On-disk blocklist supports IPv4 only. In-memory supports both IPv4 and IPv6.

  • Field Details

    • _log

      private final Log _log
      Logger for blocklist operations.

    • _context

      private final RouterContext _context
      The router context.

    • _banLogger

      private final BanLogger _banLogger
      Records why routers were banned.

    • _blocklist

      private long[] _blocklist
      Blocked network ranges as packed longs.

    • _blocklistSize

      private volatile int _blocklistSize
      Entries held in the blocklist.

    • _countryBlocklist

      private long[] _countryBlocklist
      Blocked country ranges as packed longs.

    • _countryBlocklistSize

      private int _countryBlocklistSize
      Entries held in the country blocklist.

    • _torBlocklist

      private long[] _torBlocklist
      Known Tor exit ranges as packed longs.

    • _torBlocklistSize

      private volatile int _torBlocklistSize
      Entries held in the Tor blocklist.

    • _lock

      private final Object _lock
      Lock guarding the blocklist arrays.

    • _wrapSave

      private Blocklist.Entry _wrapSave
      Entry held open while writing a range across lines.

    • _inProcess

      private final Set<Hash> _inProcess
      Routers whose banlisting is already queued.

    • _blocklistFeedFile

      private final File _blocklistFeedFile
      File listing the blocklist feeds to fetch.

    • _haveIPv6

      private volatile boolean _haveIPv6
      Whether any IPv6 ranges are blocked.

    • _started

      private volatile boolean _started
      True once the blocklist has been read in.

    • _lastExpired

      private long _lastExpired
      When expired entries were last removed.

    • _peerBlocklist

      private final Map<Hash,String> _peerBlocklist
      Temporarily banned routers by hash, with reasons.

    • _blocklistEnabled

      private volatile boolean _blocklistEnabled
      Whether the blocklist is enabled.

    • _blocklistTorEnabled

      private volatile boolean _blocklistTorEnabled
      Whether the Tor blocklist is enabled.

    • _blocklistCountryEnabled

      private volatile boolean _blocklistCountryEnabled
      Whether the country blocklist is enabled.

    • PROP_BLOCKLIST_ENABLED

      private static final String PROP_BLOCKLIST_ENABLED
      Config key turning the blocklist on or off.
      See Also:

    • PROP_BLOCKLIST_FEEDLIST_ENABLED

      private static final String PROP_BLOCKLIST_FEEDLIST_ENABLED
      Config key turning feed fetching on or off.
      See Also:

    • PROP_BLOCKLIST_TOR_ENABLED

      private static final String PROP_BLOCKLIST_TOR_ENABLED
      Config key turning the Tor blocklist on or off.
      See Also:

    • PROP_BLOCKLIST_COUNTRIES_ENABLED

      private static final String PROP_BLOCKLIST_COUNTRIES_ENABLED
      Config key turning the country blocklist on or off.
      See Also:

    • PROP_BLOCKLIST_DETAIL

      private static final String PROP_BLOCKLIST_DETAIL
      Config key for the detail level in ban messages.
      See Also:

    • PROP_BLOCKLIST_FILE

      private static final String PROP_BLOCKLIST_FILE
      Config key for the blocklist file name.
      See Also:

    • PROP_BLOCKLIST_EXPIRE_INTERVAL

      private static final String PROP_BLOCKLIST_EXPIRE_INTERVAL
      Config key for how long bans stay in effect.
      See Also:

    • BLOCKLIST_FILE_DEFAULT

      public static final String BLOCKLIST_FILE_DEFAULT
      Default blocklist filename in the installation directory.
      See Also:

    • BLOCKLIST_FILE_TOR_EXITS

      public static final String BLOCKLIST_FILE_TOR_EXITS
      Tor exit nodes blocklist filename.
      See Also:

    • BLOCKLIST_FEED_FILE

      private static final String BLOCKLIST_FEED_FILE
      Feed blocklist file path.
      See Also:

    • BLOCKLIST_COUNTRY_FILE

      public static final String BLOCKLIST_COUNTRY_FILE
      Country-based blocklist filename.
      See Also:

    • MAX_IPV4_SINGLES

      private static final int MAX_IPV4_SINGLES
      Limits of transient (in-memory) blocklists. Note that it's impossible to prevent clogging up the tables by a determined attacker, esp. on IPv6.

    • MAX_IPV6_SINGLES

      private static final int MAX_IPV6_SINGLES
      IPv6 addresses held in the singles blocklist.

    • _singleIPBlocklist

      private final Map<Integer,Object> _singleIPBlocklist
      Singly blocked IPv4 addresses.

    • _singleIPv6Blocklist

      private final Map<BigInteger, Object> _singleIPv6Blocklist
      Singly blocked IPv6 addresses.

    • DUMMY

      private static final Object DUMMY
      Placeholder value in the singles maps.

    • ID_FEED

      public static final String ID_FEED
      For Update Manager.
      See Also:

    • ID_SYSTEM

      private static final String ID_SYSTEM
      System blocklist ID.
      See Also:

    • ID_LOCAL

      private static final String ID_LOCAL
      User-configured blocklist ID.
      See Also:

    • ID_COUNTRY

      private static final String ID_COUNTRY
      Country-based blocklist ID.
      See Also:

    • ID_USER

      private static final String ID_USER
      User-specified blocklist ID.
      See Also:

    • ID_SYBIL

      public static final String ID_SYBIL
      Sybil attack blocklist ID.
      See Also:

    • ID_TOR

      public static final String ID_TOR
      Tor exit nodes blocklist ID.
      See Also:

  • Constructor Details

    • Blocklist

      public Blocklist(RouterContext context)
      Constructor. Router MUST call startup().
      Parameters:
      context - the router context

    • Blocklist

      private Blocklist()
      Only for testing with main().

  • Method Details

    • initConfig

      private void initConfig()
      Initialize configurable properties.

    • reloadConfig

      public void reloadConfig()
      Reload configuration from properties. Called when settings are changed in the console.

    • clearAll

      public void clearAll()
      Clear all blocked IPs and peers.

    • isBlocklistEnabled

      public boolean isBlocklistEnabled()
      Check if blocklist is enabled.
      Returns:
      true if blocklist is enabled

    • isTorBlocklistEnabled

      public boolean isTorBlocklistEnabled()
      Check if Tor blocklist is enabled.
      Returns:
      true if Tor blocklist is enabled

    • isCountryBlocklistEnabled

      public boolean isCountryBlocklistEnabled()
      Check if country blocklist is enabled.
      Returns:
      true if country blocklist is enabled

    • expireInterval

      private int expireInterval()
      Get the blocklist expiration interval from configuration.
      Returns:
      the expiration interval in milliseconds, or 0 if disabled

    • startup

      public void startup()
      Loads the following files in-order: $I2P/blocklist.txt $I2P/blocklist_tor.txt ~/.i2p/blocklist.txt ~/.i2p/docs/feed/blocklist/blocklist.txt ~/.i2p/blocklist-countries.txt File if specified with router.blocklist.file.

    • clear

      private void clear()
      Empties all the blocklists and their caches.

    • banlistRouter

      private void banlistRouter(Hash peer, String reason, String comment)
      Queues a ban for a router by hash.
      Parameters:
      peer - the router hash
      reason - the ban category
      comment - extra detail for the log

    • banlistRouterWithRI

      private void banlistRouterWithRI(Hash peer, String reason, String comment, RouterInfo ri, String ip)
      Queues a ban for a router with its info and address.
      Parameters:
      peer - the router hash
      reason - the ban category
      comment - extra detail for the log
      ri - the router info, may be null
      ip - the offending IP address, may be null

    • banlistRouter

      private void banlistRouter(Hash peer, String reason, String comment, RouterInfo ri)
      Queues a ban for a router given its info.
      Parameters:
      peer - the router hash
      reason - the ban category
      comment - extra detail for the log
      ri - the router info, may be null

    • addCountryFile

      public void addCountryFile()
      The blocklist-country.txt file was created or updated. Read it in. Not required normally, as the country file is read by startup().

    • disable

      public void disable()
      Disable the blocklist and clear all entries.

    • allocate

      private long[] allocate(List<Blocklist.BLFile> files)
      Allocate an array for blocklist entries.
      Parameters:
      files - the list of blocklist files to read
      Returns:
      the allocated array, or null on out of memory error

    • readBlocklistFile

      private int readBlocklistFile(Blocklist.BLFile blf, long[] blocklist, int count)
      Read in and parse the blocklist. The blocklist need not be sorted, and may contain overlapping entries.

      Acceptable formats (IPV4 only): #comment (# must be in column 1) comment:IP-IP comment:morecomments:IP-IP IP-IP (comments also allowed before any of the following) IP/masklength IP hostname (DNS looked up at list readin time, not dynamically, so may not be much use) 44-byte Base64 router hash

      Acceptable formats (IPV6 only): comment:IPv6 (must replace : with ; e.g. abcd;1234;0;12;;ff) IPv6 (must replace : with ; e.g. abcd;1234;0;12;;ff)

      No whitespace allowed after the last ':'.

      For further information and downloads: IBlockList supportCymru bogon reference

      Must call allocate(List) before and merge() after.

      Parameters:
      blf - the file to read, its id and its version; blf.version is set to the file's last-modified time when at least one entry was read
      blocklist - out parameter, entries stored here
      count - current number of entries
      Returns:
      new number of entries

    • merge

      private int merge(long[] blocklist, int count)
      Merge a new blocklist into the existing one, returning the new entry count.
      Parameters:
      blocklist - the entries read so far, sorted and de-overlapped in place
      count - valid entries in blocklist before merge
      Returns:
      count valid entries in blocklist after merge

    • parse

      private Blocklist.Entry parse(String buf, boolean shouldLog)
      Parse one line, returning a temp data structure with the result.
      Parameters:
      buf - the line to parse, without the line terminator
      shouldLog - true to log a format-error warning for an unparseable line
      Returns:
      the parsed entry, or null if the line is blank, a comment, or not a valid IP, range, CIDR block or router hash

    • getSize

      private int getSize(File blFile)
      Count the number of entries in a blocklist file.
      Parameters:
      blFile - the blocklist file to count
      Returns:
      the number of non-comment, non-blank lines

    • removeOverlap

      private int removeOverlap(long[] blist, int count)
      Merge and remove overlapping entries from a sorted list. Caller must re-sort if return code is > 0.
      Parameters:
      blist - the sorted blocklist array
      count - the number of valid entries in the array
      Returns:
      the number of overlapping entries removed

    • add

      public void add(String ip)
      Maintain a simple in-memory single-IP blocklist This is used for new additions, NOT for the main list of IP ranges read in from the file.
      Parameters:
      ip - IPv4 or IPv6

    • add

      public void add(String ip, String source)
      Maintain a simple in-memory single-IP blocklist This is used for new additions, NOT for the main list of IP ranges read in from the file.
      Parameters:
      ip - IPv4 or IPv6
      source - for logging only, may be null

    • add

      public void add(byte[] ip)
      Maintain a simple in-memory single-IP blocklist This is used for new additions, NOT for the main list of IP ranges read in from the file.
      Parameters:
      ip - IPv4 or IPv6

    • add

      public void add(byte[] ip, String source)
      Maintain a simple in-memory single-IP blocklist This is used for new additions, NOT for the main list of IP ranges read in from the file.
      Parameters:
      ip - IPv4 or IPv6
      source - for logging only, may be null

    • remove

      public void remove(byte[] ip)
      Remove from the in-memory single-IP blocklist. This is only works to undo add()s, NOT for the main list of IP ranges read in from the file.
      Parameters:
      ip - IPv4 or IPv6

    • add

      private boolean add(int ip)
      Add a single IPv4 address to the blocklist.
      Parameters:
      ip - the IPv4 address packed into an int, network byte order (negative for addresses above 127.255.255.255)
      Returns:
      true if it was NOT previously on the list

    • remove

      private void remove(int ip)
      Remove an IP from the in-memory blocklist.
      Parameters:
      ip - the IPv4 address packed into an int, network byte order

    • isOnSingleList

      private boolean isOnSingleList(int ip)
      Checks whether an IPv4 address is singly blocked.
      Parameters:
      ip - the address as an int
      Returns:
      true if the address is blocked

    • add

      private boolean add(BigInteger ip)
      Add an IPv6 address to the blocklist.
      Parameters:
      ip - IPv6 non-negative
      Returns:
      true if it was NOT previously on the list

    • remove

      private void remove(BigInteger ip)
      Remove an IPv6 address from the blocklist.
      Parameters:
      ip - IPv6 non-negative

    • isOnSingleList

      private boolean isOnSingleList(BigInteger ip)
      Check whether an IPv6 address is on the single IP blocklist.
      Parameters:
      ip - IPv6 non-negative
      Returns:
      whether on single list

    • getAddresses

      private List<byte[]> getAddresses(Hash peer)
      Will not contain duplicates.
      Parameters:
      peer - the router whose published addresses are wanted
      Returns:
      the peer's IP addresses with duplicates removed, empty if the RouterInfo is not held locally

    • getAddresses

      private List<byte[]> getAddresses(RouterInfo pinfo)
      Will not contain duplicates.
      Parameters:
      pinfo - the router info whose published addresses are collected
      Returns:
      the addresses published in the RouterInfo, IPv6 ones only when this router has IPv6 enabled, with duplicates removed

    • isBlocklisted

      public boolean isBlocklisted(Hash peer)
      Check if a peer is blocklisted by IP address. If so, and it isn't banlisted, banlist it forever or for the configured override period.
      Parameters:
      peer - the router hash to check
      Returns:
      true if the peer's IP is in the blocklist

    • isBlocklisted

      public boolean isBlocklisted(RouterInfo pinfo)
      Check if a peer is blocklisted by IP address. If so, and it isn't banlisted, banlist it forever or for the configured override period.
      Parameters:
      pinfo - the router info to check
      Returns:
      true if the peer's IP is in the blocklist

    • isBlocklisted

      public boolean isBlocklisted(String ip)
      Check if an IP address is blocklisted. Calling this externally won't banlist the peer, this is just an IP check.
      Parameters:
      ip - the IP address as a string (IPv4 or IPv6)
      Returns:
      true if the IP is blocklisted

    • isBlocklisted

      public boolean isBlocklisted(byte[] ip)
      Check if an IP address is blocklisted. Calling this externally won't banlist the peer, this is just an IP check.
      Parameters:
      ip - the IP address as a byte array (IPv4 or IPv6)
      Returns:
      true if the IP is blocklisted

    • isBlocklisted

      private boolean isBlocklisted(int ip)
      First check the single-IP list. Then do a binary search through the in-memory range list which is a sorted array of longs. The array is sorted in signed order, but we don't care. Each long is ((from << 32) | to).
      Parameters:
      ip - the IPv4 address packed into an int, network byte order
      Returns:
      true if the IP is on a single-IP, country, permanent or Tor list

    • isPermanentlyBlocklisted

      public boolean isPermanentlyBlocklisted(int ip)
      Check if an IP is permanently blocklisted using binary search. Public for console only, not a public API.
      Parameters:
      ip - the IPv4 address as an integer
      Returns:
      true if the IP is in the permanent blocklist

    • isPermanentlyBlocklisted

      private static boolean isPermanentlyBlocklisted(int ip, long[] blocklist, int blocklistSize)
      Do a binary search through the in-memory range list which is a sorted array of longs. The array is sorted in signed order, but we don't care. Each long is ((from << 32) | to).
      Parameters:
      ip - the IPv4 address packed into an int, network byte order
      blocklist - the sorted range entries, only the first blocklistSize are used
      blocklistSize - how many entries at the start of blocklist are valid
      Returns:
      whether permanently blocklisted

    • match

      private static boolean match(int ip, long entry)
      Check if an IP is included in a compressed blocklist entry.
      Parameters:
      ip - the IP to check
      entry - the compressed blocklist entry
      Returns:
      true if the IP is within the entry's range

    • isHigher

      private static boolean isHigher(int ip, long entry)
      Check if an IP is higher than the entry's starting IP.
      Parameters:
      ip - the IP to check
      entry - the compressed blocklist entry
      Returns:
      true if the IP is higher than the entry's starting IP

    • getFrom

      public static int getFrom(long entry)
      Extract the starting IP from a compressed blocklist entry. Public for console only, not a public API.
      Parameters:
      entry - the compressed blocklist entry
      Returns:
      the starting IP as an integer

    • getTo

      public static int getTo(long entry)
      Extract the ending IP from a compressed blocklist entry. Public for console only, not a public API.
      Parameters:
      entry - the compressed blocklist entry
      Returns:
      the ending IP as an integer

    • toEntry

      private static long toEntry(byte[] ip1, byte[] ip2)
      The in-memory blocklist is an array of longs, with the format ((from IP) << 32) | (to IP). The array is sorted with a signed sort, so the high addresses come first.

      So the size is (cough) almost 2MB for the 240,000 line splist.txt.

      Parameters:
      ip1 - the starting IP of the range, 4 bytes
      ip2 - the ending IP of the range, 4 bytes
      Returns:
      the range as one entry, ip1 in the high 32 bits and ip2 in the low 32 bits

    • store

      private static void store(byte[] ip1, byte[] ip2, long[] blocklist, int idx)
      Store an IPv4 range as a compressed entry in the blocklist array.
      Parameters:
      ip1 - the starting IP as a byte array
      ip2 - the ending IP as a byte array
      blocklist - the array to write the compressed entry into
      idx - the index to store at

    • store

      private static void store(int ip1, int ip2, long[] blocklist, int idx)
      Store an IPv4 range as a compressed entry in the blocklist array.
      Parameters:
      ip1 - the starting IP as an integer
      ip2 - the ending IP as an integer
      blocklist - the array to write the compressed entry into
      idx - the index to store at

    • toInt

      private static int toInt(byte[] ip)
      Convert an IPv4 address from byte array to integer.
      Parameters:
      ip - the IP address as a byte array
      Returns:
      the IP address as an integer

    • toStr

      private static String toStr(long entry)
      Convert a compressed blocklist entry to a string representation.
      Parameters:
      entry - the compressed blocklist entry
      Returns:
      the string representation (e.g., "127.0.0.1-127.0.0.255")

    • toStr

      public static String toStr(int ip)
      Convert an IPv4 address to a string representation. Public for console only, not a public API.
      Parameters:
      ip - the IPv4 address as an integer
      Returns:
      the string representation (e.g., "192.168.1.1")

    • banlist

      private void banlist(Hash peer, byte[] ip)
      We don't keep the comment field in-memory, so we have to go back out to the file to find it.

      Put this in a job because we're looking for the actual line in the blocklist file, this could take a while.

      Parameters:
      peer - the router to ban
      ip - the blocklisted address to look up, IPv4 or IPv6

    • banlist

      private void banlist(Hash peer, byte[] ip, RouterInfo ri)
      Queues a ban for a peer matching a blocklist entry.
      Parameters:
      peer - the router hash
      ip - the offending address
      ri - the router info, may be null

    • banlistRouter

      private void banlistRouter(Hash peer, String reason, String reasonCode, long duration)
      Look up the original record so we can record the reason in the banlist. That's the only reason to do this. Only synchronize to cut down on the I/O load. Additional jobs can wait. Although could this clog up the job queue runners? Yes. So we also stagger these jobs.
      Parameters:
      peer - the router to ban
      reason - the text recorded in the banlist as the reason for the ban
      reasonCode - the blocklisted IP string stored alongside the reason
      duration - positive to ban for expireInterval() ms, 0 or less to ban forever

    • banlistRouter

      private void banlistRouter(Hash peer, List<byte[]> ips, long duration)
      Bans a router and its addresses for a duration.
      Parameters:
      peer - the router hash
      ips - the offending addresses
      duration - how long the ban lasts

    • getTransientIPv4Blocks

      public List<Integer> getTransientIPv4Blocks()
      Single IPs blocked until restart. Unsorted.

      Public for console only, not a public API As of 0.9.57, will not contain IPs permanently banned, except for ones banned permanently after being added to the transient list.

      Returns:
      a copy, unsorted

    • getTransientIPv6Blocks

      public List<BigInteger> getTransientIPv6Blocks()
      Single IPs blocked until restart. Unsorted.

      Public for console only, not a public API.

      Returns:
      a copy, unsorted

    • getPermanentBlocks

      public long[] getPermanentBlocks(int max)
      IP ranges blocked until restart. Sorted, but as signed longs, so 128-255 are first

      Public for console only, not a public API.

      Parameters:
      max - maximum entries to return
      Returns:
      a copy, sorted

    • getBlocklistSize

      public int getBlocklistSize()
      Get the size of the permanent blocklist. Public for console only, not a public API.
      Returns:
      the number of entries in the permanent blocklist

    • _x

      private static final String _x(String s)
      Mark a string for extraction by xgettext and translation. Use this only in static initializers. It does not translate!
      Parameters:
      s - the literal to record for translation
      Returns:
      the string unchanged; marking it is what lets xgettext find it