DataHelper provides essential utilities for working with I2P data structures:
- Serialization: Read/write primitive types to/from streams
- Encoding: Hex encoding/decoding (Base64 and Base32 live in their own classes)
- Validation: Data format verification and bounds checking
- Conversion: Between different data representations
- Utilities: Common operations on collections and arrays
Core Operations:
- Primitive I/O:
readLong(InputStream, int), writeLong(), readString(), writeString() - Hex:
toHexString(byte[]), fromHexString() for display and debugging - Properties:
readProperties(InputStream), writeProperties() for metadata - Collections: toMap(),
Object.toString()for data structures
Encoding Support:
- Base64/Base32: provided by the
Base64andBase32classes, not here - Hex: Hexadecimal encoding for debugging and display
- UTF-8: String encoding with proper byte handling
Data Structure Support:
- Arrays: Efficient reading/writing of byte arrays
- Collections: Map, List, and Set serialization
- Dates: Timestamp handling with proper timezones
- Properties: Key-value pair serialization
Validation and Safety:
- Bounds Checking: Prevent buffer overflows and underflows
- Null Handling: Safe null value processing
- Size Limits: Protection against excessive memory allocation
- Format Validation: Data structure integrity checks
Performance Features:
- Buffer Reuse: ByteArrayStream for efficient operations
- Caching: Frequently used constants and computations
- Bulk Operations: Optimized for large data sets
- Memory Management: Careful allocation and cleanup
Common Use Cases:
- Network Protocol: I2NP message serialization
- Configuration: Properties file handling
- Address Display: via the Base32 class for .b32.i2p
- Data Storage: File format serialization
- Debugging: Hex dumps and data inspection
Thread Safety:
- Static Methods: All operations are thread-safe
- No State: Class maintains no mutable instance state
- Immutable Data: Input data is not modified
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static final classThe shared buffer forcopy(InputStream, OutputStream), resolved once for the same reason and in the same way asDataHelper.DecompressCacheHolder.private static final classThe shared output buffer fordecompress(byte[]).Field Summary
Modifier and TypeFieldDescriptionMap of String to itself to cache common keys in RouterInfo, RouterAddress, and BlockfileNamingService properties.private static final StringName of the translation resource bundle.private static final DateFormatThe default formatting for date/time, current locale, local time zone.static final intNumber of bytes in a date field (8).private static booleanTrue once the date formatter's time zone is set.private static final byte[]Shared empty byte array, to avoid allocating one.private static final String[]The characters escaped for HTML output.private static final String[]The HTML codes for each escaped character.private static final char[]Lower-case hex digits, so toString(byte[], int) never callsInteger.toHexString(int).static final intAppx.private static final PatternCharacters not allowed in a property key.private static final PatternCharacters not allowed in a property value.static final intAppx.private static final intRidiculously long, just to prevent OOM DOS.static final intMaximum uncompressed size.static final intNew default as of 0.9.47.static final intNo compression (level 0).private static final ConcurrentHashMap<String, Pattern> Cache of the compiled patterns.private static final booleanprivate static final DateFormatFormatter for the date and time strings.private static booleanTrue once the time formatter's time zone is set.Constructor Summary
ConstructorsMethod Summary
Modifier and TypeMethodDescriptionprivate static StringTranslates a string in the default bundle.static final intcompareTo(byte[] lhs, byte[] rhs) Big endian compare, treats bytes as unsigned.static byte[]compress(byte[] orig) Compress the data and return a new GZIP compressed byte array.static byte[]compress(byte[] orig, int offset, int size) Compress the data and return a new GZIP compressed byte array.static byte[]compress(byte[] orig, int offset, int size, int level) Compress the data and return a new GZIP compressed byte array.static voidcopy(InputStream in, OutputStream out) Copy in to out.static byte[]copyOfRange(byte[] src, int from, int to) Copy a range of a byte array into a new array, mirroringArrays.copyOfRange(byte[], int, int)but throwing DataFormatException on invalid bounds so callers in the data layer can handle it with their existing exception flow.static byte[]decompress(byte[] orig) Decompress the GZIP compressed data (returning null on error).static byte[]decompress(byte[] orig, int offset, int length) Decompress the GZIP compressed data (returning null on error).static final booleaneq(byte[] lhs, byte[] rhs) Run a comparison on the byte arrays, byte by byte.static final booleaneq(byte[] lhs, int offsetLeft, byte[] rhs, int offsetRight, int length) Unlike eq(byte[], byte[]), this returns false if either lhs or rhs is null.static final booleanHelper util to compare two objects, including null handling.static final booleaneq(Collection<?> lhs, Collection<?> rhs) Run a deep comparison across the two collections.static final booleaneqCT(byte[] lhs, int offsetLeft, byte[] rhs, int offsetRight, int length) Unlike eq(), this throws NPE if either lhs or rhs is null.static final booleanConstant time string comparison.static StringescapeHTML(String unescaped) Escape a string for inclusion in HTML.static StringformatDate(long now) The default formatting for date, current locale, local time zone.static StringformatDuration(long ms) Format a duration in milliseconds to a human-readable string.static StringformatDuration2(double ms) Like formatDuration2(long) but with microsec and nanosec also.static StringformatDuration2(long ms) Like formatDuration but with a non-breaking space after the number, 0 is unitless, and the unit is translated.static StringformatSize(long bytes) This is binary, i.e.static StringformatSize2(long bytes) This is binary, i.e.static StringformatSize2(long bytes, boolean nonBreaking) This is binary, i.e.static StringformatSize2Decimal(long bytes) This is decimal, i.e.static StringformatSize2Decimal(long bytes, boolean nonBreaking) This is decimal, i.e.static StringformatTime(long now) The default formatting for date/time, current locale, local time zone.static DatefromDate(byte[] src, int offset) Read a date (8-byte timestamp) from a byte array.static final byte[]fromHexString(String val) Decode a hex string to a byte array.static longfromLong(byte[] src, int offset, int numBytes) Big endian.static longfromLong8(byte[] src, int offset) Big endian.static longfromLongLE(byte[] src, int offset, int numBytes) Little endian, i.e.static intfromProperties(byte[] source, int offset, Properties target) Reads the props from the byte array and puts them in the Properties target See readProperties() for the format.static byte[]Roughly the same as orig.getBytes("ISO-8859-1") but much faster and will not throw an exception.static StringgetUTF8(byte[] orig) Same as new String(orig, "UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use.static StringgetUTF8(byte[] orig, int offset, int len) Same as new String(orig, "UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use.static byte[]Same as orig.getBytes("UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use.static inthashCode(byte[] b) Calculate the hashcode of the byte array, using 0 for null.static intCalculate the hashcode of the object, using 0 for null.static inthashCode(Collection<?> col) Calculate the hashcode of the collection, using 0 for null.static intCalculate the hashcode of the date, using 0 for null.static voidloadProps(Properties props, File file) A more efficient Properties.loadstatic voidloadProps(Properties props, File file, boolean forceLowerCase) Load properties from a file, optionally forcing keys to lower case.static voidloadProps(Properties props, InputStream inStr) Load properties from a stream.static voidloadProps(Properties props, InputStream inStr, boolean forceLowerCase) Load properties from a stream, optionally forcing keys to lower case.static StringThe throwable's message, never null.private static StringTranslates a countable string, picking the plural form.static intread(InputStream in, byte[] target) This is different thanread(target), in that it does repeated reads until the full data is received.static intread(InputStream in, byte[] target, int offset, int length) WARNING - This is different thanread(target, offset, length)for a nonzero offset, in that it returns the new offset (== old offset + length).static DatereadDate(InputStream in) Read in a date from the stream as specified by the I2P data structure spec.static StringreadLine(InputStream in) Read a newline delimited line from the stream, returning the line without the newline, or null if EOF is reached on an empty line.static booleanreadLine(InputStream in, StringBuilder buf) Read in a line, placing it into the buffer (excluding the newline).static booleanreadLine(InputStream in, StringBuilder buf, MessageDigest hash) Update the hash along the way Warning - strips \n but not \r Warning - 8KB line length limit as of 0.7.13, @throws IOException if exceeded Warning - not UTF-8.static StringreadLine(InputStream in, MessageDigest hash) Update the hash along the way Warning - strips \n but not \r Warning - 8KB line length limit as of 0.7.13, @throws IOException if exceeded Warning - not UTF-8.static longreadLong(InputStream rawStream, int numBytes) Read the stream for an integer as defined by the I2P data structure specification.static PropertiesreadProperties(InputStream rawStream) Read a mapping from the stream, as defined by the I2P data structure spec, and store it into a Properties object.static PropertiesreadProperties(InputStream rawStream, Properties props) Ditto, load into an existing propertiesstatic PropertiesreadProperties(InputStream rawStream, Properties props, boolean enforceOrder) Ditto, load into an existing propertiesstatic StringRead in a string from the stream as specified by the I2P data structure spec.static voidreplace(StringBuilder buf, String from, String to) Replace all instances of "from" with "to" in the StringBuilder buf.static voidskip(InputStream in, long n) This is different thanInputStream.skip(long), in that it does repeated reads until the full amount is skipped.static <T> voidsort(List<T> list, Comparator<? super T> c) Same asCollections.sort(List), but guaranteed not to throw an IllegalArgumentException if the sort is unstable.static <T> voidsort(T[] a, Comparator<? super T> c) Same asArrays.sort(int[]), but guaranteed not to throw an IllegalArgumentException if the sort is unstable.static String[]Same as s.split(regex) but caches the compiled pattern for speed.static String[]Same as s.split(regex, limit) but caches the compiled pattern for speed.static voidstoreProps(Properties props, File file) Writes the props to the file, unsorted (unless props is an OrderedProperties) Note that this does not escape the \r or \n in values.static StringStrip out any HTML (simply removing any less than / greater than symbols).static voidtoDate(byte[] target, int offset, long when) Write a date (8-byte timestamp) to a byte array.static byte[]Prefer toDate(byte[], int, long).static final StringtoHexString(byte[] data) Lower-case hex without leading zeros.static voidtoLong(byte[] target, int offset, int numBytes, long value) Big endian.static byte[]toLong(int numBytes, long value) Big endian.static voidtoLong8(byte[] target, int offset, long value) Big endian.static voidtoLongLE(byte[] target, int offset, int numBytes, long value) Little endian, i.e.static inttoProperties(byte[] target, int offset, Properties props) Writes the props to the byte array, sorted See readProperties() for the format.static byte[]toProperties(Properties opts) Writes the props to returned byte array, not sorted (unless the opts param is an OrderedProperties) Strings will be UTF-8 encoded in the byte array.static StringtoString(byte[] buf) Lower-case hex with leading zeros.static StringtoString(byte[] buf, int len) Lower-case hex with leading zeros.static StringtoString(Collection<?> col) Pretty print the collection.static StringPretty print the mapping, unsorted (unless the options param is an OrderedProperties).static StringtoString(Properties options) Pretty print the mapping, unsorted (unless the options param is an OrderedProperties).static voidwrite(OutputStream out, byte[] data, MessageDigest hash) Write data to the output stream while updating the MessageDigest along the way.static voidwriteDate(OutputStream out, Date date) Write out a date to the stream as specified by the I2P data structure spec.static voidwriteLong(OutputStream rawStream, int numBytes, long value) Write an integer as defined by the I2P data structure specification to the stream.static voidwriteProperties(OutputStream rawStream, Properties props) Write a mapping to the stream, as defined by the I2P data structure spec, and store it into a Properties object.static voidwriteProperties(OutputStream rawStream, Properties props, boolean utf8) Writes the props to the stream, sorted by property name.static voidwriteProperties(OutputStream rawStream, Properties props, boolean utf8, boolean sort) Writes the props to the stream, sorted by property name if sort == true or if props is an OrderedProperties.static voidwriteString(OutputStream out, String string) Write out a string to the stream as specified by the I2P data structure spec.static voidwriteStringUTF8(OutputStream out, String string) Write out a string to the stream as specified by the I2P data structure spec.static final byte[]xor(byte[] lhs, byte[] rhs) XOR two byte arrays.static final voidxor(byte[] lhs, int startLeft, byte[] rhs, int startRight, byte[] out, int startOut, int len) Xor the lhs with the rhs, storing the result in out.private static byte[]zeroCompress(byte[] in, int off, int len) Fast NO_COMPRESSION.private static byte[]zeroDecompress(byte[] in, int off, int len) Fast NO_COMPRESSION.
Field Details
SHOULD_SYNC
private static final boolean SHOULD_SYNCSeestoreProps(Properties, File). 600-750 ms on RPi._propertiesKeyCache
ILLEGAL_KEY
Characters not allowed in a property key.ILLEGAL_VALUE
Characters not allowed in a property value.DATE_FORMAT
The default formatting for date/time, current locale, local time zone.TIME_FORMAT
Formatter for the date and time strings.dateTzSet
private static boolean dateTzSetTrue once the date formatter's time zone is set.timeTzSet
private static boolean timeTzSetTrue once the time formatter's time zone is set.EMPTY_BUFFER
private static final byte[] EMPTY_BUFFERShared empty byte array, to avoid allocating one.HEX_CHARS
private static final char[] HEX_CHARSLower-case hex digits, so toString(byte[], int) never callsInteger.toHexString(int).DATE_LENGTH
public static final int DATE_LENGTHNumber of bytes in a date field (8).- See Also:
MAX_LINE_LENGTH
private static final int MAX_LINE_LENGTHRidiculously long, just to prevent OOM DOS.- See Also:
BUNDLE_NAME
Name of the translation resource bundle.- See Also:
escapeChars
The characters escaped for HTML output.escapeCodes
The HTML codes for each escaped character.MAX_UNCOMPRESSED
MAX_COMPRESSION
public static final int MAX_COMPRESSIONAppx. 30% slower, 2.5% smaller than MEDIUM_COMPRESSION.- See Also:
HIGH_COMPRESSION
public static final int HIGH_COMPRESSIONAppx. 15% slower, 1.5% smaller than MEDIUM_COMPRESSION.- See Also:
MEDIUM_COMPRESSION
NO_COMPRESSION
patterns
Cache of the compiled patterns.
Constructor Details
DataHelper
private DataHelper()Utility class.
Method Details
readProperties
public static Properties readProperties(InputStream rawStream) throws DataFormatException, IOException Read a mapping from the stream, as defined by the I2P data structure spec, and store it into a Properties object.A mapping is a set of key / value pairs. It starts with a 2 byte Integer (ala readLong(rawStream, 2)) defining how many bytes make up the mapping. After that comes that many bytes making up a set of UTF-8 encoded characters. The characters are organized as key=value;. The key is a String (ala readString(rawStream)) unique as a key within the current mapping that does not include the UTF-8 characters '=' or ';'. After the key comes the literal UTF-8 character '='. After that comes a String (ala readString(rawStream)) for the value. Finally after that comes the literal UTF-8 character ';'. This key=value; is repeated until there are no more bytes (not characters!) left as defined by the first two byte integer.
As of 0.9.18, throws DataFormatException on duplicate key.
- Parameters:
rawStream- stream to read the mapping from- Returns:
- an
OrderedPropertiesholding the pairs read from the stream, whose key set and entry set iterate in lexicographical key order rather than stream order - Throws:
DataFormatException- if the format is invalidIOException- if there is a problem reading the data
readProperties
public static Properties readProperties(InputStream rawStream, Properties props) throws DataFormatException, IOException Ditto, load into an existing propertiesAs of 0.9.18, throws DataFormatException on duplicate key Does NOT enforce key ordering.
- Parameters:
rawStream- stream to read the mapping fromprops- The Properties to load into. As of 0.9.38, if null, a new OrderedProperties will be created.- Returns:
- the parameter props, or (as of 0.9.38) a new OrderedProperties if props is null, and an immutable EmptyProperties if empty.
- Throws:
DataFormatException- if the format is invalidIOException- if there is a problem reading the data
readProperties
public static Properties readProperties(InputStream rawStream, Properties props, boolean enforceOrder) throws DataFormatException, IOException Ditto, load into an existing propertiesAs of 0.9.18, throws DataFormatException on duplicate key.
- Parameters:
rawStream- stream to read the mapping fromprops- The Properties to load into. As of 0.9.38, if null, a new OrderedProperties will be created.enforceOrder- if true, throw DataFormatException if keys are not ordered- Returns:
- the parameter props, or (as of 0.9.38) a new OrderedProperties if props is null, and an immutable EmptyProperties if empty.
- Throws:
DataFormatException- if the format is invalidIOException- if there is a problem reading the data
writeProperties
public static void writeProperties(OutputStream rawStream, Properties props) throws DataFormatException, IOException Write a mapping to the stream, as defined by the I2P data structure spec, and store it into a Properties object. See readProperties for the format. Output is sorted by property name. Property keys and values must not contain '=' or ';', this is not checked and they are not escaped Keys and values must be 255 bytes or less, Formatted length must not exceed 65535 bytesProperties from the defaults table of props (if any) are not written out by this method.
- Parameters:
rawStream- stream to write toprops- properties to write out, may be null- Throws:
DataFormatException- if there is not enough valid data to write out, or a length limit is exceededIOException- if there is an IO error writing out the data
writeProperties
public static void writeProperties(OutputStream rawStream, Properties props, boolean utf8) throws DataFormatException, IOException Writes the props to the stream, sorted by property name. See readProperties() for the format. Property keys and values must not contain '=' or ';', this is not checked and they are not escaped Keys and values must be 255 bytes or less, Formatted length must not exceed 65535 bytesProperties from the defaults table of props (if any) are not written out by this method.
jrandom disabled UTF-8 in mid-2004, for performance reasons, i.e. slow foo.getBytes("UTF-8") Re-enable it so we can pass UTF-8 tunnel names through the I2CP SessionConfig.
Use utf8 = false for RouterAddress (fast, non UTF-8) Use utf8 = true for SessionConfig (slow, UTF-8).
- Parameters:
rawStream- stream to write toprops- source may be nullutf8- if true, encode strings as UTF-8- Throws:
DataFormatException- if a length limit is exceededIOException- if there is an IO error
writeProperties
public static void writeProperties(OutputStream rawStream, Properties props, boolean utf8, boolean sort) throws DataFormatException, IOException Writes the props to the stream, sorted by property name if sort == true or if props is an OrderedProperties. See readProperties() for the format. Property keys and values must not contain '=' or ';', this is not checked and they are not escaped Keys and values must be 255 bytes or less, Formatted length must not exceed 65535 bytesProperties from the defaults table of props (if any) are not written out by this method.
jrandom disabled UTF-8 in mid-2004, for performance reasons, i.e. slow foo.getBytes("UTF-8") Re-enable it so we can pass UTF-8 tunnel names through the I2CP SessionConfig.
Use utf8 = false for RouterAddress (fast, non UTF-8) Use utf8 = true for SessionConfig (slow, UTF-8).
- Parameters:
rawStream- stream to write toprops- source may be nullutf8- if true, encode strings as UTF-8sort- should we sort the properties? (set to false if already sorted, e.g. OrderedProperties)- Throws:
DataFormatException- if any string is over 255 bytes long, or if the total length (not including the two length bytes) is greater than 65535 bytes.IOException- if there is an IO error
toProperties
public static int toProperties(byte[] target, int offset, Properties props) throws DataFormatException, IOException Writes the props to the byte array, sorted See readProperties() for the format. Property keys and values must not contain '=' or ';', this is not checked and they are not escaped Keys and values must be 255 bytes or less, Formatted length must not exceed 65535 bytes Strings will be UTF-8 encoded in the byte array. Warning - confusing method name, Properties is the source.Properties from the defaults table of props (if any) are not written out by this method.
- Parameters:
target- returned array as specified in data structure specoffset- starting offset in targetprops- source may be null- Returns:
- the offset just past the last byte written, ready for the next field
- Throws:
DataFormatException- if any string is over 255 bytes long, or if the total length (not including the two length bytes) is greater than 65535 bytes.IOException- never thrown here; declared because the ByteArrayStream used to assemble the properties inheritsOutputStream.write(int)
fromProperties
public static int fromProperties(byte[] source, int offset, Properties target) throws DataFormatException Reads the props from the byte array and puts them in the Properties target See readProperties() for the format. Warning - confusing method name, Properties is the target. Strings must be UTF-8 encoded in the byte array.As of 0.9.18, throws DataFormatException on duplicate key.
- Parameters:
source- byte array holding the UTF-8 encoded props, in readProperties() formatoffset- starting offset in sourcetarget- thePropertiesto fill in; existing keys are read to detect duplicates- Returns:
- the offset just past the last byte consumed, ready for the next field
- Throws:
DataFormatException- if the data is malformed
toProperties
Writes the props to returned byte array, not sorted (unless the opts param is an OrderedProperties) Strings will be UTF-8 encoded in the byte array. See readProperties() for the format. Property keys and values must not contain '=' or ';', this is not checked and they are not escaped Keys and values must be 255 bytes or less, Formatted length must not exceed 65535 bytes Warning - confusing method name, Properties is the source.Properties from the defaults table of props (if any) are not written out by this method.
- Parameters:
opts- properties to serialize- Returns:
- byte array containing the serialized properties
- Throws:
DataFormatException- if key, value, or total is too long
toString
Pretty print the mapping, unsorted (unless the options param is an OrderedProperties).- Parameters:
options- properties to print, may be null- Returns:
- string representation
toString
loadProps
A more efficient Properties.loadSome of the other differences: - UTF-8 encoding, not ISO-8859-1 - No escaping! This does not process or drop backslashes - '#' or ';' starts a comment line, but '!' does not - Leading whitespace is not trimmed - '=' is the only key-termination character (not ':' or whitespace)
As of 0.9.10, an empty value is allowed.
As in Java Properties, duplicate keys are allowed, last one wins.
- Parameters:
props- the properties object to load intofile- the file to read from- Throws:
IOException- if there is an error reading
loadProps
public static void loadProps(Properties props, File file, boolean forceLowerCase) throws IOException Load properties from a file, optionally forcing keys to lower case.- Parameters:
props- the properties object to load intofile- the file to read fromforceLowerCase- if true forces the keys to lower case (not the values)- Throws:
IOException- if there is an error reading
loadProps
Load properties from a stream.- Parameters:
props- the properties object to load intoinStr- the input stream to read from- Throws:
IOException- if there is an error reading
loadProps
public static void loadProps(Properties props, InputStream inStr, boolean forceLowerCase) throws IOException Load properties from a stream, optionally forcing keys to lower case.- Parameters:
props- the properties object to load intoinStr- the input stream to read fromforceLowerCase- if true forces the keys to lower case (not the values)- Throws:
IOException- if there is an error reading
storeProps
Writes the props to the file, unsorted (unless props is an OrderedProperties) Note that this does not escape the \r or \n in values. As of 0.8.1, file will be mode 600 (best effort; group read/write is also granted where POSIX permissions are supported, e.g. 660 on Linux).Properties from the defaults table of props (if any) are not written out by this method.
Leading or trailing whitespace in values is not checked but will be trimmed by
loadProps(Properties, File).- Parameters:
props- properties to storefile- the file to write to- Throws:
IllegalArgumentException- if a key contains any of "#=; " or a CR or LF, or a value contains '#' or a CR or LFIOException- if there is an IO error
toString
Pretty print the collection.- Parameters:
col- the collection to print, may be null- Returns:
- string representation
toString
Lower-case hex with leading zeros. Use toHexString(byte[]) to not get leading zeros.- Parameters:
buf- may be null (returns "")- Returns:
- String of length 2*buf.length
toString
Lower-case hex with leading zeros. Use toHexString(byte[]) to not get leading zeros.- Parameters:
buf- may be nulllen- number of bytes. If greater than buf.length, additional zeros will be prepended- Returns:
- String of length 2*len
toHexString
Lower-case hex without leading zeros. Use toString(byte[]) to get leading zeros.- Parameters:
data- may be null (returns "00")- Returns:
- hex string representation
fromHexString
Decode a hex string to a byte array. UsetoString(byte[])for the reverse operation.- Parameters:
val- non-null, may have leading minus sign- Returns:
- minimum-length representation (with possible leading 0 byte)
readLong
public static long readLong(InputStream rawStream, int numBytes) throws DataFormatException, IOException Read the stream for an integer as defined by the I2P data structure specification. Integers are a fixed number of bytes (numBytes), stored as unsigned integers in network byte order.- Parameters:
rawStream- stream to read fromnumBytes- number of bytes to read and format into a number, 1 to 8- Returns:
- the value, interpreted as an unsigned big-endian integer
- Throws:
DataFormatException- if negative (only possible if numBytes = 8) (since 0.8.12)EOFException- since 0.8.2, if there aren't enough bytes to read the numberIOException- if there is an IO error reading the number
writeLong
public static void writeLong(OutputStream rawStream, int numBytes, long value) throws DataFormatException, IOException Write an integer as defined by the I2P data structure specification to the stream. Integers are a fixed number of bytes (numBytes), stored as unsigned integers in network byte order.- Parameters:
rawStream- stream to write tonumBytes- number of bytes to write the number into, 1-8 (padding as necessary)value- value to write out, non-negative- Throws:
DataFormatException- if value is negative or if numBytes not 1-8IOException- if there is an IO error writing to the stream
toLong
Big endian.- Parameters:
numBytes- 1-8value- non-negative- Returns:
- an array of length numBytes
- Throws:
IllegalArgumentException- if numBytes is out of range or value is negative
toLong
public static void toLong(byte[] target, int offset, int numBytes, long value) throws IllegalArgumentException Big endian.- Parameters:
target- byte array the numBytes long value is written intooffset- starting offset in targetnumBytes- 1-8value- non-negative- Throws:
IllegalArgumentException- if numBytes is out of range or value is negative
toLongLE
public static void toLongLE(byte[] target, int offset, int numBytes, long value) Little endian, i.e. backwards. Not for use in I2P protocols.- Parameters:
target- byte array the numBytes long value is written into, backwardsoffset- starting offset in targetnumBytes- 1-8value- non-negative
fromLong
public static long fromLong(byte[] src, int offset, int numBytes) Big endian.- Parameters:
src- source byte array, may be null (returns 0)offset- starting offset in srcnumBytes- 1-8- Returns:
- non-negative
- Throws:
ArrayIndexOutOfBoundsException- if the source array is too shortIllegalArgumentException- if negative (only possible if numBytes = 8)
fromLongLE
public static long fromLongLE(byte[] src, int offset, int numBytes) Little endian, i.e. backwards. Not for use in I2P protocols.- Parameters:
src- source byte array, may be null (returns 0)offset- starting offset in srcnumBytes- 1-8- Returns:
- non-negative
- Throws:
ArrayIndexOutOfBoundsException- if the source array is too shortIllegalArgumentException- if negative (only possible if numBytes = 8)
fromLong8
public static long fromLong8(byte[] src, int offset) Big endian. Same as fromLong(src, offset, 8) but allows negative result.- Parameters:
src- source byte arrayoffset- starting offset in src- Returns:
- the long value, possibly negative
- Throws:
ArrayIndexOutOfBoundsException- if the source array is too short
toLong8
public static void toLong8(byte[] target, int offset, long value) Big endian. Same as toLong(target, offset, 8, value) but allows negative value.- Parameters:
target- byte array the 8 byte long value is written intooffset- starting offset in targetvalue- may be negative, unlike the toLong variants- Throws:
ArrayIndexOutOfBoundsException- if the target array is too short
readDate
Read in a date from the stream as specified by the I2P data structure spec. A date is an 8 byte unsigned integer in network byte order specifying the number of milliseconds since midnight on January 1, 1970 in the GMT timezone. If the number is 0, the date is undefined or null. (yes, this means you can't represent midnight on 1/1/1970).- Parameters:
in- stream to read from- Returns:
- date read, or null
- Throws:
DataFormatException- if the stream doesn't contain a validly formatted dateIOException- if there is an IO error reading the date
writeDate
Write out a date to the stream as specified by the I2P data structure spec.- Parameters:
out- stream to write todate- date to write (can be null)- Throws:
DataFormatException- if the date is not validIOException- if there is an IO error writing the date
toDate
Prefer toDate(byte[], int, long).- Parameters:
date- may be null (returns zero date)- Returns:
- a newly-allocated byte array of length DATE_LENGTH
- Throws:
IllegalArgumentException- if the time does not fit in DATE_LENGTH bytes
toDate
Write a date (8-byte timestamp) to a byte array.- Parameters:
target- byte array the 8 byte timestamp is written intooffset- the starting offsetwhen- the time in milliseconds since epoch- Throws:
IllegalArgumentException- if the offset is invalid
fromDate
Read a date (8-byte timestamp) from a byte array.- Parameters:
src- the source byte arrayoffset- the starting offset- Returns:
- the date, or null if the time is zero or negative
- Throws:
DataFormatException- if there is not enough data
readString
Read in a string from the stream as specified by the I2P data structure spec. A string is 1 or more bytes where the first byte is the number of bytes (not characters!) in the string and the remaining 0-255 bytes are the non-null terminated UTF-8 encoded character array.- Parameters:
in- stream to read from- Returns:
- UTF-8 string
- Throws:
EOFException- since 0.8.2, if there aren't enough bytes to read the stringIOException- if there is an IO error reading the string
writeString
public static void writeString(OutputStream out, String string) throws DataFormatException, IOException Write out a string to the stream as specified by the I2P data structure spec. Note that the max size for a string allowed by the spec is 255 bytes.WARNING - this method destroys the encoding, and therefore violates the data structure spec.
- Parameters:
out- stream to write stringstring- string to write out: null strings are perfectly valid, but strings of excess length will cause a DataFormatException to be thrown- Throws:
DataFormatException- if the string is not validIOException- if there is an IO error writing the string
writeStringUTF8
public static void writeStringUTF8(OutputStream out, String string) throws DataFormatException, IOException Write out a string to the stream as specified by the I2P data structure spec. Note that the max size for a string allowed by the spec is 255 bytes.This method correctly uses UTF-8.
- Parameters:
out- stream to write stringstring- UTF-8 string to write out: null strings are perfectly valid, but strings of excess length will cause a DataFormatException to be thrown- Throws:
DataFormatException- if the string is not validIOException- if there is an IO error writing the string
eq
eq
Run a deep comparison across the two collections.This treats (null == null) as true, (null == (!null)) as false, and then comparing each element via eq(object, object).
If the size of the collections are not equal, the comparison returns false. The collection order should be consistent, as this simply iterates across both and compares based on the value of each at each step along the way.
- Parameters:
lhs- first collection, may be nullrhs- second collection, may be null- Returns:
- true if equal
eq
public static final boolean eq(byte[] lhs, byte[] rhs) Run a comparison on the byte arrays, byte by byte.This treats (null == null) as true, (null == (!null)) as false, and unequal length arrays as false.
Variable time.
- Parameters:
lhs- first byte array, may be nullrhs- second byte array, may be null- Returns:
- true if the two arrays hold the same bytes, treating two nulls as equal
eq
public static final boolean eq(byte[] lhs, int offsetLeft, byte[] rhs, int offsetRight, int length) Unlike eq(byte[], byte[]), this returns false if either lhs or rhs is null. Variable time.- Parameters:
lhs- first byte arrayoffsetLeft- starting offset in lhsrhs- second byte arrayoffsetRight- starting offset in rhslength- number of bytes to compare- Returns:
- true if equal
- Throws:
ArrayIndexOutOfBoundsException- if either array isn't long enough
eqCT
public static final boolean eqCT(byte[] lhs, int offsetLeft, byte[] rhs, int offsetRight, int length) Unlike eq(), this throws NPE if either lhs or rhs is null. Constant time.- Parameters:
lhs- first byte arrayoffsetLeft- starting offset in lhsrhs- second byte arrayoffsetRight- starting offset in rhslength- number of bytes to compare- Returns:
- true if equal
- Throws:
NullPointerException- if lhs or rhs is nullArrayIndexOutOfBoundsException- if either array isn't long enough
copyOfRange
Copy a range of a byte array into a new array, mirroringArrays.copyOfRange(byte[], int, int)but throwing DataFormatException on invalid bounds so callers in the data layer can handle it with their existing exception flow.- Parameters:
src- source array, must not be nullfrom- starting index, inclusiveto- ending index, exclusive; must be >= from and <= src.length- Returns:
- new byte[to - from] containing src[from..to-1]
- Throws:
DataFormatException- if src is null or the range is invalid
eqCT
compareTo
public static final int compareTo(byte[] lhs, byte[] rhs) Big endian compare, treats bytes as unsigned. Shorter arg is lesser. Args may be null, null is less than non-null. Variable time.- Parameters:
lhs- first byte array, may be nullrhs- second byte array, may be null- Returns:
- negative, zero, or positive
xor
public static final byte[] xor(byte[] lhs, byte[] rhs) XOR two byte arrays.- Parameters:
lhs- first byte array, may be nullrhs- second byte array, may be null- Returns:
- an empty array if either arg is null or the args are not equal length
xor
public static final void xor(byte[] lhs, int startLeft, byte[] rhs, int startRight, byte[] out, int startOut, int len) Xor the lhs with the rhs, storing the result in out.- Parameters:
lhs- one of the source arraysstartLeft- starting index in the lhs array to begin the xorrhs- the other source arraystartRight- starting index in the rhs array to begin the xorout- output arraystartOut- starting index in the out array to store the resultlen- how many bytes into the various arrays to xor
hashCode
Calculate the hashcode of the object, using 0 for null.- Parameters:
obj- the object, may be null- Returns:
- the value of
Object.hashCode(), or 0 for null
hashCode
Calculate the hashcode of the date, using 0 for null.- Parameters:
obj- the date, may be null- Returns:
- the low 32 bits of the time in milliseconds, or 0 for null
hashCode
public static int hashCode(byte[] b) Calculate the hashcode of the byte array, using 0 for null.- Parameters:
b- byte array, may be null- Returns:
- a hash of the contents, using only the first 32 bytes for longer arrays
hashCode
Calculate the hashcode of the collection, using 0 for null.- Parameters:
col- the collection, may be null- Returns:
- an order-dependent hash of the elements, or 0 for null
skip
This is different thanInputStream.skip(long), in that it does repeated reads until the full amount is skipped. To fix findbugs issues withskip(InputStream, long).Guaranteed to skip exactly n bytes or throw an IOE.
http://stackoverflow.com/questions/14057720/robust-skipping-of-data-in-a-java-io-inputstream-and-its-subtypes http://stackoverflow.com/questions/11511093/java-inputstream-skip-return-value-near-end-of-file
- Parameters:
in- the input streamn- the number of bytes to skip- Throws:
IOException- if an I/O error occurs or EOF is reached before all bytes are skippedIllegalArgumentException- if n is negative
read
This is different thanread(target), in that it does repeated reads until the full data is received.As of 0.9.27, throws EOFException if the full length is not read.
- Parameters:
in- the input streamtarget- byte array filled with target.length bytes- Returns:
- the number of bytes read, which is always the full
target.length - Throws:
EOFException- if the full length is not read (since 0.9.27)IOException- if an I/O error occurs
read
WARNING - This is different thanread(target, offset, length)for a nonzero offset, in that it returns the new offset (== old offset + length). It also does repeated reads until the full data is received.WARNING - Broken for nonzero offset before 0.9.27. As of 0.9.27, throws EOFException if the full length is not read.
- Parameters:
in- the input streamtarget- byte array the length bytes are read intooffset- starting offset in targetlength- number of bytes to read- Returns:
- the new offset (== old offset + length)
- Throws:
EOFException- if the full length is not read (since 0.9.27)IOException- if an I/O error occurs
readLine
Read a newline delimited line from the stream, returning the line without the newline, or null if EOF is reached on an empty line. Warning - strips \n but not \r. Warning - 8KB line length limit as of 0.7.13, throws IOException if exceeded. Warning - not UTF-8.- Parameters:
in- the input stream- Returns:
- null on EOF
- Throws:
IOException- if there is an IO error
readLine
Update the hash along the way Warning - strips \n but not \r Warning - 8KB line length limit as of 0.7.13, @throws IOException if exceeded Warning - not UTF-8.- Parameters:
in- the input streamhash- null OK- Returns:
- null on EOF
- Throws:
IOException- if there is an IO error
readLine
Read in a line, placing it into the buffer (excluding the newline). Warning - strips \n but not \r Warning - 8KB line length limit as of 0.7.13, @throws IOException if exceeded Warning - not UTF-8.- Parameters:
in- the input streambuf- the buffer to append to- Returns:
- true if the line was read, false if eof was reached on an empty line (returns true for non-empty last line without a newline)
- Throws:
IOException- if there is an IO error
readLine
public static boolean readLine(InputStream in, StringBuilder buf, MessageDigest hash) throws IOException Update the hash along the way Warning - strips \n but not \r Warning - 8KB line length limit as of 0.7.13, @throws IOException if exceeded Warning - not UTF-8.- Parameters:
in- the input streambuf- the buffer to append tohash- null OK- Returns:
- true if the line was read, false if eof was reached on an empty line (returns true for non-empty last line without a newline)
- Throws:
IOException- if there is an IO error
write
Write data to the output stream while updating the MessageDigest along the way.- Parameters:
out- the output stream to write todata- the byte array to writehash- the MessageDigest to update (non-null)- Throws:
IOException- if an I/O error occurs
formatDuration
Format a duration in milliseconds to a human-readable string. NOTE:formatDuration2(long)recommended in most cases for readability.- Parameters:
ms- the duration in milliseconds- Returns:
- a compact string like "5s", "30m", "2h", "3d"
formatDuration2
Like formatDuration but with a non-breaking space after the number, 0 is unitless, and the unit is translated. This seems consistent with most style guides out there. Use only in HTML. Thresholds are a little lower than informatDuration(long)also, as precision is less important in the GUI than in logging.Negative numbers handled correctly.
- Parameters:
ms- the duration in milliseconds- Returns:
- a translated string like "5 sec", "30 min", "2 hours"
formatDuration2
Like formatDuration2(long) but with microsec and nanosec also.- Parameters:
ms- the duration in milliseconds (fractional values supported)- Returns:
- a translated string with appropriate units (ns, μs, ms, sec, min, etc.)
_t
ngettext
formatSize
This is binary, i.e. multiples of 1024. For decimal, seeformatSize2Decimal(long).Caller should append 'B' or 'b' as appropriate.
No space between the number and the letter. NOTE:
formatSize2(long)recommended in most cases for readability.- Parameters:
bytes- the number of bytes- Returns:
- e.g. "123.05Ki"
formatSize2
This is binary, i.e. multiples of 1024. For decimal, see formatSize2Decimal().Caller should append 'B' or 'b' as appropriate. Like formatSize but with a non-breaking space after the number This seems consistent with most style guides out there. Use only in HTML, and not inside form values (use formatSize2(bytes, false) there instead).
- Parameters:
bytes- the number of bytes- Returns:
- e.g. "123.05 Ki"
formatSize2
This is binary, i.e. multiples of 1024. For decimal, see formatSize2Decimal().Caller should append 'B' or 'b' as appropriate, Like formatSize but with a space after the number This seems consistent with most style guides out there.
- Parameters:
bytes- the number of bytesnonBreaking- use an HTML thin non-breaking space ( )- Returns:
- e.g. "123.05 Ki" or "123.05 Ki"
formatSize2Decimal
This is decimal, i.e. multiples of 1000. For binary, see formatSize2().Caller should append 'B' or 'b' as appropriate. Like formatSize but with a space after the number This seems consistent with most style guides out there.
- Parameters:
bytes- the number of bytes- Returns:
- e.g. "123.05 K"
formatSize2Decimal
This is decimal, i.e. multiples of 1000. For binary, see formatSize2().Caller should append 'B' or 'b' as appropriate. Like formatSize but with a space after the number This seems consistent with most style guides out there.
- Parameters:
bytes- the number of bytesnonBreaking- use an HTML thin non-breaking space ( )- Returns:
- e.g. "123.05 K" or "123.05 K"
formatDate
The default formatting for date, current locale, local time zone. Warning - NOT UTC! Examples: en: Aug 30, 2019 de: 30.08.2019.- Parameters:
now- the timestamp in milliseconds since epoch- Returns:
- the formatted date string
formatTime
The default formatting for date/time, current locale, local time zone. Warning - NOT UTC! Examples: en: Aug 30, 2019 12:38 PM de: 30.08.2019 12:38.- Parameters:
now- the timestamp in milliseconds since epoch- Returns:
- the formatted date/time string
stripHTML
escapeHTML
compress
public static byte[] compress(byte[] orig) Compress the data and return a new GZIP compressed byte array. The compressed data conforms to RFC 1952, with a 10-byte gzip header and a 8-byte gzip checksum footer.Prior to 0.9.29, this would return a zero-length output for a zero-length input. As of 0.9.29, output is valid for a zero-length input also.
As of 0.9.47, this uses a level of MEDIUM_COMPRESSION, which is a good space/speed tradeoff. Prior to that, it used MAX_COMPRESSION.
- Parameters:
orig- the data to compress- Returns:
- null if orig is null
- Throws:
IllegalArgumentException- if input size is over 40KBIllegalStateException- on compression failure, as of 0.9.29
compress
public static byte[] compress(byte[] orig, int offset, int size) Compress the data and return a new GZIP compressed byte array. The compressed data conforms to RFC 1952, with a 10-byte gzip header and a 8-byte gzip checksum footer.Prior to 0.9.29, this would return a zero-length output for a zero-length input. As of 0.9.29, output is valid for a zero-length input also.
As of 0.9.47, this uses a level of MEDIUM_COMPRESSION, which is a good space/speed tradeoff. Prior to that, it used MAX_COMPRESSION.
- Parameters:
orig- the data to compressoffset- the offset into the datasize- the number of bytes to compress- Returns:
- null if orig is null
- Throws:
IllegalArgumentException- if size is over 40KBIllegalStateException- on compression failure, as of 0.9.29
compress
public static byte[] compress(byte[] orig, int offset, int size, int level) Compress the data and return a new GZIP compressed byte array. The compressed data conforms to RFC 1952, with a 10-byte gzip header and a 8-byte gzip checksum footer.Prior to 0.9.29, this would return a zero-length output for a zero-length input. As of 0.9.29, output is valid for a zero-length input also.
- Parameters:
orig- the data to compressoffset- the offset into orig at which to startsize- the number of bytes to compresslevel- the compression level, 0 to 9- Returns:
- null if orig is null
- Throws:
IllegalArgumentException- if size is over 40KBIllegalStateException- on compression failure, as of 0.9.29
zeroCompress
private static byte[] zeroCompress(byte[] in, int off, int len) Fast NO_COMPRESSION. 135x faster for 23 bytes, 15x faster for 1752 bytes.Ref: RFC 1951, RFC 1952, ResettableGzipOutputStream.
- Parameters:
in- the data to wrap in a stored gzip streamoff- the offset in in where the data beginslen- 32767 max- Returns:
- 23 bytes longer
decompress
Decompress the GZIP compressed data (returning null on error).- Parameters:
orig- the GZIP compressed data- Returns:
- null if orig is null
- Throws:
IOException- if uncompressed is over 40 KB, or on a decompression error
decompress
Decompress the GZIP compressed data (returning null on error).- Parameters:
orig- the GZIP compressed dataoffset- the offset into orig at which the compressed data startslength- the number of compressed bytes to decompress- Returns:
- null if orig is null
- Throws:
IOException- if uncompressed is over 40 KB, or on a decompression error
zeroDecompress
Fast NO_COMPRESSION. ~30x faster for typ. lengths, ~20x faster for 4 KB or more.Ref: RFC 1951, RFC 1952, ResettableGzipOutputStream.
- Parameters:
in- in[off + 10] MUST BE VALIDATED == 1 before calling this.off- the offset in in where the gzip stream beginslen- 65558 max- Returns:
- 23 bytes shorter
- Throws:
IOException- on all errors
getUTF8
Same as orig.getBytes("UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use.- Parameters:
orig- the string to encode as UTF-8- Returns:
- the UTF-8 bytes, an empty array if orig is null
- Throws:
RuntimeException- never thrown - StandardCharsets.UTF_8 is always available
getUTF8
Same as new String(orig, "UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use. Used by Syndie.- Parameters:
orig- the UTF-8 bytes to decode into a String- Returns:
- null if orig is null
- Throws:
RuntimeException- never thrown - StandardCharsets.UTF_8 is always available
getUTF8
Same as new String(orig, "UTF-8") but throws an unchecked RuntimeException instead of an UnsupportedEncodingException if no UTF-8, for ease of use.- Parameters:
orig- the UTF-8 bytes to decode into a Stringoffset- the offset into orig at which the encoded string startslen- the number of bytes to decode- Returns:
- null if orig is null
- Throws:
RuntimeException- never thrown - StandardCharsets.UTF_8 is always available
getASCII
Roughly the same as orig.getBytes("ISO-8859-1") but much faster and will not throw an exception.Warning - misnamed, converts to ISO-8859-1.
- Parameters:
orig- non-null, truncates to 8-bit chars- Returns:
- the ISO-8859-1 bytes of orig
split
Same as s.split(regex) but caches the compiled pattern for speed. This saves about 10 microseconds (Bulldozer) on subsequent invocations.Note: For an input "" this returns [""], not a zero-length array. This is the same behavior as
String.split(String, int).- Parameters:
s- non-nullregex- non-null, don't forget to enclose multiple choices with []- Returns:
- the array of strings
- Throws:
PatternSyntaxException- unchecked, if the regex is not a valid pattern
split
Same as s.split(regex, limit) but caches the compiled pattern for speed. This saves about 10 microseconds (Bulldozer) on subsequent invocations.Note: For an input "" this returns [""], not a zero-length array. This is the same behavior as
String.split(String, int).- Parameters:
s- non-nullregex- non-null, don't forget to enclose multiple choices with []limit- passed through toString.split(): a positive value caps the number of results, zero leaves the result uncapped, and a negative value keeps trailing empty strings- Returns:
- the array of strings
- Throws:
PatternSyntaxException- unchecked, if the regex is not a valid pattern
copy
Copy in to out. Caller MUST close the streams.- Parameters:
in- non-nullout- non-null- Throws:
IOException- if in cannot be read or out cannot be written
sort
Same asCollections.sort(List), but guaranteed not to throw an IllegalArgumentException if the sort is unstable. As of Java 7, TimSort will throw an IAE if the underlying sort order changes during the sort.This catches the IAE, retries once, and then returns. If an IAE is thrown twice, this method will return, with the list possibly unsorted.
- Type Parameters:
T- the element type of the list and of the comparator- Parameters:
list- the list to be sorted.c- the comparator to determine the order of the list. A null value indicates that the elements' natural ordering should be used.
sort
Same asArrays.sort(int[]), but guaranteed not to throw an IllegalArgumentException if the sort is unstable. As of Java 7, TimSort will throw an IAE if the underlying sort order changes during the sort.This catches the IAE, retries once, and then returns. If an IAE is thrown twice, this method will return, with the array possibly unsorted.
- Type Parameters:
T- the element type of the array and of the comparator- Parameters:
a- the array to be sorted.c- the comparator to determine the order of the array. A null value indicates that the elements' natural ordering should be used.
messageOf
The throwable's message, never null.Throwable.getMessage()returns null routinely, and not only for exotic exceptions: a stream that dies mid-read produces anIOExceptionwith no message at all. Four sites in the HTTP server and client tunnels calledgetMessage().indexOf(...)on it anyway, and each threw aNullPointerExceptionfrom inside its own catch block - which replaced the diagnostic it was trying to log with a stack trace, and left the original failure unexplained.Falls back to the throwable's class name, because "what failed" is recoverable from the type even when there is no message, whereas the bare string "null" would make two different failures read identically in a log.
- Parameters:
t- the throwable, which may be null- Returns:
- the message, or the simple class name when there is none, or the empty string when the throwable itself is null
replace
Replace all instances of "from" with "to" in the StringBuilder buf. Same asString.replace(char, char), but in-memory with no object churn, as long as "to" is equal size or smaller than "from", or buf has capacity. Use for large Strings or for multiple replacements in a row.- Parameters:
buf- contains the string to be searchedfrom- the string to be replacedto- the replacement string