SSLEepGet

java.lang.Object

net.i2p.util.EepGet

net.i2p.util.SSLEepGet

public class SSLEepGetextends EepGet

HTTPS only, no retries, no min and max size options, no timeout option As of 0.9.45, supports redirect to https (redirect to http will still fail). Fails on bad certs (must have a valid cert chain) Self-signed certs or CAs not in the JVM key store must be loaded to be trusted.

Since 0.8.2, loads additional trusted CA certs from $I2P/certificates/ssl/ and ~/.i2p/certificates/ssl/

Since 0.9.33, HTTP proxies (CONNECT) supported. Proxy auth not supported.

Nested Class Summary
Nested Classes
Modifier and Type
Class
Description
static enum
SSLEepGet.ProxyType
Not all may be supported.
static class
SSLEepGet.SSLState
An opaque class for the caller to pass to repeated instantiations of SSLEepGet.
Nested classes/interfaces inherited from class EepGet
EepGet.AUTH_MODE, EepGet.AuthState, EepGet.CLIStatusListener, EepGet.ContentRange, EepGet.Gunzipper, EepGet.StatusListener
Modifier and Type
Class
Description
protected static enum
EepGet.AUTH_MODE
Authentication mode enumeration.
protected class
EepGet.AuthState
Manage the authentication parameters Ref: RFC 2617 Supports both Basic and Digest, however i2ptunnel HTTP proxy has migrated all previous Basic support to Digest.
protected class
EepGet.CLIStatusListener
Reports fetch progress to the command line, one mark per markSize bytes and a summary line every lineSize marks.
(package private) static final class
EepGet.ContentRange
Parsed Content-Range header value.
protected class
EepGet.Gunzipper
Decompressor thread.
static interface
EepGet.StatusListener
Callback interface for monitoring EepGet transfer progress.
Field Summary
Fields inherited from class EepGet
_aborted, _acceptRanges, _actualURL, _alreadyTransferred, _authState, _bytesRemaining, _bytesTransferred, _cacheControl, _contentEncoding, _contentLanguage, _contentRange, _contentType, _context, _cookie, _csp, _currentAttempt, _decompressException, _encodingChunked, _etag, _etagOrig, _expiryDate, _extraHeaders, _fetchHeaderTimeout, _fetchInactivityTimeout, _fetchTotalTimeout, _isGzippedResponse, _keepFetching, _lastModified, _lastModifiedOrig, _listeners, _log, _maxCompleteFails, _notModified, _numRetries, _out, _outputFile, _outputStream, _proxy, _proxyHost, _proxyIn, _proxyOut, _proxyPort, _redirectLocation, _redirects, _referrerPolicy, _responseCode, _responseText, _retryDelayMs, _server, _shouldProxy, _shouldWriteErrorToOutput, _status, _transferEncoding, _transferFailed, _url, _vary, _xContentTypeOptions, _xframeOptions, _xPoweredBy, _xssProtection, DEFAULT_CONNECT_TIMEOUT, DEFAULT_INACTIVITY_TIMEOUT, DEFAULT_MAX_COMPLETE_FAILS, DEFAULT_NUM_RETRIES, DEFAULT_RETRY_DELAY, DEFAULT_RETRY_JITTER, GZIP_PARTIAL_MARKER_SUFFIX, PROP_CONNECT_TIMEOUT, PROP_DEFAULT_RETRIES, PROP_INACTIVITY_TIMEOUT, PROP_MAX_COMPLETE_FAILS, PROP_RETRY_DELAY, USER_AGENT
Modifier and Type
Field
Description
protected boolean
_aborted
Whether the fetch was aborted due to timeout or user request.
protected String
_acceptRanges
Accept-Ranges header value from the response.
protected String
_actualURL
The URL we actually fetch from (may differ from _url in case of redirect).
protected long
_alreadyTransferred
Total bytes transferred across all attempts, including resumed content.
protected EepGet.AuthState
_authState
Authentication state for proxy digest/basic auth; created via addAuthorization().
protected long
_bytesRemaining
Remaining bytes on the current attempt, -1 if unknown (chunked or no Content-Length).
protected long
_bytesTransferred
Body bytes transferred in the current fetch(), summed across attempts.
protected String
_cacheControl
Cache-Control header value from the response.
protected String
_contentEncoding
Content-Encoding header value from the response.
protected String
_contentLanguage
Content-Language header value from the response.
protected String
_contentRange
Content-Range header value from the response, for validating 206/416 resumes.
protected String
_contentType
Content-Type header value from the response.
protected final I2PAppContext
_context
I2P application context for configuration, logging, and random number generation.
protected String
_cookie
Set-Cookie header value from the response.
protected String
_csp
Content-Security-Policy header value from the response.
protected int
_currentAttempt
Zero-based attempt counter for the current fetch; read by the watchdog command.
protected IOException
_decompressException
IOException from the background decompressor thread, if any.
protected boolean
_encodingChunked
Whether the response is using chunked transfer encoding.
protected String
_etag
ETag received from the server (for conditional requests).
protected final String
_etagOrig
Original ETag passed in constructor, preserved across redirects.
protected String
_expiryDate
Expires header value from the response.
protected List<String>
_extraHeaders
Additional HTTP headers to include in the request.
protected int
_fetchHeaderTimeout
Timeout in ms for receiving the HTTP response headers.
protected int
_fetchInactivityTimeout
Inactivity timeout in ms between data packets.
protected int
_fetchTotalTimeout
Total timeout in ms for the entire fetch operation.
protected boolean
_isGzippedResponse
Whether the response body is gzip-compressed.
protected boolean
_keepFetching
Whether to continue fetching; set false to abort.
protected String
_lastModified
Last-Modified header received from the server.
protected final String
_lastModifiedOrig
Original Last-Modified passed in constructor, preserved across redirects.
protected final List<EepGet.StatusListener>
_listeners
Registered status listeners for progress callbacks.
protected final Log
_log
Logger for this instance.
protected int
_maxCompleteFails
Maximum number of complete (zero-byte) failures before giving up.
protected boolean
_notModified
Whether the server returned 304 Not Modified.
protected final int
_numRetries
Maximum number of retry attempts per fetch.
protected OutputStream
_out
Output stream for writing fetched data (file or pipe).
protected final String
_outputFile
Path to the output file for saving the fetched content.
protected final OutputStream
_outputStream
Output stream destination, overrides outputFile when non-null.
protected Socket
_proxy
The socket connection to the proxy or the target server (if not proxied).
protected final String
_proxyHost
Proxy hostname or IP address.
protected InputStream
_proxyIn
Input stream from the proxy/server socket.
protected OutputStream
_proxyOut
Output stream to the proxy/server socket.
protected final int
_proxyPort
Proxy port number.
protected String
_redirectLocation
Location header value from redirect or 407 proxy auth response.
protected AtomicInteger
_redirects
Counter tracking the number of HTTP redirects followed.
protected String
_referrerPolicy
Referrer-Policy header value from the response.
protected int
_responseCode
HTTP response status code, -1 if no response received.
protected String
_responseText
HTTP response status text (e.g.
protected final long
_retryDelayMs
Fixed retry delay in ms, -1 selects the jittered default @since 0.9.71+.
protected String
_server
Server header value from the response.
protected final boolean
_shouldProxy
Whether to route requests through an HTTP proxy.
protected boolean
_shouldWriteErrorToOutput
Whether to write error response bodies to the output.
protected String
_status
Status header value from the response.
protected String
_transferEncoding
Transfer-Encoding header value from the response.
protected boolean
_transferFailed
Whether the current transfer attempt failed (non-2xx response code).
protected final String
_url
The URL we were asked to fetch.
protected String
_vary
Vary header value from the response.
protected String
_xContentTypeOptions
X-Content-Type-Options header value from the response.
protected String
_xframeOptions
X-Frame-Options header value from the response.
protected String
_xPoweredBy
X-Powered-By header value from the response.
protected String
_xssProtection
X-XSS-Protection header value from the response.
protected static final int
DEFAULT_CONNECT_TIMEOUT
Default connect timeout in milliseconds (90 seconds).
protected static final int
DEFAULT_INACTIVITY_TIMEOUT
Default inactivity timeout in milliseconds (5 minutes).
protected static final int
DEFAULT_MAX_COMPLETE_FAILS
Maximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14.
protected static final int
DEFAULT_NUM_RETRIES
Default number of retry attempts.
protected static final int
DEFAULT_RETRY_DELAY
Base retry delay in milliseconds when the jittered default is used @since 0.9.71+.
protected static final int
DEFAULT_RETRY_JITTER
Maximum additional jitter in milliseconds added to the base retry delay @since 0.9.71+.
(package private) static final String
GZIP_PARTIAL_MARKER_SUFFIX
Suffix of the sidecar marker file recording that the output file holds transparently-decompressed (gunzipped) bytes rather than raw response bytes, so its length must not be used as a Range resume offset.
(package private) static final String
PROP_CONNECT_TIMEOUT
Property name for connect timeout configuration.
(package private) static final String
PROP_DEFAULT_RETRIES
Property name for default retries configuration.
(package private) static final String
PROP_INACTIVITY_TIMEOUT
Property name for inactivity timeout configuration.
(package private) static final String
PROP_MAX_COMPLETE_FAILS
Property name for max complete fails configuration.
(package private) static final String
PROP_RETRY_DELAY
Property name for the fixed retry delay in milliseconds.
protected static final String
USER_AGENT
Default User-Agent header value.
Constructor Summary
Constructors
Constructor
Description
SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url)
A new SSLEepGet with a new SSLState.
SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url, long maxSize, SSLEepGet.SSLState state)
Create an SSLEepGet that writes to a stream, using an existing SSL state, limited to a maximum response size.
SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url, SSLEepGet.SSLState state)
Create an SSLEepGet that writes to a stream, using an existing SSL state.
SSLEepGet(I2PAppContext ctx, String outputFile, String url)
A new SSLEepGet with a new SSLState.
SSLEepGet(I2PAppContext ctx, String outputFile, String url, SSLEepGet.SSLState state)
Create an SSLEepGet that writes to a file, using an existing SSL state.
SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, OutputStream outputStream, String url)
Use a proxy.
SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, OutputStream outputStream, String url, SSLEepGet.SSLState state)
Use a proxy.
SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, String outputFile, String url)
Use a proxy.
SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, String outputFile, String url, SSLEepGet.SSLState state)
Use a proxy.
Method Summary
All Methods
Modifier and Type
Method
Description
protected void
doFetch(SocketTimeout timeout)
Fetch the resource over SSL.
void
forceDNSOverHTTPS(boolean on)
Override the config setting, force DNSoverHTTPS on or off Call before the fetch.
void
forceDNSOverHTTPS(boolean on, boolean forceIPv6)
Override the config setting, force DNSoverHTTPS on or off Call before the fetch.
SSLEepGet.SSLState
getSSLState()
Pass this back to the next SSLEepGet constructor for faster fetches.
protected void
sendRequest(SocketTimeout timeout)
Send the HTTP request over SSL.
Methods inherited from class EepGet
addAuthorization, addHeader, addStatusListener, fatalAbort, fetch, fetch, fetch, getAcceptRanges, getCacheControl, getContentEncoding, getContentLanguage, getContentType, getCookie, getCSP, getEtag, getExpiryDate, getLastModified, getNotModified, getReferrerPolicy, getRequest, getResumeOffset, getServer, getStatus, getStatusCode, getStatusText, getTransferEncoding, getVary, getXContentTypeOptions, getXframeOptions, getXPoweredBy, getXSSProtection, gzipMarkerExists, isStalledButProgressing, isTotalBudgetExhausted, main, markGzipPartial, parseAuthArgs, parseChunkLength, parseContentRange, readChunkLength, readHeaders, redactedUrl, requestedRangeStart, setPostData, setPostData, setPostData, setWriteErrorToOutput, shouldReadBody, stopFetching, suggestName, usageText, validateResumeResponse
Modifier and Type
Method
Description
void
addAuthorization(String userName, String password)
Add basic authorization header for the proxy.
void
addHeader(String name, String value)
Add an extra header to the request.
void
addStatusListener(EepGet.StatusListener lsnr)
Add a status listener.
(package private) static boolean
fatalAbort(boolean aborted, boolean strictSize, long remaining, boolean sawEof)
Decide whether a watchdog abort observed after the body loop must fail the attempt.
boolean
fetch()
Blocking fetch, returning true if the URL was retrieved, false if all retries failed.
boolean
fetch(long fetchHeaderTimeout)
Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given.
boolean
fetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout)
Blocking fetch.
String
getAcceptRanges()
Show the Accept-Ranges field.
String
getCacheControl()
Show the Cache-Control field.
String
getContentEncoding()
Show the Content-Encoding field.
String
getContentLanguage()
Show the Content-Language field.
String
getContentType()
After fetch, the received value from the server, or null if none.
String
getCookie()
Show the Set-Cookie field.
String
getCSP()
Content-Security-Policy header.
String
getEtag()
After fetch, the received value from the server, or null if none.
String
getExpiryDate()
Show the Expires field.
String
getLastModified()
After fetch, the received value from the server, or null if none.
boolean
getNotModified()
Whether the server returned 304.
String
getReferrerPolicy()
Referrer-Policy header.
protected String
getRequest()
Build the HTTP request string for the current URL and state.
protected long
getResumeOffset(File outFile)
Byte offset to resume from for a file-based fetch, accounting for the persisted gunzip state: a marker means the file holds decompressed bytes, whose length is not a valid offset into the compressed representation, so the fetch restarts instead of resuming.
String
getServer()
Show the Server field.
String
getStatus()
Combined server status message e.g.
int
getStatusCode()
The server response (200, etc).
String
getStatusText()
The server text ("OK", "Not Found", etc).
String
getTransferEncoding()
Show the Transfer-Encoding field.
String
getVary()
Vary header.
String
getXContentTypeOptions()
X-Content-Type-Options header.
String
getXframeOptions()
X-Frame-Options header.
String
getXPoweredBy()
X-Powered-By header.
String
getXSSProtection()
X-XSS-Protection header.
(package private) boolean
gzipMarkerExists()
Whether this fetch's output file was written by the transparent gunzipper, so its length is a decompressed size that must not be used as a Range resume offset.
(package private) static boolean
isStalledButProgressing(IOException ioe, boolean aborted, long attemptStartBytes, long bytesTransferred)
Whether a failed attempt was a data-phase stall on an otherwise working path: the read timed out or our SocketTimeout watchdog aborted the transfer (which closes the socket first, surfacing as a raw SocketException, so the explicit aborted signal.
(package private) static boolean
isTotalBudgetExhausted(long totalTimeoutMs, long attemptStartMs, long nowMs)
Whether an attempt has already consumed its total-timeout budget.
static void
main(String[] args)
Command-line entry point for EepGet.
(package private) void
markGzipPartial(File outFile)
Record that the output file now holds decompressed bytes.
static Map<String,String>
parseAuthArgs(String args)
Parse the args in an authentication header.
(package private) static long
parseChunkLength(String line)
Extract the chunk size from a chunk-size line, dropping any chunk extensions.
(package private) static EepGet.ContentRange
parseContentRange(String val)
Parse a Content-Range header value.
protected long
readChunkLength()
Parse a chunk length from the HTTP response stream.
protected void
readHeaders()
Read and parse HTTP response headers.
(package private) static String
redactedUrl(String url)
The request URL for log output.
(package private) static long
requestedRangeStart(long alreadyTransferred, List<String> extraHeaders)
Determine the start offset this fetch requested, so a 206's Content-Range can be validated against what was actually asked for.
protected void
setPostData(String contentType, byte[] data)
Set post data (binary body).
protected void
setPostData(String contentType, File data)
Set post data (file body).
protected void
setPostData(String contentType, String data)
Set post data (string body).
void
setWriteErrorToOutput()
If called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc).
protected boolean
shouldReadBody()
Whether to read the response body.
void
stopFetching()
Stop the fetch in progress.
static String
suggestName(String url)
Parse URL for a viable filename.
(package private) static String
usageText()
The command-line help text, listing every option accepted by main().
(package private) boolean
validateResumeResponse()
Validate a resume-sensitive response once all headers are parsed.
Methods inherited from class Object
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url)
  A new SSLEepGet with a new SSLState.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  outputStream - stream to write the response body to; takes precedence over outputFile
  url - the https URL to fetch; redirects back to http are not followed
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url, SSLEepGet.SSLState state)
  Create an SSLEepGet that writes to a stream, using an existing SSL state.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  outputStream - stream to write the response body to; takes precedence over outputFile
  url - the https URL to fetch; redirects back to http are not followed
  state - an SSLState retrieved from a previous SSLEepGet with getSSLState(), or null. This makes repeated fetches from the same host MUCH faster, and prevents repeated key store loads even for different hosts.
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, OutputStream outputStream, String url, long maxSize, SSLEepGet.SSLState state)
  Create an SSLEepGet that writes to a stream, using an existing SSL state, limited to a maximum response size.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  outputStream - stream to write the response body to; takes precedence over outputFile
  url - the https URL to fetch; redirects back to http are not followed
  maxSize - The maximum size of the response
  state - an SSLState retrieved from a previous SSLEepGet with getSSLState(), or null. This makes repeated fetches from the same host MUCH faster, and prevents repeated key store loads even for different hosts.
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, String outputFile, String url)
  A new SSLEepGet with a new SSLState.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  outputFile - path of the file to write the response body to; takes precedence over outputStream
  url - the https URL to fetch; redirects back to http are not followed
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, String outputFile, String url, SSLEepGet.SSLState state)
  Create an SSLEepGet that writes to a file, using an existing SSL state.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  outputFile - path of the file to write the response body to; takes precedence over outputStream
  url - the https URL to fetch; redirects back to http are not followed
  state - an SSLState retrieved from a previous SSLEepGet with getSSLState(), or null. This makes repeated fetches from the same host MUCH faster, and prevents repeated key store loads even for different hosts.
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, OutputStream outputStream, String url)
  Use a proxy.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  type - proxy scheme used to reach the origin, or NONE for a direct connection
  proxyHost - Must be valid hostname or literal IPv4/v6. If type is INTERNAL, set to "localhost".
  proxyPort - Must be valid, -1 disallowed, no default. If type is INTERNAL, set to 4444.
  outputStream - stream to write the response body to; takes precedence over outputFile
  url - the https URL to fetch; redirects back to http are not followed
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, OutputStream outputStream, String url, SSLEepGet.SSLState state)
  Use a proxy.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  type - proxy scheme used to reach the origin, or NONE for a direct connection
  proxyHost - Must be valid hostname or literal IPv4/v6. If type is INTERNAL, set to "localhost".
  proxyPort - Must be valid, -1 disallowed, no default. If type is INTERNAL, set to 4444.
  outputStream - stream to write the response body to; takes precedence over outputFile
  url - the https URL to fetch; redirects back to http are not followed
  state - an SSLState retrieved from a previous SSLEepGet with getSSLState(), or null. This makes repeated fetches from the same host MUCH faster, and prevents repeated key store loads even for different hosts.
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, String outputFile, String url)
  Use a proxy.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  type - proxy scheme used to reach the origin, or NONE for a direct connection
  proxyHost - Must be valid hostname or literal IPv4/v6. If type is INTERNAL, set to "localhost".
  proxyPort - Must be valid, -1 disallowed, no default. If type is INTERNAL, set to 4444.
  outputFile - path of the file to write the response body to; takes precedence over outputStream
  url - the https URL to fetch; redirects back to http are not followed
- SSLEepGet
  public SSLEepGet(I2PAppContext ctx, SSLEepGet.ProxyType type, String proxyHost, int proxyPort, String outputFile, String url, SSLEepGet.SSLState state)
  Use a proxy.
  Parameters:
  ctx - the router context supplying config, logging, and the cert directories
  type - proxy scheme used to reach the origin, or NONE for a direct connection
  proxyHost - Must be valid hostname or literal IPv4/v6. If type is INTERNAL, set to "localhost".
  proxyPort - Must be valid, -1 disallowed, no default. If type is INTERNAL, set to 4444.
  outputFile - path of the file to write the response body to; takes precedence over outputStream
  url - the https URL to fetch; redirects back to http are not followed
  state - an SSLState retrieved from a previous SSLEepGet with getSSLState(), or null. This makes repeated fetches from the same host MUCH faster, and prevents repeated key store loads even for different hosts.
Method Details
- getSSLState
  public SSLEepGet.SSLState getSSLState()
  Pass this back to the next SSLEepGet constructor for faster fetches. This may be called either after the constructor or after the fetch.
  Returns:
  the SSL state
- forceDNSOverHTTPS
  public void forceDNSOverHTTPS(boolean on)
  Override the config setting, force DNSoverHTTPS on or off Call before the fetch.
  Parameters:
  on - true to force DoH on, false to force it off, overriding eepget.useDNSOverHTTPS
- forceDNSOverHTTPS
  public void forceDNSOverHTTPS(boolean on, boolean forceIPv6)
  Override the config setting, force DNSoverHTTPS on or off Call before the fetch.
  Parameters:
  on - true to force DoH on, false to force it off, overriding eepget.useDNSOverHTTPS
  forceIPv6 - use IPv6 for BOTH the connection to the DoH server AND for the queried address. on must be true.
- doFetch
  protected void doFetch(SocketTimeout timeout) throws IOException
  Fetch the resource over SSL.
  Overrides:
  doFetch in class EepGet
  Parameters:
  timeout - may be null
  Throws:
  IOException - on IO error
- sendRequest
  protected void sendRequest(SocketTimeout timeout) throws IOException
  Send the HTTP request over SSL.
  Overrides:
  sendRequest in class EepGet
  Parameters:
  timeout - may be null
  Throws:
  IOException - on IO error

Nested Class Summary

Nested classes/interfaces inherited from class EepGet

Field Summary

Fields inherited from class EepGet

Constructor Summary

Method Summary

Methods inherited from class EepGet

Methods inherited from class Object

Constructor Details

SSLEepGet

SSLEepGet

SSLEepGet

SSLEepGet

SSLEepGet

SSLEepGet

SSLEepGet

SSLEepGet

Method Details

getSSLState

forceDNSOverHTTPS

forceDNSOverHTTPS

doFetch

sendRequest