public class EepGet extends Object
| Modifier and Type | Class and Description |
|---|---|
protected static class | EepGet.AUTH_MODEAuthentication mode enumeration. |
protected class | EepGet.AuthStateManage the authentication parameters
Ref: RFC 2617
Supports both Basic and Digest, however i2ptunnel HTTP proxy
has migrated all previous Basic support to Digest. |
protected class | EepGet.CLIStatusListenerCLIStatusListener. |
protected class | EepGet.GunzipperDecompressor thread. |
static interface | EepGet.StatusListenerCallback interface for monitoring EepGet transfer progress. |
| Modifier and Type | Field and Description |
|---|---|
protected boolean | _abortedWhether the fetch was aborted due to timeout or user request |
protected String | _acceptRangesAccept-Ranges header value from the response |
protected String | _actualURLThe URL we actually fetch from (may differ from _url in case of redirect) |
protected long | _alreadyTransferredTotal bytes transferred across all attempts, including resumed content |
protected EepGet.AuthState | _authStateAuthentication state for proxy digest/basic auth; created via addAuthorization() |
protected long | _bytesRemainingRemaining bytes on the current attempt, -1 if unknown (chunked or no Content-Length) |
protected long | _bytesTransferredTotal bytes transferred including headers, retries, redirects, and discarded partial downloads |
protected String | _cacheControlCache-Control header value from the response |
protected String | _contentEncodingContent-Encoding header value from the response |
protected String | _contentLanguageContent-Language header value from the response |
protected String | _contentTypeContent-Type header value from the response |
protected I2PAppContext | _contextI2P application context for configuration, logging, and random number generation |
protected String | _cookieSet-Cookie header value from the response |
protected String | _cspContent-Security-Policy header value from the response |
protected int | _currentAttemptZero-based attempt counter for the current fetch |
protected IOException | _decompressExceptionIOException from the background decompressor thread, if any |
protected boolean | _encodingChunkedWhether the response is using chunked transfer encoding |
protected String | _etagETag received from the server (for conditional requests) |
protected String | _etagOrigOriginal ETag passed in constructor, preserved across redirects |
protected String | _expiryDateExpires header value from the response |
protected List<String> | _extraHeadersAdditional HTTP headers to include in the request |
protected int | _fetchHeaderTimeoutTimeout in ms for receiving the HTTP response headers |
protected int | _fetchInactivityTimeoutInactivity timeout in ms between data packets |
protected int | _fetchTotalTimeoutTotal timeout in ms for the entire fetch operation |
protected boolean | _isGzippedResponseWhether the response body is gzip-compressed |
protected boolean | _keepFetchingWhether to continue fetching; set false to abort |
protected String | _lastModifiedLast-Modified header received from the server |
protected String | _lastModifiedOrigOriginal Last-Modified passed in constructor, preserved across redirects |
protected List<EepGet.StatusListener> | _listenersRegistered status listeners for progress callbacks |
protected Log | _logLogger for this instance |
protected int | _maxCompleteFailsMaximum number of complete (zero-byte) failures before giving up |
protected boolean | _notModifiedWhether the server returned 304 Not Modified |
protected int | _numRetriesMaximum number of retry attempts per fetch |
protected OutputStream | _outOutput stream for writing fetched data (file or pipe) |
protected String | _outputFilePath to the output file for saving the fetched content |
protected OutputStream | _outputStreamOutput stream destination, overrides outputFile when non-null |
protected Socket | _proxyThe socket connection to the proxy or the target server (if not proxied). |
protected String | _proxyHostProxy hostname or IP address |
protected InputStream | _proxyInInput stream from the proxy/server socket |
protected OutputStream | _proxyOutOutput stream to the proxy/server socket |
protected int | _proxyPortProxy port number |
protected String | _redirectLocationLocation header value from redirect or 407 proxy auth response |
protected AtomicInteger | _redirectsCounter tracking the number of HTTP redirects followed |
protected String | _referrerPolicyReferrer-Policy header value from the response |
protected int | _responseCodeHTTP response status code, -1 if no response received |
protected String | _responseTextHTTP response status text (e.g. |
protected String | _serverServer header value from the response |
protected boolean | _shouldProxyWhether to route requests through an HTTP proxy |
protected boolean | _shouldWriteErrorToOutputWhether to write error response bodies to the output |
protected String | _statusStatus header value from the response |
protected String | _transferEncodingTransfer-Encoding header value from the response |
protected boolean | _transferFailedWhether the current transfer attempt failed (non-2xx response code) |
protected String | _urlThe URL we were asked to fetch |
protected String | _varyVary header value from the response |
protected String | _xContentTypeOptionsX-Content-Type-Options header value from the response |
protected String | _xframeOptionsX-Frame-Options header value from the response |
protected String | _xPoweredByX-Powered-By header value from the response |
protected String | _xssProtectionX-XSS-Protection header value from the response |
protected static int | DEFAULT_CONNECT_TIMEOUTDefault connect timeout in milliseconds (90 seconds) |
protected static int | DEFAULT_INACTIVITY_TIMEOUTDefault inactivity timeout in milliseconds (5 minutes) |
protected static int | DEFAULT_MAX_COMPLETE_FAILSMaximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14 |
protected static int | DEFAULT_NUM_RETRIESDefault number of retry attempts |
(package private) static String | PROP_CONNECT_TIMEOUTProperty name for connect timeout configuration |
(package private) static String | PROP_DEFAULT_RETRIESProperty name for default retries configuration |
(package private) static String | PROP_INACTIVITY_TIMEOUTProperty name for inactivity timeout configuration |
(package private) static String | PROP_MAX_COMPLETE_FAILSProperty name for max complete fails configuration |
protected static String | USER_AGENTDefault User-Agent header value. |
| Constructor and Description |
|---|
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
long minSize,
long maxSize,
String outputFile,
OutputStream outputStream,
String url,
boolean allowCaching,
String etag,
String postData)Creates a new EepGet instance with size limits, output stream, and POST data. |
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
long minSize,
long maxSize,
String outputFile,
OutputStream outputStream,
String url,
boolean allowCaching,
String etag,
String lastModified,
String postData)Full constructor for EepGet with all parameters. |
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url)Creates a new EepGet instance with explicit proxy control, defaulting to allow caching. |
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url,
boolean allowCaching,
String etag)Creates a new EepGet instance with conditional fetch via ETag. |
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url,
boolean allowCaching,
String etag,
String lastModified)Creates a new EepGet instance with conditional fetch via ETag and Last-Modified. |
EepGet(I2PAppContext ctx,
boolean shouldProxy,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url,
String postData)Creates a new EepGet instance with POST data support. |
EepGet(I2PAppContext ctx,
int numRetries,
String outputFile,
String url)Creates a new EepGet instance without a proxy, defaulting to allow caching. |
EepGet(I2PAppContext ctx,
int numRetries,
String outputFile,
String url,
boolean allowCaching)Creates a new EepGet instance without a proxy, with configurable caching. |
EepGet(I2PAppContext ctx,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url)Creates a new EepGet instance with proxy support, defaulting to allow caching. |
EepGet(I2PAppContext ctx,
String proxyHost,
int proxyPort,
int numRetries,
String outputFile,
String url,
boolean allowCaching)Creates a new EepGet instance with proxy support and configurable caching. |
| Modifier and Type | Method and Description |
|---|---|
void | addAuthorization(String userName,
String password)Add basic authorization header for the proxy. |
void | addHeader(String name,
String value)Add an extra header to the request. |
void | addStatusListener(EepGet.StatusListener lsnr)Add a status listener. |
protected void | doFetch(SocketTimeout timeout)Read response body for a single fetch. |
boolean | fetch()Blocking fetch, returning true if the URL was retrieved, false if all retries failed. |
boolean | fetch(long fetchHeaderTimeout)Blocking fetch, timing out individual attempts if the HTTP response headers
don't come back in the time given. |
boolean | fetch(long fetchHeaderTimeout,
long totalTimeout,
long inactivityTimeout)Blocking fetch. |
String | getAcceptRanges()Show the Accept-Ranges field |
String | getCacheControl()Show the Cache-Control field |
String | getContentEncoding()Show the Content-Encoding field |
String | getContentLanguage()Show the Content-Language field |
String | getContentType()After fetch, the received value from the server, or null if none. |
String | getCookie()Show the Set-Cookie field |
String | getCSP()Content-Security-Policy header. |
String | getEtag()After fetch, the received value from the server, or null if none. |
String | getExpiryDate()Show the Expires field |
String | getLastModified()After fetch, the received value from the server, or null if none. |
boolean | getNotModified()Whether the server returned 304. |
String | getReferrerPolicy()Referrer-Policy header. |
protected String | getRequest()Build the HTTP request string for the current URL and state. |
String | getServer()Show the Server field |
String | getStatus()Combined server status message e.g. |
int | getStatusCode()The server response (200, etc). |
String | getStatusText()The server text ("OK", "Not Found", etc). |
String | getTransferEncoding()Show the Transfer-Encoding field |
String | getVary()Vary header. |
String | getXContentTypeOptions()X-Content-Type-Options header. |
String | getXframeOptions()X-Frame-Options header. |
String | getXPoweredBy()X-Powered-By header. |
String | getXSSProtection()X-XSS-Protection header. |
static void | main(String[] args)Command-line entry point for EepGet. |
static Map<String,String> | parseAuthArgs(String args)Parse the args in an authentication header. |
protected long | readChunkLength()Parse a chunk length from the HTTP response stream. |
protected void | readHeaders()Read and parse HTTP response headers. |
protected void | sendRequest(SocketTimeout timeout)Open connection and send HTTP request. |
protected void | setPostData(String contentType,
byte[] data)Set post data (binary body). |
protected void | setPostData(String contentType,
File data)Set post data (file body). |
protected void | setPostData(String contentType,
String data)Set post data (string body). |
void | setWriteErrorToOutput()If called (before calling fetch()),
data from the server or proxy will be written to the
output file or stream even on an error response code (4xx, 5xx, etc). |
protected boolean | shouldReadBody()Whether to read the response body. |
void | stopFetching()Stop the fetch in progress. |
static String | suggestName(String url)Parse URL for a viable filename. |
protected volatile boolean _aborted
protected String _acceptRanges
protected String _actualURL
protected long _alreadyTransferred
protected EepGet.AuthState _authState
protected long _bytesRemaining
protected long _bytesTransferred
protected String _cacheControl
protected String _contentEncoding
protected String _contentLanguage
protected String _contentType
protected final I2PAppContext _context
protected String _cookie
protected String _csp
protected int _currentAttempt
protected IOException _decompressException
protected volatile boolean _encodingChunked
protected String _etag
protected final String _etagOrig
protected String _expiryDate
protected volatile int _fetchHeaderTimeout
protected volatile int _fetchInactivityTimeout
protected volatile int _fetchTotalTimeout
protected boolean _isGzippedResponse
protected volatile boolean _keepFetching
protected String _lastModified
protected final String _lastModifiedOrig
protected final List<EepGet.StatusListener> _listeners
protected final Log _log
protected volatile int _maxCompleteFails
protected volatile boolean _notModified
protected final int _numRetries
protected OutputStream _out
protected final String _outputFile
protected final OutputStream _outputStream
protected Socket _proxy
protected final String _proxyHost
protected InputStream _proxyIn
protected OutputStream _proxyOut
protected final int _proxyPort
protected String _redirectLocation
protected AtomicInteger _redirects
protected String _referrerPolicy
protected volatile int _responseCode
protected String _responseText
protected String _server
protected final boolean _shouldProxy
protected volatile boolean _shouldWriteErrorToOutput
protected String _status
protected String _transferEncoding
protected volatile boolean _transferFailed
protected final String _url
protected String _vary
protected String _xContentTypeOptions
protected String _xframeOptions
protected String _xPoweredBy
protected String _xssProtection
protected static final int DEFAULT_CONNECT_TIMEOUT
protected static final int DEFAULT_INACTIVITY_TIMEOUT
protected static final int DEFAULT_MAX_COMPLETE_FAILS
protected static final int DEFAULT_NUM_RETRIES
static final String PROP_CONNECT_TIMEOUT
static final String PROP_DEFAULT_RETRIES
static final String PROP_INACTIVITY_TIMEOUT
static final String PROP_MAX_COMPLETE_FAILS
protected static final String USER_AGENT
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String postData)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesminSize - minimum acceptable response size, -1 for unlimitedmaxSize - maximum acceptable response size, -1 for unlimitedoutputFile - not used if outputStream is non-nulloutputStream - takes precedence over outputFileurl - URL to fetchallowCaching - whether to allow cachingetag - ETag for conditional fetchpostData - POST data stringpublic EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String lastModified, String postData)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesminSize - minimum acceptable response size, -1 for unlimitedmaxSize - maximum acceptable response size, -1 for unlimitedoutputFile - not used if outputStream is non-nulloutputStream - takes precedence over outputFileurl - URL to fetchallowCaching - whether to allow cachingetag - ETag for conditional fetchlastModified - last-modified for conditional fetchpostData - POST data stringpublic EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchpublic EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching, String etag)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchallowCaching - whether to allow cachingetag - ETag for conditional fetchpublic EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching, String etag, String lastModified)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchallowCaching - whether to allow cachingetag - ETag for conditional fetchlastModified - last-modified for conditional fetchpublic EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, String postData)
ctx - I2P app contextshouldProxy - whether to use a proxyproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchpostData - POST data stringpublic EepGet(I2PAppContext ctx, int numRetries, String outputFile, String url)
ctx - I2P app contextnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchpublic EepGet(I2PAppContext ctx, int numRetries, String outputFile, String url, boolean allowCaching)
ctx - I2P app contextnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchallowCaching - whether to allow cachingpublic EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url)
ctx - I2P app contextproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchpublic EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching)
ctx - I2P app contextproxyHost - proxy hostnameproxyPort - proxy portnumRetries - number of retriesoutputFile - output file pathurl - URL to fetchallowCaching - whether to allow cachingpublic void addAuthorization(String userName, String password)
userName - proxy usernamepassword - proxy passwordpublic void addHeader(String name, String value)
name - header namevalue - header valuepublic void addStatusListener(EepGet.StatusListener lsnr)
lsnr - the listener to addprotected void doFetch(SocketTimeout timeout) throws IOException
timeout - may be nullIOException - on IO errorpublic boolean fetch()
public boolean fetch(long fetchHeaderTimeout)
fetchHeaderTimeout - timeout in mspublic boolean fetch(long fetchHeaderTimeout,
long totalTimeout,
long inactivityTimeout)fetchHeaderTimeout - <= 0 for none (proxy will timeout if none, none isn't recommended if no proxy)totalTimeout - <= 0 for default noneinactivityTimeout - <= 0 for default 60 secpublic String getAcceptRanges()
public String getCacheControl()
public String getContentEncoding()
public String getContentLanguage()
public String getContentType()
public String getCookie()
public String getCSP()
public String getEtag()
public String getExpiryDate()
public String getLastModified()
public boolean getNotModified()
public String getReferrerPolicy()
protected String getRequest() throws IOException
IOException - on bad URLpublic String getServer()
public String getStatus()
public int getStatusCode()
public String getStatusText()
public String getTransferEncoding()
public String getVary()
public String getXContentTypeOptions()
public String getXframeOptions()
public String getXPoweredBy()
public String getXSSProtection()
public static void main(String[] args)
args - command-line argumentspublic static Map<String,String> parseAuthArgs(String args)
args - non-null, starting after "Digest " or "Basic "protected long readChunkLength()
throws IOExceptionIOException - on IO errorprotected void readHeaders()
throws IOExceptionIOException - on IO errorprotected void sendRequest(SocketTimeout timeout) throws IOException
timeout - may be nullIOException - on IO errorprotected void setPostData(String contentType, byte[] data)
contentType - Content-Type header valuedata - POST body bytesIllegalStateException - if already setprotected void setPostData(String contentType, File data)
contentType - Content-Type header valuedata - file containing POST bodyIllegalStateException - if already setprotected void setPostData(String contentType, String data)
contentType - Content-Type header valuedata - POST bodyIllegalStateException - if already setpublic void setWriteErrorToOutput()
protected boolean shouldReadBody()
public void stopFetching()