I2PSocketEepGet

java.lang.Object

net.i2p.util.EepGet

net.i2p.client.streaming.I2PSocketEepGet

public class I2PSocketEepGetextends EepGet

Fetch a URL using a socket from the supplied I2PSocketManager. Hostname must resolve to an i2p destination - no routing to an outproxy. Does not support response gzip decompression (unlike I2PTunnelHTTPProxy) (yet), but of course there is still gzip at the I2CP layer. This is designed for Java apps such as bittorrent clients that wish to do HTTP fetches and use other protocols on a single set of tunnels. This may provide anonymity benefits over using the shared clients HTTP proxy, preventing inadvertent outproxy usage, reduce resource usage by eliminating a second set of tunnels, and eliminate the requirement to to separately configure the proxy host and port. For additional documentation see the superclass. Supports http://example.i2p/blah Supports http://B32KEY.b32.i2p/blah Supports http://i2p/B64KEY/blah for compatibility with the eepproxy Supports http://B64KEY/blah Warning - does not support /eepproxy/blah, address helpers, http://B64KEY.i2p/blah, or other odd things that may be found in the HTTP proxy.

Nested Class Summary
Nested classes/interfaces inherited from class EepGet
EepGet.AUTH_MODE, EepGet.AuthState, EepGet.CLIStatusListener, EepGet.Gunzipper, EepGet.StatusListener
Modifier and Type
Class
Description
protected static enum
EepGet.AUTH_MODE
Authentication mode enumeration.
protected class
EepGet.AuthState
Manage the authentication parameters Ref: RFC 2617 Supports both Basic and Digest, however i2ptunnel HTTP proxy has migrated all previous Basic support to Digest.
protected class
EepGet.CLIStatusListener
Reports fetch progress to the command line, one mark per markSize bytes and a summary line every lineSize marks.
protected class
EepGet.Gunzipper
Decompressor thread.
static interface
EepGet.StatusListener
Callback interface for monitoring EepGet transfer progress.
Field Summary
Fields inherited from class EepGet
_aborted, _acceptRanges, _actualURL, _alreadyTransferred, _authState, _bytesRemaining, _bytesTransferred, _cacheControl, _contentEncoding, _contentLanguage, _contentRange, _contentType, _context, _cookie, _csp, _currentAttempt, _decompressException, _encodingChunked, _etag, _etagOrig, _expiryDate, _extraHeaders, _fetchHeaderTimeout, _fetchInactivityTimeout, _fetchTotalTimeout, _isGzippedResponse, _keepFetching, _lastModified, _lastModifiedOrig, _listeners, _log, _maxCompleteFails, _notModified, _numRetries, _out, _outputFile, _outputStream, _proxy, _proxyHost, _proxyIn, _proxyOut, _proxyPort, _redirectLocation, _redirects, _referrerPolicy, _responseCode, _responseText, _retryDelayMs, _server, _shouldProxy, _shouldWriteErrorToOutput, _status, _transferEncoding, _transferFailed, _url, _vary, _xContentTypeOptions, _xframeOptions, _xPoweredBy, _xssProtection, DEFAULT_CONNECT_TIMEOUT, DEFAULT_INACTIVITY_TIMEOUT, DEFAULT_MAX_COMPLETE_FAILS, DEFAULT_NUM_RETRIES, DEFAULT_RETRY_DELAY, DEFAULT_RETRY_JITTER, USER_AGENT
Modifier and Type
Field
Description
protected boolean
_aborted
Whether the fetch was aborted due to timeout or user request.
protected String
_acceptRanges
Accept-Ranges header value from the response.
protected String
_actualURL
The URL we actually fetch from (may differ from _url in case of redirect).
protected long
_alreadyTransferred
Total bytes transferred across all attempts, including resumed content.
protected EepGet.AuthState
_authState
Authentication state for proxy digest/basic auth; created via EepGet.addAuthorization(String, String).
protected long
_bytesRemaining
Remaining bytes on the current attempt, -1 if unknown (chunked or no Content-Length).
protected long
_bytesTransferred
Body bytes transferred in the current EepGet.fetch(), summed across attempts.
protected String
_cacheControl
Cache-Control header value from the response.
protected String
_contentEncoding
Content-Encoding header value from the response.
protected String
_contentLanguage
Content-Language header value from the response.
protected String
_contentRange
Content-Range header value from the response, for validating 206/416 resumes.
protected String
_contentType
Content-Type header value from the response.
protected final I2PAppContext
_context
I2P application context for configuration, logging, and random number generation.
protected String
_cookie
Set-Cookie header value from the response.
protected String
_csp
Content-Security-Policy header value from the response.
protected int
_currentAttempt
Zero-based attempt counter for the current fetch; read by the watchdog command.
protected IOException
_decompressException
IOException from the background decompressor thread, if any.
protected boolean
_encodingChunked
Whether the response is using chunked transfer encoding.
protected String
_etag
ETag received from the server (for conditional requests).
protected final String
_etagOrig
Original ETag passed in constructor, preserved across redirects.
protected String
_expiryDate
Expires header value from the response.
protected List<String>
_extraHeaders
Additional HTTP headers to include in the request.
protected int
_fetchHeaderTimeout
Timeout in ms for receiving the HTTP response headers.
protected int
_fetchInactivityTimeout
Inactivity timeout in ms between data packets.
protected int
_fetchTotalTimeout
Total timeout in ms for the entire fetch operation.
protected boolean
_isGzippedResponse
Whether the response body is gzip-compressed.
protected boolean
_keepFetching
Whether to continue fetching; set false to abort.
protected String
_lastModified
Last-Modified header received from the server.
protected final String
_lastModifiedOrig
Original Last-Modified passed in constructor, preserved across redirects.
protected final List<EepGet.StatusListener>
_listeners
Registered status listeners for progress callbacks.
protected final Log
_log
Logger for this instance.
protected int
_maxCompleteFails
Maximum number of complete (zero-byte) failures before giving up.
protected boolean
_notModified
Whether the server returned 304 Not Modified.
protected final int
_numRetries
Maximum number of retry attempts per fetch.
protected OutputStream
_out
Output stream for writing fetched data (file or pipe).
protected final String
_outputFile
Path to the output file for saving the fetched content.
protected final OutputStream
_outputStream
Output stream destination, overrides outputFile when non-null.
protected Socket
_proxy
The socket connection to the proxy or the target server (if not proxied).
protected final String
_proxyHost
Proxy hostname or IP address.
protected InputStream
_proxyIn
Input stream from the proxy/server socket.
protected OutputStream
_proxyOut
Output stream to the proxy/server socket.
protected final int
_proxyPort
Proxy port number.
protected String
_redirectLocation
Location header value from redirect or 407 proxy auth response.
protected AtomicInteger
_redirects
Counter tracking the number of HTTP redirects followed.
protected String
_referrerPolicy
Referrer-Policy header value from the response.
protected int
_responseCode
HTTP response status code, -1 if no response received.
protected String
_responseText
HTTP response status text (e.g.
protected final long
_retryDelayMs
Fixed retry delay in ms, -1 selects the jittered default @since 0.9.71+.
protected String
_server
Server header value from the response.
protected final boolean
_shouldProxy
Whether to route requests through an HTTP proxy.
protected boolean
_shouldWriteErrorToOutput
Whether to write error response bodies to the output.
protected String
_status
Status header value from the response.
protected String
_transferEncoding
Transfer-Encoding header value from the response.
protected boolean
_transferFailed
Whether the current transfer attempt failed (non-2xx response code).
protected final String
_url
The URL we were asked to fetch.
protected String
_vary
Vary header value from the response.
protected String
_xContentTypeOptions
X-Content-Type-Options header value from the response.
protected String
_xframeOptions
X-Frame-Options header value from the response.
protected String
_xPoweredBy
X-Powered-By header value from the response.
protected String
_xssProtection
X-XSS-Protection header value from the response.
protected static final int
DEFAULT_CONNECT_TIMEOUT
Default connect timeout in milliseconds (90 seconds).
protected static final int
DEFAULT_INACTIVITY_TIMEOUT
Default inactivity timeout in milliseconds (5 minutes).
protected static final int
DEFAULT_MAX_COMPLETE_FAILS
Maximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14.
protected static final int
DEFAULT_NUM_RETRIES
Default number of retry attempts.
protected static final int
DEFAULT_RETRY_DELAY
Base retry delay in milliseconds when the jittered default is used @since 0.9.71+.
protected static final int
DEFAULT_RETRY_JITTER
Maximum additional jitter in milliseconds added to the base retry delay @since 0.9.71+.
protected static final String
USER_AGENT
Default User-Agent header value.
Constructor Summary
Constructors
Constructor
Description
I2PSocketEepGet(I2PAppContext ctx, I2PSocketManager mgr, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url)
Creates an I2PSocketEepGet with full configuration.
I2PSocketEepGet(I2PAppContext ctx, I2PSocketManager mgr, int numRetries, String outputFile, String url)
Creates an I2PSocketEepGet with default size limits.
Method Summary
All Methods
Modifier and Type
Method
Description
boolean
fetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout)
We have to override this to close _socket, since we can't use _proxy in super as the I2PSocket.
protected String
getRequest()
Guess we have to override this since super doesn't strip the http://host from the GET line which hoses some servers (opentracker) HTTP proxy was kind enough to do this for us.
protected void
readHeaders()
Overridden to disable inline gunzipping.
protected void
sendRequest(SocketTimeout timeout)
Look up the address, get a socket from the I2PSocketManager supplied in the constructor, and send the request.
Methods inherited from class EepGet
addAuthorization, addHeader, addStatusListener, doFetch, fetch, fetch, getAcceptRanges, getCacheControl, getContentEncoding, getContentLanguage, getContentType, getCookie, getCSP, getEtag, getExpiryDate, getLastModified, getNotModified, getReferrerPolicy, getResumeOffset, getServer, getStatus, getStatusCode, getStatusText, getTransferEncoding, getVary, getXContentTypeOptions, getXframeOptions, getXPoweredBy, getXSSProtection, main, parseAuthArgs, readChunkLength, setPostData, setPostData, setPostData, setWriteErrorToOutput, shouldReadBody, stopFetching, suggestName
Modifier and Type
Method
Description
void
addAuthorization(String userName, String password)
Add basic authorization header for the proxy.
void
addHeader(String name, String value)
Add an extra header to the request.
void
addStatusListener(EepGet.StatusListener lsnr)
Add a status listener.
protected void
doFetch(SocketTimeout timeout)
Read response body for a single fetch.
boolean
fetch()
Blocking fetch, returning true if the URL was retrieved, false if all retries failed.
boolean
fetch(long fetchHeaderTimeout)
Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given.
String
getAcceptRanges()
Show the Accept-Ranges field.
String
getCacheControl()
Show the Cache-Control field.
String
getContentEncoding()
Show the Content-Encoding field.
String
getContentLanguage()
Show the Content-Language field.
String
getContentType()
After fetch, the received value from the server, or null if none.
String
getCookie()
Show the Set-Cookie field.
String
getCSP()
Content-Security-Policy header.
String
getEtag()
After fetch, the received value from the server, or null if none.
String
getExpiryDate()
Show the Expires field.
String
getLastModified()
After fetch, the received value from the server, or null if none.
boolean
getNotModified()
Whether the server returned 304.
String
getReferrerPolicy()
Referrer-Policy header.
protected long
getResumeOffset(File outFile)
Byte offset to resume from for a file-based fetch, accounting for the persisted gunzip state: a marker means the file holds decompressed bytes, whose length is not a valid offset into the compressed representation, so the fetch restarts instead of resuming.
String
getServer()
Show the Server field.
String
getStatus()
Combined server status message e.g.
int
getStatusCode()
The server response (200, etc).
String
getStatusText()
The server text ("OK", "Not Found", etc).
String
getTransferEncoding()
Show the Transfer-Encoding field.
String
getVary()
Vary header.
String
getXContentTypeOptions()
X-Content-Type-Options header.
String
getXframeOptions()
X-Frame-Options header.
String
getXPoweredBy()
X-Powered-By header.
String
getXSSProtection()
X-XSS-Protection header.
static void
main(String[] args)
Command-line entry point for EepGet.
static Map<String,String>
parseAuthArgs(String args)
Parse the args in an authentication header.
protected long
readChunkLength()
Parse a chunk length from the HTTP response stream.
protected void
setPostData(String contentType, byte[] data)
Set post data (binary body).
protected void
setPostData(String contentType, File data)
Set post data (file body).
protected void
setPostData(String contentType, String data)
Set post data (string body).
void
setWriteErrorToOutput()
If called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc).
protected boolean
shouldReadBody()
Whether to read the response body.
void
stopFetching()
Stop the fetch in progress.
static String
suggestName(String url)
Parse URL for a viable filename.
Methods inherited from class Object
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details
- I2PSocketEepGet
  public I2PSocketEepGet(I2PAppContext ctx, I2PSocketManager mgr, int numRetries, String outputFile, String url)
  Creates an I2PSocketEepGet with default size limits.
  Parameters:
  ctx - the I2P app context
  mgr - the socket manager
  numRetries - number of retries on failure
  outputFile - file to write output to
  url - the URL to fetch
- I2PSocketEepGet
  public I2PSocketEepGet(I2PAppContext ctx, I2PSocketManager mgr, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url)
  Creates an I2PSocketEepGet with full configuration.
  Parameters:
  ctx - the I2P app context
  mgr - the socket manager
  numRetries - number of retries on failure
  minSize - minimum acceptable size
  maxSize - maximum acceptable size
  outputFile - file to write output to
  outputStream - stream to write output to
  url - the URL to fetch
Method Details
- fetch
  public boolean fetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout)
  We have to override this to close _socket, since we can't use _proxy in super as the I2PSocket.
  Overrides:
  fetch in class EepGet
  Parameters:
  fetchHeaderTimeout - <= 0 for none (proxy will timeout if none, none isn't recommended if no proxy) forced to totalTimeout if totalTimeout is set and this is <= 0
  totalTimeout - operation-wide deadline for the whole fetch, including retries and backoff between attempts; <= 0 for default none
  inactivityTimeout - <= 0 for default EepGet.DEFAULT_INACTIVITY_TIMEOUT (5 minutes)
  Returns:
  whether the URL was retrieved; false if every attempt failed
- readHeaders
  protected void readHeaders() throws IOException
  Overridden to disable inline gunzipping.
  Overrides:
  readHeaders in class EepGet
  Throws:
  IOException - on IO error
- sendRequest
  protected void sendRequest(SocketTimeout timeout) throws IOException
  Look up the address, get a socket from the I2PSocketManager supplied in the constructor, and send the request.
  Overrides:
  sendRequest in class EepGet
  Parameters:
  timeout - connect and inactivity timeout, used to detect hung connections
  Throws:
  IOException - on IO error
- getRequest
  protected String getRequest() throws IOException
  Guess we have to override this since super doesn't strip the http://host from the GET line which hoses some servers (opentracker) HTTP proxy was kind enough to do this for us.
  Overrides:
  getRequest in class EepGet
  Returns:
  the HTTP request line and headers, with the scheme and host stripped from the GET line
  Throws:
  IOException - on bad URL

Nested Class Summary

Nested classes/interfaces inherited from class EepGet

Field Summary

Fields inherited from class EepGet

Constructor Summary

Method Summary

Methods inherited from class EepGet

Methods inherited from class Object

Constructor Details

I2PSocketEepGet

Method Details

fetch

readHeaders

sendRequest

getRequest