java.lang.Object
net.i2p.util.EepGet
net.i2p.util.PartialEepGet
Fetch exactly the first 'size' bytes into a stream Anything less or more will throw an IOException Retries may be given; zero for no retries. If the server does not return a Content-Length header of the correct size, the fetch will fail.
Useful for checking .sud versions.
Nested Class Summary
Nested classes/interfaces inherited from class EepGet
EepGet.AUTH_MODE, EepGet.AuthState, EepGet.CLIStatusListener, EepGet.ContentRange, EepGet.Gunzipper, EepGet.StatusListenerModifier and TypeClassDescriptionprotected static enumAuthentication mode enumeration.protected classManage the authentication parameters Ref: RFC 2617 Supports both Basic and Digest, however i2ptunnel HTTP proxy has migrated all previous Basic support to Digest.protected classReports fetch progress to the command line, one mark per markSize bytes and a summary line every lineSize marks.(package private) static final classParsed Content-Range header value.protected classDecompressor thread.static interfaceCallback interface for monitoring EepGet transfer progress.Field Summary
FieldsFields inherited from class EepGet
_aborted, _acceptRanges, _actualURL, _alreadyTransferred, _authState, _bytesRemaining, _bytesTransferred, _cacheControl, _contentEncoding, _contentLanguage, _contentRange, _contentType, _context, _cookie, _csp, _currentAttempt, _decompressException, _encodingChunked, _etag, _etagOrig, _expiryDate, _extraHeaders, _fetchHeaderTimeout, _fetchInactivityTimeout, _fetchTotalTimeout, _isGzippedResponse, _keepFetching, _lastModified, _lastModifiedOrig, _listeners, _log, _maxCompleteFails, _notModified, _numRetries, _out, _outputFile, _outputStream, _proxy, _proxyHost, _proxyIn, _proxyOut, _proxyPort, _redirectLocation, _redirects, _referrerPolicy, _responseCode, _responseText, _retryDelayMs, _server, _shouldProxy, _shouldWriteErrorToOutput, _status, _transferEncoding, _transferFailed, _url, _vary, _xContentTypeOptions, _xframeOptions, _xPoweredBy, _xssProtection, DEFAULT_CONNECT_TIMEOUT, DEFAULT_INACTIVITY_TIMEOUT, DEFAULT_MAX_COMPLETE_FAILS, DEFAULT_NUM_RETRIES, DEFAULT_RETRY_DELAY, DEFAULT_RETRY_JITTER, GZIP_PARTIAL_MARKER_SUFFIX, PROP_CONNECT_TIMEOUT, PROP_DEFAULT_RETRIES, PROP_INACTIVITY_TIMEOUT, PROP_MAX_COMPLETE_FAILS, PROP_RETRY_DELAY, USER_AGENTModifier and TypeFieldDescriptionprotected booleanWhether the fetch was aborted due to timeout or user request.protected StringAccept-Ranges header value from the response.protected StringThe URL we actually fetch from (may differ from _url in case of redirect).protected longTotal bytes transferred across all attempts, including resumed content.protected EepGet.AuthStateAuthentication state for proxy digest/basic auth; created viaEepGet.addAuthorization(String, String).protected longRemaining bytes on the current attempt, -1 if unknown (chunked or no Content-Length).protected longBody bytes transferred in the currentEepGet.fetch(), summed across attempts.protected StringCache-Control header value from the response.protected StringContent-Encoding header value from the response.protected StringContent-Language header value from the response.protected StringContent-Range header value from the response, for validating 206/416 resumes.protected StringContent-Type header value from the response.protected final I2PAppContextI2P application context for configuration, logging, and random number generation.protected StringSet-Cookie header value from the response.protected StringContent-Security-Policy header value from the response.protected intZero-based attempt counter for the current fetch; read by the watchdog command.protected IOExceptionIOException from the background decompressor thread, if any.protected booleanWhether the response is using chunked transfer encoding.protected StringETag received from the server (for conditional requests).protected final StringOriginal ETag passed in constructor, preserved across redirects.protected StringExpires header value from the response.Additional HTTP headers to include in the request.protected intTimeout in ms for receiving the HTTP response headers.protected intInactivity timeout in ms between data packets.protected intTotal timeout in ms for the entire fetch operation.protected booleanWhether the response body is gzip-compressed.protected booleanWhether to continue fetching; set false to abort.protected StringLast-Modified header received from the server.protected final StringOriginal Last-Modified passed in constructor, preserved across redirects.protected final List<EepGet.StatusListener> Registered status listeners for progress callbacks.protected final LogLogger for this instance.protected intMaximum number of complete (zero-byte) failures before giving up.protected booleanWhether the server returned 304 Not Modified.protected final intMaximum number of retry attempts per fetch.protected OutputStreamOutput stream for writing fetched data (file or pipe).protected final StringPath to the output file for saving the fetched content.protected final OutputStreamOutput stream destination, overrides outputFile when non-null.protected SocketThe socket connection to the proxy or the target server (if not proxied).protected final StringProxy hostname or IP address.protected InputStreamInput stream from the proxy/server socket.protected OutputStreamOutput stream to the proxy/server socket.protected final intProxy port number.protected StringLocation header value from redirect or 407 proxy auth response.protected AtomicIntegerCounter tracking the number of HTTP redirects followed.protected StringReferrer-Policy header value from the response.protected intHTTP response status code, -1 if no response received.protected StringHTTP response status text (e.g.protected final longFixed retry delay in ms, -1 selects the jittered default @since 0.9.71+.protected StringServer header value from the response.protected final booleanWhether to route requests through an HTTP proxy.protected booleanWhether to write error response bodies to the output.protected StringStatus header value from the response.protected StringTransfer-Encoding header value from the response.protected booleanWhether the current transfer attempt failed (non-2xx response code).protected final StringThe URL we were asked to fetch.protected StringVary header value from the response.protected StringX-Content-Type-Options header value from the response.protected StringX-Frame-Options header value from the response.protected StringX-Powered-By header value from the response.protected StringX-XSS-Protection header value from the response.protected static final intDefault connect timeout in milliseconds (90 seconds).protected static final intDefault inactivity timeout in milliseconds (5 minutes).protected static final intMaximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14.protected static final intDefault number of retry attempts.protected static final intBase retry delay in milliseconds when the jittered default is used @since 0.9.71+.protected static final intMaximum additional jitter in milliseconds added to the base retry delay @since 0.9.71+.(package private) static final StringSuffix of the sidecar marker file recording that the output file holds transparently-decompressed (gunzipped) bytes rather than raw response bytes, so its length must not be used as a Range resume offset.(package private) static final StringProperty name for connect timeout configuration.(package private) static final StringProperty name for default retries configuration.(package private) static final StringProperty name for inactivity timeout configuration.(package private) static final StringProperty name for max complete fails configuration.(package private) static final StringProperty name for the fixed retry delay in milliseconds.protected static final StringDefault User-Agent header value.Constructor Summary
ConstructorsConstructorDescriptionPartialEepGet(I2PAppContext ctx, String proxyHost, int proxyPort, OutputStream outputStream, String url, long size) Instantiate an EepGet that will fetch exactly size bytes when fetch() is called.PartialEepGet(I2PAppContext ctx, String proxyHost, int proxyPort, OutputStream outputStream, String url, long size, int numRetries) Instantiate an EepGet that will fetch exactly size bytes when fetch() is called.Method Summary
Methods inherited from class EepGet
addAuthorization, addHeader, addStatusListener, doFetch, failureReason, fatalAbort, fetch, fetch, fetch, getAcceptRanges, getCacheControl, getContentEncoding, getContentLanguage, getContentType, getCookie, getCSP, getEtag, getExpiryDate, getLastModified, getNotModified, getReferrerPolicy, getResumeOffset, getServer, getStatus, getStatusCode, getStatusText, getTransferEncoding, getVary, getXContentTypeOptions, getXframeOptions, getXPoweredBy, getXSSProtection, gzipMarkerExists, isStalledButProgressing, isTotalBudgetExhausted, markGzipPartial, parseAuthArgs, parseChunkLength, parseContentRange, readChunkLength, readHeaders, redactedUrl, requestedRangeStart, seconds, sendRequest, setPostData, setPostData, setPostData, setWriteErrorToOutput, shouldReadBody, stopFetching, suggestName, usageText, validateResumeResponseModifier and TypeMethodDescriptionvoidaddAuthorization(String userName, String password) Add basic authorization header for the proxy.voidAdd an extra header to the request.voidAdd a status listener.protected voiddoFetch(SocketTimeout timeout) Read response body for a single fetch.(package private) static StringfailureReason(Throwable t, long inactivityMs) The failure reason for a transfer-failed log line.(package private) static booleanfatalAbort(boolean aborted, boolean strictSize, long remaining, boolean sawEof) Decide whether a watchdog abort observed after the body loop must fail the attempt.booleanfetch()Blocking fetch, returning true if the URL was retrieved, false if all retries failed.booleanfetch(long fetchHeaderTimeout) Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given.booleanfetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout) Blocking fetch.Show the Accept-Ranges field.Show the Cache-Control field.Show the Content-Encoding field.Show the Content-Language field.After fetch, the received value from the server, or null if none.Show the Set-Cookie field.getCSP()Content-Security-Policy header.getEtag()After fetch, the received value from the server, or null if none.Show the Expires field.After fetch, the received value from the server, or null if none.booleanWhether the server returned 304.Referrer-Policy header.protected longgetResumeOffset(File outFile) Byte offset to resume from for a file-based fetch, accounting for the persisted gunzip state: a marker means the file holds decompressed bytes, whose length is not a valid offset into the compressed representation, so the fetch restarts instead of resuming.Show the Server field.Combined server status message e.g.intThe server response (200, etc).The server text ("OK", "Not Found", etc).Show the Transfer-Encoding field.getVary()Vary header.X-Content-Type-Options header.X-Frame-Options header.X-Powered-By header.X-XSS-Protection header.(package private) booleanWhether this fetch's output file was written by the transparent gunzipper, so its length is a decompressed size that must not be used as a Range resume offset.(package private) static booleanisStalledButProgressing(IOException ioe, boolean aborted, long attemptStartBytes, long bytesTransferred) Whether a failed attempt was a data-phase stall on an otherwise working path: the read timed out or our SocketTimeout watchdog aborted the transfer (which closes the socket first, surfacing as a raw SocketException, so the explicitabortedsignal.(package private) static booleanisTotalBudgetExhausted(long totalTimeoutMs, long attemptStartMs, long nowMs) Whether an attempt has already consumed its total-timeout budget.(package private) voidmarkGzipPartial(File outFile) Record that the output file now holds decompressed bytes.parseAuthArgs(String args) Parse the args in an authentication header.(package private) static longparseChunkLength(String line) Extract the chunk size from a chunk-size line, dropping any chunk extensions.(package private) static EepGet.ContentRangeparseContentRange(String val) Parse a Content-Range header value.protected longParse a chunk length from the HTTP response stream.protected voidRead and parse HTTP response headers.(package private) static StringredactedUrl(String url) The request URL for log output.(package private) static longrequestedRangeStart(long alreadyTransferred, List<String> extraHeaders) Determine the start offset this fetch requested, so a 206's Content-Range can be validated against what was actually asked for.(package private) static Stringseconds(long ms) Render a millisecond duration in seconds for log output, keeping milliseconds only for sub-second or non-whole-second values.protected voidsendRequest(SocketTimeout timeout) Open connection and send HTTP request.protected voidsetPostData(String contentType, byte[] data) Set post data (binary body).protected voidsetPostData(String contentType, File data) Set post data (file body).protected voidsetPostData(String contentType, String data) Set post data (string body).voidIf called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc).protected booleanWhether to read the response body.voidStop the fetch in progress.static StringsuggestName(String url) Parse URL for a viable filename.(package private) static StringThe command-line help text, listing every option accepted byEepGet.main(String[]).(package private) booleanValidate a resume-sensitive response once all headers are parsed.
Field Details
_fetchSize
private final long _fetchSizeSize of the byte range to fetch.
Constructor Details
PartialEepGet
public PartialEepGet(I2PAppContext ctx, String proxyHost, int proxyPort, OutputStream outputStream, String url, long size) Instantiate an EepGet that will fetch exactly size bytes when fetch() is called. UsesEepGet.DEFAULT_NUM_RETRIESretries so a transient empty or flaky upstream response does not immediately declare the source dead.- Parameters:
ctx- the I2P app contextproxyHost- use null or "" for no proxyproxyPort- use 0 for no proxyoutputStream- the stream to write output tourl- the URL to fetchsize- fetch exactly this many bytes
PartialEepGet
public PartialEepGet(I2PAppContext ctx, String proxyHost, int proxyPort, OutputStream outputStream, String url, long size, int numRetries) Instantiate an EepGet that will fetch exactly size bytes when fetch() is called.- Parameters:
ctx- the I2P app contextproxyHost- use null or "" for no proxyproxyPort- use 0 for no proxyoutputStream- the stream to write output tourl- the URL to fetchsize- fetch exactly this many bytesnumRetries- number of attempts (beyond the first) before giving up; 0 for no retries
Method Details
main
Entry point for command-line usage.- Parameters:
args- command line arguments
usage
private static void usage()Prints how to fetch part of a file from the command line.getRequest
Description copied from class:EepGetBuild the HTTP request string for the current URL and state.- Overrides:
getRequestin classEepGet- Returns:
- the HTTP request string
- Throws:
IOException- on bad URL