Package net.i2p.util
Class EepGet
java.lang.Object
net.i2p.util.EepGet
- Direct Known Subclasses:
EepHead
,I2PSocketEepGet
,PartialEepGet
,SSLEepGet
public class EepGet extends Object
EepGet [-p 127.0.0.1:4444]
[-n #retries]
[-o outputFile]
[-m markSize lineLen]
url
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
EepGet.AUTH_MODE
protected class
EepGet.AuthState
Manage the authentication parameters Ref: RFC 2617 Supports both Basic and Digest, however i2ptunnel HTTP proxy has migrated all previous Basic support to Digest.protected class
EepGet.CLIStatusListener
protected class
EepGet.Gunzipper
Decompressor thread.static interface
EepGet.StatusListener
Callback interface -
Field Summary
Fields Modifier and Type Field Description protected boolean
_aborted
protected String
_acceptRanges
protected String
_actualURL
the URL we actually fetch from (may differ from the _url in case of redirect)protected long
_alreadyTransferred
protected EepGet.AuthState
_authState
protected long
_bytesRemaining
protected long
_bytesTransferred
protected String
_cacheControl
protected String
_contentEncoding
protected String
_contentLanguage
protected String
_contentType
protected I2PAppContext
_context
protected String
_cookie
protected String
_csp
protected int
_currentAttempt
protected IOException
_decompressException
protected boolean
_encodingChunked
protected String
_etag
protected String
_etagOrig
protected String
_expiryDate
protected List<String>
_extraHeaders
protected int
_fetchHeaderTimeout
protected int
_fetchInactivityTimeout
protected int
_fetchTotalTimeout
protected boolean
_isGzippedResponse
protected boolean
_keepFetching
protected String
_lastModified
protected String
_lastModifiedOrig
protected List<EepGet.StatusListener>
_listeners
protected Log
_log
protected boolean
_notModified
protected int
_numRetries
protected OutputStream
_out
protected String
_outputFile
protected OutputStream
_outputStream
protected Socket
_proxy
protected String
_proxyHost
protected InputStream
_proxyIn
protected OutputStream
_proxyOut
protected int
_proxyPort
protected String
_redirectLocation
protected int
_redirects
protected String
_referrerPolicy
protected int
_responseCode
protected String
_responseText
protected String
_server
protected boolean
_shouldProxy
protected boolean
_shouldWriteErrorToOutput
protected String
_status
protected String
_transferEncoding
protected boolean
_transferFailed
protected String
_url
url we were asked to fetchprotected String
_vary
protected String
_xContentTypeOptions
protected String
_xframeOptions
protected String
_xPoweredBy
protected String
_xssProtection
protected static int
CONNECT_TIMEOUT
protected static int
INACTIVITY_TIMEOUT
protected static int
MAX_COMPLETE_FAILS
maximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14protected static String
USER_AGENT
this will be replaced by the HTTP Proxy if we are using it -
Constructor Summary
Constructors Constructor Description EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String postData)
EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String lastModified, String postData)
EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url)
EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching, String etag)
EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching, String etag, String lastModified)
EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, String postData)
EepGet(I2PAppContext ctx, int numRetries, String outputFile, String url)
EepGet(I2PAppContext ctx, int numRetries, String outputFile, String url, boolean allowCaching)
EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url)
EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching)
-
Method Summary
Modifier and Type Method Description void
addAuthorization(String userName, String password)
Add basic authorization header for the proxy.void
addHeader(String name, String value)
Add an extra header to the request.void
addStatusListener(EepGet.StatusListener lsnr)
protected void
doFetch(SocketTimeout timeout)
This reads the response to a single fetch.boolean
fetch()
Blocking fetch, returning true if the URL was retrieved, false if all retries failed.boolean
fetch(long fetchHeaderTimeout)
Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given.boolean
fetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout)
Blocking fetch.String
getAcceptRanges()
Show the Accept-Ranges fieldString
getCacheControl()
Show the Cache-Control fieldString
getContentEncoding()
Show the Content-Encoding fieldString
getContentLanguage()
Show the Content-Language fieldString
getContentType()
After fetch, the received value from the server, or null if none.String
getCookie()
Show the Set-Cookie fieldString
getCSP()
Show the Content-Security-Policy fieldString
getEtag()
After fetch, the received value from the server, or null if none.String
getExpiryDate()
Show the Expires fieldString
getLastModified()
After fetch, the received value from the server, or null if none.boolean
getNotModified()
String
getReferrerPolicy()
Show the Referrer-Policy fieldprotected String
getRequest()
String
getServer()
Show the Server fieldString
getStatus()
Show a combined server status message e.g.int
getStatusCode()
The server response (200, etc).String
getStatusText()
The server text ("OK", "Not Found", etc).String
getTransferEncoding()
Show the Transfer-Encoding fieldString
getVary()
Show the Vary fieldString
getXContentTypeOptions()
Show the X-Content-Type-Options fieldString
getXframeOptions()
Show the X-frame-Options fieldString
getXPoweredBy()
Show the X-Powered-By fieldString
getXSSProtection()
Show the X-XSS-Protection fieldstatic void
main(String[] args)
EepGet [-p 127.0.0.1:4444] [-n #retries] [-e etag] [-o outputFile] [-m markSize lineLen] url As of 0.9.45, supports https and redirect to httpsstatic Map<String,String>
parseAuthArgs(String args)
Parse the args in an authentication header.protected long
readChunkLength()
TODO this does not skip over chunk extensions (RFC 2616 sec.protected void
readHeaders()
protected void
sendRequest(SocketTimeout timeout)
void
setWriteErrorToOutput()
If called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc).protected boolean
shouldReadBody()
Should we read the body of the response?void
stopFetching()
static String
suggestName(String url)
Parse URL for a viable filename.
-
Field Details
-
_context
-
_log
-
_shouldProxy
protected final boolean _shouldProxy -
_proxyHost
-
_proxyPort
protected final int _proxyPort -
_numRetries
protected final int _numRetries -
_outputFile
-
_outputStream
-
_url
url we were asked to fetch -
_actualURL
the URL we actually fetch from (may differ from the _url in case of redirect) -
_listeners
-
_extraHeaders
-
_keepFetching
protected boolean _keepFetching -
_proxy
-
_proxyOut
-
_proxyIn
-
_out
-
_alreadyTransferred
protected long _alreadyTransferred -
_bytesTransferred
protected long _bytesTransferred -
_bytesRemaining
protected long _bytesRemaining -
_currentAttempt
protected int _currentAttempt -
_responseCode
protected int _responseCode -
_responseText
-
_shouldWriteErrorToOutput
protected boolean _shouldWriteErrorToOutput -
_etag
-
_lastModified
-
_etagOrig
-
_lastModifiedOrig
-
_encodingChunked
protected boolean _encodingChunked -
_notModified
protected boolean _notModified -
_contentType
-
_server
-
_status
-
_transferEncoding
-
_contentEncoding
-
_contentLanguage
-
_cacheControl
-
_acceptRanges
-
_vary
-
_expiryDate
-
_cookie
-
_referrerPolicy
-
_xframeOptions
-
_csp
-
_xssProtection
-
_xContentTypeOptions
-
_xPoweredBy
-
_transferFailed
protected boolean _transferFailed -
_aborted
protected volatile boolean _aborted -
_fetchHeaderTimeout
protected int _fetchHeaderTimeout -
_fetchTotalTimeout
protected int _fetchTotalTimeout -
_fetchInactivityTimeout
protected int _fetchInactivityTimeout -
_redirects
protected int _redirects -
_redirectLocation
-
_isGzippedResponse
protected boolean _isGzippedResponse -
_decompressException
-
_authState
-
USER_AGENT
this will be replaced by the HTTP Proxy if we are using it- See Also:
- Constant Field Values
-
CONNECT_TIMEOUT
protected static final int CONNECT_TIMEOUT- See Also:
- Constant Field Values
-
INACTIVITY_TIMEOUT
protected static final int INACTIVITY_TIMEOUT- See Also:
- Constant Field Values
-
MAX_COMPLETE_FAILS
protected static final int MAX_COMPLETE_FAILSmaximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14- See Also:
- Constant Field Values
-
-
Constructor Details
-
EepGet
public EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url) -
EepGet
public EepGet(I2PAppContext ctx, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching) -
EepGet
-
EepGet
public EepGet(I2PAppContext ctx, int numRetries, String outputFile, String url, boolean allowCaching) -
EepGet
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url) -
EepGet
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, String postData) -
EepGet
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, String outputFile, String url, boolean allowCaching, String etag) -
EepGet
-
EepGet
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String postData)- Parameters:
outputFile
- ignored if outputStream is non-nulloutputStream
- takes precedence over outputFile
-
EepGet
public EepGet(I2PAppContext ctx, boolean shouldProxy, String proxyHost, int proxyPort, int numRetries, long minSize, long maxSize, String outputFile, OutputStream outputStream, String url, boolean allowCaching, String etag, String lastModified, String postData)- Parameters:
outputFile
- ignored if outputStream is non-nulloutputStream
- takes precedence over outputFile
-
-
Method Details
-
main
EepGet [-p 127.0.0.1:4444] [-n #retries] [-e etag] [-o outputFile] [-m markSize lineLen] url As of 0.9.45, supports https and redirect to https -
suggestName
Parse URL for a viable filename.- Parameters:
url
- a URL giving the location of an online resource- Returns:
- a filename to save the resource as on local filesystem
-
addStatusListener
-
stopFetching
public void stopFetching() -
fetch
public boolean fetch()Blocking fetch, returning true if the URL was retrieved, false if all retries failed. Header timeout default 45 sec, total timeout default none, inactivity timeout default 60 sec.- Returns:
- success
-
fetch
public boolean fetch(long fetchHeaderTimeout)Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given. If the timeout is zero or less, this will wait indefinitely. Total timeout default none, inactivity timeout default 60 sec.- Returns:
- success
-
fetch
public boolean fetch(long fetchHeaderTimeout, long totalTimeout, long inactivityTimeout)Blocking fetch.- Parameters:
fetchHeaderTimeout
- <= 0 for none (proxy will timeout if none, none isn't recommended if no proxy)totalTimeout
- <= 0 for default noneinactivityTimeout
- <= 0 for default 60 sec- Returns:
- success
-
doFetch
This reads the response to a single fetch. Call after sendRequest()- Parameters:
timeout
- may be null- Throws:
IOException
-
readHeaders
- Throws:
IOException
-
shouldReadBody
protected boolean shouldReadBody()Should we read the body of the response?- Returns:
- true always, overridden in EepHead
- Since:
- 0.9.50
-
readChunkLength
TODO this does not skip over chunk extensions (RFC 2616 sec. 3.6.1)- Throws:
IOException
-
sendRequest
- Parameters:
timeout
- may be null- Throws:
IOException
-
getRequest
- Throws:
IOException
-
getEtag
After fetch, the received value from the server, or null if none. Before fetch, and after some errors, may be the value passed in the constructor. -
getLastModified
After fetch, the received value from the server, or null if none. Before fetch, and after some errors, may be the value passed in the constructor. -
getNotModified
public boolean getNotModified()- Returns:
- true if the server returned 304
-
getContentType
After fetch, the received value from the server, or null if none. -
getServer
Show the Server field- Since:
- I2P+ 0.9.47
-
getContentLanguage
Show the Content-Language field- Since:
- I2P+ 0.9.47
-
getTransferEncoding
Show the Transfer-Encoding field- Since:
- I2P+ 0.9.47
-
getContentEncoding
Show the Content-Encoding field- Since:
- I2P+ 0.9.47
-
getCacheControl
Show the Cache-Control field- Since:
- I2P+ 0.9.47
-
getAcceptRanges
Show the Accept-Ranges field- Since:
- I2P+ 0.9.47
-
getExpiryDate
Show the Expires field- Since:
- I2P+ 0.9.47
-
getCookie
Show the Set-Cookie field- Since:
- I2P+ 0.9.47
-
getReferrerPolicy
Show the Referrer-Policy field- Since:
- I2P+ 0.9.47
-
getVary
Show the Vary field- Since:
- I2P+ 0.9.47
-
getXframeOptions
Show the X-frame-Options field- Since:
- I2P+ 0.9.47
-
getCSP
Show the Content-Security-Policy field- Since:
- I2P+ 0.9.47
-
getXSSProtection
Show the X-XSS-Protection field- Since:
- I2P+ 0.9.47
-
getXContentTypeOptions
Show the X-Content-Type-Options field- Since:
- I2P+ 0.9.47
-
getXPoweredBy
Show the X-Powered-By field- Since:
- I2P+ 0.9.47
-
getStatusCode
public int getStatusCode()The server response (200, etc).- Returns:
- -1 if invalid, or if the proxy never responded, or if no proxy was used and the server never responded. If a non-proxied request partially succeeded (for example a redirect followed by a fail, or a partial fetch followed by a fail), this will be the last status code received. Note that fetch() may return false even if this returns 200.
- Since:
- 0.8.8
-
getStatusText
The server text ("OK", "Not Found", etc). Note that the text may contain % encoding.- Returns:
- null if invalid, or if the proxy never responded, or if no proxy was used and the server never responded. If a non-proxied request partially succeeded (for example a redirect followed by a fail, or a partial fetch followed by a fail), this will be the last status code received. Note that fetch() may return false even if this returns "OK".
- Since:
- 0.9.9
-
getStatus
Show a combined server status message e.g. "200 OK"- Since:
- I2P+ 0.9.47
-
setWriteErrorToOutput
public void setWriteErrorToOutput()If called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc). The error data will only be written if no previous data was written on an earlier try. Caller must of course check getStatusCode() or the fetch() return value.- Since:
- 0.8.8
-
addHeader
Add an extra header to the request. Must be called before fetch(). Not supported by EepHead. As of 0.9.10, If name is User-Agent, this will replace the default User-Agent header. As of 0.9.14, If name is If-None-Match or If-Modified-Since, this will replace the etag or last-modified value given in the constructor. Note that headers may be subsequently modified or removed in the I2PTunnel HTTP Client proxy. In proxied SSLEepGet, these headers are sent to the remote server, NOT the proxy.- Since:
- 0.8.8
-
addAuthorization
Add basic authorization header for the proxy. Only added if the request is going through a proxy. Must be called before fetch().- Since:
- 0.8.9
-
parseAuthArgs
Parse the args in an authentication header. Modified from LoadClientAppsJob. All keys are mapped to lower case. Double quotes around values are stripped. Ref: RFC 2617 Public for I2PTunnelHTTPClientBase; use outside of tree at own risk, subject to change or removal- Parameters:
args
- non-null, starting after "Digest " or "Basic "- Since:
- 0.9.4, moved from I2PTunnelHTTPClientBase in 0.9.12
-