FetchAndAdd

java.lang.Object

org.klomp.snark.Snark

org.klomp.snark.web.FetchAndAdd

All Implemented Interfaces:: Runnable, EepGet.StatusListener, CoordinatorListener, ShutdownListener, StorageListener

public class FetchAndAddextends Snark implements EepGet.StatusListener, Runnable

A cancellable torrent file downloader. We extend Snark so its status may be easily listed in the web table without adding a lot of code there.

Upon successful download, this Snark will be deleted and a "real" Snark created.

The methods return values similar to a Snark in magnet mode. A fake info hash, which is the SHA1 of the URL, is returned to prevent duplicates.

This Snark may be stopped and restarted, although a partially downloaded file is discarded.

Field Summary
Fields
Modifier and Type
Field
Description
private boolean
_active
True while the fetch is being shown.
private final I2PAppContext
_ctx
The app context.
private final File
_dataDir
Directory the torrent downloads to.
private EepGet
_eepGet
The client doing the fetch.
private String
_failCause
Reason the fetch failed.
private final byte[]
_fakeHash
Placeholder hash shown while fetching.
private boolean
_isRunning
True while the fetch thread runs.
private final Log
_log
Logger for the torrent fetch.
private final SnarkManager
_mgr
The manager the torrent is added to.
private final String
_name
Name given to the torrent.
private long
_remaining
Bytes left to fetch, -1 until known.
private long
_started
When the fetch started.
private Thread
_thread
The thread running the fetch.
private long
_total
Total size of the fetch, -1 until known.
private long
_transferred
Bytes fetched so far.
private final String
_url
The URL the torrent is fetched from.
private static final int
RETRIES
Tries allowed to fetch the torrent.
Fields inherited from class Snark
_util, PROP_MAX_CONNECTIONS, storageCompleted
Modifier and Type
Field
Description
protected final I2PSnarkUtil
_util
The util.
static final String
PROP_MAX_CONNECTIONS
Maximum connections.
boolean
storageCompleted
Whether the torrent storage has reported completion.
Constructor Summary
Constructors
Constructor
Description
FetchAndAdd(I2PAppContext ctx, SnarkManager mgr, String url, File dataDir)
Create the downloader; the caller should call _mgr.addDownloader(this), which starts things off.
Method Summary
Modifier and Type
Method
Description
private String
_t(String s)
Translates a string for the page.
private String
_t(String s, String o)
Translates a string with one parameter.
private void
add(File file)
Tell SnarkManager to copy the torrent file over and add it to the Snarks list.
(package private) static String
appendCause(String prefix, Throwable t)
Append the exception message, stripped of HTML, to a localized notice.
void
attemptFailed(String url, long bytesTransferred, long bytesRemaining, int currentAttempt, int numRetries, Exception cause)
Called when a download attempt fails.
void
attempting(String url)
Called when a download attempt is starting.
void
bytesTransferred(long alreadyTransferred, int currentWrite, long bytesTransferred, long bytesRemaining, String url)
Called when bytes are transferred during download.
(package private) static long
downloaded(long total, long remaining)
Bytes downloaded so far, given the reported total and remaining.
private String
failMessage()
Build the "failed to retrieve" user message, appending the sanitized failure cause if any.
private File
get()
Fetch the torrent file from _url into a temp file, connecting the I2P tunnel on demand.
String
getBaseName()
The base name of the torrent.
long
getDataLength()
The torrent file size, or -1.
long
getDownloaded()
The total number of bytes downloaded.
long
getDownloadRate()
The current download rate in bytes per second.
byte[]
getInfoHash()
The fake info hash (SHA1 of the URL), which makes re-downloading the same URL a duplicate.
String
getName()
The file name of the .torrent file.
long
getNeededLength()
The torrent file bytes remaining, or -1 before the size is known.
int
getPeerCount()
The peer count.
long
getRemainingLength()
The remaining bytes, or -1 when done so the web lists us as "complete" instead of "seeding".
long
getTotalLength()
The torrent file size, or -1 before the size is known.
int
getTrackerSeenPeers()
The tracker seen peers count.
void
headerReceived(String url, int attemptNum, String key, String val)
Called when an HTTP header is received.
boolean
isStopped()
Check if the torrent is stopped.
void
run()
Begin fetching; called by startTorrent().
void
startTorrent()
Begin (or restart) the fetch on a fresh background thread.
void
stopTorrent()
Stop contacting the tracker and talking with peers.
void
transferComplete(long alreadyTransferred, long bytesTransferred, long bytesRemaining, String url, String outputFile, boolean notModified)
Called when a transfer completes successfully.
void
transferFailed(String url, long bytesTransferred, long bytesRemaining, int currentAttempt)
Called when a transfer fails.
private void
updateState(long transferred, long remaining, long total, boolean active)
Update the shared progress state reported by the Snark overrides.
private static String
urlify(String s)
Turns a URL in the text into a link.
Methods inherited from class Snark
addComments, addMessage, ban, getCheckingProgress, getComments, getDest, getID, getLastTrackerResponse, getMetaInfo, getNeeded, getPeerList, getPieceLength, getPieces, getRPCID, getScrapeLeechers, getScrapePartialSeeds, getScrapeSeeders, getSkippedLength, getStartedTime, getStorage, getSwarmPeerCount, getTrackerProblems, getTrackerURL, getUploaded, getUploadRate, gotMetaInfo, isAllocating, isAutoStoppable, isBanned, isChecking, isNotificationSent, isStarting, isStorageCompleted, overUploadLimit, peerChange, replaceMetaInfo, restartAcceptor, setAutoStoppable, setLastTrackerResponse, setNotificationSent, setStarting, setTrackerProblems, setTrackerSeenPeers, setWantedPieces, shutdown, stopTorrent, storageAllChecked, storageAllocated, storageChecked, storageCompleted, storageCreateFile, updatePiecePriorities, updateScrape, updateScrape
Modifier and Type
Method
Description
boolean
addComments(List<Comment> comments)
Add to the current comment set for this torrent, creating it if it didn't previously exist.
void
addMessage(String message)
StorageListener and CoordinatorListener callback.
void
ban(Hash h)
Ban the given hash.
double
getCheckingProgress()
The progress of file checking as a percentage.
CommentSet
getComments()
The current comment set for this torrent.
TorrentDest
getDest()
The transient multi-dest destination, or null when multi-dest is disabled or the torrent is stopped.
byte[]
getID()
The peer ID.
long
getLastTrackerResponse()
The wall-clock time of the last valid tracker announce or scrape response, or 0 if none was ever received.
MetaInfo
getMetaInfo()
The metainfo.
long
getNeeded()
The number of pieces still missing.
List<Peer>
getPeerList()
The peer list.
int
getPieceLength(int p)
The piece length.
int
getPieces()
The number of pieces.
int
getRPCID()
A unique ID for this torrent, useful for RPC.
int
getScrapeLeechers()
The best-known leech count from tracker scrapes.
int
getScrapePartialSeeds()
The best-known partial seed count from tracker scrapes.
int
getScrapeSeeders()
The best-known seed count from tracker scrapes.
long
getSkippedLength()
Bytes not received and set to skipped.
long
getStartedTime()
When did we start this torrent For RPC.
Storage
getStorage()
The storage.
int
getSwarmPeerCount()
Distinct swarm peers seen recently (via announce, PEX or DHT) or currently connected, for the torrent view peer count.
String
getTrackerProblems()
Not HTML escaped.
String
getTrackerURL()
The tracker URL.
long
getUploaded()
The total number of bytes uploaded.
long
getUploadRate()
The current upload rate in bytes per second.
void
gotMetaInfo(PeerCoordinator coordinator, MetaInfo metainfo)
Called when the PeerCoordinator got the MetaInfo via magnet.
boolean
isAllocating()
Check if disk allocation (ballooning) is in progress.
boolean
isAutoStoppable()
Check if the torrent is auto-stoppable.
boolean
isBanned(Hash h)
Is the given hash banned?
boolean
isChecking()
Check if file checking is in progress.
boolean
isNotificationSent()
Whether a completion notification has already been sent for this torrent.
boolean
isStarting()
Check if startup is in progress.
boolean
isStorageCompleted()
Whether the torrent storage has completed.
boolean
overUploadLimit(int uploaders)
CoordinatorListener callback to check if the total uploaders across all torrents exceeds the configured limit.
void
peerChange(PeerCoordinator coordinator, Peer peer)
CoordinatorListener no-op.
void
replaceMetaInfo(MetaInfo metainfo)
Call after editing torrent.
boolean
restartAcceptor()
Restart the connection acceptor.
void
setAutoStoppable(boolean yes)
Store whether the torrent is auto-stoppable.
void
setLastTrackerResponse(long t)
Record a valid tracker announce or scrape response.
void
setNotificationSent(boolean sent)
Mark that a completion notification has been sent.
void
setStarting()
Mark startup in progress.
void
setTrackerProblems(String p)
Store the tracker problems string.
void
setTrackerSeenPeers(int p)
Store the tracker seen peers count.
void
setWantedPieces(Storage storage)
StorageListener callback to update the coordinator's wanted piece set.
void
shutdown()
SnarkShutdown callback unused.
void
stopTorrent(boolean fast)
Stop contacting the tracker and talking with peers.
void
storageAllChecked(Storage storage)
StorageListener callback called when all pieces have been checked.
void
storageAllocated(Storage storage, long length)
No-op; deliberately empty.
void
storageChecked(Storage storage, int num, boolean checked)
StorageListener callback called when a piece check completes.
void
storageCompleted(Storage storage)
StorageListener callback called when the torrent has completed.
void
storageCreateFile(Storage storage, String name, long length)
StorageListener no-op.
void
updatePiecePriorities()
Recalculate piece priorities.
void
updateScrape(int seeders, int leechers)
Update the best-known swarm composition from a tracker scrape (BEP 15/48).
void
updateScrape(int seeders, int leechers, int partials)
Update the best-known swarm composition from a tracker scrape (BEP 15/48/21).
Methods inherited from class Object
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- RETRIES
  private static final int RETRIES
  Tries allowed to fetch the torrent.
  See Also:
  Constant Field Values
- _ctx
  private final I2PAppContext _ctx
  The app context.
- _log
  private final Log _log
  Logger for the torrent fetch.
- _mgr
  private final SnarkManager _mgr
  The manager the torrent is added to.
- _url
  private final String _url
  The URL the torrent is fetched from.
- _fakeHash
  private final byte[] _fakeHash
  Placeholder hash shown while fetching.
- _name
  private final String _name
  Name given to the torrent.
- _dataDir
  private final File _dataDir
  Directory the torrent downloads to.
- _remaining
  private volatile long _remaining
  Bytes left to fetch, -1 until known.
- _total
  private volatile long _total
  Total size of the fetch, -1 until known.
- _transferred
  private volatile long _transferred
  Bytes fetched so far.
- _isRunning
  private volatile boolean _isRunning
  True while the fetch thread runs.
- _active
  private volatile boolean _active
  True while the fetch is being shown.
- _started
  private volatile long _started
  When the fetch started.
- _failCause
  private String _failCause
  Reason the fetch failed.
- _thread
  private Thread _thread
  The thread running the fetch.
- _eepGet
  private EepGet _eepGet
  The client doing the fetch.
Constructor Details
- FetchAndAdd
  public FetchAndAdd(I2PAppContext ctx, SnarkManager mgr, String url, File dataDir)
  Create the downloader; the caller should call _mgr.addDownloader(this), which starts things off.
  The fake info hash is the SHA1 of the URL, so downloading the same URL twice is detected as a duplicate.
  Parameters:
  ctx - the I2P application context supplying the logger
  mgr - the manager that owns this downloader
  url - the torrent file URL to fetch
  dataDir - null to default to snark data directory
Method Details
- run
  public void run()
  Begin fetching; called by startTorrent().
  Specified by:
  run in interface Runnable
- failMessage
  private String failMessage()
  Build the "failed to retrieve" user message, appending the sanitized failure cause if any.
  Returns:
  the localized failure message
- get
  private File get()
  Fetch the torrent file from _url into a temp file, connecting the I2P tunnel on demand. The returned temp file is registered for deletion on JVM exit.
  This is modeled on the equivalent logic in I2PSnarkUtil, but registers this class as a status listener so the web page shows live progress.
  Returns:
  the populated temp file, or null on failure (the temp file is already deleted)
- add
  private void add(File file)
  Tell SnarkManager to copy the torrent file over and add it to the Snarks list. This Snark may then be deleted.
  Parameters:
  file - the downloaded torrent file to copy over and add
- appendCause
  static String appendCause(String prefix, Throwable t)
  Append the exception message, stripped of HTML, to a localized notice.
  Parameters:
  prefix - the base message
  t - the throwable whose message to append
  Returns:
  the combined message
- startTorrent
  public void startTorrent()
  Begin (or restart) the fetch on a fresh background thread.
  Because a partially downloaded file is discarded on stop, restart simply resets all progress state and fetches again from scratch.
  Overrides:
  startTorrent in class Snark
- stopTorrent
  public void stopTorrent()
  Description copied from class: Snark
  Stop contacting the tracker and talking with peers.
  Overrides:
  stopTorrent in class Snark
- isStopped
  public boolean isStopped()
  Description copied from class: Snark
  Check if the torrent is stopped.
  Overrides:
  isStopped in class Snark
  Returns:
  true if stopped
- getName
  public String getName()
  Description copied from class: Snark
  The file name of the .torrent file.
  Overrides:
  getName in class Snark
  Returns:
  file name of .torrent file (should be full absolute path), or a fake name if in magnet mode.
- getBaseName
  public String getBaseName()
  Description copied from class: Snark
  The base name of the torrent.
  Overrides:
  getBaseName in class Snark
  Returns:
  base name of torrent [filtered version of getMetaInfo.Snark.getName()], or a fake name if in magnet mode
- getInfoHash
  public byte[] getInfoHash()
  The fake info hash (SHA1 of the URL), which makes re-downloading the same URL a duplicate.
  Overrides:
  getInfoHash in class Snark
  Returns:
  the fake info hash
- getTotalLength
  public long getTotalLength()
  The torrent file size, or -1 before the size is known.
  Overrides:
  getTotalLength in class Snark
  Returns:
  torrent file size or -1
- getDataLength
  public long getDataLength()
  The torrent file size, or -1. No padding information is available while fetching, so this equals getTotalLength().
  Overrides:
  getDataLength in class Snark
  Returns:
  torrent file size or -1
- getRemainingLength
  public long getRemainingLength()
  The remaining bytes, or -1 when done so the web lists us as "complete" instead of "seeding".
  Overrides:
  getRemainingLength in class Snark
  Returns:
  -1 when done so the web lists us as "complete" instead of "seeding"
- getNeededLength
  public long getNeededLength()
  The torrent file bytes remaining, or -1 before the size is known.
  Overrides:
  getNeededLength in class Snark
  Returns:
  torrent file bytes remaining or -1
- getDownloadRate
  public long getDownloadRate()
  Description copied from class: Snark
  The current download rate in bytes per second.
  Overrides:
  getDownloadRate in class Snark
  Returns:
  download rate in bytes per second, or 0 if not available
- getDownloaded
  public long getDownloaded()
  Description copied from class: Snark
  The total number of bytes downloaded.
  Overrides:
  getDownloaded in class Snark
  Returns:
  total bytes downloaded, or 0 if not available
- downloaded
  static long downloaded(long total, long remaining)
  Bytes downloaded so far, given the reported total and remaining.
  Before the first byte is received the total is -1 (size unknown); in that case 0 is reported rather than a bogus negative/positive value derived from the -1 sentinel.
  Parameters:
  total - the total size, possibly -1 while the size is unknown
  remaining - the bytes remaining, possibly -1 while unknown
  Returns:
  the bytes downloaded, 0 while the total is unknown
- getPeerCount
  public int getPeerCount()
  Description copied from class: Snark
  The peer count.
  Overrides:
  getPeerCount in class Snark
  Returns:
  connected peers, 0 if the coordinator has not been created yet
- getTrackerSeenPeers
  public int getTrackerSeenPeers()
  Description copied from class: Snark
  The tracker seen peers count.
  Overrides:
  getTrackerSeenPeers in class Snark
  Returns:
  count returned from tracker
- attemptFailed
  public void attemptFailed(String url, long bytesTransferred, long bytesRemaining, int currentAttempt, int numRetries, Exception cause)
  Called when a download attempt fails.
  Specified by:
  attemptFailed in interface EepGet.StatusListener
  Parameters:
  url - the URL being downloaded
  bytesTransferred - bytes transferred so far
  bytesRemaining - bytes remaining to transfer
  currentAttempt - the current attempt number
  numRetries - the maximum number of retries
  cause - the exception that caused the failure, or null
  See Also:
  EepGet.StatusListener.bytesTransferred(long, int, long, long, String)
- bytesTransferred
  public void bytesTransferred(long alreadyTransferred, int currentWrite, long bytesTransferred, long bytesRemaining, String url)
  Called when bytes are transferred during download.
  Specified by:
  bytesTransferred in interface EepGet.StatusListener
  Parameters:
  alreadyTransferred - total bytes transferred before this write
  currentWrite - number of bytes in this write
  bytesTransferred - total bytes transferred so far
  bytesRemaining - bytes remaining to transfer
  url - the URL being downloaded
- transferComplete
  public void transferComplete(long alreadyTransferred, long bytesTransferred, long bytesRemaining, String url, String outputFile, boolean notModified)
  Called when a transfer completes successfully.
  Specified by:
  transferComplete in interface EepGet.StatusListener
  Parameters:
  alreadyTransferred - total bytes transferred before this call
  bytesTransferred - bytes transferred in this call
  bytesRemaining - bytes remaining (should be 0)
  url - the URL being downloaded
  outputFile - the output file path
  notModified - true if the server returned 304
  See Also:
  EepGet.StatusListener.bytesTransferred(long, int, long, long, String)
- transferFailed
  public void transferFailed(String url, long bytesTransferred, long bytesRemaining, int currentAttempt)
  Called when a transfer fails.
  Specified by:
  transferFailed in interface EepGet.StatusListener
  Parameters:
  url - the URL being downloaded
  bytesTransferred - bytes transferred so far
  bytesRemaining - bytes remaining to transfer
  currentAttempt - the current attempt number
  See Also:
  EepGet.StatusListener.bytesTransferred(long, int, long, long, String)
- headerReceived
  public void headerReceived(String url, int attemptNum, String key, String val)
  Called when an HTTP header is received. Not needed here.
  Specified by:
  headerReceived in interface EepGet.StatusListener
  Parameters:
  url - the address the header arrived on
  attemptNum - zero-based index of the attempt the header arrived on
  key - header key
  val - header value
- attempting
  public void attempting(String url)
  Called when a download attempt is starting. Not needed here.
  Specified by:
  attempting in interface EepGet.StatusListener
  Parameters:
  url - the URL being attempted
- updateState
  private void updateState(long transferred, long remaining, long total, boolean active)
  Update the shared progress state reported by the Snark overrides.
  The remaining and total sizes are only updated once a known non-negative size is reported; a total of -1 leaves the last known total in place (used by the failure callbacks, which report remaining bytes but no total).
  Parameters:
  transferred - bytes transferred so far
  remaining - bytes remaining to transfer
  total - the total size, or -1 to leave the current total unchanged
  active - whether a transfer is currently in progress
- _t
  private String _t(String s)
  Translates a string for the page.
  Parameters:
  s - the string to translate
  Returns:
  the translated text
- _t
  private String _t(String s, String o)
  Translates a string with one parameter.
  Parameters:
  s - the string to translate
  o - the parameter to substitute
  Returns:
  the translated text
- urlify
  private static String urlify(String s)
  Turns a URL in the text into a link.
  Parameters:
  s - the text to convert
  Returns:
  the text with the link added

Field Summary

Fields inherited from class Snark

Constructor Summary

Method Summary

Methods inherited from class Snark

Methods inherited from class Object

Field Details

RETRIES

_ctx

_log

_mgr

_url

_fakeHash

_name

_dataDir

_remaining

_total

_transferred

_isRunning

_active

_started

_failCause

_thread

_eepGet

Constructor Details

Method Details

run

failMessage

get

add

appendCause

startTorrent

stopTorrent

isStopped

getName

getBaseName

getInfoHash

getTotalLength

getDataLength

getRemainingLength

getNeededLength

getDownloadRate

getDownloaded

downloaded

getPeerCount

getTrackerSeenPeers

attemptFailed

bytesTransferred

transferComplete

transferFailed

headerReceived

attempting

updateState

_t

_t

urlify