Storage (Javadocs [I2P+ 2.13.0+] - API Version 0.9.70+)

java.lang.Object
- org.klomp.snark.Storage

All Implemented Interfaces:
Closeable, AutoCloseable
```
public class Storage
extends Object
implements Closeable
```
Storage manages torrent data files on disk for I2PSnark.
This class handles all aspects of storing and retrieving torrent pieces:
- Creating and managing files/directories for torrent data
- Reading and writing piece data to disk
- Tracking downloaded pieces via BitField
- Validating downloaded pieces against piece hashes
- File pre-allocation for sparse file support
- Priority-based file selection for partial downloads
- File checking and rechecking for data integrity
Storage implements Closeable and should be properly closed after use. The storage must be checked via check() before reading or writing pieces.
Thread safety: This class uses synchronized methods and concurrent collections for thread-safe access to file handles and metadata.
See Also:
MetaInfo, BitField, StorageListener

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`MAX_PIECE_SIZE` Maximum permitted piece size.
`static int`	`MAX_PIECES` The maximum number of pieces allowed in a single torrent.
`static long`	`MAX_TOTAL_SIZE` Maximum total torrent size (MAX_PIECE_SIZE * MAX_PIECES).
`(package private) static String`	`PAD_DIR` BEP 47 directory holding synthetic padding files; skipped when walking a source dir.
`static int`	`PRIORITY_NORMAL` Default priority value for files.
`static int`	`PRIORITY_SKIP` Priority value indicating a file should be skipped during download.

Constructor Summary

Constructors
Constructor and Description
`Storage(I2PSnarkUtil util, File baseFile, MetaInfo metainfo, StorageListener listener, boolean preserveFileNames)` Creates a new storage based on the supplied MetaInfo.
`Storage(I2PSnarkUtil util, File baseFile, String announce, List<List<String>> announce_list, String created_by, boolean privateTorrent, List<String> url_list, String comment, StorageListener listener, List<TorrentCreateFilter> filters)` Creates a storage from the existing file or directory.
`Storage(I2PSnarkUtil util, File baseFile, String announce, List<List<String>> announce_list, String created_by, boolean privateTorrent, StorageListener listener, List<TorrentCreateFilter> filters)` Creates a storage from the existing file or directory.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`check()` Creates (and/or checks) all files from the metainfo file list.
`void`	`check(long savedTime, BitField savedBitField)` Creates (and/or checks) all files from the metainfo file list.
`void`	`cleanRAFs()` Close unused RAFs - call periodically
`(package private) void`	`clearChanged()` Clear the storage changed variable
`void`	`close()` Closes the Storage and makes sure that all RandomAccessFiles are closed.
`boolean`	`complete()` Whether or not this storage contains all pieces if the MetaInfo.
`(package private) static int`	`countNonPad(File[] files)` Count the entries in a directory listing for the max files per torrent check, excluding padding directories.
`void`	`deleteStagingData()` Removes all partially downloaded data from the staging directory for this torrent.
`static String`	`filterName(String name)` Removes 'suspicious' characters from the given file name.
`long`	`getActivity()` The last activity timestamp.
`File`	`getBase()` The base file or directory.
`String`	`getBaseName()` The base file or directory name of the data, as specified in the .torrent file, but filtered to remove illegal characters.
`BitField`	`getBitField()` The BitField that tells which pieces this storage contains.
`double`	`getCheckingProgress()` If checking is in progress, return completion 0.0 ...
`SortedSet<File>`	`getDirectories()` Includes the base for a multi-file torrent.
`List<String>`	`getExcludedFiles(File base)` The excluded files.
`int`	`getFileCount()` Does not include directories.
`int[]`	`getFilePriorities()` The file priorities array.
`List<File>`	`getFiles()` Does not include directories.
`boolean`	`getInOrder()` Whether in-order download mode is enabled.
`MetaInfo`	`getMetaInfo()` Returns the MetaInfo associated with this Storage.
`ByteArray`	`getPiece(int piece, int off, int len)` Returns a byte array containing a portion of the requested piece or null if the storage doesn't contain the piece yet.
`int[]`	`getPiecePriorities()` Call setPriority() for all changed files first, then call this.
`boolean`	`getPreserveFileNames()` Whether original file names are being preserved, false if filtered.
`int`	`getPriority(int fileIndex)` The priority of the file at the index.
`long`	`getSkippedLength()` Call setPriority() for all changed files first, then call this.
`(package private) static File`	`getStagingBase(String tempDir, String baseName, MetaInfo metainfo)` The staging directory for a torrent: the configured temp dir plus a subdirectory unique to the torrent.
`File`	`getStagingDir()` The directory this torrent's incomplete files are staged in while downloading, or null when the staging feature is disabled.
`(package private) static int`	`getVerifyThreads(I2PAppContext ctx)` Worker threads for a parallel piece verification: the i2psnark.verifyThreads property when set, otherwise scaled to the CPU count (at least 4).
`int`	`indexOf(File file)` Index to pass to remaining(), getPriority(), setPriority()
`boolean`	`isAllocating()` Disk allocation (ballooning) in progress.
`boolean`	`isChanged()` Has the storage changed since instantiation?
`boolean`	`isChecking()` File checking in progress.
`(package private) static boolean`	`isPadDir(String name)` Is this a padding directory (BEP 47 ".pad" or a parse-renamed "_pad")? Padding entries hold only synthetic zero files: they must not count toward the max files per torrent limit, nor be included as data when creating a torrent.
`File[]`	`listMerged(String pathInTorrent)` Lists a directory within the torrent, merging the staging tree while the download is incomplete, so completed files already moved to the data directory and files still staged in the temp dir both appear.
`static void`	`main(String[] args)` Create a metainfo.
`int`	`needed()` How many pieces are still missing from this storage.
`(package private) static String`	`noDataMessage(File base, List<String> unreadable)` The message for a failed torrent creation with no readable data, so a permission problem is reported as such rather than as an empty folder.
`boolean`	`putPiece(PartialPiece pp)` Put the piece in the Storage if it is correct.
`boolean`	`recheck()` Blocking.
`long[]`	`remaining()` For efficiency, calculate remaining bytes for all files at once
`long[][]`	`remaining2()` For efficiency, calculate remaining bytes for all files at once.
`void`	`reopen()` Doesn't really reopen the file descriptors for a restart.
`void`	`setActivity(long time)` Store the activity timestamp.
`(package private) void`	`setFilePriorities(int[] p)` The file priorities array.
`void`	`setInOrder(boolean yes)` Enable or disable in-order download mode.
`void`	`setPriority(int fileIndex, int pri)` Must call Snark.updatePiecePriorities() (which calls getPiecePriorities()) after calling this.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - MAX_PIECE_SIZE
```
public static final int MAX_PIECE_SIZE
```
    Maximum permitted piece size. Torrents with pieces larger than this will be rejected.
    See Also:
    Constant Field Values
  - MAX_PIECES
```
public static final int MAX_PIECES
```
    The maximum number of pieces allowed in a single torrent.
    See Also:
    Constant Field Values
  - MAX_TOTAL_SIZE
```
public static final long MAX_TOTAL_SIZE
```
    Maximum total torrent size (MAX_PIECE_SIZE * MAX_PIECES).
    See Also:
    Constant Field Values
  - PAD_DIR
```
static final String PAD_DIR
```
    BEP 47 directory holding synthetic padding files; skipped when walking a source dir.
    See Also:
    Constant Field Values
  - PRIORITY_NORMAL
```
public static final int PRIORITY_NORMAL
```
    Default priority value for files.
    See Also:
    Constant Field Values
  - PRIORITY_SKIP
```
public static final int PRIORITY_SKIP
```
    Priority value indicating a file should be skipped during download.
    See Also:
    Constant Field Values
- Constructor Detail
  - Storage
```
public Storage(I2PSnarkUtil util,
               File baseFile,
               MetaInfo metainfo,
               StorageListener listener,
               boolean preserveFileNames)
```
    Creates a new storage based on the supplied MetaInfo.
    Does not check storage. Caller MUST call check(), which will try to create and/or check all needed files in the MetaInfo.
    Parameters:
    baseFile - the torrent data file or dir
    preserveFileNames - if true, do not remap names to a 'safe' charset
  - Storage
```
public Storage(I2PSnarkUtil util,
               File baseFile,
               String announce,
               List<List<String>> announce_list,
               String created_by,
               boolean privateTorrent,
               List<String> url_list,
               String comment,
               StorageListener listener,
               List<TorrentCreateFilter> filters)
        throws IOException
```
    Creates a storage from the existing file or directory. Creates an in-memory metainfo but does not save it to a file, caller must do that.
    Creates the metainfo, this may take a LONG time. BLOCKING.
    Parameters:
    announce - may be null
    listener - may be null
    created_by - may be null
    url_list - may be null
    comment - may be null
    Throws:
    IOException - when creating and/or checking files fails.
  - Storage
```
public Storage(I2PSnarkUtil util,
               File baseFile,
               String announce,
               List<List<String>> announce_list,
               String created_by,
               boolean privateTorrent,
               StorageListener listener,
               List<TorrentCreateFilter> filters)
        throws IOException
```
    Creates a storage from the existing file or directory. Creates an in-memory metainfo but does not save it to a file, caller must do that.
    Creates the metainfo, this may take a LONG time. BLOCKING.
    Parameters:
    announce - may be null
    listener - may be null
    created_by - may be null
    Throws:
    IOException - when creating and/or checking files fails.
- Method Detail
  - check
```
public void check()
           throws IOException
```
    Creates (and/or checks) all files from the metainfo file list. Only call this once, and only after the constructor with the metainfo. Use recheck() to check again later.
    Throws:
    IllegalStateException - if called more than once
    IOException
  - check
```
public void check(long savedTime,
                  BitField savedBitField)
           throws IOException
```
    Creates (and/or checks) all files from the metainfo file list. Use a saved bitfield and timestamp from a config file. Only call this once, and only after the constructor with the metainfo. Use recheck() to check again later.
    Throws:
    IllegalStateException - if called more than once
    IOException
  - cleanRAFs
```
public void cleanRAFs()
```
    Close unused RAFs - call periodically
  - clearChanged
```
void clearChanged()
```
    Clear the storage changed variable
  - close
```
public void close()
           throws IOException
```
    Closes the Storage and makes sure that all RandomAccessFiles are closed. The Storage is unusable after this.
    Specified by:
    close in interface Closeable
    Specified by:
    close in interface AutoCloseable
    Throws:
    IOException
  - complete
```
public boolean complete()
```
    Whether or not this storage contains all pieces if the MetaInfo.
  - countNonPad
```
static int countNonPad(File[] files)
```
    Count the entries in a directory listing for the max files per torrent check, excluding padding directories.
    Parameters:
    files - directory listing, may be null
    Returns:
    the number of non-padding entries
  - deleteStagingData
```
public void deleteStagingData()
```
    Removes all partially downloaded data from the staging directory for this torrent. No-op when the staging feature is disabled.
  - filterName
```
public static String filterName(String name)
```
    Removes 'suspicious' characters from the given file name. http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx Then replace chars not supported in the charset.
    This is called frequently and it can be pretty slow so cache the result.
    TODO: If multiple files in the same torrent map to the same filter name, the whole torrent will blow up. Check at torrent creation?
  - getActivity
```
public long getActivity()
```
    The last activity timestamp.
    Returns:
    timestamp in milliseconds since epoch, or 0 if never set
  - getBase
```
public File getBase()
```
    The base file or directory.
    Returns:
    the File
  - getBaseName
```
public String getBaseName()
```
    The base file or directory name of the data, as specified in the .torrent file, but filtered to remove illegal characters. This is where the data actually is, relative to the snark base dir.
    Returns:
    the base name
  - getBitField
```
public BitField getBitField()
```
    The BitField that tells which pieces this storage contains. Do not change this since this is the current state of the storage.
    Returns:
    the bit field
  - getCheckingProgress
```
public double getCheckingProgress()
```
    If checking is in progress, return completion 0.0 ... 1.0, else return 1.0.
    Returns:
    the checking progress
  - getDirectories
```
public SortedSet<File> getDirectories()
```
    Includes the base for a multi-file torrent. Sorted bottom-up for easy deletion. Slow. Use for deletion only.
    Returns:
    a new Set or null for a single-file torrent
  - getExcludedFiles
```
public List<String> getExcludedFiles(File base)
```
    The excluded files.
    Returns:
    the excluded files
  - getFileCount
```
public int getFileCount()
```
    Does not include directories.
    Returns:
    the file count
  - getFilePriorities
```
public int[] getFilePriorities()
```
    The file priorities array.
    Returns:
    null on error, if complete, or if only one file
  - getFiles
```
public List<File> getFiles()
```
    Does not include directories. Unsorted.
    In staging mode, returns the final data-directory location of each file, even while the data is still being written to the staging dir.
    Returns:
    a new List
  - getInOrder
```
public boolean getInOrder()
```
    Whether in-order download mode is enabled.
    Returns:
    whether in-order download mode is enabled
  - getMetaInfo
```
public MetaInfo getMetaInfo()
```
    Returns the MetaInfo associated with this Storage.
  - getPiece
```
public ByteArray getPiece(int piece,
                          int off,
                          int len)
                   throws IOException
```
    Returns a byte array containing a portion of the requested piece or null if the storage doesn't contain the piece yet.
    Returns:
    the piece
    Throws:
    IOException
  - getPiecePriorities
```
public int[] getPiecePriorities()
```
    Call setPriority() for all changed files first, then call this. Set the piece priority to the highest priority of all files spanning the piece. Caller must pass array to the PeerCoordinator.
    Returns:
    null on error, if complete, or if only one file and inOrder not set.
  - getPreserveFileNames
```
public boolean getPreserveFileNames()
```
    Whether original file names are being preserved, false if filtered.
    Returns:
    true if original file names are being preserved, false if filtered
  - getPriority
```
public int getPriority(int fileIndex)
```
    The priority of the file at the index.
    Parameters:
    fileIndex - as obtained from indexOf
    Returns:
    the priority
  - getSkippedLength
```
public long getSkippedLength()
```
    Call setPriority() for all changed files first, then call this. The length of all the pieces that are not yet downloaded, and are set to skipped. This is not the same as the total of all skipped files, since pieces may span multiple files.
    Returns:
    0 on error, if complete, or if only one file
  - getStagingBase
```
static File getStagingBase(String tempDir,
                           String baseName,
                           MetaInfo metainfo)
```
    The staging directory for a torrent: the configured temp dir plus a subdirectory unique to the torrent. Null when the feature is disabled or the infohash is unavailable.
    Parameters:
    tempDir - the configured staging directory, or null
    baseName - the torrent's base name
    metainfo - the torrent
    Returns:
    the per-torrent staging dir, or null
  - getStagingDir
```
public File getStagingDir()
```
    The directory this torrent's incomplete files are staged in while downloading, or null when the staging feature is disabled. Paths inside it mirror the data-directory layout of the torrent.
  - getVerifyThreads
```
static int getVerifyThreads(I2PAppContext ctx)
```
    Worker threads for a parallel piece verification: the i2psnark.verifyThreads property when set, otherwise scaled to the CPU count (at least 4).
  - indexOf
```
public int indexOf(File file)
```
    Index to pass to remaining(), getPriority(), setPriority()
    Parameters:
    file - non-canonical path (non-directory)
    Returns:
    internal index of file; -1 if unknown file
  - isAllocating
```
public boolean isAllocating()
```
    Disk allocation (ballooning) in progress. Always false on Windows.
    Returns:
    whether allocating
  - isChanged
```
public boolean isChanged()
```
    Has the storage changed since instantiation?
    Returns:
    whether changed
  - isChecking
```
public boolean isChecking()
```
    File checking in progress.
    Returns:
    whether checking
  - isPadDir
```
static boolean isPadDir(String name)
```
    Is this a padding directory (BEP 47 ".pad" or a parse-renamed "_pad")? Padding entries hold only synthetic zero files: they must not count toward the max files per torrent limit, nor be included as data when creating a torrent.
    Parameters:
    name - directory name
    Returns:
    true if it is a padding directory
  - listMerged
```
public File[] listMerged(String pathInTorrent)
```
    Lists a directory within the torrent, merging the staging tree while the download is incomplete, so completed files already moved to the data directory and files still staged in the temp dir both appear. Entries are deduplicated by name, data-directory entries first.
    Parameters:
    pathInTorrent - the directory path within the torrent, "/" or empty for the root
    Returns:
    the merged children, or null if neither tree contains the directory
  - main
```
public static void main(String[] args)
```
    Create a metainfo. Used in the installer build process; do not comment out.
  - needed
```
public int needed()
```
    How many pieces are still missing from this storage.
  - noDataMessage
```
static String noDataMessage(File base,
                            List<String> unreadable)
```
    The message for a failed torrent creation with no readable data, so a permission problem is reported as such rather than as an empty folder.
    Parameters:
    base - the data folder or file
    unreadable - paths that exist but could not be read, may be empty
    Returns:
    the message
  - putPiece
```
public boolean putPiece(PartialPiece pp)
                 throws IOException
```
    Put the piece in the Storage if it is correct. Warning - takes a LONG time if complete as it does the recheck here. TODO thread the recheck?
    Returns:
    true if the piece was correct (sha metainfo hash matches), otherwise false.
    Throws:
    IOException - when some storage related error occurs.
  - recheck
```
public boolean recheck()
                throws IOException
```
    Blocking. Holds lock. Recommend running only when stopped. Caller should thread. Calls listener.setWantedPieces() on completion if anything changed.
    Returns:
    true if anything changed, false otherwise
    Throws:
    IOException
  - remaining
```
public long[] remaining()
```
    For efficiency, calculate remaining bytes for all files at once
    Returns:
    number of bytes remaining for each file, use indexOf() to get index for a file
  - remaining2
```
public long[][] remaining2()
```
    For efficiency, calculate remaining bytes for all files at once. Remaining bytes is rv[0]. Preview bytes is rv[1].
    Returns:
    number of bytes remaining and number of bytes available for a preview for each file, use indexOf() to get index for a file
  - reopen
```
public void reopen()
            throws IOException
```
    Doesn't really reopen the file descriptors for a restart. Just does an existence check but no length check or data reverification
    Throws:
    IOException - on fail
  - setActivity
```
public void setActivity(long time)
```
    Store the activity timestamp.
    Parameters:
    time - timestamp in milliseconds since epoch
  - setFilePriorities
```
void setFilePriorities(int[] p)
```
    The file priorities array. Only call this when stopped, but after check()
    Parameters:
    p - may be null
  - setInOrder
```
public void setInOrder(boolean yes)
```
    Enable or disable in-order download mode. When enabled, pieces within each file are prioritized sequentially. Must call setFilePriorities() BEFORE this method.
    Parameters:
    yes - true to enable in-order mode, false to disable
  - setPriority
```
public void setPriority(int fileIndex,
                        int pri)
```
    Must call Snark.updatePiecePriorities() (which calls getPiecePriorities()) after calling this.
    Parameters:
    fileIndex - as obtained from indexOf
    pri - default 0; <0 to disable

Class Storage

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

MAX_PIECE_SIZE

MAX_PIECES

MAX_TOTAL_SIZE

PAD_DIR

PRIORITY_NORMAL

PRIORITY_SKIP

Constructor Detail

Storage

Storage

Storage

Method Detail

check

check

cleanRAFs

clearChanged

close

complete

countNonPad

deleteStagingData

filterName

getActivity

getBase

getBaseName

getBitField

getCheckingProgress

getDirectories

getExcludedFiles

getFileCount

getFilePriorities

getFiles

getInOrder

getMetaInfo

getPiece

getPiecePriorities

getPreserveFileNames

getPriority

getSkippedLength

getStagingBase

getStagingDir

getVerifyThreads

indexOf

isAllocating

isChanged

isChecking

isPadDir

listMerged

main

needed

noDataMessage

putPiece

recheck

remaining

remaining2

reopen

setActivity

setFilePriorities

setInOrder

setPriority