Package org.apache.pdfbox.io

Class ScratchFile

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, RandomAccessStreamCache
    public class ScratchFile
extends java.lang.Object
implements RandomAccessStreamCache
    Implements a memory page handling mechanism as base for creating (multiple) RandomAccess buffers each having its set of pages (implemented by ScratchFileBuffer). A buffer is created calling createBuffer().

    Pages can be stored in main memory or in a temporary file. A mixed mode is supported storing a certain amount of pages in memory and only the additional ones in temporary file (defined by maximum main memory to be used).

    Pages can be marked as 'free' in order to re-use them. For in-memory pages this will release the used memory while for pages in temporary file this simply marks the area as free to re-use.

    If a temporary file was created (done with the first page to be stored in temporary file) it is deleted when close() is called.

    Using this class for RandomAccess buffers allows for a direct control on the maximum memory usage and allows processing large files for which we otherwise would get an OutOfMemoryError in case of using RandomAccessReadBuffer.

    This base class for providing pages is thread safe (the buffer implementations are not).

    • Field Detail

      • LOG

        private static final org.apache.commons.logging.Log LOG

      • ENLARGE_PAGE_COUNT

        private static final int ENLARGE_PAGE_COUNT
        number of pages by which we enlarge the scratch file (reduce I/O-operations)
        See Also:
        Constant Field Values

      • INIT_UNRESTRICTED_MAINMEM_PAGECOUNT

        private static final int INIT_UNRESTRICTED_MAINMEM_PAGECOUNT
        in case of unrestricted main memory usage this is the initial number of pages inMemoryPages is setup for
        See Also:
        Constant Field Values

      • ioLock

        private final java.lang.Object ioLock

      • scratchFileDirectory

        private final java.io.File scratchFileDirectory

      • file

        private java.io.File file
        scratch file; only to be accessed under synchronization of ioLock

      • raf

        private java.io.RandomAccessFile raf
        random access to scratch file; only to be accessed under synchronization of ioLock

      • pageCount

        private volatile int pageCount

      • freePages

        private final java.util.BitSet freePages

      • inMemoryPages

        private volatile byte[][] inMemoryPages
        holds pointers to in-memory page content; will be initialized once in case of restricted main memory, otherwise it is enlarged as needed and first initialized to a size of INIT_UNRESTRICTED_MAINMEM_PAGECOUNT

      • inMemoryMaxPageCount

        private final int inMemoryMaxPageCount

      • maxPageCount

        private final int maxPageCount

      • useScratchFile

        private final boolean useScratchFile

      • maxMainMemoryIsRestricted

        private final boolean maxMainMemoryIsRestricted

      • isClosed

        private volatile boolean isClosed

    • Constructor Detail

      • ScratchFile

        public ScratchFile​(java.io.File scratchFileDirectory)
            throws java.io.IOException
        Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

        All pages will be stored in the scratch file.

        Parameters:
        scratchFileDirectory - The directory in which to create the scratch file or null to created it in the default temporary directory.
        Throws:
        java.io.IOException - If scratch file directory was given but don't exist.

      • ScratchFile

        public ScratchFile​(MemoryUsageSetting memUsageSetting)
            throws java.io.IOException
        Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

        Depending on the size of allowed memory usage a number of pages (memorySize/PAGE_SIZE) will be stored in-memory and only additional pages will be written to/read from scratch file.

        Parameters:
        memUsageSetting - set how memory/temporary files are used for buffering streams etc.
        Throws:
        java.io.IOException - If scratch file directory was given but don't exist.

    • Method Detail

      • initPages

        private void initPages()

      • getMainMemoryOnlyInstance

        public static ScratchFile getMainMemoryOnlyInstance()
        Getter for an instance using only unrestricted main memory for buffering (same as new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())).
        Returns:
        instance configured to only use main memory with no size restriction

      • getMainMemoryOnlyInstance

        public static ScratchFile getMainMemoryOnlyInstance​(long maxMainMemoryBytes)
        Getter for an instance to only use main-memory with the defined maximum.
        Parameters:
        maxMainMemoryBytes - maximum number of main-memory to be used; -1 for no restriction; 0 will also be interpreted here as no restriction
        Returns:
        instance configured to only use main memory with no size restriction or null if cannot create scratch file

      • getNewPage

        int getNewPage()
        throws java.io.IOException
        Returns a new free page, either from free page pool or by enlarging scratch file (may be created).
        Returns:
        index of new page
        Throws:
        java.io.IOException

      • enlarge

        private void enlarge()
              throws java.io.IOException
        This will provide new free pages by either enlarging the scratch file by a number of pages defined by ENLARGE_PAGE_COUNT - in case scratch file usage is allowed - or increase the inMemoryPages array in case main memory was not restricted. If neither of both is allowed/the case than free pages count won't be changed. The same is true if no new pages could be added because we reached the maximum of Integer.MAX_VALUE pages.

        If scratch file usage is allowed and scratch file does not exist already it will be created.

        Only to be called under synchronization on freePages.

        Throws:
        java.io.IOException

      • getPageSize

        int getPageSize()
        Returns byte size of a page.
        Returns:
        byte size of a page

      • readPage

        byte[] readPage​(int pageIdx)
         throws java.io.IOException
        Reads the page with specified index.
        Parameters:
        pageIdx - index of page to read
        Returns:
        byte array of size PAGE_SIZE filled with page data read from file
        Throws:
        java.io.IOException

      • writePage

        void writePage​(int pageIdx,
               byte[] page)
        throws java.io.IOException
        Writes updated page. Page is either kept in-memory if pageIdx < inMemoryMaxPageCount or is written to scratch file.

        Provided page byte array must not be re-used for other pages since we store it as is in case of in-memory handling.

        Parameters:
        pageIdx - index of page to write
        page - page to write (length has to be 4096)
        Throws:
        java.io.IOException - in case page index is out of range or page has wrong length or writing to file failed

      • checkClosed

        void checkClosed()
          throws java.io.IOException
        Checks if this page handler has already been closed. If so, an IOException is thrown.
        Throws:
        java.io.IOException - If close() has already been called.

      • createBuffer

        public RandomAccess createBuffer()
                          throws java.io.IOException
        Creates a new buffer using this page handler.
        Specified by:
        createBuffer in interface RandomAccessStreamCache
        Returns:
        A new buffer.
        Throws:
        java.io.IOException - If an error occurred.

      • markPagesAsFree

        void markPagesAsFree​(int[] pageIndexes,
                     int off,
                     int count)
        Allows a buffer which is cleared/closed to release its pages to be re-used.
        Parameters:
        pageIndexes - pages indexes of pages to release
        count - number of page indexes contained in provided array

      • close

        public void close()
           throws java.io.IOException
        Closes and deletes the temporary file. No further interaction with the scratch file or associated buffers can happen after this method is called. It also releases in-memory pages.
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException - If there was a problem closing or deleting the temporary file.