Package org.apache.lucene.store

Class FSDirectory

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable
    Direct Known Subclasses:
    MMapDirectory, NativeUnixDirectory, NIOFSDirectory, SimpleFSDirectory, WindowsDirectory
    public abstract class FSDirectory
extends BaseDirectory
    Base class for Directory implementations that store index files in the file system. There are currently three core subclasses:
    • SimpleFSDirectory is a straightforward implementation using java.io.RandomAccessFile. However, it has poor concurrent performance (multiple threads will bottleneck) as it synchronizes when multiple threads read from the same file.
    • NIOFSDirectory uses java.nio's FileChannel's positional io when reading to avoid synchronization when reading from the same file. Unfortunately, due to a Windows-only Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using Thread.interrupt() or Future.cancel(boolean) should use SimpleFSDirectory instead. See NIOFSDirectory java doc for details.
    • MMapDirectory uses memory-mapped IO when reading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's IndexInput.close() is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using Thread.interrupt() or Future.cancel(boolean) should use SimpleFSDirectory instead. See MMapDirectory java doc for details.
    Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the open(java.io.File) method, to allow Lucene to choose the best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use open(java.io.File). For all others, you should instantiate the desired implementation directly.

    The locking implementation is by default NativeFSLockFactory, but can be changed by passing in a custom LockFactory instance.

    See Also:
    Directory

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static int DEFAULT_READ_CHUNK_SIZE
      Deprecated.
      This constant is no longer used since Lucene 4.5.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      void close()
      Closes the store to future operations.
      IndexOutput createOutput​(java.lang.String name, IOContext context)
      Creates an IndexOutput for the file with the given name.
      void deleteFile​(java.lang.String name)
      Removes an existing file in the directory.
      boolean fileExists​(java.lang.String name)
      Returns true iff a file with the given name exists.
      long fileLength​(java.lang.String name)
      Returns the length in bytes of a file in the directory.
      java.io.File getDirectory()  
      java.lang.String getLockID()
      Return a string identifier that uniquely differentiates this Directory instance from other Directory instances.
      int getReadChunkSize()
      Deprecated.
      This is no longer used since Lucene 4.5.
      java.lang.String[] listAll()
      Lists all files (not subdirectories) in the directory.
      static java.lang.String[] listAll​(java.io.File dir)
      Lists all files (not subdirectories) in the directory.
      static FSDirectory open​(java.io.File path)
      Creates an FSDirectory instance, trying to pick the best implementation given the current environment.
      static FSDirectory open​(java.io.File path, LockFactory lockFactory)
      Just like open(File), but allows you to also specify a custom LockFactory.
      void setLockFactory​(LockFactory lockFactory)
      Set the LockFactory that this Directory instance should use for its locking implementation.
      void setReadChunkSize​(int chunkSize)
      Deprecated.
      This is no longer used since Lucene 4.5.
      void sync​(java.util.Collection<java.lang.String> names)
      Ensure that any writes to these files are moved to stable storage.
      java.lang.String toString()
      For debug output.

      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Field Detail

      • DEFAULT_READ_CHUNK_SIZE

        @Deprecated
public static final int DEFAULT_READ_CHUNK_SIZE
        Deprecated.
        This constant is no longer used since Lucene 4.5.
        Default read chunk size: 8192 bytes (this is the size up to which the JDK does not allocate additional arrays while reading/writing)
        See Also:
        Constant Field Values

    • Method Detail

      • open

        public static FSDirectory open​(java.io.File path)
                        throws java.io.IOException
        Creates an FSDirectory instance, trying to pick the best implementation given the current environment. The directory returned uses the NativeFSLockFactory.

        Currently this returns MMapDirectory for most Solaris and Windows 64-bit JREs, NIOFSDirectory for other non-Windows JREs, and SimpleFSDirectory for other JREs on Windows. It is highly recommended that you consult the implementation's documentation for your platform before using this method.

        NOTE: this method may suddenly change which implementation is returned from release to release, in the event that higher performance defaults become possible; if the precise implementation is important to your application, please instantiate it directly, instead. For optimal performance you should consider using MMapDirectory on 64 bit JVMs.

        See above

        Throws:
        java.io.IOException

      • open

        public static FSDirectory open​(java.io.File path,
                               LockFactory lockFactory)
                        throws java.io.IOException
        Just like open(File), but allows you to also specify a custom LockFactory.
        Throws:
        java.io.IOException

      • setLockFactory

        public void setLockFactory​(LockFactory lockFactory)
                    throws java.io.IOException
        Description copied from class: Directory
        Set the LockFactory that this Directory instance should use for its locking implementation. Each * instance of LockFactory should only be used for one directory (ie, do not share a single instance across multiple Directories).
        Overrides:
        setLockFactory in class BaseDirectory
        Parameters:
        lockFactory - instance of LockFactory.
        Throws:
        java.io.IOException

      • listAll

        public static java.lang.String[] listAll​(java.io.File dir)
                                  throws java.io.IOException
        Lists all files (not subdirectories) in the directory. This method never returns null (throws IOException instead).
        Throws:
        NoSuchDirectoryException - if the directory does not exist, or does exist but is not a directory.
        java.io.IOException - if list() returns null

      • fileExists

        public boolean fileExists​(java.lang.String name)
        Returns true iff a file with the given name exists.
        Specified by:
        fileExists in class Directory

      • fileLength

        public long fileLength​(java.lang.String name)
                throws java.io.IOException
        Returns the length in bytes of a file in the directory.
        Specified by:
        fileLength in class Directory
        Parameters:
        name - the name of the file for which to return the length.
        Throws:
        java.io.IOException - if there was an IO error while retrieving the file's length.

      • deleteFile

        public void deleteFile​(java.lang.String name)
                throws java.io.IOException
        Removes an existing file in the directory.
        Specified by:
        deleteFile in class Directory
        Throws:
        java.io.IOException

      • createOutput

        public IndexOutput createOutput​(java.lang.String name,
                                IOContext context)
                         throws java.io.IOException
        Creates an IndexOutput for the file with the given name.
        Specified by:
        createOutput in class Directory
        Throws:
        java.io.IOException

      • sync

        public void sync​(java.util.Collection<java.lang.String> names)
          throws java.io.IOException
        Description copied from class: Directory
        Ensure that any writes to these files are moved to stable storage. Lucene uses this to properly commit changes to the index, to prevent a machine/OS crash from corrupting the index.

        NOTE: Clients may call this method for same files over and over again, so some impls might optimize for that. For other impls the operation can be a noop, for various reasons.
        Specified by:
        sync in class Directory
        Throws:
        java.io.IOException

      • getLockID

        public java.lang.String getLockID()
        Description copied from class: Directory
        Return a string identifier that uniquely differentiates this Directory instance from other Directory instances. This ID should be the same if two Directory instances (even in different JVMs and/or on different machines) are considered "the same index". This is how locking "scopes" to the right index.
        Overrides:
        getLockID in class Directory

      • close

        public void close()
        Closes the store to future operations.
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in class Directory

      • getDirectory

        public java.io.File getDirectory()
        Returns:
        the underlying filesystem directory

      • toString

        public java.lang.String toString()
        For debug output.
        Overrides:
        toString in class Directory

      • setReadChunkSize

        @Deprecated
public final void setReadChunkSize​(int chunkSize)
        Deprecated.
        This is no longer used since Lucene 4.5.
        This setting has no effect anymore.

      • getReadChunkSize

        @Deprecated
public final int getReadChunkSize()
        Deprecated.
        This is no longer used since Lucene 4.5.
        This setting has no effect anymore.