[jira] Resolved: (LUCENE-414) Java NIO patch against Lucene 1.9

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] Resolved: (LUCENE-414) Java NIO patch against Lucene 1.9

Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-414.

    Resolution: Duplicate
      Assignee:     (was: Lucene Developers)

I believe most of this is a dup of LUCENE-753, and/or separately incporated into Lucene already.

> Java NIO patch against Lucene 1.9
> ---------------------------------
>                 Key: LUCENE-414
>                 URL: https://issues.apache.org/jira/browse/LUCENE-414
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: unspecified
>         Environment: Operating System: All
> Platform: All
>            Reporter: Chris Lamprecht
>            Priority: Minor
>         Attachments: MemoryLRUCache.java, nio-lucene-1.9.patch, NioFile.java
> Robert Engels previously submitted a patch against Lucene 1.4 for a Java NIO-
> based Directory implementation.  It also included some changes to FSDirectory
> to allow better concurrency when searching from multiple threads.  The
> complete thread is at:
> <a href="http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200505.mbox/%">http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200505.mbox/%
> [hidden email]%3e
> This thread ended with Doug Cutting suggesting that someone port Robert's
> changes to the SVN trunk.  This is what I've done in this patch.
> There are two parts to the patch.  The first part modifies FieldsReader,
> CompoundFileReader, and SegmentReader, to allow better concurrency when
> reading an index.  The second part includes the new NioFSDirectory
> implementation, and makes small changes to FSDirectory and IndexInput to
> accomodate this change.  I'll put a more detailed outline of the changes to
> each file in a separate message.
> To use the new NioFSDirectory, set the system property
> org.apache.lucene.FSDirectory.class to
> org.apache.lucene.store.NioFSDirectory.  This will cause
> FSDirectory.getDirectory() to return an NioFSDirectory instance.  By default,
> NioFile limits the number of concurrent channels to 4, but you can override
> this by setting the system property org.apache.lucene.nio.channels.  
> I did some performance tests with these patches.  The biggest improvement came
> from the concurrency improvements.  NioFSDirectory performed about the same as
> FSDirectory (with the concurrency improvements).  
> I ran my tests under Fedora Core 1; uname -a reports:
> Linux myhost 2.4.22-1.2199.nptlsmp #1 SMP Wed Aug 4 11:48:29 EDT 2004 i686
> i686 i386 GNU/Linux
> The machine is a dual xeon 2.8GHz with 4GB RAM, and the tests were run against
> a 9GB compound index file.  The tests were run "hot" -- with everything
> already cached by linux's filesystem cache.  The numbers are:
> FSDirectory without patch:          13.3 searches per second
> FSDirectory WITH concurrency patch: 14.3 searches per second
> Both tests were run with 6 concurrent threads, which gave the highest numbers
> in each case.  I suspect that the concurrency improvements would make a bigger
> difference on a more realistic test where the index isn't all cached in RAM
> already, since the I/O happens whild holding the sychronized lock.  Patches to
> follow...
> Thoughts?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]