in memory lucene

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

in memory lucene

Jonathan Willis
Hi, i'm looking into using Lucene 7.7.0 and noticed that the RAMDirectory
has been deprecated because of inefficient synchronization issues and that
we are encouraged to use MMapDirectory instead. I was hoping to use an in
memory only directory and was wondering if that would be possible without
RAMDirectory? How would I go about doing this?

Thank you
--
Jonathan Willis
Reply | Threaded
Open this post in threaded view
|

RE: in memory lucene

Uwe Schindler
Hi,

In Lucene 7.7 you can use ByteBuffersDirectory, the default constructor behaves like RAMDirectory (allocates on heap), but has much better concurrency and garbage collection behaviour (no millions of byte[8192] instances holding the data)
http://lucene.apache.org/core/7_7_0/core/org/apache/lucene/store/ByteBuffersDirectory.html

By using the complex constructor you can also make it behave funny things like allocation of DirectBuffers off-heap (using Java's ByteBuffer::allocateDirect method passed as reference to ByteBuffersDataOutput blockAllocate parameter) or similar stuff. But when doing this you are on your own.

Nevertheless, MMapDirectory on a Linux "tmpfs" filesystem is the better choice for most use cases.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Jonathan Willis <[hidden email]>
> Sent: Wednesday, February 27, 2019 12:19 AM
> To: [hidden email]
> Subject: in memory lucene
>
> Hi, i'm looking into using Lucene 7.7.0 and noticed that the RAMDirectory
> has been deprecated because of inefficient synchronization issues and that
> we are encouraged to use MMapDirectory instead. I was hoping to use an in
> memory only directory and was wondering if that would be possible without
> RAMDirectory? How would I go about doing this?
>
> Thank you
> --
> Jonathan Willis


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: in memory lucene

wmartinusa
In reply to this post by Jonathan Willis
One of the folks at OpenSource Connections put a lucene derivative on an Intel Edison (?) (tiny board computer; raspberry pi like) a couple of years ago. That project might have something to offer you in terms of ideas.

Sorry was only able to find a link for their project on same for Cassandra cluster. Another interesting search result link snippet was from  eClubPrague.

-----Original Message-----
From: Jonathan Willis <[hidden email]>
Sent: Tuesday, February 26, 2019 6:19 PM
To: [hidden email]
Subject: in memory lucene

Hi, i'm looking into using Lucene 7.7.0 and noticed that the RAMDirectory has been deprecated because of inefficient synchronization issues and that we are encouraged to use MMapDirectory instead. I was hoping to use an in memory only directory and was wondering if that would be possible without RAMDirectory? How would I go about doing this?

Thank you
--
Jonathan Willis


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]