[lucy-user] C library RAMFolder

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[lucy-user] C library RAMFolder

serkanmulayim@gmail.com
Hi guys,

In my project I might need to have the index in memory. But I have some doubts about it considering that it might not be tested well. Would it be possible for you guys to address the following questions?

1- Is it stable :)?
2- Is there a limit (smaller than the file index) in the number of indexed documents.
3- Is the index always optimized considering that it is in memory.
4- How do the indexing and searching performances compared to the in disk implementation.
5- Is it possible to merge an index on disk to an index in memory?
6- Are there any concerns in using the in memory index?

Thanks,
Serkan
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] C library RAMFolder

Peter Karman
[hidden email] wrote on 10/30/17 3:49 PM:

> Hi guys,
>
> In my project I might need to have the index in memory. But I have some doubts about it considering that it might not be tested well. Would it be possible for you guys to address the following questions?
>
> 1- Is it stable :)?
> 2- Is there a limit (smaller than the file index) in the number of indexed documents.
> 3- Is the index always optimized considering that it is in memory.
> 4- How do the indexing and searching performances compared to the in disk implementation.
> 5- Is it possible to merge an index on disk to an index in memory?
> 6- Are there any concerns in using the in memory index?
>


I'm not going to answer your question directly, other than to say that
Lucy::Store::RAMFolder wasn't really designed for production use, and so I don't
know if things like merging were ever considered. Whether is meets your needs is
probably a try-it-and-see thing.

I should note that the entire architecture is built around mmap so you're
getting the benefits of in-memory OS-level performance with persistence on disk.
So without digging more deeply into why you "might need to have the index
memory" if it is assumption about performance, you might not see much gain by
using RAMFolder.


--
Peter Karman  .  https://karpet.github.io  .  https://keybase.io/peterkarman