Indexing documents from S3 bucket

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Indexing documents from S3 bucket

marotosg
Hi,

At the moment I have a SolrCloud Cluster with a documents collection being
populated indexing documents coming from a DFS server. Linux boxes are
mounting that DFS server using samba.

There is a request to move that DFS server to a AWS S3 bucket.
Does anyone have previous experience about it? Speed etc.

Thanks for the advice.




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Indexing documents from S3 bucket

☼ R Nair
S3 gives listeners. So tap those listeners when objects are added, updated
or deleted and use Solr API to push. That's high level, but I believe
doable.

I worked on Minio, an open source object storage supporting S3 and could do
this because Minio gives me good and stable listeners.

Best, Ravion

On Mon, Oct 8, 2018, 11:26 AM marotosg <[hidden email]> wrote:

> Hi,
>
> At the moment I have a SolrCloud Cluster with a documents collection being
> populated indexing documents coming from a DFS server. Linux boxes are
> mounting that DFS server using samba.
>
> There is a request to move that DFS server to a AWS S3 bucket.
> Does anyone have previous experience about it? Speed etc.
>
> Thanks for the advice.
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>