is SearchComponent the correct way?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

is SearchComponent the correct way?

John Thorhauer
We have a need to check the results of a search against a set of security
lists that are maintained in a redis cache.  I need to be able to take each
document that is returned for a search and check the redis cache to see if
the document should be displayed or not.

I am attempting to do this by creating a SearchComponent.  I am able to
iterate thru the results and identify the items I want to remove from the
results but I am not sure how to proceed in removing them.

Is SearchComponent the best way to do this?  If so, any thoughts on how to
proceed?


Thanks,
John Thorhauer
Reply | Threaded
Open this post in threaded view
|

Re: is SearchComponent the correct way?

Mikhail Khludnev-2
It should be postfilter
https://lucidworks.com/2017/11/27/caching-and-filters-and-post-filters/, I
believe.


On Tue, Nov 6, 2018 at 2:24 PM John Thorhauer <[hidden email]>
wrote:

> We have a need to check the results of a search against a set of security
> lists that are maintained in a redis cache.  I need to be able to take each
> document that is returned for a search and check the redis cache to see if
> the document should be displayed or not.
>
> I am attempting to do this by creating a SearchComponent.  I am able to
> iterate thru the results and identify the items I want to remove from the
> results but I am not sure how to proceed in removing them.
>
> Is SearchComponent the best way to do this?  If so, any thoughts on how to
> proceed?
>
>
> Thanks,
> John Thorhauer
>


--
Sincerely yours
Mikhail Khludnev
Reply | Threaded
Open this post in threaded view
|

Re: is SearchComponent the correct way?

John Thorhauer
In reply to this post by John Thorhauer
Mikhail,

Thanks for the suggestion.  After looking over the PostFilter interface and
the DelegatingCollector, it appears that this would require me to query my
outside datastore (redis) for security information once for each document.
This would be a big performance issue.  I would like to be able to iterate
through the documents, gathering all the critical ID's and then send a
single query to redis, getting back my security related data, and then
iterate through the documents, pulling out the ones that the user should
not see.

Is this possible?

Thanks again for your help!
John


On Tue, Nov 6, 2018 at 6:24 AM John Thorhauer <[hidden email]>
wrote:

> We have a need to check the results of a search against a set of security
> lists that are maintained in a redis cache.  I need to be able to take each
> document that is returned for a search and check the redis cache to see if
> the document should be displayed or not.
>
> I am attempting to do this by creating a SearchComponent.  I am able to
> iterate thru the results and identify the items I want to remove from the
> results but I am not sure how to proceed in removing them.
>
> Is SearchComponent the best way to do this?  If so, any thoughts on how to
> proceed?
>
>
> Thanks,
> John Thorhauer
>
>

--
John Thorhauer
Vice President, Software Development
Yakabod, Inc.
Cell: 240-818-9050
Office: 301-662-4554 x2105
Reply | Threaded
Open this post in threaded view
|

Re: is SearchComponent the correct way?

Mikhail Khludnev-2
Not really. It expect to work segment by segment. So it can buffer all doc
from one segment, hit redis and push all results into delegating collector.

On Tue, Nov 6, 2018 at 8:29 PM John Thorhauer <[hidden email]>
wrote:

> Mikhail,
>
> Thanks for the suggestion.  After looking over the PostFilter interface and
> the DelegatingCollector, it appears that this would require me to query my
> outside datastore (redis) for security information once for each document.
> This would be a big performance issue.  I would like to be able to iterate
> through the documents, gathering all the critical ID's and then send a
> single query to redis, getting back my security related data, and then
> iterate through the documents, pulling out the ones that the user should
> not see.
>
> Is this possible?
>
> Thanks again for your help!
> John
>
>
> On Tue, Nov 6, 2018 at 6:24 AM John Thorhauer <[hidden email]>
> wrote:
>
> > We have a need to check the results of a search against a set of security
> > lists that are maintained in a redis cache.  I need to be able to take
> each
> > document that is returned for a search and check the redis cache to see
> if
> > the document should be displayed or not.
> >
> > I am attempting to do this by creating a SearchComponent.  I am able to
> > iterate thru the results and identify the items I want to remove from the
> > results but I am not sure how to proceed in removing them.
> >
> > Is SearchComponent the best way to do this?  If so, any thoughts on how
> to
> > proceed?
> >
> >
> > Thanks,
> > John Thorhauer
> >
> >
>
> --
> John Thorhauer
> Vice President, Software Development
> Yakabod, Inc.
> Cell: 240-818-9050
> Office: 301-662-4554 x2105
>


--
Sincerely yours
Mikhail Khludnev
Reply | Threaded
Open this post in threaded view
|

Re: is SearchComponent the correct way?

John Thorhauer
Mikhail,

Where do I implement the buffering?  I can not do it in then collect()
method.  I can not see how I can get access to what I need in the finish()
method.

Thanks,
John

On Tue, Nov 6, 2018 at 12:44 PM Mikhail Khludnev <[hidden email]> wrote:

> Not really. It expect to work segment by segment. So it can buffer all doc
> from one segment, hit redis and push all results into delegating collector.
>
> On Tue, Nov 6, 2018 at 8:29 PM John Thorhauer <[hidden email]>
> wrote:
>
> > Mikhail,
> >
> > Thanks for the suggestion.  After looking over the PostFilter interface
> and
> > the DelegatingCollector, it appears that this would require me to query
> my
> > outside datastore (redis) for security information once for each
> document.
> > This would be a big performance issue.  I would like to be able to
> iterate
> > through the documents, gathering all the critical ID's and then send a
> > single query to redis, getting back my security related data, and then
> > iterate through the documents, pulling out the ones that the user should
> > not see.
> >
> > Is this possible?
> >
> > Thanks again for your help!
> > John
> >
> >
> > On Tue, Nov 6, 2018 at 6:24 AM John Thorhauer <[hidden email]>
> > wrote:
> >
> > > We have a need to check the results of a search against a set of
> security
> > > lists that are maintained in a redis cache.  I need to be able to take
> > each
> > > document that is returned for a search and check the redis cache to see
> > if
> > > the document should be displayed or not.
> > >
> > > I am attempting to do this by creating a SearchComponent.  I am able to
> > > iterate thru the results and identify the items I want to remove from
> the
> > > results but I am not sure how to proceed in removing them.
> > >
> > > Is SearchComponent the best way to do this?  If so, any thoughts on how
> > to
> > > proceed?
> > >
> > >
> > > Thanks,
> > > John Thorhauer
> > >
> > >
> >
> > --
> > John Thorhauer
> > Vice President, Software Development
> > Yakabod, Inc.
> > Cell: 240-818-9050
> > Office: 301-662-4554 x2105
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


--
John Thorhauer
Vice President, Software Development
Yakabod, Inc.
Cell: 240-818-9050
Office: 301-662-4554 x2105
Reply | Threaded
Open this post in threaded view
|

Re: is SearchComponent the correct way?

Mikhail Khludnev-2
On Tue, Nov 13, 2018 at 6:36 AM John Thorhauer <[hidden email]>
wrote:

> Mikhail,
>
> Where do I implement the buffering?  I can not do it in then collect()
> method.

Please clarify why exactly? Notice my statement about one segment only.


> I can not see how I can get access to what I need in the finish()
> method.
>
> Thanks,
> John
>
> On Tue, Nov 6, 2018 at 12:44 PM Mikhail Khludnev <[hidden email]> wrote:
>
> > Not really. It expect to work segment by segment. So it can buffer all
> doc
> > from one segment, hit redis and push all results into delegating
> collector.
> >
> > On Tue, Nov 6, 2018 at 8:29 PM John Thorhauer <[hidden email]>
> > wrote:
> >
> > > Mikhail,
> > >
> > > Thanks for the suggestion.  After looking over the PostFilter interface
> > and
> > > the DelegatingCollector, it appears that this would require me to query
> > my
> > > outside datastore (redis) for security information once for each
> > document.
> > > This would be a big performance issue.  I would like to be able to
> > iterate
> > > through the documents, gathering all the critical ID's and then send a
> > > single query to redis, getting back my security related data, and then
> > > iterate through the documents, pulling out the ones that the user
> should
> > > not see.
> > >
> > > Is this possible?
> > >
> > > Thanks again for your help!
> > > John
> > >
> > >
> > > On Tue, Nov 6, 2018 at 6:24 AM John Thorhauer <[hidden email]>
> > > wrote:
> > >
> > > > We have a need to check the results of a search against a set of
> > security
> > > > lists that are maintained in a redis cache.  I need to be able to
> take
> > > each
> > > > document that is returned for a search and check the redis cache to
> see
> > > if
> > > > the document should be displayed or not.
> > > >
> > > > I am attempting to do this by creating a SearchComponent.  I am able
> to
> > > > iterate thru the results and identify the items I want to remove from
> > the
> > > > results but I am not sure how to proceed in removing them.
> > > >
> > > > Is SearchComponent the best way to do this?  If so, any thoughts on
> how
> > > to
> > > > proceed?
> > > >
> > > >
> > > > Thanks,
> > > > John Thorhauer
> > > >
> > > >
> > >
> > > --
> > > John Thorhauer
> > > Vice President, Software Development
> > > Yakabod, Inc.
> > > Cell: 240-818-9050
> > > Office: 301-662-4554 x2105
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
>
> --
> John Thorhauer
> Vice President, Software Development
> Yakabod, Inc.
> Cell: 240-818-9050
> Office: 301-662-4554 x2105
>


--
Sincerely yours
Mikhail Khludnev