Get results in multiple orders (multiple boosts)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Get results in multiple orders (multiple boosts)

Luca Dall'Osto
Hello,
I'm new in Solr (and in mailing lists..), and I have a question about querying contents in multiple custom orders.
I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to make a search of 2 day and return results boosted by category field, like this:

?indent=on
&defType=edismax
&q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
&bq=category:9500^2
&bq=category:1100^1
&rows=40
&wt=jsonThis will return all documents of category 9500 first, and 1100 in after.Now I would like to get this documents with a second boost based on another field, called source.I would like to have documents in this order:1) category:9500 AND source:5
2) category:9500 AND source:9
3) category:9500 AND source:7
4) category:1100 AND source:5
5) category:1100 AND source:9
6) category:1100 AND source:7
To get this order, I tied with this query:?indent=on
&defType=edismax
&q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
&bq=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
&bq=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
&rows=40
&wt=json
How can I apply a double boosts to get the documents in my correct order? Is boost the correct tool for my purpose?Any help will be greatly appreciated. Thanks
Luca
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: Get results in multiple orders (multiple boosts)

Florian Waltersdorfer
Hi,

I am quite the SolR newbie myself, but have you looked at the resulting scores, e.g. via fl=*,score (that way, you can see/test how your boosting affects the results)?
In a similar scenario, I am using fixed value boosts for specific field values; "^=[boost]" instead of "^[factor]", for example:

category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)

(Actual fixed values open for experimentation.)

Regards,
Florian

-----Ursprüngliche Nachricht-----
Von: Luca Dall'Osto [mailto:[hidden email]]
Gesendet: Montag, 17. Juli 2017 12:20
An: [hidden email]
Betreff: Get results in multiple orders (multiple boosts)

Hello,
I'm new in Solr (and in mailing lists..), and I have a question about querying contents in multiple custom orders.
I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to make a search of 2 day and return results boosted by category field, like this:

?indent=on
&defType=edismax
&q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
&bq=category:9500^2
&bq=category:1100^1
&rows=40
&wt=jsonThis will return all documents of category 9500 first, and 1100 in after.Now I would like to get this documents with a second boost based on another field, called source.I would like to have documents in this order:1) category:9500 AND source:5
2) category:9500 AND source:9
3) category:9500 AND source:7
4) category:1100 AND source:5
5) category:1100 AND source:9
6) category:1100 AND source:7
To get this order, I tied with this query:?indent=on &defType=edismax &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
&bq=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
&bq=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
&rows=40
&wt=json
How can I apply a double boosts to get the documents in my correct order? Is boost the correct tool for my purpose?Any help will be greatly appreciated. Thanks Luca
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Erick Erickson
I don't think boosting is really what you want here. Boosting
_influences_ the score, it does not impose an ordering.

Sorting _does_ impose an ordering, the question is how to sort and the
answer depends on how fixed (or not) the sorting criteria are. Do they
change with different queries? If not, the very simplest thing to do
is to index a field with a pre-computed sort value. IOW, if your
ordering is _always_ source 5, 9, 7 index a source_sort field that
orders things that way and sort on that. Then I'd have a secondary
sort by score as a tie-breaker.

If that's not the case, perhaps sorting by function (perhaps a custom
function) would work.

Best,
Erick

On Mon, Jul 17, 2017 at 4:30 AM, Florian Waltersdorfer
<[hidden email]> wrote:

> Hi,
>
> I am quite the SolR newbie myself, but have you looked at the resulting scores, e.g. via fl=*,score (that way, you can see/test how your boosting affects the results)?
> In a similar scenario, I am using fixed value boosts for specific field values; "^=[boost]" instead of "^[factor]", for example:
>
> category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)
>
> (Actual fixed values open for experimentation.)
>
> Regards,
> Florian
>
> -----Ursprüngliche Nachricht-----
> Von: Luca Dall'Osto [mailto:[hidden email]]
> Gesendet: Montag, 17. Juli 2017 12:20
> An: [hidden email]
> Betreff: Get results in multiple orders (multiple boosts)
>
> Hello,
> I'm new in Solr (and in mailing lists..), and I have a question about querying contents in multiple custom orders.
> I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to make a search of 2 day and return results boosted by category field, like this:
>
> ?indent=on
> &defType=edismax
> &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> &bq=category:9500^2
> &bq=category:1100^1
> &rows=40
> &wt=jsonThis will return all documents of category 9500 first, and 1100 in after.Now I would like to get this documents with a second boost based on another field, called source.I would like to have documents in this order:1) category:9500 AND source:5
> 2) category:9500 AND source:9
> 3) category:9500 AND source:7
> 4) category:1100 AND source:5
> 5) category:1100 AND source:9
> 6) category:1100 AND source:7
> To get this order, I tied with this query:?indent=on &defType=edismax &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> &bq=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
> &bq=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
> &rows=40
> &wt=json
> How can I apply a double boosts to get the documents in my correct order? Is boost the correct tool for my purpose?Any help will be greatly appreciated. Thanks Luca
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Luca Dall'Osto
In reply to this post by Luca Dall'Osto
Hello everyone,thanks for the prompt reply!
In response to Florian, I can get correct score only when boost for 1 filed (for example category): the score are correctly increased by the factor.But when I try to make a double boost, the score are not great as expected (for example, if the greatest boost factor for category is 3 and for source is 3,  sometimes I've got documents boosted with category^3 and source^1 **before** documents boosted with category^3 and source^3).
I tired you snippet code "^=[boost]" instead of "^[factor]" but seems not work for me: SyntaxError: Cannot parse 'category:9500^': Encountered \"<EOF>\" at line 1 (...)
In response to Erik, I have different "sort preferences", so I can't build a index and use for sorting.Maybe I have to sort by category then by source and by language or by source, then by category and by date.The sort function that you are talking is a custom sort function of Solr?I build a custom PHP method that sort documents queried from Solr and it works good: the problem is that if I choose this way I have to get **all** the results in a single query and I can't paginate.Today I have millions of record and query could be very large, I would like to paginate results.To paginate results I need that Solr give me the results query in the correct order.Thank you very much

Luca

 

    On Monday, July 17, 2017 6:04 PM, Erick Erickson <[hidden email]> wrote:
 

 I don't think boosting is really what you want here. Boosting
_influences_ the score, it does not impose an ordering.

Sorting _does_ impose an ordering, the question is how to sort and the
answer depends on how fixed (or not) the sorting criteria are. Do they
change with different queries? If not, the very simplest thing to do
is to index a field with a pre-computed sort value. IOW, if your
ordering is _always_ source 5, 9, 7 index a source_sort field that
orders things that way and sort on that. Then I'd have a secondary
sort by score as a tie-breaker.

If that's not the case, perhaps sorting by function (perhaps a custom
function) would work.

Best,
Erick

On Mon, Jul 17, 2017 at 4:30 AM, Florian Waltersdorfer
<[hidden email]> wrote:

> Hi,
>
> I am quite the SolR newbie myself, but have you looked at the resulting scores, e.g. via fl=*,score (that way, you can see/test how your boosting affects the results)?
> In a similar scenario, I am using fixed value boosts for specific field values; "^=[boost]" instead of "^[factor]", for example:
>
> category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)
>
> (Actual fixed values open for experimentation.)
>
> Regards,
> Florian
>
> -----Ursprüngliche Nachricht-----
> Von: Luca Dall'Osto [mailto:[hidden email]]
> Gesendet: Montag, 17. Juli 2017 12:20
> An: [hidden email]
> Betreff: Get results in multiple orders (multiple boosts)
>
> Hello,
> I'm new in Solr (and in mailing lists..), and I have a question about querying contents in multiple custom orders.
> I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to make a search of 2 day and return results boosted by category field, like this:
>
> ?indent=on
> &defType=edismax
> &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> &bq=category:9500^2
> &bq=category:1100^1
> &rows=40
> &wt=jsonThis will return all documents of category 9500 first, and 1100 in after.Now I would like to get this documents with a second boost based on another field, called source.I would like to have documents in this order:1) category:9500 AND source:5
> 2) category:9500 AND source:9
> 3) category:9500 AND source:7
> 4) category:1100 AND source:5
> 5) category:1100 AND source:9
> 6) category:1100 AND source:7
> To get this order, I tied with this query:?indent=on &defType=edismax &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> &bq=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
> &bq=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
> &rows=40
> &wt=json
> How can I apply a double boosts to get the documents in my correct order? Is boost the correct tool for my purpose?Any help will be greatly appreciated. Thanks Luca

   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Susheel Kumar-3
As Eric suggested, its possible by sorting using custom function. You may
have to use if, sum and exists function etc. to come up with custom score
field and sort using this field. The if condition would check for the
conditions mentioned and keep adding the score etc.

Thanks,
Susheel

On Tue, Jul 18, 2017 at 6:23 AM, Luca Dall'Osto <
[hidden email]> wrote:

> Hello everyone,thanks for the prompt reply!
> In response to Florian, I can get correct score only when boost for 1
> filed (for example category): the score are correctly increased by the
> factor.But when I try to make a double boost, the score are not great as
> expected (for example, if the greatest boost factor for category is 3 and
> for source is 3,  sometimes I've got documents boosted with category^3 and
> source^1 **before** documents boosted with category^3 and source^3).
> I tired you snippet code "^=[boost]" instead of "^[factor]" but seems not
> work for me: SyntaxError: Cannot parse 'category:9500^': Encountered
> \"<EOF>\" at line 1 (...)
> In response to Erik, I have different "sort preferences", so I can't build
> a index and use for sorting.Maybe I have to sort by category then by source
> and by language or by source, then by category and by date.The sort
> function that you are talking is a custom sort function of Solr?I build a
> custom PHP method that sort documents queried from Solr and it works good:
> the problem is that if I choose this way I have to get **all** the results
> in a single query and I can't paginate.Today I have millions of record and
> query could be very large, I would like to paginate results.To paginate
> results I need that Solr give me the results query in the correct
> order.Thank you very much
>
> Luca
>
>
>
>     On Monday, July 17, 2017 6:04 PM, Erick Erickson <
> [hidden email]> wrote:
>
>
>  I don't think boosting is really what you want here. Boosting
> _influences_ the score, it does not impose an ordering.
>
> Sorting _does_ impose an ordering, the question is how to sort and the
> answer depends on how fixed (or not) the sorting criteria are. Do they
> change with different queries? If not, the very simplest thing to do
> is to index a field with a pre-computed sort value. IOW, if your
> ordering is _always_ source 5, 9, 7 index a source_sort field that
> orders things that way and sort on that. Then I'd have a secondary
> sort by score as a tie-breaker.
>
> If that's not the case, perhaps sorting by function (perhaps a custom
> function) would work.
>
> Best,
> Erick
>
> On Mon, Jul 17, 2017 at 4:30 AM, Florian Waltersdorfer
> <[hidden email]> wrote:
> > Hi,
> >
> > I am quite the SolR newbie myself, but have you looked at the resulting
> scores, e.g. via fl=*,score (that way, you can see/test how your boosting
> affects the results)?
> > In a similar scenario, I am using fixed value boosts for specific field
> values; "^=[boost]" instead of "^[factor]", for example:
> >
> > category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)
> >
> > (Actual fixed values open for experimentation.)
> >
> > Regards,
> > Florian
> >
> > -----Ursprüngliche Nachricht-----
> > Von: Luca Dall'Osto [mailto:[hidden email]]
> > Gesendet: Montag, 17. Juli 2017 12:20
> > An: [hidden email]
> > Betreff: Get results in multiple orders (multiple boosts)
> >
> > Hello,
> > I'm new in Solr (and in mailing lists..), and I have a question about
> querying contents in multiple custom orders.
> > I 'm trying to query some documents boosted by 2 (or more) fields: I'm
> able to make a search of 2 day and return results boosted by category
> field, like this:
> >
> > ?indent=on
> > &defType=edismax
> > &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> > &bq=category:9500^2
> > &bq=category:1100^1
> > &rows=40
> > &wt=jsonThis will return all documents of category 9500 first, and 1100
> in after.Now I would like to get this documents with a second boost based
> on another field, called source.I would like to have documents in this
> order:1) category:9500 AND source:5
> > 2) category:9500 AND source:9
> > 3) category:9500 AND source:7
> > 4) category:1100 AND source:5
> > 5) category:1100 AND source:9
> > 6) category:1100 AND source:7
> > To get this order, I tied with this query:?indent=on &defType=edismax
> &q=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> > &bq=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
> > &bq=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
> > &rows=40
> > &wt=json
> > How can I apply a double boosts to get the documents in my correct
> order? Is boost the correct tool for my purpose?Any help will be greatly
> appreciated. Thanks Luca
>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

alessandro.benedetti
"I have different "sort preferences", so I can't build a index and use for sorting.Maybe I have to sort by category then by source and by language or by source, then by category and by date"

I would like to focus on this bit.
It is ok to go for a custom function and sort at query time, but I am curious to explore why an index time solution should not be ok.
You can have these distinct fields :
source_priority
language_priority
category_priority
ect

This values can be assigned at the documents at indexing time ( using for example a custom update request processor).
Then at query time you can easily sort on those values in a multi layered approach :
sort:source_priority desc, category_priority  desc
Of course, if the priority for a source changes quite often or if it's user dependent, a query time solution would be preferred.

---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Luca Dall'Osto
In reply to this post by Luca Dall'Osto
Hello,The problem of build an index is that each user has a custom source order and category order: are not static orders (for example user X could have category:5 as most important category but user Y could have category:9 as most important).
Has anyone ever written a custom sort function in solr?Maybe a link of a tutorial or an example could be very helpful. Thanks 

Luca

    On Tuesday, July 18, 2017 4:18 PM, alessandro.benedetti <[hidden email]> wrote:
 

 "I have different "sort preferences", so I can't build a index and use for
sorting.Maybe I have to sort by category then by source and by language or
by source, then by category and by date"

I would like to focus on this bit.
It is ok to go for a custom function and sort at query time, but I am
curious to explore why an index time solution should not be ok.
You can have these distinct fields :
source_priority
language_priority
category_priority
ect

This values can be assigned at the documents at indexing time ( using for
example a custom update request processor).
Then at query time you can easily sort on those values in a multi layered
approach :
sort:source_priority desc, category_priority  desc
Of course, if the priority for a source changes quite often or if it's user
dependent, a query time solution would be preferred.





-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
View this message in context: http://lucene.472066.n3.nabble.com/Get-results-in-multiple-orders-multiple-boosts-tp4346304p4346559.html
Sent from the Solr - User mailing list archive at Nabble.com.


   
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Rick Leir-2
Luca,
You can pass a sort parameter in the query. User A could sort=date%20desc and user b could sort=foofield%20asc.

Maybe query functions can also help with this. Cheers -- Rick

On July 19, 2017 4:39:59 AM EDT, Luca Dall'Osto <[hidden email]> wrote:

>Hello,The problem of build an index is that each user has a custom
>source order and category order: are not static orders (for example
>user X could have category:5 as most important category but user Y
>could have category:9 as most important).
>Has anyone ever written a custom sort function in solr?Maybe a link of
>a tutorial or an example could be very helpful. Thanks 
>
>Luca
>
>On Tuesday, July 18, 2017 4:18 PM, alessandro.benedetti
><[hidden email]> wrote:
>
>
>"I have different "sort preferences", so I can't build a index and use
>for
>sorting.Maybe I have to sort by category then by source and by language
>or
>by source, then by category and by date"
>
>I would like to focus on this bit.
>It is ok to go for a custom function and sort at query time, but I am
>curious to explore why an index time solution should not be ok.
>You can have these distinct fields :
>source_priority
>language_priority
>category_priority
>ect
>
>This values can be assigned at the documents at indexing time ( using
>for
>example a custom update request processor).
>Then at query time you can easily sort on those values in a multi
>layered
>approach :
>sort:source_priority desc, category_priority  desc
>Of course, if the priority for a source changes quite often or if it's
>user
>dependent, a query time solution would be preferred.
>
>
>
>
>
>-----
>---------------
>Alessandro Benedetti
>Search Consultant, R&D Software Engineer, Director
>Sease Ltd. - www.sease.io
>--
>View this message in context:
>http://lucene.472066.n3.nabble.com/Get-results-in-multiple-orders-multiple-boosts-tp4346304p4346559.html
>Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>  

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Get results in multiple orders (multiple boosts)

Susheel Kumar-3
Let me try to put an example for custom sort.

On Wed, Jul 19, 2017 at 6:34 AM, Rick Leir <[hidden email]> wrote:

> Luca,
> You can pass a sort parameter in the query. User A could sort=date%20desc
> and user b could sort=foofield%20asc.
>
> Maybe query functions can also help with this. Cheers -- Rick
>
> On July 19, 2017 4:39:59 AM EDT, Luca Dall'Osto
> <[hidden email]> wrote:
> >Hello,The problem of build an index is that each user has a custom
> >source order and category order: are not static orders (for example
> >user X could have category:5 as most important category but user Y
> >could have category:9 as most important).
> >Has anyone ever written a custom sort function in solr?Maybe a link of
> >a tutorial or an example could be very helpful. Thanks
> >
> >Luca
> >
> >On Tuesday, July 18, 2017 4:18 PM, alessandro.benedetti
> ><[hidden email]> wrote:
> >
> >
> >"I have different "sort preferences", so I can't build a index and use
> >for
> >sorting.Maybe I have to sort by category then by source and by language
> >or
> >by source, then by category and by date"
> >
> >I would like to focus on this bit.
> >It is ok to go for a custom function and sort at query time, but I am
> >curious to explore why an index time solution should not be ok.
> >You can have these distinct fields :
> >source_priority
> >language_priority
> >category_priority
> >ect
> >
> >This values can be assigned at the documents at indexing time ( using
> >for
> >example a custom update request processor).
> >Then at query time you can easily sort on those values in a multi
> >layered
> >approach :
> >sort:source_priority desc, category_priority  desc
> >Of course, if the priority for a source changes quite often or if it's
> >user
> >dependent, a query time solution would be preferred.
> >
> >
> >
> >
> >
> >-----
> >---------------
> >Alessandro Benedetti
> >Search Consultant, R&D Software Engineer, Director
> >Sease Ltd. - www.sease.io
> >--
> >View this message in context:
> >http://lucene.472066.n3.nabble.com/Get-results-in-
> multiple-orders-multiple-boosts-tp4346304p4346559.html
> >Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
> >
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com
Loading...