Solution for reverse order of year facets?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Solution for reverse order of year facets?

Michael Lackhoff-2
If I understand the docs right, it is only possible to sort facets by
count or value in ascending order. Both variants are not very helpful
for year facets if I want the most recent years at the top (or appear at
all if I restrict the number of facet entries).

It looks like a requirement that was articulated repeatedly and the
recommended solution seems to be to do some math like 10000 - year and
index that. So far so good. Only problem is that I have many data
sources and I would like to avoid to change every connector to include
the new field. I think a better solution would be to have a custom
TokenFilterFactory that does it.

Since it seems a common request, did someone already build such a
TokenFilterFactory? If not, do you think I could build one myself? I do
some (script-)programming but have no experience with Java, so I think I
could adapt an example. Are there any guides out there?

Or even better, is there a built-in solution I haven't heard of?

-Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

iorixxx
Hi,

Currently there are two storing criteria available. However sort by index - to return the constraints sorted in their index order (lexicographic by indexed term) - should return most recent year at top, no?

Ahmet



On Monday, March 3, 2014 4:36 PM, Michael Lackhoff <[hidden email]> wrote:
If I understand the docs right, it is only possible to sort facets by
count or value in ascending order. Both variants are not very helpful
for year facets if I want the most recent years at the top (or appear at
all if I restrict the number of facet entries).

It looks like a requirement that was articulated repeatedly and the
recommended solution seems to be to do some math like 10000 - year and
index that. So far so good. Only problem is that I have many data
sources and I would like to avoid to change every connector to include
the new field. I think a better solution would be to have a custom
TokenFilterFactory that does it.

Since it seems a common request, did someone already build such a
TokenFilterFactory? If not, do you think I could build one myself? I do
some (script-)programming but have no experience with Java, so I think I
could adapt an example. Are there any guides out there?

Or even better, is there a built-in solution I haven't heard of?

-Michael

Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

Michael Lackhoff-2
On 03.03.2014 16:33 Ahmet Arslan wrote:

> Currently there are two storing criteria available. However sort by index - to return the constraints sorted in their index order (lexicographic by indexed term) - should return most recent year at top, no?

No, it returns them -- as you say -- in lexicographic order and that
means oldest first, like:
1815
1820
...
2012
2013
(might well stop before we get here)
2014

-Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

iorixxx
Hi Michael,

Yes you are correct, oldest comes fist. 

There is no built in solution for this.

Two workaround :

1) use facet.limit=-1 and invert the list (faceting response) at client side

2) use multiples facet.query
   a)facet.query=year:[2012 TO 2014]&facet.query=year:[2010 TO 2012] 
   b)facet.query=year:2014&facet.query=year:2013 ...



On Monday, March 3, 2014 5:45 PM, Michael Lackhoff <[hidden email]> wrote:
On 03.03.2014 16:33 Ahmet Arslan wrote:

> Currently there are two storing criteria available. However sort by index - to return the constraints sorted in their index order (lexicographic by indexed term) - should return most recent year at top, no?

No, it returns them -- as you say -- in lexicographic order and that
means oldest first, like:
1815
1820
...
2012
2013
(might well stop before we get here)

2014

-Michael

Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

Shawn Heisey-4
In reply to this post by Michael Lackhoff-2
On 3/3/2014 7:35 AM, Michael Lackhoff wrote:
> If I understand the docs right, it is only possible to sort facets by
> count or value in ascending order. Both variants are not very helpful
> for year facets if I want the most recent years at the top (or appear at
> all if I restrict the number of facet entries).

There's already an issue in Jira.

https://issues.apache.org/jira/browse/SOLR-1672

I can't take a look now, but I will later if someone else hasn't taken
it up.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

Michael Lackhoff-2
In reply to this post by iorixxx
Hi Ahmet,

> There is no built in solution for this.

Yes, I know, that's why I would like the TokenFilterFactory

> Two workaround :
>
> 1) use facet.limit=-1 and invert the list (faceting response) at client side
>
> 2) use multiples facet.query
>    a)facet.query=year:[2012 TO 2014]&facet.query=year:[2010 TO 2012]
>    b)facet.query=year:2014&facet.query=year:2013 ...

I thought about these but they have the disadvantage that 1) could
return hundreds of facet entries. 2b) is better but would need about 30
facet-queries which makes quite a long URL and it wouldn't always work
as expected. There are subjects that were very popular in the past but
with no (or very few) recent publications. For these I would get empty
results for my 2014-1985 facet-queries but miss all the stuff from the
1960s.

From all these thoughts I came to the conclusion that a custom
TokenFilterFactory could do exactly what I want. In effect it would give
me a reverse sort:
10000 - 2014 = 7986
10000 - 2013 = 7987
...
The client code can easily regain the original year values for display.

And I think it shouldn't be too difficult to write such a beast, only
problem is I am not a Java programmer. That is why I asked if someone
has done it already or if there is a guide I could use.
After all it is just a simple subtraction...

-Michael

Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

Michael Lackhoff-2
In reply to this post by Shawn Heisey-4
On 03.03.2014 19:58 Shawn Heisey wrote:

> There's already an issue in Jira.
>
> https://issues.apache.org/jira/browse/SOLR-1672

Thanks, this is of course the best solution. Only problem is that I use
a custom verson from a vendor (based on version 4.3) I want to enhance.
But perhaps they apply the patch. In the meantime I still think the
custom filter could be a workaround.

> I can't take a look now, but I will later if someone else hasn't taken
> it up.

That would be great!

Thanks
-Michael

Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

iorixxx
In reply to this post by Michael Lackhoff-2
Hi,

Regarding "just a simple subtraction" you do it in indexer code or in a update prcessor too. You can either modify original field or you can create an additional one. Java-script could be used : http://wiki.apache.org/solr/ScriptUpdateProcessor

Ahmet


On Monday, March 3, 2014 9:11 PM, Michael Lackhoff <[hidden email]> wrote:
Hi Ahmet,

> There is no built in solution for this.

Yes, I know, that's why I would like the TokenFilterFactory

> Two workaround :
>
> 1) use facet.limit=-1 and invert the list (faceting response) at client side
>
> 2) use multiples facet.query
>    a)facet.query=year:[2012 TO 2014]&facet.query=year:[2010 TO 2012]
>    b)facet.query=year:2014&facet.query=year:2013 ...

I thought about these but they have the disadvantage that 1) could
return hundreds of facet entries. 2b) is better but would need about 30
facet-queries which makes quite a long URL and it wouldn't always work
as expected. There are subjects that were very popular in the past but
with no (or very few) recent publications. For these I would get empty
results for my 2014-1985 facet-queries but miss all the stuff from the
1960s.

From all these thoughts I came to the conclusion that a custom
TokenFilterFactory could do exactly what I want. In effect it would give
me a reverse sort:
10000 - 2014 = 7986
10000 - 2013 = 7987
...
The client code can easily regain the original year values for display.

And I think it shouldn't be too difficult to write such a beast, only
problem is I am not a Java programmer. That is why I asked if someone
has done it already or if there is a guide I could use.
After all it is just a simple subtraction...


-Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

iorixxx
Hi Michael,


I forgot to include what I did for one customer :

1) Using StatsComponent I get min and max values of the field (year)
2) Calculate "smart gap/range values" according to minimum and maximum.
3) Re-issue the same query (for thee second time) that includes a set of facet.query.

Ahmet



On Monday, March 3, 2014 10:30 PM, Ahmet Arslan <[hidden email]> wrote:
Hi,

Regarding "just a simple subtraction" you do it in indexer code or in a update prcessor too. You can either modify original field or you can create an additional one. Java-script could be used : http://wiki.apache.org/solr/ScriptUpdateProcessor

Ahmet



On Monday, March 3, 2014 9:11 PM, Michael Lackhoff <[hidden email]> wrote:
Hi Ahmet,

> There is no built in solution for this.

Yes, I know, that's why I would like the TokenFilterFactory

> Two workaround :
>
> 1) use facet.limit=-1 and invert the list (faceting response) at client side
>
> 2) use multiples facet.query
>    a)facet.query=year:[2012 TO 2014]&facet.query=year:[2010 TO 2012]
>    b)facet.query=year:2014&facet.query=year:2013 ...

I thought about these but they have the disadvantage that 1) could
return hundreds of facet entries. 2b) is better but would need about 30
facet-queries which makes quite a long URL and it wouldn't always work
as expected. There are subjects that were very popular in the past but
with no (or very few) recent publications. For these I would get empty
results for my 2014-1985 facet-queries but miss all the stuff from the
1960s.

From all these thoughts I came to the conclusion that a custom
TokenFilterFactory could do exactly what I want. In effect it would give
me a reverse sort:
10000 - 2014 = 7986
10000 - 2013 = 7987
...
The client code can easily regain the original year values for display.

And I think it shouldn't be too difficult to write such a beast, only
problem is I am not a Java programmer. That is why I asked if someone
has done it already or if there is a guide I could use.
After all it is just a simple subtraction...


-Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solution for reverse order of year facets?

Michael Lackhoff-2
Hi Ahmet,

> I forgot to include what I did for one customer :
>
> 1) Using StatsComponent I get min and max values of the field (year)
> 2) Calculate "smart gap/range values" according to minimum and maximum.
> 3) Re-issue the same query (for thee second time) that includes a set of facet.query.

It's amazing, everyone I am talking with about this problem seems to
remember some hack(s) to work around the problem ;-)

On one hand it shows, there are some options (and thanks for giving me
some more!) but on the other hand it also shows how much need there is
for a real solution like SOLR-1672. I really hope Shawn finds some time
to make it work.

-Michael