Sorting using "packed" fields?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Sorting using "packed" fields?

Christopher Schultz
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

All,

I have documents that need to appear to have different attributes
depending upon which user is trying to search them. One of the fields
I currently have in the document is called "latest_submission" and
it's a multi-valued text field that contains fields packed with a
numeric identifier prefix and then the real data. Something like this:

101:2018-04-16T16:41:00Z
102:2017-01-25T22:08:17Z
103:2018-11-19T02:52:28Z

When searching, I will know which prefixes are valid for a certain
user, so I know I can search by *other* fields and then pull-out the
values that are appropriate for a particular user.

But if I want Solr/Lucene to searcg/sort by the "latest submission", I
need to be able to tell Solr/Lucene which values are appropriate to
use for that user.

Is this kind of thing possible? I'd like to be able to issue a search
that says e.g.:

  find documents matching name:foo sort by latest_submission starting
with ("102:" or "103:")

I'm just starting out with this data set, so I can completely change
the organization of the data within the index if necessary.

Does anyone have any suggestions?

I've seen some questions on the list about "child documents", and it
seems like that might be relevant. Right now, my input data looks like
this:

{
  { "name" : "document name",
    "latest_submission" : [ "prefix:date", "prefix:date", etc. ]
  }
}

But that could easily be changed to be:

{
  { "name" : "document name",
    "latest_submission" : { "prefix" : "101",
                            "date" : "[date]" },
                          { "prefix" : "103",
                            "date" : "[date]" },
  }
}


Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlrVDAQACgkQHPApP6U8
pFj7Mw//dnM0ZMRhbvAlMptYSH3LEj08I0l/oJWMQYilWOIltpZ148QOJp+5Iqu/
Q9uYfkItdv0Fw77Ebgtmm7N5PUzH7utiyDfKNayvL9d9+MtfFzx4CKPyqdjNDXvC
2LLUks9ABTX93h7AUdeO5rM4NsPci6LMY8dcxU6fbVDbDT5nYTRULUrbGfDxmY6E
SyMwk25DOzmrIoFCOJcyhuluvHhax753mOQCCljuFaCM3J8ap0+2ZqX8Nl5D2NLz
CqU5ROTGxm+qMVQ8dbqhT6LRdbjj6KqazutOxZl+H+Ix6yVeWZG/9TiAtkKZklvJ
6wjMB2te4utj35YPhpMkghkIYwo7s6jt9DXyBaf2gv1fbiNKmvPN2eqhsI870f0t
UmknH8Atx3ygeru3ddjIvb2Fn17E7EpKHWxkmmrexKE8uzCo9Ith6BWqL8ae19o/
LtBQ7RNCNjIbyNk3GcUJmvboM+PAAvUWbnpwQ4V2oI8b5sO9zeopE4JlzbWmG89H
WVmtPpIdw0H8AwLNbJuGaaksY5ZIcYg2iFH56BHvvu1ri3ArSgcQuyHfxEZD7gs3
cjh+mX9QEgbCVrz2i0CwRkgAMMIffG2SjBsHhUs5ESYqeskkDcyFDi70Q+5wNJ71
GhAESSbgpI31lpbhkGwh7gdXiJyKJG3EMFDEEZVN5sLhFYv96Q8=
=V+EE
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: Sorting using "packed" fields?

Alessandro Benedetti
Hi Christopher,
if you model your documents with a nested document approach ( like the one
you mentioned) you should be able to achieve your requirement following
this interesting blog [1] :



*" ToParentBlockJoinQuery supports several score calculation modes. For
example, a score for a parent could be calculated as a min(max) score among
of all its children’s scores. So, with the piece of code below we can sort
parent documents by their children’s prices in descending
order....sort={!parent which=doc_type:parent score=max v=’+doc_type:child
+{!func}price’} desc… "*

Instead of using just the plain price function you could design your own
function, such as :

{!func}if(gt(query(prefix:<user_prefix>),0),latest_submission,0)

it's just a quick attempt to give you the idea, the function query I posted
may need some refinement but it could work

Cheers

[1]
https://blog.griddynamics.com/how-to-sort-parent-documents-by-child-attributes-in-solr/

--------------------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
www.sease.io

On Mon, Apr 16, 2018 at 9:48 PM, Christopher Schultz <
[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> I have documents that need to appear to have different attributes
> depending upon which user is trying to search them. One of the fields
> I currently have in the document is called "latest_submission" and
> it's a multi-valued text field that contains fields packed with a
> numeric identifier prefix and then the real data. Something like this:
>
> 101:2018-04-16T16:41:00Z
> 102:2017-01-25T22:08:17Z
> 103:2018-11-19T02:52:28Z
>
> When searching, I will know which prefixes are valid for a certain
> user, so I know I can search by *other* fields and then pull-out the
> values that are appropriate for a particular user.
>
> But if I want Solr/Lucene to searcg/sort by the "latest submission", I
> need to be able to tell Solr/Lucene which values are appropriate to
> use for that user.
>
> Is this kind of thing possible? I'd like to be able to issue a search
> that says e.g.:
>
>   find documents matching name:foo sort by latest_submission starting
> with ("102:" or "103:")
>
> I'm just starting out with this data set, so I can completely change
> the organization of the data within the index if necessary.
>
> Does anyone have any suggestions?
>
> I've seen some questions on the list about "child documents", and it
> seems like that might be relevant. Right now, my input data looks like
> this:
>
> {
>   { "name" : "document name",
>     "latest_submission" : [ "prefix:date", "prefix:date", etc. ]
>   }
> }
>
> But that could easily be changed to be:
>
> {
>   { "name" : "document name",
>     "latest_submission" : { "prefix" : "101",
>                             "date" : "[date]" },
>                           { "prefix" : "103",
>                             "date" : "[date]" },
>   }
> }
>
>
> Thanks,
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlrVDAQACgkQHPApP6U8
> pFj7Mw//dnM0ZMRhbvAlMptYSH3LEj08I0l/oJWMQYilWOIltpZ148QOJp+5Iqu/
> Q9uYfkItdv0Fw77Ebgtmm7N5PUzH7utiyDfKNayvL9d9+MtfFzx4CKPyqdjNDXvC
> 2LLUks9ABTX93h7AUdeO5rM4NsPci6LMY8dcxU6fbVDbDT5nYTRULUrbGfDxmY6E
> SyMwk25DOzmrIoFCOJcyhuluvHhax753mOQCCljuFaCM3J8ap0+2ZqX8Nl5D2NLz
> CqU5ROTGxm+qMVQ8dbqhT6LRdbjj6KqazutOxZl+H+Ix6yVeWZG/9TiAtkKZklvJ
> 6wjMB2te4utj35YPhpMkghkIYwo7s6jt9DXyBaf2gv1fbiNKmvPN2eqhsI870f0t
> UmknH8Atx3ygeru3ddjIvb2Fn17E7EpKHWxkmmrexKE8uzCo9Ith6BWqL8ae19o/
> LtBQ7RNCNjIbyNk3GcUJmvboM+PAAvUWbnpwQ4V2oI8b5sO9zeopE4JlzbWmG89H
> WVmtPpIdw0H8AwLNbJuGaaksY5ZIcYg2iFH56BHvvu1ri3ArSgcQuyHfxEZD7gs3
> cjh+mX9QEgbCVrz2i0CwRkgAMMIffG2SjBsHhUs5ESYqeskkDcyFDi70Q+5wNJ71
> GhAESSbgpI31lpbhkGwh7gdXiJyKJG3EMFDEEZVN5sLhFYv96Q8=
> =V+EE
> -----END PGP SIGNATURE-----
>
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io