How to index and search (integer or float) vector.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to index and search (integer or float) vector.

Jason
Hi,I have specific documents that consist of integer vector with fixed
length.But I have no idea how to index integer vector and search similar
vector.Which fieldType should I use to solve this problem?And can I get any
example for how to search?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: How to index and search (integer or float) vector.

Rick Leir-2
Jason
One way is simply to use a multi value field. But this is not officially a vector, and the order might not be guaranteed. I suspect you can just post a document with the values, and see them in order.

Search for a single value would not be very useful.

Another way is to choose a textual representation for the vector, and save it as a string (not tokenized). This is more complicated.

What's the use case? Do you want the vector to be searchable?
Cheers -- Rick


On April 12, 2018 8:44:10 PM EDT, Jason <[hidden email]> wrote:

>Hi,I have specific documents that consist of integer vector with fixed
>length.But I have no idea how to index integer vector and search
>similar
>vector.Which fieldType should I use to solve this problem?And can I get
>any
>example for how to search?
>
>
>
>--
>Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

Re: How to index and search (integer or float) vector.

Alexandre Rafalovitch
In reply to this post by Jason
I think that perhaps something along PointType or
IntPointField/FloatPointField would be the right answer to store
things: https://lucene.apache.org/solr/guide/7_3/field-types-included-with-solr.html

On the other hand, I am not sure what "searching similar vectors" mean
for you, I suspect there are different meanings. It may be something
like using a dist function:
https://lucene.apache.org/solr/guide/7_3/function-queries.html#dist-function

Or it may be a custom similarity.

The other two avenues may be Learning-to-rank:
https://lucene.apache.org/solr/7_3_0/solr-ltr/index.html
https://lucene.apache.org/solr/guide/7_3/learning-to-rank.html
Or streaming expressions
https://lucene.apache.org/solr/guide/7_3/streaming-expressions.html
(the documentation is lagging a bit, as individual evaluators are
developed very quickly....)

I haven't seen any examples in the wild using the above, which is why
I am a bit hesitant. So, I suspect some further digging would be
needed. And I have a feeling that each of the approaches above are not
cross-linked, so you really need to be clear on what those vectors are
for you and what 'search' means.

Regards,
   Alex
P.s. If you do get it to work, a follow-up summary email would be
fantastic resources for others searching this kind of information
later.

On 12 April 2018 at 20:44, Jason <[hidden email]> wrote:
> Hi,I have specific documents that consist of integer vector with fixed
> length.But I have no idea how to index integer vector and search similar
> vector.Which fieldType should I use to solve this problem?And can I get any
> example for how to search?
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html