Fetch a binary field

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Fetch a binary field

Barbet Alain
Hi !


I've a Lucene base coming from a C++ program linked with Lucene++, a
port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
2.9. Can see a binary field I have in Luke, with data encoded in
base64.


I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
with Solr 6.5.1. I rebuild a schema for this base, who have one field
in binary:


<field name="document" type="binary" indexed="false" stored="true"/>


When I try to retrieve this field with a query (with SOLR admin
interface) it's fail with "can not use FieldCache on a field which is
neither indexed nor has doc values: document"


I can retrieve others fields, but can't find a way for this one. Does
someone have an idea ? (At the end I want do this with php, but as it
fail with Solr interface too ....).

Thank you for any help !
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fetch a binary field

Barbet Alain
Re,
I take a look on the source code where this msg happen
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186

I use version 6.5 who differ from master.
In 6.5:
if (! (indexed() || hasDocValues()) ) {
As my field is not indexed (and hasDocValues() is false as binary),
this test fail.
Ok but so, what is the way to get this field as I can see it in Luke
(in hex format) ?


2017-08-11 15:41 GMT+02:00 Barbet Alain <[hidden email]>:

> Hi !
>
>
> I've a Lucene base coming from a C++ program linked with Lucene++, a
> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
> 2.9. Can see a binary field I have in Luke, with data encoded in
> base64.
>
>
> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
> with Solr 6.5.1. I rebuild a schema for this base, who have one field
> in binary:
>
>
> <field name="document" type="binary" indexed="false" stored="true"/>
>
>
> When I try to retrieve this field with a query (with SOLR admin
> interface) it's fail with "can not use FieldCache on a field which is
> neither indexed nor has doc values: document"
>
>
> I can retrieve others fields, but can't find a way for this one. Does
> someone have an idea ? (At the end I want do this with php, but as it
> fail with Solr interface too ....).
>
> Thank you for any help !
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fetch a binary field

David Hastings
Why didn't you set it to be indexed? Sure it would be a small dent in an index

> On Aug 11, 2017, at 5:20 PM, Barbet Alain <[hidden email]> wrote:
>
> Re,
> I take a look on the source code where this msg happen
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
>
> I use version 6.5 who differ from master.
> In 6.5:
> if (! (indexed() || hasDocValues()) ) {
> As my field is not indexed (and hasDocValues() is false as binary),
> this test fail.
> Ok but so, what is the way to get this field as I can see it in Luke
> (in hex format) ?
>
>
> 2017-08-11 15:41 GMT+02:00 Barbet Alain <[hidden email]>:
>> Hi !
>>
>>
>> I've a Lucene base coming from a C++ program linked with Lucene++, a
>> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
>> 2.9. Can see a binary field I have in Luke, with data encoded in
>> base64.
>>
>>
>> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
>> with Solr 6.5.1. I rebuild a schema for this base, who have one field
>> in binary:
>>
>>
>> <field name="document" type="binary" indexed="false" stored="true"/>
>>
>>
>> When I try to retrieve this field with a query (with SOLR admin
>> interface) it's fail with "can not use FieldCache on a field which is
>> neither indexed nor has doc values: document"
>>
>>
>> I can retrieve others fields, but can't find a way for this one. Does
>> someone have an idea ? (At the end I want do this with php, but as it
>> fail with Solr interface too ....).
>>
>> Thank you for any help !
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fetch a binary field

Barbet Alain
Hi !

Because this field containt a zipped xml that is bigger than all
others fields & I don't need it for searching, just for display. Yes
it would be better if this field is outside the Lucene base, but as I
have hundred of bases like that, with millions of documents for each,
no I can't change this & reindex the stuff ...

Any other idea ?

2017-08-11 23:55 GMT+02:00 Dave <[hidden email]>:

> Why didn't you set it to be indexed? Sure it would be a small dent in an index
>
>> On Aug 11, 2017, at 5:20 PM, Barbet Alain <[hidden email]> wrote:
>>
>> Re,
>> I take a look on the source code where this msg happen
>> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
>>
>> I use version 6.5 who differ from master.
>> In 6.5:
>> if (! (indexed() || hasDocValues()) ) {
>> As my field is not indexed (and hasDocValues() is false as binary),
>> this test fail.
>> Ok but so, what is the way to get this field as I can see it in Luke
>> (in hex format) ?
>>
>>
>> 2017-08-11 15:41 GMT+02:00 Barbet Alain <[hidden email]>:
>>> Hi !
>>>
>>>
>>> I've a Lucene base coming from a C++ program linked with Lucene++, a
>>> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
>>> 2.9. Can see a binary field I have in Luke, with data encoded in
>>> base64.
>>>
>>>
>>> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
>>> with Solr 6.5.1. I rebuild a schema for this base, who have one field
>>> in binary:
>>>
>>>
>>> <field name="document" type="binary" indexed="false" stored="true"/>
>>>
>>>
>>> When I try to retrieve this field with a query (with SOLR admin
>>> interface) it's fail with "can not use FieldCache on a field which is
>>> neither indexed nor has doc values: document"
>>>
>>>
>>> I can retrieve others fields, but can't find a way for this one. Does
>>> someone have an idea ? (At the end I want do this with php, but as it
>>> fail with Solr interface too ....).
>>>
>>> Thank you for any help !
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fetch a binary field

Rick Leir-2
On 2017-08-12 04:19 AM, Barbet Alain wrote:
> Hi !
>
> Because this field containt a zipped xml that is bigger than all
> others fields & I don't need it for searching, just for display. Yes
> it would be better if this field is outside the Lucene base, but as I
> have hundred of bases like that, with millions of documents for each,
> no I can't change this & reindex the stuff ...
>
> Any other idea ?
Alain,
Since nobody else said it, after a long while...
Your zipped xml could be opened before indexing. You should just index
the data from the xml which will be needed for display. Hopefully that
will consume less index space than the whole xml.
cheers -- Rick
GW
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Fetch a binary field

GW
Had the same issue with long base64_encoded images. Binary & string failed.
Set my field type to field type ignored. Doesn't seem right (or wrong) but
it worked.

On 17 August 2017 at 03:58, Rick Leir <[hidden email]> wrote:

> On 2017-08-12 04:19 AM, Barbet Alain wrote:
>
>> Hi !
>>
>> Because this field containt a zipped xml that is bigger than all
>> others fields & I don't need it for searching, just for display. Yes
>> it would be better if this field is outside the Lucene base, but as I
>> have hundred of bases like that, with millions of documents for each,
>> no I can't change this & reindex the stuff ...
>>
>> Any other idea ?
>>
> Alain,
> Since nobody else said it, after a long while...
> Your zipped xml could be opened before indexing. You should just index the
> data from the xml which will be needed for display. Hopefully that will
> consume less index space than the whole xml.
> cheers -- Rick
>
Loading...