Retrieve field from docValues

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Retrieve field from docValues

weiwang19
Hi,

I have a few questions about using the useDocValuesAsStored option to
retrieve field from docValues:

1. For schema version 1.6, useDocValuesAsStored=true is default, so there
is no need to explicitly set it in schema.xml?

2.  With useDocValuesAsStored=true and the following definition, will Solr
retrieve id from docValues instead of stored field? if fl= id, title,
score,   both id and title are single value field:

  <field name="id" type="string" indexed="true" stored="true"
docValues="true" required="true"/>

 <field name="title" type="string" indexed="true" stored="true"
docValues="true" required="true"/>

  Do I need to have all fields stored="false" docValues="true" to make solr
retrieve from docValues only? I am using Solr 6.6.

Thanks,
Wei
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

Yasufumi Mizoguchi
Hi,

> 1. For schema version 1.6, useDocValuesAsStored=true is default, so there
> is no need to explicitly set it in schema.xml?

Yes.

> 2.  With useDocValuesAsStored=true and the following definition, will Solr
> retrieve id from docValues instead of stored field?

No.
AFAIK, if you define both docValues="true" and stored="true" in your
schema,
Solr tries to retrieve stored value.
(Except using streaming expressions or /export handler etc...
See:
https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
)

Thanks,
Yasufumi


2018年11月6日(火) 9:54 Wei <[hidden email]>:

> Hi,
>
> I have a few questions about using the useDocValuesAsStored option to
> retrieve field from docValues:
>
> 1. For schema version 1.6, useDocValuesAsStored=true is default, so there
> is no need to explicitly set it in schema.xml?
>
> 2.  With useDocValuesAsStored=true and the following definition, will Solr
> retrieve id from docValues instead of stored field? if fl= id, title,
> score,   both id and title are single value field:
>
>   <field name="id" type="string" indexed="true" stored="true"
> docValues="true" required="true"/>
>
>  <field name="title" type="string" indexed="true" stored="true"
> docValues="true" required="true"/>
>
>   Do I need to have all fields stored="false" docValues="true" to make solr
> retrieve from docValues only? I am using Solr 6.6.
>
> Thanks,
> Wei
>
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

Erick Erickson
2. "it depends". Solr  will try to do the most efficient thing
possible. If _all_ the fields are docValues, it will return the stored
values from the docValues  structure. This prevents a disk seek and
decompress cycle.

However, if even one field is docValues=false Solr will by default
return the stored values. For the multiValued case, you can explicitly
tell Solr to return the docValues field.

Best,
Erick
On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
<[hidden email]> wrote:

>
> Hi,
>
> > 1. For schema version 1.6, useDocValuesAsStored=true is default, so there
> > is no need to explicitly set it in schema.xml?
>
> Yes.
>
> > 2.  With useDocValuesAsStored=true and the following definition, will Solr
> > retrieve id from docValues instead of stored field?
>
> No.
> AFAIK, if you define both docValues="true" and stored="true" in your
> schema,
> Solr tries to retrieve stored value.
> (Except using streaming expressions or /export handler etc...
> See:
> https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> )
>
> Thanks,
> Yasufumi
>
>
> 2018年11月6日(火) 9:54 Wei <[hidden email]>:
>
> > Hi,
> >
> > I have a few questions about using the useDocValuesAsStored option to
> > retrieve field from docValues:
> >
> > 1. For schema version 1.6, useDocValuesAsStored=true is default, so there
> > is no need to explicitly set it in schema.xml?
> >
> > 2.  With useDocValuesAsStored=true and the following definition, will Solr
> > retrieve id from docValues instead of stored field? if fl= id, title,
> > score,   both id and title are single value field:
> >
> >   <field name="id" type="string" indexed="true" stored="true"
> > docValues="true" required="true"/>
> >
> >  <field name="title" type="string" indexed="true" stored="true"
> > docValues="true" required="true"/>
> >
> >   Do I need to have all fields stored="false" docValues="true" to make solr
> > retrieve from docValues only? I am using Solr 6.6.
> >
> > Thanks,
> > Wei
> >
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

weiwang19
Thanks Yasufumi and Erick.

---. 2. "it depends". Solr  will try to do the most efficient thing
possible. If _all_ the fields are docValues, it will return the stored
values from the docValues  structure.

I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344    Does
this mean "Solr  will try to do the most efficient thing possible" only
working for 7.x?  Is the behavior available for 6.6?

-- This prevents a disk seek and  decompress cycle.

Does this still hold if whole index is loaded into memory?  Also for the
benefit of performance improvement,  does the uniqueKey field need to be
always docValues? Since it is used in the first phase of distributed
search.

Thanks,
Wei



On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
wrote:

> 2. "it depends". Solr  will try to do the most efficient thing
> possible. If _all_ the fields are docValues, it will return the stored
> values from the docValues  structure. This prevents a disk seek and
> decompress cycle.
>
> However, if even one field is docValues=false Solr will by default
> return the stored values. For the multiValued case, you can explicitly
> tell Solr to return the docValues field.
>
> Best,
> Erick
> On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
> <[hidden email]> wrote:
> >
> > Hi,
> >
> > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> there
> > > is no need to explicitly set it in schema.xml?
> >
> > Yes.
> >
> > > 2.  With useDocValuesAsStored=true and the following definition, will
> Solr
> > > retrieve id from docValues instead of stored field?
> >
> > No.
> > AFAIK, if you define both docValues="true" and stored="true" in your
> > schema,
> > Solr tries to retrieve stored value.
> > (Except using streaming expressions or /export handler etc...
> > See:
> >
> https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> > )
> >
> > Thanks,
> > Yasufumi
> >
> >
> > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
> >
> > > Hi,
> > >
> > > I have a few questions about using the useDocValuesAsStored option to
> > > retrieve field from docValues:
> > >
> > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> there
> > > is no need to explicitly set it in schema.xml?
> > >
> > > 2.  With useDocValuesAsStored=true and the following definition, will
> Solr
> > > retrieve id from docValues instead of stored field? if fl= id, title,
> > > score,   both id and title are single value field:
> > >
> > >   <field name="id" type="string" indexed="true" stored="true"
> > > docValues="true" required="true"/>
> > >
> > >  <field name="title" type="string" indexed="true" stored="true"
> > > docValues="true" required="true"/>
> > >
> > >   Do I need to have all fields stored="false" docValues="true" to make
> solr
> > > retrieve from docValues only? I am using Solr 6.6.
> > >
> > > Thanks,
> > > Wei
> > >
>
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

Erick Erickson
Yes, "the most efficient possible" is associated with that JIRA, so only in 7x.

"Does this still hold if whole index is loaded into memory?"
The decompression part yes, the disk seek part no. And it's also
sensitive to whether the documentCache already has the document.

I'd also make uniqueKey ant the _version_ fields docValues.

Best,
Erick
On Tue, Nov 6, 2018 at 10:44 AM Wei <[hidden email]> wrote:

>
> Thanks Yasufumi and Erick.
>
> ---. 2. "it depends". Solr  will try to do the most efficient thing
> possible. If _all_ the fields are docValues, it will return the stored
> values from the docValues  structure.
>
> I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344    Does
> this mean "Solr  will try to do the most efficient thing possible" only
> working for 7.x?  Is the behavior available for 6.6?
>
> -- This prevents a disk seek and  decompress cycle.
>
> Does this still hold if whole index is loaded into memory?  Also for the
> benefit of performance improvement,  does the uniqueKey field need to be
> always docValues? Since it is used in the first phase of distributed
> search.
>
> Thanks,
> Wei
>
>
>
> On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
> wrote:
>
> > 2. "it depends". Solr  will try to do the most efficient thing
> > possible. If _all_ the fields are docValues, it will return the stored
> > values from the docValues  structure. This prevents a disk seek and
> > decompress cycle.
> >
> > However, if even one field is docValues=false Solr will by default
> > return the stored values. For the multiValued case, you can explicitly
> > tell Solr to return the docValues field.
> >
> > Best,
> > Erick
> > On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
> > <[hidden email]> wrote:
> > >
> > > Hi,
> > >
> > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > there
> > > > is no need to explicitly set it in schema.xml?
> > >
> > > Yes.
> > >
> > > > 2.  With useDocValuesAsStored=true and the following definition, will
> > Solr
> > > > retrieve id from docValues instead of stored field?
> > >
> > > No.
> > > AFAIK, if you define both docValues="true" and stored="true" in your
> > > schema,
> > > Solr tries to retrieve stored value.
> > > (Except using streaming expressions or /export handler etc...
> > > See:
> > >
> > https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> > > )
> > >
> > > Thanks,
> > > Yasufumi
> > >
> > >
> > > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
> > >
> > > > Hi,
> > > >
> > > > I have a few questions about using the useDocValuesAsStored option to
> > > > retrieve field from docValues:
> > > >
> > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > there
> > > > is no need to explicitly set it in schema.xml?
> > > >
> > > > 2.  With useDocValuesAsStored=true and the following definition, will
> > Solr
> > > > retrieve id from docValues instead of stored field? if fl= id, title,
> > > > score,   both id and title are single value field:
> > > >
> > > >   <field name="id" type="string" indexed="true" stored="true"
> > > > docValues="true" required="true"/>
> > > >
> > > >  <field name="title" type="string" indexed="true" stored="true"
> > > > docValues="true" required="true"/>
> > > >
> > > >   Do I need to have all fields stored="false" docValues="true" to make
> > solr
> > > > retrieve from docValues only? I am using Solr 6.6.
> > > >
> > > > Thanks,
> > > > Wei
> > > >
> >
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

weiwang19
I see there is also a docValuesFormat option, what's the default for this
setting? Performance wise is it good to set docValuesFormat="Memory" ?

Best,
Wei


On Tue, Nov 6, 2018 at 11:55 AM Erick Erickson <[hidden email]>
wrote:

> Yes, "the most efficient possible" is associated with that JIRA, so only
> in 7x.
>
> "Does this still hold if whole index is loaded into memory?"
> The decompression part yes, the disk seek part no. And it's also
> sensitive to whether the documentCache already has the document.
>
> I'd also make uniqueKey ant the _version_ fields docValues.
>
> Best,
> Erick
> On Tue, Nov 6, 2018 at 10:44 AM Wei <[hidden email]> wrote:
> >
> > Thanks Yasufumi and Erick.
> >
> > ---. 2. "it depends". Solr  will try to do the most efficient thing
> > possible. If _all_ the fields are docValues, it will return the stored
> > values from the docValues  structure.
> >
> > I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344
> Does
> > this mean "Solr  will try to do the most efficient thing possible" only
> > working for 7.x?  Is the behavior available for 6.6?
> >
> > -- This prevents a disk seek and  decompress cycle.
> >
> > Does this still hold if whole index is loaded into memory?  Also for the
> > benefit of performance improvement,  does the uniqueKey field need to be
> > always docValues? Since it is used in the first phase of distributed
> > search.
> >
> > Thanks,
> > Wei
> >
> >
> >
> > On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
> > wrote:
> >
> > > 2. "it depends". Solr  will try to do the most efficient thing
> > > possible. If _all_ the fields are docValues, it will return the stored
> > > values from the docValues  structure. This prevents a disk seek and
> > > decompress cycle.
> > >
> > > However, if even one field is docValues=false Solr will by default
> > > return the stored values. For the multiValued case, you can explicitly
> > > tell Solr to return the docValues field.
> > >
> > > Best,
> > > Erick
> > > On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
> > > <[hidden email]> wrote:
> > > >
> > > > Hi,
> > > >
> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > > there
> > > > > is no need to explicitly set it in schema.xml?
> > > >
> > > > Yes.
> > > >
> > > > > 2.  With useDocValuesAsStored=true and the following definition,
> will
> > > Solr
> > > > > retrieve id from docValues instead of stored field?
> > > >
> > > > No.
> > > > AFAIK, if you define both docValues="true" and stored="true" in your
> > > > schema,
> > > > Solr tries to retrieve stored value.
> > > > (Except using streaming expressions or /export handler etc...
> > > > See:
> > > >
> > >
> https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> > > > )
> > > >
> > > > Thanks,
> > > > Yasufumi
> > > >
> > > >
> > > > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
> > > >
> > > > > Hi,
> > > > >
> > > > > I have a few questions about using the useDocValuesAsStored option
> to
> > > > > retrieve field from docValues:
> > > > >
> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > > there
> > > > > is no need to explicitly set it in schema.xml?
> > > > >
> > > > > 2.  With useDocValuesAsStored=true and the following definition,
> will
> > > Solr
> > > > > retrieve id from docValues instead of stored field? if fl= id,
> title,
> > > > > score,   both id and title are single value field:
> > > > >
> > > > >   <field name="id" type="string" indexed="true" stored="true"
> > > > > docValues="true" required="true"/>
> > > > >
> > > > >  <field name="title" type="string" indexed="true" stored="true"
> > > > > docValues="true" required="true"/>
> > > > >
> > > > >   Do I need to have all fields stored="false" docValues="true" to
> make
> > > solr
> > > > > retrieve from docValues only? I am using Solr 6.6.
> > > > >
> > > > > Thanks,
> > > > > Wei
> > > > >
> > >
>
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

Erick Erickson
docValuesFormat="Memory" has been deprecated, so you shouldn't use it.
On Tue, Nov 6, 2018 at 2:14 PM Wei <[hidden email]> wrote:

>
> I see there is also a docValuesFormat option, what's the default for this
> setting? Performance wise is it good to set docValuesFormat="Memory" ?
>
> Best,
> Wei
>
>
> On Tue, Nov 6, 2018 at 11:55 AM Erick Erickson <[hidden email]>
> wrote:
>
> > Yes, "the most efficient possible" is associated with that JIRA, so only
> > in 7x.
> >
> > "Does this still hold if whole index is loaded into memory?"
> > The decompression part yes, the disk seek part no. And it's also
> > sensitive to whether the documentCache already has the document.
> >
> > I'd also make uniqueKey ant the _version_ fields docValues.
> >
> > Best,
> > Erick
> > On Tue, Nov 6, 2018 at 10:44 AM Wei <[hidden email]> wrote:
> > >
> > > Thanks Yasufumi and Erick.
> > >
> > > ---. 2. "it depends". Solr  will try to do the most efficient thing
> > > possible. If _all_ the fields are docValues, it will return the stored
> > > values from the docValues  structure.
> > >
> > > I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344
> > Does
> > > this mean "Solr  will try to do the most efficient thing possible" only
> > > working for 7.x?  Is the behavior available for 6.6?
> > >
> > > -- This prevents a disk seek and  decompress cycle.
> > >
> > > Does this still hold if whole index is loaded into memory?  Also for the
> > > benefit of performance improvement,  does the uniqueKey field need to be
> > > always docValues? Since it is used in the first phase of distributed
> > > search.
> > >
> > > Thanks,
> > > Wei
> > >
> > >
> > >
> > > On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
> > > wrote:
> > >
> > > > 2. "it depends". Solr  will try to do the most efficient thing
> > > > possible. If _all_ the fields are docValues, it will return the stored
> > > > values from the docValues  structure. This prevents a disk seek and
> > > > decompress cycle.
> > > >
> > > > However, if even one field is docValues=false Solr will by default
> > > > return the stored values. For the multiValued case, you can explicitly
> > > > tell Solr to return the docValues field.
> > > >
> > > > Best,
> > > > Erick
> > > > On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
> > > > <[hidden email]> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > > > there
> > > > > > is no need to explicitly set it in schema.xml?
> > > > >
> > > > > Yes.
> > > > >
> > > > > > 2.  With useDocValuesAsStored=true and the following definition,
> > will
> > > > Solr
> > > > > > retrieve id from docValues instead of stored field?
> > > > >
> > > > > No.
> > > > > AFAIK, if you define both docValues="true" and stored="true" in your
> > > > > schema,
> > > > > Solr tries to retrieve stored value.
> > > > > (Except using streaming expressions or /export handler etc...
> > > > > See:
> > > > >
> > > >
> > https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> > > > > )
> > > > >
> > > > > Thanks,
> > > > > Yasufumi
> > > > >
> > > > >
> > > > > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I have a few questions about using the useDocValuesAsStored option
> > to
> > > > > > retrieve field from docValues:
> > > > > >
> > > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default, so
> > > > there
> > > > > > is no need to explicitly set it in schema.xml?
> > > > > >
> > > > > > 2.  With useDocValuesAsStored=true and the following definition,
> > will
> > > > Solr
> > > > > > retrieve id from docValues instead of stored field? if fl= id,
> > title,
> > > > > > score,   both id and title are single value field:
> > > > > >
> > > > > >   <field name="id" type="string" indexed="true" stored="true"
> > > > > > docValues="true" required="true"/>
> > > > > >
> > > > > >  <field name="title" type="string" indexed="true" stored="true"
> > > > > > docValues="true" required="true"/>
> > > > > >
> > > > > >   Do I need to have all fields stored="false" docValues="true" to
> > make
> > > > solr
> > > > > > retrieve from docValues only? I am using Solr 6.6.
> > > > > >
> > > > > > Thanks,
> > > > > > Wei
> > > > > >
> > > >
> >
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

weiwang19
In reply to this post by weiwang19
Also I notice this issue is still open:
https://issues.apache.org/jira/browse/SOLR-10816
Does that mean we still need to have stored=true for uniqueKey?

On Tue, Nov 6, 2018 at 2:14 PM Wei <[hidden email]> wrote:

> I see there is also a docValuesFormat option, what's the default for this
> setting? Performance wise is it good to set docValuesFormat="Memory" ?
>
> Best,
> Wei
>
>
> On Tue, Nov 6, 2018 at 11:55 AM Erick Erickson <[hidden email]>
> wrote:
>
>> Yes, "the most efficient possible" is associated with that JIRA, so only
>> in 7x.
>>
>> "Does this still hold if whole index is loaded into memory?"
>> The decompression part yes, the disk seek part no. And it's also
>> sensitive to whether the documentCache already has the document.
>>
>> I'd also make uniqueKey ant the _version_ fields docValues.
>>
>> Best,
>> Erick
>> On Tue, Nov 6, 2018 at 10:44 AM Wei <[hidden email]> wrote:
>> >
>> > Thanks Yasufumi and Erick.
>> >
>> > ---. 2. "it depends". Solr  will try to do the most efficient thing
>> > possible. If _all_ the fields are docValues, it will return the stored
>> > values from the docValues  structure.
>> >
>> > I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344
>> Does
>> > this mean "Solr  will try to do the most efficient thing possible" only
>> > working for 7.x?  Is the behavior available for 6.6?
>> >
>> > -- This prevents a disk seek and  decompress cycle.
>> >
>> > Does this still hold if whole index is loaded into memory?  Also for the
>> > benefit of performance improvement,  does the uniqueKey field need to be
>> > always docValues? Since it is used in the first phase of distributed
>> > search.
>> >
>> > Thanks,
>> > Wei
>> >
>> >
>> >
>> > On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
>> > wrote:
>> >
>> > > 2. "it depends". Solr  will try to do the most efficient thing
>> > > possible. If _all_ the fields are docValues, it will return the stored
>> > > values from the docValues  structure. This prevents a disk seek and
>> > > decompress cycle.
>> > >
>> > > However, if even one field is docValues=false Solr will by default
>> > > return the stored values. For the multiValued case, you can explicitly
>> > > tell Solr to return the docValues field.
>> > >
>> > > Best,
>> > > Erick
>> > > On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
>> > > <[hidden email]> wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default,
>> so
>> > > there
>> > > > > is no need to explicitly set it in schema.xml?
>> > > >
>> > > > Yes.
>> > > >
>> > > > > 2.  With useDocValuesAsStored=true and the following definition,
>> will
>> > > Solr
>> > > > > retrieve id from docValues instead of stored field?
>> > > >
>> > > > No.
>> > > > AFAIK, if you define both docValues="true" and stored="true" in your
>> > > > schema,
>> > > > Solr tries to retrieve stored value.
>> > > > (Except using streaming expressions or /export handler etc...
>> > > > See:
>> > > >
>> > >
>> https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
>> > > > )
>> > > >
>> > > > Thanks,
>> > > > Yasufumi
>> > > >
>> > > >
>> > > > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I have a few questions about using the useDocValuesAsStored
>> option to
>> > > > > retrieve field from docValues:
>> > > > >
>> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default,
>> so
>> > > there
>> > > > > is no need to explicitly set it in schema.xml?
>> > > > >
>> > > > > 2.  With useDocValuesAsStored=true and the following definition,
>> will
>> > > Solr
>> > > > > retrieve id from docValues instead of stored field? if fl= id,
>> title,
>> > > > > score,   both id and title are single value field:
>> > > > >
>> > > > >   <field name="id" type="string" indexed="true" stored="true"
>> > > > > docValues="true" required="true"/>
>> > > > >
>> > > > >  <field name="title" type="string" indexed="true" stored="true"
>> > > > > docValues="true" required="true"/>
>> > > > >
>> > > > >   Do I need to have all fields stored="false" docValues="true" to
>> make
>> > > solr
>> > > > > retrieve from docValues only? I am using Solr 6.6.
>> > > > >
>> > > > > Thanks,
>> > > > > Wei
>> > > > >
>> > >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Retrieve field from docValues

Erick Erickson
You should until this is resolved. The original purpose of that JIRA
doesn't count any longer, i.e. the speedup aspects since that's been
taken care of though.
On Tue, Nov 6, 2018 at 3:50 PM Wei <[hidden email]> wrote:

>
> Also I notice this issue is still open:
> https://issues.apache.org/jira/browse/SOLR-10816
> Does that mean we still need to have stored=true for uniqueKey?
>
> On Tue, Nov 6, 2018 at 2:14 PM Wei <[hidden email]> wrote:
>
> > I see there is also a docValuesFormat option, what's the default for this
> > setting? Performance wise is it good to set docValuesFormat="Memory" ?
> >
> > Best,
> > Wei
> >
> >
> > On Tue, Nov 6, 2018 at 11:55 AM Erick Erickson <[hidden email]>
> > wrote:
> >
> >> Yes, "the most efficient possible" is associated with that JIRA, so only
> >> in 7x.
> >>
> >> "Does this still hold if whole index is loaded into memory?"
> >> The decompression part yes, the disk seek part no. And it's also
> >> sensitive to whether the documentCache already has the document.
> >>
> >> I'd also make uniqueKey ant the _version_ fields docValues.
> >>
> >> Best,
> >> Erick
> >> On Tue, Nov 6, 2018 at 10:44 AM Wei <[hidden email]> wrote:
> >> >
> >> > Thanks Yasufumi and Erick.
> >> >
> >> > ---. 2. "it depends". Solr  will try to do the most efficient thing
> >> > possible. If _all_ the fields are docValues, it will return the stored
> >> > values from the docValues  structure.
> >> >
> >> > I find this jira:   https://issues.apache.org/jira/browse/SOLR-8344
> >> Does
> >> > this mean "Solr  will try to do the most efficient thing possible" only
> >> > working for 7.x?  Is the behavior available for 6.6?
> >> >
> >> > -- This prevents a disk seek and  decompress cycle.
> >> >
> >> > Does this still hold if whole index is loaded into memory?  Also for the
> >> > benefit of performance improvement,  does the uniqueKey field need to be
> >> > always docValues? Since it is used in the first phase of distributed
> >> > search.
> >> >
> >> > Thanks,
> >> > Wei
> >> >
> >> >
> >> >
> >> > On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson <[hidden email]>
> >> > wrote:
> >> >
> >> > > 2. "it depends". Solr  will try to do the most efficient thing
> >> > > possible. If _all_ the fields are docValues, it will return the stored
> >> > > values from the docValues  structure. This prevents a disk seek and
> >> > > decompress cycle.
> >> > >
> >> > > However, if even one field is docValues=false Solr will by default
> >> > > return the stored values. For the multiValued case, you can explicitly
> >> > > tell Solr to return the docValues field.
> >> > >
> >> > > Best,
> >> > > Erick
> >> > > On Tue, Nov 6, 2018 at 1:46 AM Yasufumi Mizoguchi
> >> > > <[hidden email]> wrote:
> >> > > >
> >> > > > Hi,
> >> > > >
> >> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default,
> >> so
> >> > > there
> >> > > > > is no need to explicitly set it in schema.xml?
> >> > > >
> >> > > > Yes.
> >> > > >
> >> > > > > 2.  With useDocValuesAsStored=true and the following definition,
> >> will
> >> > > Solr
> >> > > > > retrieve id from docValues instead of stored field?
> >> > > >
> >> > > > No.
> >> > > > AFAIK, if you define both docValues="true" and stored="true" in your
> >> > > > schema,
> >> > > > Solr tries to retrieve stored value.
> >> > > > (Except using streaming expressions or /export handler etc...
> >> > > > See:
> >> > > >
> >> > >
> >> https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-EnablingDocValues
> >> > > > )
> >> > > >
> >> > > > Thanks,
> >> > > > Yasufumi
> >> > > >
> >> > > >
> >> > > > 2018年11月6日(火) 9:54 Wei <[hidden email]>:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > I have a few questions about using the useDocValuesAsStored
> >> option to
> >> > > > > retrieve field from docValues:
> >> > > > >
> >> > > > > 1. For schema version 1.6, useDocValuesAsStored=true is default,
> >> so
> >> > > there
> >> > > > > is no need to explicitly set it in schema.xml?
> >> > > > >
> >> > > > > 2.  With useDocValuesAsStored=true and the following definition,
> >> will
> >> > > Solr
> >> > > > > retrieve id from docValues instead of stored field? if fl= id,
> >> title,
> >> > > > > score,   both id and title are single value field:
> >> > > > >
> >> > > > >   <field name="id" type="string" indexed="true" stored="true"
> >> > > > > docValues="true" required="true"/>
> >> > > > >
> >> > > > >  <field name="title" type="string" indexed="true" stored="true"
> >> > > > > docValues="true" required="true"/>
> >> > > > >
> >> > > > >   Do I need to have all fields stored="false" docValues="true" to
> >> make
> >> > > solr
> >> > > > > retrieve from docValues only? I am using Solr 6.6.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Wei
> >> > > > >
> >> > >
> >>
> >