TextField size limit

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

TextField size limit

revenge-2
Hi all,

i have a TextField containing over 400k of text....

when i try to search a word solr doesn't return any result.... but if I search for a single document, I can see that the word exists there....

So I suppose that solr has a textfield size limit.... (the field is indexed using a tokenizer and some filters)

Could anyone help me to undestand the problem? and if is it possible to solve?

Thanks in advance,
  Antonio


Reply | Threaded
Open this post in threaded view
|

Re: TextField size limit

Erik Hatcher
Check your solrconfig.xml:

      <maxFieldLength>10000</maxFieldLength>

That's probably the truncating factor.  That's the maximum number of  
terms, not bytes or characters.

        Erik


On Dec 15, 2008, at 5:00 PM, Antonio Zippo wrote:

> Hi all,
>
> i have a TextField containing over 400k of text....
>
> when i try to search a word solr doesn't return any result.... but  
> if I search for a single document, I can see that the word exists  
> there....
>
> So I suppose that solr has a textfield size limit.... (the field is  
> indexed using a tokenizer and some filters)
>
> Could anyone help me to undestand the problem? and if is it possible  
> to solve?
>
> Thanks in advance,
>  Antonio
>
>

Reply | Threaded
Open this post in threaded view
|

Re: TextField size limit

revenge-2
>
> Check your solrconfig.xml:
>
>      <maxFieldLength>10000</maxFieldLength>
>
> That's probably the truncating factor.  That's the maximum number of terms, not bytes or characters.
>
>     Erik
>


Thanks... I think it could be the problem.....
i tried to count whitespace in a single text and it's over 55.000 ... but solr truncates to 10.000

do you know if I can change the value to 100.000 without recreate the index? (when I modify schema.xml I need to create the index again.... but with solrconfig.xml?)

Thanks,
  Antonio



Reply | Threaded
Open this post in threaded view
|

Re: TextField size limit

Yonik Seeley
On Mon, Dec 15, 2008 at 5:28 PM, Antonio Zippo <[hidden email]> wrote:

>>
>> Check your solrconfig.xml:
>>
>>      <maxFieldLength>10000</maxFieldLength>
>>
>> That's probably the truncating factor.  That's the maximum number of terms, not bytes or characters.
>>
> Thanks... I think it could be the problem.....
> i tried to count whitespace in a single text and it's over 55.000 ... but solr truncates to 10.000
>
> do you know if I can change the value to 100.000 without recreate the index? (when I modify schema.xml I need to create the index again.... but with solrconfig.xml?)

No need to re-index with this change.
But you will have to re-index any documents that got cut off of course.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: TextField size limit

revenge-2
>
> No need to re-index with this change.
> But you will have to re-index any documents that got cut off of course.
>
> -Yonik
>

Ok, thanks...
I hoped to reindex the documents over the existent index (with incremental update...while solr is running) ...and without delete the index folder....

But the important is to solve the problem ;-)

Thanks...
  Antonio