SpellCheck Reload

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

SpellCheck Reload

Sadiki Latty
Greetings list,

I had question regarding the spellcheck.reload parameter. I am using the IndexBasedSpellChecker which creates it's dictionary based on content from a field. I built the spell check (in error) with a field that has stemming and other filters associated to it.

Regarding the spellcheck.reload parameter, the guide states "If set to true, this parameter reloads the spellchecker. The results depend on the implementation of SolrSpellChecker.reload(). In a typical implementation, reloading the spellchecker means reloading the dictionary."

My question is, does "reloading the dictionary" mean completely erasing the current dictionary and starting from scratch (which is what I want), or does it simply reload the dictionary into some form of memory which would include what was there before (the stemmed and filtered data based on the initial field)?


Thanks,

Sid
Reply | Threaded
Open this post in threaded view
|

RE: SpellCheck Reload

Sadiki Latty
Hello,

Just bumping this question up regarding the spellcheck reload. Can anyone provide some insight on this question?


Thanks in advance

Sid

-----Original Message-----
From: Sadiki Latty [mailto:[hidden email]]
Sent: March-12-18 1:38 PM
To: [hidden email]
Subject: SpellCheck Reload

Greetings list,

I had question regarding the spellcheck.reload parameter. I am using the IndexBasedSpellChecker which creates it's dictionary based on content from a field. I built the spell check (in error) with a field that has stemming and other filters associated to it.

Regarding the spellcheck.reload parameter, the guide states "If set to true, this parameter reloads the spellchecker. The results depend on the implementation of SolrSpellChecker.reload(). In a typical implementation, reloading the spellchecker means reloading the dictionary."

My question is, does "reloading the dictionary" mean completely erasing the current dictionary and starting from scratch (which is what I want), or does it simply reload the dictionary into some form of memory which would include what was there before (the stemmed and filtered data based on the initial field)?


Thanks,

Sid
Reply | Threaded
Open this post in threaded view
|

RE: SpellCheck Reload

Alessandro Benedetti
Hi Sadiki,
the kind of spellchecker you are using built an auxiliary Lucene index as a
support data structure.
That is going to be used to provide the spellcheck suggestions.

"My question is, does "reloading the dictionary" mean completely erasing the
current dictionary and starting from scratch (which is what I want)? "

What you want is re-build the spellchecker.
In the case of the the IndexBasedSpellChecker, the index is used to build
the dictionary.
When the spellchecker is initialized a reader is opened from the latest
index version available.

if in the meantime your index has changed and commits have happened, just
building the spellchecker *should* use the old reader :

@Override
  public void build(SolrCore core, SolrIndexSearcher searcher) throws
IOException {
    IndexReader reader = null;
    if (sourceLocation == null) {
      // Load from Solr's index
      reader = searcher.getIndexReader();
    } else {
      // Load from Lucene index at given sourceLocation
      reader = this.reader;
    }

This means your dictionary is not going to see any substantial changes.

So what you need to do is :

1) reload the spellchecker -> which will initialise again the source for the
dictionary to the latest index commit
2) re-build the dictionary



Cheers







-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

RE: SpellCheck Reload

Sadiki Latty
Thanks Alessandro, I'll give this a try next time. I ended up deleting the spell folder after trying the reload option without success. Next time I will try the reload then build method you suggested.

Thanks again for the info.

-----Original Message-----
From: Alessandro Benedetti [mailto:[hidden email]]
Sent: March-15-18 1:34 PM
To: [hidden email]
Subject: RE: SpellCheck Reload

Hi Sadiki,
the kind of spellchecker you are using built an auxiliary Lucene index as a support data structure.
That is going to be used to provide the spellcheck suggestions.

"My question is, does "reloading the dictionary" mean completely erasing the current dictionary and starting from scratch (which is what I want)? "

What you want is re-build the spellchecker.
In the case of the the IndexBasedSpellChecker, the index is used to build the dictionary.
When the spellchecker is initialized a reader is opened from the latest index version available.

if in the meantime your index has changed and commits have happened, just building the spellchecker *should* use the old reader :

@Override
  public void build(SolrCore core, SolrIndexSearcher searcher) throws IOException {
    IndexReader reader = null;
    if (sourceLocation == null) {
      // Load from Solr's index
      reader = searcher.getIndexReader();
    } else {
      // Load from Lucene index at given sourceLocation
      reader = this.reader;
    }

This means your dictionary is not going to see any substantial changes.

So what you need to do is :

1) reload the spellchecker -> which will initialise again the source for the dictionary to the latest index commit
2) re-build the dictionary



Cheers







-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html