removing write.lock file in solr after indexing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

removing write.lock file in solr after indexing

shyam bhaskaran-2
Hi,

We are using Solr 4.0 and after indexing every time it is observed that the write.lock remains without getting cleared and for the next indexing we have to delete the file to get the indexing process running.

We use SolrServer for our indexing and I do not see any  methods to close or clear the indexes on completion of indexing.


I have seen that adding the below lines into solrconfig.xml file avoids the issue of physically removing the write.lock file when doing indexing.



<indexDefaults>

  <lockType>simple</lockType>

  <unlockOnStartup>true</unlockOnStartup>

</indexDefaults>


But I am hesitant in adding this directive, as it might not be a good idea to set this directive in production as it would defeat the purpose of locking the index while another process writes into it.

Let me know if we can do this programmatically, is there something like close() which would remove the write.lock file after completion of indexing using SolrServer?

Thanks
Shyam
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: removing write.lock file in solr after indexing

Erick Erickson
On Mon, Jan 30, 2012 at 2:42 AM, Shyam Bhaskaran
<[hidden email]> wrote:

> Hi,
>
> We are using Solr 4.0 and after indexing every time it is observed that the write.lock remains without getting cleared and for the next indexing we have to delete the file to get the indexing process running.
>
> We use SolrServer for our indexing and I do not see any  methods to close or clear the indexes on completion of indexing.
>
>
> I have seen that adding the below lines into solrconfig.xml file avoids the issue of physically removing the write.lock file when doing indexing.
>
>
>
> <indexDefaults>
>
>  <lockType>simple</lockType>
>
>  <unlockOnStartup>true</unlockOnStartup>
>
> </indexDefaults>
>
>
> But I am hesitant in adding this directive, as it might not be a good idea to set this directive in production as it would defeat the purpose of locking the index while another process writes into it.
>
> Let me know if we can do this programmatically, is there something like close() which would remove the write.lock file after completion of indexing using SolrServer?
>
> Thanks
> Shyam
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: removing write.lock file in solr after indexing

Erick Erickson
Oops, fat fingers... Anyway, this is surprising. Can you provide
more details on how you do your indexing?

Best
Erick

On Tue, Jan 31, 2012 at 8:59 AM, Erick Erickson <[hidden email]> wrote:

> On Mon, Jan 30, 2012 at 2:42 AM, Shyam Bhaskaran
> <[hidden email]> wrote:
>> Hi,
>>
>> We are using Solr 4.0 and after indexing every time it is observed that the write.lock remains without getting cleared and for the next indexing we have to delete the file to get the indexing process running.
>>
>> We use SolrServer for our indexing and I do not see any  methods to close or clear the indexes on completion of indexing.
>>
>>
>> I have seen that adding the below lines into solrconfig.xml file avoids the issue of physically removing the write.lock file when doing indexing.
>>
>>
>>
>> <indexDefaults>
>>
>>  <lockType>simple</lockType>
>>
>>  <unlockOnStartup>true</unlockOnStartup>
>>
>> </indexDefaults>
>>
>>
>> But I am hesitant in adding this directive, as it might not be a good idea to set this directive in production as it would defeat the purpose of locking the index while another process writes into it.
>>
>> Let me know if we can do this programmatically, is there something like close() which would remove the write.lock file after completion of indexing using SolrServer?
>>
>> Thanks
>> Shyam
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: removing write.lock file in solr after indexing

shyam bhaskaran-2
Hi Erick,


Below is the sample flow.


String solrHome = "/opt/solr/home";

File solrXml = new File( solrHome, "solr.xml" );

container = new CoreContainer();

container.load(solrHome, solrXml);

SolrServer solr = new EmbeddedSolrServer(container, "core1");

solr.deleteByQuery("*:*");

SolrInputDocument doc1 = new SolrInputDocument();

doc1.addField( "id", "id1", 1.0f );

doc1.addField( "name", "doc1", 1.0f );

Collection<SolrInputDocument> docs = new ArrayList<SolrInputDocument>();

docs.add( doc1 );

solr.commit();

SolrCore curCore = container.getCore("core1");
 
curCore.close();



I have also seen that EmbeddedSolrServer process is not terminating after completion of the indexing process, can this be a reason. But even after manual termination of the process the 'write.lock' file stays in the index directory.


-Shyam

-----Original Message-----
From: Erick Erickson [mailto:[hidden email]]
Sent: Tuesday, January 31, 2012 7:30 PM
To: [hidden email]
Subject: Re: removing write.lock file in solr after indexing

Oops, fat fingers... Anyway, this is surprising. Can you provide
more details on how you do your indexing?

Best
Erick

On Tue, Jan 31, 2012 at 8:59 AM, Erick Erickson <[hidden email]> wrote:

> On Mon, Jan 30, 2012 at 2:42 AM, Shyam Bhaskaran
> <[hidden email]> wrote:
>> Hi,
>>
>> We are using Solr 4.0 and after indexing every time it is observed that the write.lock remains without getting cleared and for the next indexing we have to delete the file to get the indexing process running.
>>
>> We use SolrServer for our indexing and I do not see any  methods to close or clear the indexes on completion of indexing.
>>
>>
>> I have seen that adding the below lines into solrconfig.xml file avoids the issue of physically removing the write.lock file when doing indexing.
>>
>>
>>
>> <indexDefaults>
>>
>>  <lockType>simple</lockType>
>>
>>  <unlockOnStartup>true</unlockOnStartup>
>>
>> </indexDefaults>
>>
>>
>> But I am hesitant in adding this directive, as it might not be a good idea to set this directive in production as it would defeat the purpose of locking the index while another process writes into it.
>>
>> Let me know if we can do this programmatically, is there something like close() which would remove the write.lock file after completion of indexing using SolrServer?
>>
>> Thanks
>> Shyam
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: removing write.lock file in solr after indexing

shyam bhaskaran-2
In reply to this post by Erick Erickson
Hi Erick,

I was able to resolve the issue with 'write.lock' files.

Using container.remove("core1") or using container.shutdown() is helping to remove the 'write.lock' files.

-Shyam

Loading...