[jira] Created: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

JIRA jira@apache.org
Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr
------------------------------------------------------------------------------------------------------

                 Key: SOLR-1976
                 URL: https://issues.apache.org/jira/browse/SOLR-1976
             Project: Solr
          Issue Type: Bug
          Components: clients - java
    Affects Versions: 1.4.1
            Reporter: Stephen Duncan Jr


Because of it's multi-threaded nature, calling commit on StreamingUpdateSolrServer  can send the commit before all the added documents have been sent to Solr.  Calling blockUntilFinished() does not change this.  It needs to be possible to send a commit that will commit all the documents that have been added previously.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886441#action_12886441 ]

Mark Miller commented on SOLR-1976:
-----------------------------------

Can you expound on this? It should block until all threads are done adding before doing the commit - are you seeing a bug?

> Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1976
>                 URL: https://issues.apache.org/jira/browse/SOLR-1976
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 1.4.1
>            Reporter: Stephen Duncan Jr
>
> Because of it's multi-threaded nature, calling commit on StreamingUpdateSolrServer  can send the commit before all the added documents have been sent to Solr.  Calling blockUntilFinished() does not change this.  It needs to be possible to send a commit that will commit all the documents that have been added previously.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886444#action_12886444 ]

Stephen Duncan Jr commented on SOLR-1976:
-----------------------------------------

Yes, I'm seeing a bug.  In my testing the commit is happening immediately, not waiting for the threads to be done adding.  So I can see the following in my logs (and matching behavior from Solr):

add
add
commit
add

Even though my code was:

add
add
add
blockUntilFinished
commit

> Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1976
>                 URL: https://issues.apache.org/jira/browse/SOLR-1976
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 1.4.1
>            Reporter: Stephen Duncan Jr
>
> Because of it's multi-threaded nature, calling commit on StreamingUpdateSolrServer  can send the commit before all the added documents have been sent to Solr.  Calling blockUntilFinished() does not change this.  It needs to be possible to send a commit that will commit all the documents that have been added previously.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886455#action_12886455 ]

Mark Miller commented on SOLR-1976:
-----------------------------------

Interesting - well to start, there does look to be at least one synchronization issue -

runners is protected by a lock on runners itself in every place but blockUntilFinished - where it is protected by a lock on the StreamUpdateSolrServer instance.
That looks like a problem, though I'm not sure its the only one, or yours.

> Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1976
>                 URL: https://issues.apache.org/jira/browse/SOLR-1976
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 1.4.1
>            Reporter: Stephen Duncan Jr
>
> Because of it's multi-threaded nature, calling commit on StreamingUpdateSolrServer  can send the commit before all the added documents have been sent to Solr.  Calling blockUntilFinished() does not change this.  It needs to be possible to send a commit that will commit all the documents that have been added previously.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1976) Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886845#action_12886845 ]

Stephen Duncan Jr commented on SOLR-1976:
-----------------------------------------

That does look like a problem, but I tested after changing blockUntilFinished to have a synchronized block on runners around its body, and it did not fix my issue.

> Commit on StreamingUpdateSolrServer can happen before all previously added docs have been sent to Solr
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1976
>                 URL: https://issues.apache.org/jira/browse/SOLR-1976
>             Project: Solr
>          Issue Type: Bug
>          Components: clients - java
>    Affects Versions: 1.4.1
>            Reporter: Stephen Duncan Jr
>
> Because of it's multi-threaded nature, calling commit on StreamingUpdateSolrServer  can send the commit before all the added documents have been sent to Solr.  Calling blockUntilFinished() does not change this.  It needs to be possible to send a commit that will commit all the documents that have been added previously.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]