[jira] [Created] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)
DIH - multithreaded DocBuilder ignores onError Attribute
--------------------------------------------------------

                 Key: SOLR-2668
                 URL: https://issues.apache.org/jira/browse/SOLR-2668
             Project: Solr
          Issue Type: Bug
          Components: contrib - DataImportHandler
    Affects Versions: 3.3
            Reporter: Frank Wesemann


If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Frank Wesemann updated SOLR-2668:
---------------------------------

    Attachment: SOLR-2668.patch

patch for the unittest to clarify the problem

> DIH - multithreaded DocBuilder ignores onError Attribute
> --------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>         Attachments: SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Assigned] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar reassigned SOLR-2668:
-------------------------------------------

    Assignee: Shalin Shekhar Mangar

> DIH - multithreaded DocBuilder ignores onError Attribute
> --------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070767#comment-13070767 ]

Shalin Shekhar Mangar commented on SOLR-2668:
---------------------------------------------

Thanks for the test Frank. There are more problems. In single thread mode, an error causes a rollback but in threaded mode, a commit is called.

> DIH - multithreaded DocBuilder ignores onError Attribute
> --------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar updated SOLR-2668:
----------------------------------------

    Attachment: SOLR-2668.patch

I was looking at this problem again today. The onError attributes are not used to deal with exceptions from EntityProcessor.init() method. They are only used for reading rows, applying transformers and inserting documents into Solr.

The real problem was that in multi-threaded mode, the exceptions from EntityProcessor.init were being eaten up so a commit was called instead of rolling back the changes. I've fixed that to re-throw the exception up the hierarchy.

> DIH - multithreaded DocBuilder ignores onError Attribute
> --------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch, SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2668) DIH - multithreaded DocBuilder ignores onError Attribute

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar updated SOLR-2668:
----------------------------------------

    Attachment: SOLR-2668.patch

Fixed a bug in the tests. The index was not being emptied in tearDown so even after rollback, the index had 4 documents.

Just to show that multi-threaded does honor the onError attribute, I've added a test which uses a transformer which always fails.

The older two tests are used to demonstrate that both multi-threaded and serial mode show the same characteristic for a failing entity processor.

> DIH - multithreaded DocBuilder ignores onError Attribute
> --------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch, SOLR-2668.patch, SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
>  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2668) DIH - multithreaded does not rollback on errors from EntityProcessor

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar updated SOLR-2668:
----------------------------------------

    Description: If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder logs "Import completed successfully" and calls commit instead of calling rollback.  (was: If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder ignores onError=continue or skip attributes on the parent entity. DocBuilder stops immediately and logs "Import completed successfully".
 )
        Summary: DIH - multithreaded does not rollback on errors from EntityProcessor  (was: DIH - multithreaded DocBuilder ignores onError Attribute)

> DIH - multithreaded does not rollback on errors from EntityProcessor
> --------------------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch, SOLR-2668.patch, SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder logs "Import completed successfully" and calls commit instead of calling rollback.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (SOLR-2668) DIH - multithreaded does not rollback on errors from EntityProcessor

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar resolved SOLR-2668.
-----------------------------------------

    Resolution: Fixed

Committed revision 1161505 on trunk and 1161510 on branch_3x.

> DIH - multithreaded does not rollback on errors from EntityProcessor
> --------------------------------------------------------------------
>
>                 Key: SOLR-2668
>                 URL: https://issues.apache.org/jira/browse/SOLR-2668
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Frank Wesemann
>            Assignee: Shalin Shekhar Mangar
>         Attachments: SOLR-2668.patch, SOLR-2668.patch, SOLR-2668.patch
>
>
> If the EntityProcessor of a subentity throws an Exception in its init() Method, DocBuilder logs "Import completed successfully" and calls commit instead of calling rollback.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]