[jira] [Created] (LUCENE-4084) unable to generate a 4.x back compat test index

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
Robert Muir created LUCENE-4084:
-----------------------------------

             Summary: unable to generate a 4.x back compat test index
                 Key: LUCENE-4084
                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
             Project: Lucene - Java
          Issue Type: Bug
          Components: core/index
            Reporter: Robert Muir
             Fix For: 4.0


It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?

stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284948#comment-13284948 ]

Robert Muir commented on LUCENE-4084:
-------------------------------------

{noformat}
  [junit4] ERROR   0.23s | TestBackwardsCompatibility.testCreateCFS
   [junit4]    > Throwable #1: org.apache.lucene.util.SetOnce$AlreadySetException: The object cannot be set twice!
   [junit4]    > at __randomizedtesting.SeedInfo.seed([2ADDAAF27DE7C63D:38D013F5650FDD60]:0)
   [junit4]    > at org.apache.lucene.util.SetOnce.set(SetOnce.java:69)
   [junit4]    > at org.apache.lucene.index.DocumentsWriterPerThreadPool.initialize(DocumentsWriterPerThreadPool.java:139)
   [junit4]    > at org.apache.lucene.index.DocumentsWriter.<init>(DocumentsWriter.java:145)
   [junit4]    > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:668)
   [junit4]    > at org.apache.lucene.index.TestBackwardsCompatibility.createIndex(TestBackwardsCompatibility.java:464)
   [junit4]    > at org.apache.lucene.index.TestBackwardsCompatibility.testCreateCFS(TestBackwardsCompatibility.java:78)
{noformat}
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>             Fix For: 4.0
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284996#comment-13284996 ]

Simon Willnauer commented on LUCENE-4084:
-----------------------------------------

same is true for MergePolicy no?
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>             Fix For: 4.0
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Assigned] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless reassigned LUCENE-4084:
------------------------------------------

    Assignee: Michael McCandless
   

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285026#comment-13285026 ]

Robert Muir commented on LUCENE-4084:
-------------------------------------

Yes:

Additionally, I think its possible this might affect 3.x as well (I didnt test). depending on the fix maybe we can also backport if its appropriate.

Today IWC documents that IW makes a private clone() so you can edit some settings live, but this IWC.clone() is shallow so this isnt really working.

               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-4084:
---------------------------------------

    Attachment: LUCENE-4084.patch

Patch.

I fixed IWC.clone to do a deepish-clone for those members that have IW-specific state, and then fixed various tests that relied on those members not being cloned ....

Tests pass, but the javadocs-lint is angry... not sure why yet.
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-4084:
--------------------------------

    Attachment: LUCENE-4084.patch

patch with javadocs-lint working.

Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285056#comment-13285056 ]

Michael McCandless commented on LUCENE-4084:
--------------------------------------------

bq. Basically the issue is that RandomDWPTThreadPool was public, but its pkg-private subclass had the new interface (Cloneable), and i think this confuses class-use etc etc.

Aha!  OK, thanks for fixing.... I think patch is ready now.
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285063#comment-13285063 ]

Simon Willnauer commented on LUCENE-4084:
-----------------------------------------

I think we need to add a changes entry too since MP now needs to impl clone in some cases?
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285073#comment-13285073 ]

Michael McCandless commented on LUCENE-4084:
--------------------------------------------

bq. I think we need to add a changes entry too since MP now needs to impl clone in some cases?

I agree, I'll add...
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-4084:
---------------------------------------

    Attachment: LUCENE-4084.patch

New patch, starting from Rob's patch and adding changes.txt entry.  I also cutover the IWC's SetOnce to AtomicBoolean, to record if this IWC is private to an IW to catch misuse.

I think it's ready.
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285234#comment-13285234 ]

Robert Muir commented on LUCENE-4084:
-------------------------------------

looks good, +1
               

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (LUCENE-4084) unable to generate a 4.x back compat test index

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-4084.
----------------------------------------

       Resolution: Fixed
    Fix Version/s: 5.0
   

> unable to generate a 4.x back compat test index
> -----------------------------------------------
>
>                 Key: LUCENE-4084
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4084
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: 4.0, 5.0
>
>         Attachments: LUCENE-4084.patch, LUCENE-4084.patch, LUCENE-4084.patch
>
>
> It seems the issue is an IWC cannot be used across 2 different indexwriters, because the DWPTThreadPool is created on the IndexWriterConfig, and you hit a SetOnce exception... seems like the exception is correct and it would be bad to have two indexwriters sharing the same DWPTThreadPool?
> stacktrace to follow in a comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]