[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC

Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749887#comment-13749887 ]

Dawid Weiss commented on LUCENE-5168:
-------------------------------------

Just a quick update that I'm still on this issue, trying to figure out which exact hotspot optimization is causing it. The problem is somewhere on the intersection of G1GC, escape analysis and C2 optimizations. I have a reproducible scenario (although *very* large, unfortunately) which definitely misses a variable update. ByteSliceReader's readByte
{code}
  @Override
  public byte readByte() {
    assert !eof();
    assert upto <= limit;
    if (upto == limit) {
      nextSlice();
    }
    return buffer[upto++];
  }
{code}

the increment in buffer[upto++] is skipped, resulting in havoc later on that eventually leads to an assertion in FreqProxTermsWriterPerField.flush. This assertion is a follow-up of incorrectly read/ interpreted input stream's data.

The "workarounds" like declaring upto volatile are not worth the gains. Once I know which exact VM bug this is we can decide what to do with it -- I'd go back to Yonik (or Robert's?) suggestion of trying to probe the VM we're running on and if it matches any of the blacklisted versions, emitting a big red warning saying the execution is unsafe. Or throwing an exception and allowing a system-property override switch that has to be manually provided (so that whoever still decides to run under such a VM knows what they're doing).

               

> ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
> ---------------------------------------------------------------
>
>                 Key: LUCENE-5168
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5168
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: java8-windows-4x-3075-console.txt, log.0025, log.0042, log.0078, log.0086, log.0100
>
>
> This assertion trips (sometimes from different tests), if you run the highlighting tests on branch_4x with r1512807.
> It reproduces about half the time, always only with 32bit + G1GC (other combinations do not seem to trip it, i didnt try looping or anything really though).
> {noformat}
> rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
> rmuir@beast:~/workspace/branch_4x$ ant clean
> rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
> otherwise master seed does not work!
> rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
> -Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs="-server
> -XX:+UseG1GC"
> {noformat}
> Originally showed up like this:
> {noformat}
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
> Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
> 1 tests failed.
> REGRESSION:  org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
> Error Message:
> Stack Trace:
> java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
>         at org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
>         at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
>         at org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
>         at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
>         at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
>         at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
>         at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
>         at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]