[jira] [Commented] (LUCENE-7966) build mr-jar and use some java 9 methods if available

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-7966) build mr-jar and use some java 9 methods if available

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162820#comment-16162820 ]

Robert Muir commented on LUCENE-7966:
-------------------------------------

Dawid: i didn't really add new checks in the patch: I converted a bunch of existing checks to the new Objects.checkXXX methods. I think any difference in exception messages is not a big deal, actually i like the consistency and like the newer exception messages (see the unit tests for those: i emulated what java 9 returns in the java 8 methods).

The only place where there are new "checks" is where we use Arrays.compare/equals/mismatch. Those methods are defined in the jdk as safe, of course, and as I mentioned here, the existing comparator/etc methods we have are not very safe. See especially the prefix-coding corner cases I ran into here :)
For those methods, the hope is that its still net/net drowned out: and hopefully definitely compensated by the fact that these methods no longer go one-byte-at-a-time.

In all cases none of this stuff should really be a hotspot for lucene. IMO we should be encouraging more safety/checks/reliability, and thats the goal here.

> build mr-jar and use some java 9 methods if available
> -----------------------------------------------------
>
>                 Key: LUCENE-7966
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7966
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: general/build
>            Reporter: Robert Muir
>         Attachments: LUCENE-7966.patch, LUCENE-7966.patch
>
>
> See background: http://openjdk.java.net/jeps/238
> It would be nice to use some of the newer array methods and range checking methods in java 9 for example, without waiting for lucene 10 or something. If we build an MR-jar, we can start migrating our code to use java 9 methods right now, it will use optimized methods from java 9 when thats available, otherwise fall back to java 8 code.  
> This patch adds:
> {code}
> Objects.checkIndex(int,int)
> Objects.checkFromToIndex(int,int,int)
> Objects.checkFromIndexSize(int,int,int)
> Arrays.mismatch(byte[],int,int,byte[],int,int)
> Arrays.compareUnsigned(byte[],int,int,byte[],int,int)
> Arrays.equal(byte[],int,int,byte[],int,int)
> // did not add char/int/long/short/etc but of course its possible if needed
> {code}
> It sets these up in {{org.apache.lucene.future}} as 1-1 mappings to java methods. This way, we can simply directly replace call sites with java 9 methods when java 9 is a minimum. Simple 1-1 mappings mean also that we only have to worry about testing that our java 8 fallback methods work.
> I found that many of the current byte array methods today are willy-nilly and very lenient for example, passing invalid offsets at times and relying on compare methods not throwing exceptions, etc. I fixed all the instances in core/codecs but have not looked at the problems with AnalyzingSuggester. Also SimpleText still uses a silly method in ArrayUtil in similar crazy way, have not removed that one yet.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]