[jira] [Commented] (LUCENE-8653) Reverse FST storage so it can be read forward

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (LUCENE-8653) Reverse FST storage so it can be read forward

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748985#comment-16748985 ]

Michael McCandless commented on LUCENE-8653:

Impressive how simple this was!  I think it's simpler to think about, reading the {{byte[]}} in forward order, and it ought to be a bit more cache friendly.  I agree jumping between FST nodes is very random access, but e.g. at a given node as we scan the arcs looking for a match that would become sequential byte reads with this change.  Curious the impact is neutral, but maybe if we combine this with LUCENE-8635 we can measure an impact?

> Reverse FST storage so it can be read forward
> ---------------------------------------------
>                 Key: LUCENE-8653
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8653
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/FSTs
>            Reporter: Mike Sokolov
>            Priority: Major
>         Attachments: fst-reverse.patch
> Discussion of keeping FST off-heap led to the idea of ensuring that FST's can be read forward in order to be more cache-friendly and align better with standard I/O practice. Today FSTs are read in reverse and this leads to some awkwardness, and you can't use standard readers so the code can be confusing to work with.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]