[jira] [Commented] (LUCENE-8653) Reverse FST storage so it can be read forward

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-8653) Reverse FST storage so it can be read forward

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748985#comment-16748985 ]

Michael McCandless commented on LUCENE-8653:
--------------------------------------------

Impressive how simple this was!  I think it's simpler to think about, reading the {{byte[]}} in forward order, and it ought to be a bit more cache friendly.  I agree jumping between FST nodes is very random access, but e.g. at a given node as we scan the arcs looking for a match that would become sequential byte reads with this change.  Curious the impact is neutral, but maybe if we combine this with LUCENE-8635 we can measure an impact?

> Reverse FST storage so it can be read forward
> ---------------------------------------------
>
>                 Key: LUCENE-8653
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8653
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/FSTs
>            Reporter: Mike Sokolov
>            Priority: Major
>         Attachments: fst-reverse.patch
>
>
> Discussion of keeping FST off-heap led to the idea of ensuring that FST's can be read forward in order to be more cache-friendly and align better with standard I/O practice. Today FSTs are read in reverse and this leads to some awkwardness, and you can't use standard readers so the code can be confusing to work with.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]