[jira] [Commented] (SOLR-11653) create next time collection based on a fixed time gap

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-11653) create next time collection based on a fixed time gap

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323486#comment-16323486 ]

Gus Heck commented on SOLR-11653:
---------------------------------

Just thought of something I missed in my review:

It looks like we would be creating collection names that look like this in the case of +1HOUR interval

alias_2014-01-14_22
alias_2014-01-14_23
alias_2014-01-15
alias_2014-01-15_01

or for +30MINUTE interval:

alias_2014-01-14_22
alias_2014-01-14_22_30
alias_2014-01-14_23
alias_2014-01-14_23_30
alias_2014-01-15
alias_2014-01-15_01

I think that's not very nice since the length is inconsistent lengths and would be hard for (users) who won't have our fancy formater definition on hand to parse or generate... maybe we should be creating names like these:

alias_2014-01-14_22
alias_2014-01-14_23
alias_2014-01-15_00
alias_2014-01-15_01

and

alias_2014-01-14_22_00
alias_2014-01-14_22_30
alias_2014-01-14_23_00
alias_2014-01-14_23_30
alias_2014-01-15_00_00
alias_2014-01-15_01_00

To do that we probably have to analyze the interval and decide what the smallest unit in the date math is and then record that format (or a value that maps to the right format) in metadata.

If someone specifies +60MINUTES however I'd say they just get the extra _00 on everything vs +1HOUR ...that actually could be viewed as a feature.

> create next time collection based on a fixed time gap
> -----------------------------------------------------
>
>                 Key: SOLR-11653
>                 URL: https://issues.apache.org/jira/browse/SOLR-11653
>             Project: Solr
>          Issue Type: Sub-task
>      Security Level: Public(Default Security Level. Issues are Public)
>          Components: SolrCloud
>            Reporter: David Smiley
>            Assignee: David Smiley
>             Fix For: 7.3
>
>         Attachments: SOLR-11653.patch, SOLR-11653.patch, SOLR-11653.patch
>
>
> For time series collections (as part of a collection Alias with certain metadata), we want to automatically add new collections. In this issue, this is about creating the next collection based on a configurable fixed time gap.  And we will also add this collection synchronously once a document flowing through the URP chain exceeds the gap, as opposed to asynchronously in advance.  There will be some Alias metadata to define in this issue.  The preponderance of the implementation will be in TimePartitionedUpdateProcessor or perhaps a helper to this URP.
> note: other issues will implement pre-emptive creation and capping collections by size.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]