[jira] [Commented] (SOLR-6671) Introduce a solr.data.home as root dir for all data

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (SOLR-6671) Introduce a solr.data.home as root dir for all data

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871741#comment-15871741 ]

Jan Høydahl commented on SOLR-6671:

I see many customer examples where there is a wish for separating data home from config home, so I'd like to push this forward again.

The default for SOLR_DATA_HOME could still be same as SOLR_HOME, but in the linux installer script, we could default to using {{/var/solr/data}} for data and {{/var/solr/home}} for home, so solr.in.sh would typically look like:

and produce this tree:

├── data
│   ├── bar
│   │   └── data
│   │       ├── index
│   │       └── tlog
│   └── foo
│       └── data
│           ├── index
│           └── tlog
├── home
│   ├── bar
│   │   ├── conf
│   │   │   ├── managed-schema
│   │   │   └── solrconfig.xml
│   │   └── core.properties
│   ├── foo
│   │   ├── conf
│   │   │   ├── managed-schema
│   │   │   └── solrconfig.xml
│   │   └── core.properties
│   ├── solr.xml
│   └── zoo.cfg
├── log4j.properties
└── logs
    └── solr.log.1

Benefit is that it is super easy move data to a new partition/disk with a single {{mv}} command. We just now have a customer who upgrade from 4.x to 6.x using Linux installer, but still want to run non-cloud. They need to separate data from config, i.e. they are not happy to have configs in /var/solr/data together with data, it makes upgrading only the config harder. Today they solve it by hardcoding {{<dir>}} in every single solrconfig.xml. In the new install I have used symlinks for each conf folder instead, so they can have a partition where they replace the {{home/<core>/conf}} folders from SCM without disturbing data.

This would also help solve SOLR-10095.

> Introduce a solr.data.home as root dir for all data
> ---------------------------------------------------
>                 Key: SOLR-6671
>                 URL: https://issues.apache.org/jira/browse/SOLR-6671
>             Project: Solr
>          Issue Type: New Feature
>          Components: SolrCloud
>    Affects Versions: 4.10.1
>            Reporter: Jan Høydahl
>            Assignee: Jan Høydahl
>             Fix For: 6.2, master (7.0)
>         Attachments: SOLR-6671.patch, SOLR-6671.patch, SOLR-6671.patch, SOLR-6671.patch, SOLR-6671.patch
> Many users prefer to deploy code, config and data on separate disk locations, so the default of placing the indexes under {{$\{solr.solr.home\}/$\{solr.core.name\}/data}} is not always wanted.
> In a multi-core/collection system, there is not much help in the {{solr.data.dir}} option, as it would set the {{dataDir}} to the same folder for all collections. One workaround, if you don't want to hardcode paths in your {{solrconfig.xml}}, is to specify the {{dataDir}} property in each {{solr.properties}} file.
> A more elegant solution would be to introduce a new Java-option {{solr.data.home}} which would be to data the same as {{solr.solr.home}} is for config. If set, all collections would default their {{dataDir}} as {{$\{solr.data.home\)/$\{solr.core.name\}/data}}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]