[jira] Created: (SOLR-646) Configuration properties in multicore.xml

classic Classic list List threaded Threaded
71 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
Configuration properties in multicore.xml
-----------------------------------------

                 Key: SOLR-646
                 URL: https://issues.apache.org/jira/browse/SOLR-646
             Project: Solr
          Issue Type: New Feature
    Affects Versions: 1.3
            Reporter: Henri Biestro
             Fix For: 1.3


This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties defined in multicore.xml.

For instance, the following multicore.xml:

{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
  <property name='revision'>33</property>  <!-- a basic property -->
  <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
  <property name='one'>1</property>  <!-- used to expand the core1 name  -->
  <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
  <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
  <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
    <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
    <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
    <property name='revision'>11</property>
  </core>
  <core name='core${one}' instanceDir='core1/'>
    <property name='id_type'>core${one}_id</property>
    <property name='updateHandler'>solr.DirectUpdateHandler2</property>
    <property name='revision'>22</property>
  </core>
</multicore>
{code}

allows this config.xml:

{code:xml}
<config>
<!-- use the defined update handler property -->
  <updateHandler class="${updateHandler}" />

  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
 
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
 
  <!-- config for the admin interface -->
  <admin>
    <defaultQuery>solr</defaultQuery>
    <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
    <pingQuery>
     qt=standard&amp;q=solrpingquery
    </pingQuery>
  </admin>

</config>
{code}

and this schema.xml:

{code:xml}
<schema name="example core zero" version="1.1">
  <types>
   <!-- define a type name dynamically -->
    <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
  </types>

 <fields>  
  <!-- the type of unique key defined above -->
  <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
 </fields>
 <uniqueKey>id</uniqueKey>
 <defaultSearchField>name</defaultSearchField>
 <solrQueryParser defaultOperator="OR"/>
</schema>
{code}


Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they were defined so).

The core descriptor properties are automatically defined in each core context, namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName

The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.

Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Biestro updated SOLR-646:
-------------------------------

    Attachment: solr-646.patch

supersedes the version in solr-350 with some improvements & bug fixes.
Multicore will attempt to load the bundle "multicore" (aka tries to load a multicore.properties) allowing to define installation wide properties.
Unrelated but convenient to test/verify persistence, solrj now allows to persist the file with an explicit name.



> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> For instance, the following multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615698#action_12615698 ]

Noble Paul commented on SOLR-646:
---------------------------------

Can we see a use case for this kind of configuration?

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> For instance, the following multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Biestro updated SOLR-646:
-------------------------------

    Description:
This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties defined in multicore.xml.

Use cases:
Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).

The following (contrived) multicore.xml:

{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
  <property name='revision'>33</property>  <!-- a basic property -->
  <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
  <property name='one'>1</property>  <!-- used to expand the core1 name  -->
  <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
  <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
  <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
    <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
    <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
    <property name='revision'>11</property>
  </core>
  <core name='core${one}' instanceDir='core1/'>
    <property name='id_type'>core${one}_id</property>
    <property name='updateHandler'>solr.DirectUpdateHandler2</property>
    <property name='revision'>22</property>
  </core>
</multicore>
{code}

allows this config.xml:

{code:xml}
<config>
<!-- use the defined update handler property -->
  <updateHandler class="${updateHandler}" />

  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
 
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
 
  <!-- config for the admin interface -->
  <admin>
    <defaultQuery>solr</defaultQuery>
    <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
    <pingQuery>
     qt=standard&amp;q=solrpingquery
    </pingQuery>
  </admin>

</config>
{code}

and this schema.xml:

{code:xml}
<schema name="example core zero" version="1.1">
  <types>
   <!-- define a type name dynamically -->
    <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
  </types>

 <fields>  
  <!-- the type of unique key defined above -->
  <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
 </fields>
 <uniqueKey>id</uniqueKey>
 <defaultSearchField>name</defaultSearchField>
 <solrQueryParser defaultOperator="OR"/>
</schema>
{code}


Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they were defined so).

The core descriptor properties are automatically defined in each core context, namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName

The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.

Reviews & comments more than welcome.

  was:
This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties defined in multicore.xml.

For instance, the following multicore.xml:

{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
  <property name='revision'>33</property>  <!-- a basic property -->
  <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
  <property name='one'>1</property>  <!-- used to expand the core1 name  -->
  <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
  <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
  <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
    <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
    <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
    <property name='revision'>11</property>
  </core>
  <core name='core${one}' instanceDir='core1/'>
    <property name='id_type'>core${one}_id</property>
    <property name='updateHandler'>solr.DirectUpdateHandler2</property>
    <property name='revision'>22</property>
  </core>
</multicore>
{code}

allows this config.xml:

{code:xml}
<config>
<!-- use the defined update handler property -->
  <updateHandler class="${updateHandler}" />

  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
 
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
 
  <!-- config for the admin interface -->
  <admin>
    <defaultQuery>solr</defaultQuery>
    <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
    <pingQuery>
     qt=standard&amp;q=solrpingquery
    </pingQuery>
  </admin>

</config>
{code}

and this schema.xml:

{code:xml}
<schema name="example core zero" version="1.1">
  <types>
   <!-- define a type name dynamically -->
    <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
  </types>

 <fields>  
  <!-- the type of unique key defined above -->
  <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
 </fields>
 <uniqueKey>id</uniqueKey>
 <defaultSearchField>name</defaultSearchField>
 <solrQueryParser defaultOperator="OR"/>
</schema>
{code}


Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they were defined so).

The core descriptor properties are automatically defined in each core context, namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName

The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.

Reviews & comments more than welcome.


> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615725#action_12615725 ]

Henri Biestro commented on SOLR-646:
------------------------------------

Added some use cases. Thanks Paul.

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Biestro updated SOLR-646:
-------------------------------

    Description:
This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties defined in multicore.xml.

h3. Use cases:
Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).

h3. Syntax:
h4. Defining properties:
Either in the multicore.properties file (usual property format):
{code:xml}
env.value=an installation value
env.other_value=another installation value
{code}
In the multicore.xml:
{code:xml}
<multicore'>
  <property name='mp0'>a value</property>  <!-- a basic property -->
  <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
  <core name='core_name'>
     <property name='p0'>some value</property>
     <property name='p1'>${mp0}-and some data</property>
  </core>
</multicore>
{code}
h4. Using properties:
Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
{code}
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
{code}
All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
{code:xml}
<dataDir>${core.dataDir}</dataDir>
{code}

h4. Technical specifications
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they were defined so).

The core descriptor properties are automatically defined in each core context, namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName

h4. Test:
The following (contrived) multicore.xml:

{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
  <property name='revision'>33</property>  <!-- a basic property -->
  <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
  <property name='one'>1</property>  <!-- used to expand the core1 name  -->
  <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
  <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
  <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
    <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
    <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
    <property name='revision'>11</property>
  </core>
  <core name='core${one}' instanceDir='core1/'>
    <property name='id_type'>core${one}_id</property>
    <property name='updateHandler'>solr.DirectUpdateHandler2</property>
    <property name='revision'>22</property>
  </core>
</multicore>
{code}

allows this config.xml:

{code:xml}
<config>
<!-- use the defined update handler property -->
  <updateHandler class="${updateHandler}" />

  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
 
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
 
  <!-- config for the admin interface -->
  <admin>
    <defaultQuery>solr</defaultQuery>
    <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
    <pingQuery>
     qt=standard&amp;q=solrpingquery
    </pingQuery>
  </admin>

</config>
{code}

and this schema.xml:

{code:xml}
<schema name="example core zero" version="1.1">
  <types>
   <!-- define a type name dynamically -->
    <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
  </types>

 <fields>  
  <!-- the type of unique key defined above -->
  <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
 </fields>
 <uniqueKey>id</uniqueKey>
 <defaultSearchField>name</defaultSearchField>
 <solrQueryParser defaultOperator="OR"/>
</schema>
{code}

Allow the trunk test to work.

h3. Coding notes:
The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.

Reviews & comments more than welcome.

  was:
This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties defined in multicore.xml.

Use cases:
Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).

The following (contrived) multicore.xml:

{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
  <property name='revision'>33</property>  <!-- a basic property -->
  <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
  <property name='one'>1</property>  <!-- used to expand the core1 name  -->
  <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
  <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
  <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
    <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
    <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
    <property name='revision'>11</property>
  </core>
  <core name='core${one}' instanceDir='core1/'>
    <property name='id_type'>core${one}_id</property>
    <property name='updateHandler'>solr.DirectUpdateHandler2</property>
    <property name='revision'>22</property>
  </core>
</multicore>
{code}

allows this config.xml:

{code:xml}
<config>
<!-- use the defined update handler property -->
  <updateHandler class="${updateHandler}" />

  <requestDispatcher handleSelect="true" >
    <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
  </requestDispatcher>
 
  <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
 
  <!-- config for the admin interface -->
  <admin>
    <defaultQuery>solr</defaultQuery>
    <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
    <pingQuery>
     qt=standard&amp;q=solrpingquery
    </pingQuery>
  </admin>

</config>
{code}

and this schema.xml:

{code:xml}
<schema name="example core zero" version="1.1">
  <types>
   <!-- define a type name dynamically -->
    <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
  </types>

 <fields>  
  <!-- the type of unique key defined above -->
  <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
  <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
  <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
 </fields>
 <uniqueKey>id</uniqueKey>
 <defaultSearchField>name</defaultSearchField>
 <solrQueryParser defaultOperator="OR"/>
</schema>
{code}


Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they were defined so).

The core descriptor properties are automatically defined in each core context, namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName

The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.

Reviews & comments more than welcome.


> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615875#action_12615875 ]

Noble Paul commented on SOLR-646:
---------------------------------

how is this evaluated? The syntax is not obvious as to what it is doing. I am referring to the type attribute in the following snippet

{code}
 <field name="id"   type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
{code}

I can't imagine a usecase where field types are dynamically assigned

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615894#action_12615894 ]

Henri Biestro commented on SOLR-646:
------------------------------------


...type='${id_type:id_t}'... : evaluate the property 'id_type', if no value found default to 'id_t''.
Let's consider you want to have the same schema (at the field level) but in different cores for different languages;  you can dynamically set a field type based on a locale (stemming/stopwords).
I agree that using the id field in the example is misleading.


> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617322#action_12617322 ]

Henri Biestro commented on SOLR-646:
------------------------------------

I'll update soon the patch to add an 'import' feature (allowing to import chunks of xml) in config & schema as in <import resource='stuff.xml'/> or even <import resource='${my.types}'/>.

Configuration not being a high-priority feature in the Solr community, let me know if anyone is interested so I put more efforts in documenting this & the follow up.

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617323#action_12617323 ]

Shalin Shekhar Mangar commented on SOLR-646:
--------------------------------------------

I haven't looked into the patch but just fyi, there's a VariableResolver, VariableResolverImpl and TemplateString classes in DataImportHandler (SOLR-469) that also support general variable substitution. However that code does not support specifying defaults the way DOMUtil does.

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Biestro updated SOLR-646:
-------------------------------

    Attachment: solr-646.patch

Besides the <property .../> syntax, the latest patch version adds the <import resource='resource'/> (where resource can be a constant or a variable).
This means config and schemas can be chunked in multiple pieces that can be reused in different cores. Combined with properties, this completely frees up core configuration.
Also added a <fragment>...</fragment> tag to allow importing XML chunks that don't have a natural root node.

The PropertyMap.imports() allows to enumerate which expressions were resolved as resources locally (aka config, schema, import) and their logical location when implicit. (Map.Entry<String,String[]> e; e.getValue()[0] is the canonical resource value, getValue()[1] the 'symbolic' one. When value[0] == value[1], it means this was absolute or found in a JAR.

Shalin, thanks, good to know; I've only been using/refactoring DOMUtil since this patch mainly deals with property expansion (besides definition & book-keeping).

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618782#action_12618782 ]

Hoss Man commented on SOLR-646:
-------------------------------

i haven't looked at the patch, but i'm +1 on the write up and the description of how things would/should work ... the only thing that jumps out at me is the automatic properties...

{quote}
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
{quote}

...first off, i'm not sure what the difference between instanceDir and instancePath are or what configName means (but that might just be my general out of touch-ness with all the multicore advancements, in which case don't worry about it.)  My main questions:
* Are the values of these automatic properties invariant, or can the be overridden?
* Should we use some special prefix/syntax for automatic properties such that we reduce/eliminate the risk of collision if we add new ones in future versions? (example: the automatic properties could all have names starting with "_", and the code that reads property declarations could ignore/warn if a user attempts to declare a property that (might) collide in the future.

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619277#action_12619277 ]

Lance Norskog commented on SOLR-646:
------------------------------------

How will these files and substituted properties show up in the admin JSPs?

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619330#action_12619330 ]

Henri Biestro commented on SOLR-646:
------------------------------------

About documentation:
I'll try to come with better (ie more meaningful) examples - although I kinda feel "impaired" to describe the 3 tags (property/import/fragment) if what I've already written does not convey 90% of how things would/should work already. I welcome any help/hint on what a useful documentation should look like (any template I could/should follow)?
On the documentation editing/releasing process, I'd tend to go for refining this issue description till the code gets committed at which time I'd copy/paste it in a Wiki page if that's OK.

About "automatic" properties:
Those reflect the Core descriptor values or in more general terms, any Solr install/multicore/core configuration/schema value used in the system but defined through another API; the core name, instance directory, configuration file name (defaults to solrconfig.xml), schema file name (defaults to schema.xml) all come form the core descriptor .
solr.core.instanceDir is strictly what the core descriptor instance directory is; solr.core.instancePath is the non-relative version of it. When solr.core.instanceDir is *not* an absolute path, is is relative to the solr.home instance directory; in this case, solr.core.instancePath is the non relative version. Otherwise, they are just the same; overall, this just reflect what the code does but removes any ambiguous implicits (which is what the PropertyMap.imports() also resolves*).

I've also added the solr.core.schemaDir & solr.core.configDir- the directories where the config & schema files reside. We have no way yet to specify those last two but they are inferred from the other values - which would be their defaults if un-specified.There is a functional choice to make about how we want to complete the "core" properties; the "missing" parts are the config & schema directories. My preference would be to extend the core descriptor and reflect those as properties but we might want&try to go the other way, put everything as properties & retain the API as shortcuts.

Regarding naming conventions/protection, I would tend to go for the natural attribute-like convention & protect those; the automatic/reserved namespaces would be any property starting with 'solr.' (including 'solr.core' and 'solr.multicore') and disallow/warn about any attempt to override. Would that be acceptable/enough ?

About files/substituted properties & admin JSPs:
As is, nothing has changed; getting the schema/config return the file as it exists in the file-system. The first natural extension would be that for config & schema, when allowed, the imported files should be downloadable too. It might be nice to have the download expanded version of each file for debugging purpose as well if this is actually useful and/or the expanded version of config & schema; any functional suggestions is welcome. Lance, what do you think would be good to have?

About PropertyMap.imports():
The rationale behind this -besides debugging/logging purpose- would be "dynamic" replication of a core (a slave Slr 'install' could automatically replicate new "master' cores. The goal of this API is to have the canonical (local) path of each file - so we can read them- and their logical/symbolic counterparts so their destination (remote) path can be determined, each core's solrconfig.xml itself being "replicable".

About the issue itself:
How much of these should be specified/implemented before the patch can be committed and/or is there any chance to put this in the 1.3 scope ?


> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619406#action_12619406 ]

Jeremy Hinegardner commented on SOLR-646:
-----------------------------------------

There seems to be some issues with this patch, I would like to use it, but it doesn't apply cleanly to svn head, patch seems to mess up in the first hunk of Config.java.  I manually applied that portion, and now I'm recieveing an the error:


{noformat}
[javac] /Users/jeremy/repos/svn/solr/apache-solr-nightly/src/java/org/apache/solr/core/MultiCore.java:106: cannot find symbol
[javac] symbol  : method keySet()
[javac] location: class java.util.ResourceBundle
[javac]         Iterator<String> ikeys = bundle.keySet().iterator();
{noformat}


My java is rusty, but this appears to be a Java 6 only call on Resource Bundle.  Is solr 1.3 going to be Java 6 only ?

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619407#action_12619407 ]

Ryan McKinley commented on SOLR-646:
------------------------------------

yes, keySet() is 1.6 only....

Solr is 1.5, so this will need to get changed before going in...

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619408#action_12619408 ]

Ryan McKinley commented on SOLR-646:
------------------------------------

{panel}
About the issue itself:
How much of these should be specified/implemented before the patch can be committed and/or is there any chance to put this in the 1.3 scope ?
{panel}
Yes, I think this should be in 1.3 -- it makes MultiCore dramatically more useful...

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619531#action_12619531 ]

Raghu kashyap commented on SOLR-646:
------------------------------------

+1 for it being in 1.3

We are using multicore and eagerly waiting for 1.3

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henri Biestro updated SOLR-646:
-------------------------------

    Attachment: solr-646.patch

Fixed dependency on Java 6; my bad, sorry Jeremy.

Solved 2 questions pending from the previous patch:

1 - Added schemaDir & configDir to solr core descriptors; this means all 'automatic' properties come as representations of values coming from the core descriptor. I removed the solr.core.instancePath and homogenized the solr.core.{instance,schema,config}Dir properties to be expanded relative to the multicore instance dir if not absolute.
2 - Added some checking/warning if some user defined properties collide with automatic ones during core creation; the system computed value is the one kept. The current list of automated properties (& example values):
solr.core.configDir = ..\..\..\example\multicore\core0\conf/
solr.core.configName = solrconfig.xml
solr.core.name = core0
solr.core.instanceDir = ..\..\..\example\multicore\core0/
solr.core.schemaName = schema.xml
solr.core.schemaDir = ..\..\..\example\multicore\core0\conf/

I'll udpate the issue description to reflect the current choices.

New (sub) issue/choice to make:
There is no way yet to specify properties through solrj when creating a core (nor a way to specify the core {config, schema}{Dir, Name}). Any preferred syntax? property.<name_of_property0>=<value_of_property0>&property.<name_of_property1>=<value_of_property1> is my current idea to pass them in a http request but this is heavy...



> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (SOLR-646) Configuration properties in multicore.xml

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan McKinley reassigned SOLR-646:
----------------------------------

    Assignee: Ryan McKinley

> Configuration properties in multicore.xml
> -----------------------------------------
>
>                 Key: SOLR-646
>                 URL: https://issues.apache.org/jira/browse/SOLR-646
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.3
>            Reporter: Henri Biestro
>            Assignee: Ryan McKinley
>             Fix For: 1.3
>
>         Attachments: solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in multicore.properties (different base/install/data directories on different hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
>   <property name='mp0'>a value</property>  <!-- a basic property -->
>   <property name='mp1'>${env.value}</property>  }<!-- a property whose value is an expression -->
>   <core name='core_name'>
>      <property name='p0'>some value</property>
>      <property name='p1'>${mp0}-and some data</property>
>   </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if they were defined so).
> The core descriptor properties are automatically defined in each core context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
>   <property name='revision'>33</property>  <!-- a basic property -->
>   <property name='zero'>0</property>  <!-- used to expand the core0 name  -->
>   <property name='one'>1</property>  <!-- used to expand the core1 name  -->
>   <property name='id_type'>bogus</property>  <!-- a bogus type that will be overriden  -->
>   <property name='updateHandler'>bogus</property>  <!-- a bogus updateHandler that will be overriden  -->
>   <core name='core${zero}' instanceDir='core0/'>    <!-- the name is expanded -->
>     <property name='id_type'>core${zero}_id</property> <!-- so is a text node -->
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a property can be overriden -->
>     <property name='revision'>11</property>
>   </core>
>   <core name='core${one}' instanceDir='core1/'>
>     <property name='id_type'>core${one}_id</property>
>     <property name='updateHandler'>solr.DirectUpdateHandler2</property>
>     <property name='revision'>22</property>
>   </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
>   <updateHandler class="${updateHandler}" />
>   <requestDispatcher handleSelect="true" >
>     <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
>   </requestDispatcher>
>  
>   <requestHandler name="standard" class="solr.StandardRequestHandler" default="true" />
>   <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>   <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
>  
>   <!-- config for the admin interface -->
>   <admin>
>     <defaultQuery>solr</defaultQuery>
>     <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
>     <pingQuery>
>      qt=standard&amp;q=solrpingquery
>     </pingQuery>
>   </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
>   <types>
>    <!-- define a type name dynamically -->
>     <fieldtype name="${id_type:id_t}"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>     <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
>   </types>
>  <fields>  
>   <!-- the type of unique key defined above -->
>   <field name="id"      type="${id_type:id_t}"   indexed="true"  stored="true"  multiValued="false" required="true"/>
>   <field name="type"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="name"    type="string"   indexed="true"  stored="true"  multiValued="false" />
>   <field name="${solr.core.name:core}"   type="string"   indexed="true"  stored="true"  multiValued="false" />
>  </fields>
>  <uniqueKey>id</uniqueKey>
>  <defaultSearchField>name</defaultSearchField>
>  <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map); those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

1234