About editing managed-schema by hand

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

About editing managed-schema by hand

Issei Nishigata
Hi, All

Similar questions may have been already asked, but just in case please let
me ask you.
According to the below URL it says as "Schema modifications via the Schema
API will now be enabled by default.",
but would there be any issues if I edited with text editor instead of
Schema API?

https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+5+to+Solr+6


In the answer to the past question, it seemed okay.

http://lucene.472066.n3.nabble.com/Solr-6-managed-schema-amp-version-control-td4289243.html


I was worried because managed-schema said "<!-- Solr managed schema -
automatically generated - DO NOT EDIT -->" when managed-schema was
automatically generated from schema.xml.
If I need to use Schema API and if I wanted to do some process that cannot
be done with Schema API(modifying unique key, etc), what should I do?


Thanks,
Issei
Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Erick Erickson
This is the sequence that gets you in trouble:
> start solr
> hand edit the schema _without_ reloading your collection or restarting all your Solr instances.
> use the managed-schema API to make modifications.

In this scenario your hand-edits can be lost since the in-memory version of the
schema is written out without re-fetching it from Zookeeper.

If you only ever hand-edit your schema, you'll be fine.

If you conscientiously reload your collection (or restart all your Solr's) after
you hand-edit your schema, you'll be fine even if you use the managed schema
API calls.

But really, if you want to hand-edit your schema why not go back to using
the ClassicSchemaFactory? See:
https://cwiki.apache.org/confluence/display/solr/Schema+Factory+Definition+in+SolrConfig#SchemaFactoryDefinitioninSolrConfig-SwitchingfromManagedSchematoManuallyEditedschema.xml

Best,
Erick

On Sun, Feb 26, 2017 at 8:22 AM, Issei Nishigata <[hidden email]> wrote:

> Hi, All
>
> Similar questions may have been already asked, but just in case please let
> me ask you.
> According to the below URL it says as "Schema modifications via the Schema
> API will now be enabled by default.",
> but would there be any issues if I edited with text editor instead of
> Schema API?
>
> https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+5+to+Solr+6
>
>
> In the answer to the past question, it seemed okay.
>
> http://lucene.472066.n3.nabble.com/Solr-6-managed-schema-amp-version-control-td4289243.html
>
>
> I was worried because managed-schema said "<!-- Solr managed schema -
> automatically generated - DO NOT EDIT -->" when managed-schema was
> automatically generated from schema.xml.
> If I need to use Schema API and if I wanted to do some process that cannot
> be done with Schema API(modifying unique key, etc), what should I do?
>
>
> Thanks,
> Issei
Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Issei Nishigata
Thank you for your reply.

If I was to say which one, I'd maybe be talking about the concept for Solr.
I understand we should use "ClassicSchemaFactory" when we want to
hand-edit, but why are there two files, schema.xml and managed-schema, in
spite that we can hand-edit managed-schema? If we can modify the schema.xml
through Schema API, I think we won't need the managed-schema, but is there
any reason why that can't be done?
Could you please let me know if there is any information that can clear
things up around those details?

Thanks,
Issei

2017-02-27 1:51 GMT+09:00 Erick Erickson <[hidden email]>:

> This is the sequence that gets you in trouble:
> > start solr
> > hand edit the schema _without_ reloading your collection or restarting
> all your Solr instances.
> > use the managed-schema API to make modifications.
>
> In this scenario your hand-edits can be lost since the in-memory version
> of the
> schema is written out without re-fetching it from Zookeeper.
>
> If you only ever hand-edit your schema, you'll be fine.
>
> If you conscientiously reload your collection (or restart all your Solr's)
> after
> you hand-edit your schema, you'll be fine even if you use the managed
> schema
> API calls.
>
> But really, if you want to hand-edit your schema why not go back to using
> the ClassicSchemaFactory? See:
> https://cwiki.apache.org/confluence/display/solr/
> Schema+Factory+Definition+in+SolrConfig#SchemaFactoryDefinitioninSolrC
> onfig-SwitchingfromManagedSchematoManuallyEditedschema.xml
>
> Best,
> Erick
>
> On Sun, Feb 26, 2017 at 8:22 AM, Issei Nishigata <[hidden email]>
> wrote:
> > Hi, All
> >
> > Similar questions may have been already asked, but just in case please
> let
> > me ask you.
> > According to the below URL it says as "Schema modifications via the
> Schema
> > API will now be enabled by default.",
> > but would there be any issues if I edited with text editor instead of
> > Schema API?
> >
> > https://cwiki.apache.org/confluence/display/solr/Major+
> Changes+from+Solr+5+to+Solr+6
> >
> >
> > In the answer to the past question, it seemed okay.
> >
> > http://lucene.472066.n3.nabble.com/Solr-6-managed-
> schema-amp-version-control-td4289243.html
> >
> >
> > I was worried because managed-schema said "<!-- Solr managed schema -
> > automatically generated - DO NOT EDIT -->" when managed-schema was
> > automatically generated from schema.xml.
> > If I need to use Schema API and if I wanted to do some process that
> cannot
> > be done with Schema API(modifying unique key, etc), what should I do?
> >
> >
> > Thanks,
> > Issei
>
Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Erick Erickson
see: https://cwiki.apache.org/confluence/display/solr/Schema+Factory+Definition+in+SolrConfig#SchemaFactoryDefinitioninSolrConfig-Switchingfromschema.xmltoManagedSchema

bq: If we can modify the schema.xml
through Schema API

You can't do this by default, but if you insist you can change
the SchemaFactory in solrcofnig.xml to use any file you
want.

Best,
Erick

On Mon, Feb 27, 2017 at 3:46 AM, Issei Nishigata <[hidden email]> wrote:

> Thank you for your reply.
>
> If I was to say which one, I'd maybe be talking about the concept for Solr.
> I understand we should use "ClassicSchemaFactory" when we want to
> hand-edit, but why are there two files, schema.xml and managed-schema, in
> spite that we can hand-edit managed-schema? If we can modify the schema.xml
> through Schema API, I think we won't need the managed-schema, but is there
> any reason why that can't be done?
> Could you please let me know if there is any information that can clear
> things up around those details?
>
> Thanks,
> Issei
>
> 2017-02-27 1:51 GMT+09:00 Erick Erickson <[hidden email]>:
>
>> This is the sequence that gets you in trouble:
>> > start solr
>> > hand edit the schema _without_ reloading your collection or restarting
>> all your Solr instances.
>> > use the managed-schema API to make modifications.
>>
>> In this scenario your hand-edits can be lost since the in-memory version
>> of the
>> schema is written out without re-fetching it from Zookeeper.
>>
>> If you only ever hand-edit your schema, you'll be fine.
>>
>> If you conscientiously reload your collection (or restart all your Solr's)
>> after
>> you hand-edit your schema, you'll be fine even if you use the managed
>> schema
>> API calls.
>>
>> But really, if you want to hand-edit your schema why not go back to using
>> the ClassicSchemaFactory? See:
>> https://cwiki.apache.org/confluence/display/solr/
>> Schema+Factory+Definition+in+SolrConfig#SchemaFactoryDefinitioninSolrC
>> onfig-SwitchingfromManagedSchematoManuallyEditedschema.xml
>>
>> Best,
>> Erick
>>
>> On Sun, Feb 26, 2017 at 8:22 AM, Issei Nishigata <[hidden email]>
>> wrote:
>> > Hi, All
>> >
>> > Similar questions may have been already asked, but just in case please
>> let
>> > me ask you.
>> > According to the below URL it says as "Schema modifications via the
>> Schema
>> > API will now be enabled by default.",
>> > but would there be any issues if I edited with text editor instead of
>> > Schema API?
>> >
>> > https://cwiki.apache.org/confluence/display/solr/Major+
>> Changes+from+Solr+5+to+Solr+6
>> >
>> >
>> > In the answer to the past question, it seemed okay.
>> >
>> > http://lucene.472066.n3.nabble.com/Solr-6-managed-
>> schema-amp-version-control-td4289243.html
>> >
>> >
>> > I was worried because managed-schema said "<!-- Solr managed schema -
>> > automatically generated - DO NOT EDIT -->" when managed-schema was
>> > automatically generated from schema.xml.
>> > If I need to use Schema API and if I wanted to do some process that
>> cannot
>> > be done with Schema API(modifying unique key, etc), what should I do?
>> >
>> >
>> > Thanks,
>> > Issei
>>
Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Shawn Heisey-2
In reply to this post by Issei Nishigata
2/27/2017 4:46 AM, Issei Nishigata wrote:
> Thank you for your reply. If I was to say which one, I'd maybe be
> talking about the concept for Solr. I understand we should use
> "ClassicSchemaFactory" when we want to hand-edit, but why are there
> two files, schema.xml and managed-schema, in spite that we can
> hand-edit managed-schema? If we can modify the schema.xml through
> Schema API, I think we won't need the managed-schema, but is there any
> reason why that can't be done? Could you please let me know if there
> is any information that can clear things up around those details?

The default filename with the Managed Schema factory is managed-schema
-- no extension.  I'm pretty sure that the reason the extension was
removed was to discourage hand-editing.  If you use both hand-editing
and API modification, you can lose some (or maybe all) of your hand edits.

The default filename for the schema with the classic factory is
schema.xml.  With this factory, API modification is not possible.

If the managed factory is in use, and a schema.xml file is found during
startup, the system will rename managed-schema (or whatever the config
says to use) to something else, then rename schema.xml to managed-schema
-- basically this is a startup-only way to support a legacy config.

I personally don't ever plan to use the managed schema API, but I will
leave the default factory in place, and hand-edit managed-schema, just
like I did in previous versions with schema.xml.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Issei Nishigata
Thank you for these information.

but I am still confusing about specification of managed-schema.

I recognize that I cannot modify "unique id" or "Similarity" by Schema API
now.
* https://issues.apache.org/jira/browse/SOLR-7242

Isn't there any other way than hand-editing in this particular case?
Do we have any other way than hand-editing?


Is my understanding correct that managed-schema is not limited that it can
be modified
only via Schema API, but that we usually modify it via Schema API, and we
also can modify
what Schema API can't do by hand-editing?

Needless to say, I understand that there is an assumption that we do not
use
Schema API and hand-editing at the same time.



Thanks,
Issei

2017-03-02 10:15 GMT+09:00 Shawn Heisey <[hidden email]>:

> 2/27/2017 4:46 AM, Issei Nishigata wrote:
> > Thank you for your reply. If I was to say which one, I'd maybe be
> > talking about the concept for Solr. I understand we should use
> > "ClassicSchemaFactory" when we want to hand-edit, but why are there
> > two files, schema.xml and managed-schema, in spite that we can
> > hand-edit managed-schema? If we can modify the schema.xml through
> > Schema API, I think we won't need the managed-schema, but is there any
> > reason why that can't be done? Could you please let me know if there
> > is any information that can clear things up around those details?
>
> The default filename with the Managed Schema factory is managed-schema
> -- no extension.  I'm pretty sure that the reason the extension was
> removed was to discourage hand-editing.  If you use both hand-editing
> and API modification, you can lose some (or maybe all) of your hand edits.
>
> The default filename for the schema with the classic factory is
> schema.xml.  With this factory, API modification is not possible.
>
> If the managed factory is in use, and a schema.xml file is found during
> startup, the system will rename managed-schema (or whatever the config
> says to use) to something else, then rename schema.xml to managed-schema
> -- basically this is a startup-only way to support a legacy config.
>
> I personally don't ever plan to use the managed schema API, but I will
> leave the default factory in place, and hand-edit managed-schema, just
> like I did in previous versions with schema.xml.
>
> Thanks,
> Shawn
>
>
Reply | Threaded
Open this post in threaded view
|

Re: About editing managed-schema by hand

Shawn Heisey-2
On 3/20/2017 9:22 AM, Issei Nishigata wrote:
> Is my understanding correct that managed-schema is not limited that it
> can be modified only via Schema API, but that we usually modify it via
> Schema API, and we also can modify what Schema API can't do by
> hand-editing? Needless to say, I understand that there is an
> assumption that we do not use Schema API and hand-editing at the same
> time.

Hand-editing is perfectly acceptable if it is done right.  The reason
that editing is discouraged is because mixing it with API usage can
result in hand edits getting lost.

The online Schema API reference now includes a note about why
hand-editing is discouraged.  The managed-schema comment could include a
link to that.

Thanks,
Shawn