Created: (SOLR-181) Support for "Required" field Property

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Created: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)
Support for "Required" field Property
-------------------------------------

                 Key: SOLR-181
                 URL: https://issues.apache.org/jira/browse/SOLR-181
             Project: Solr
          Issue Type: Improvement
          Components: update
            Reporter: Greg Ludington
            Priority: Minor


In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:

   <field name="name" type="nametext" indexed="true" stored="true" required="true"/>

A request to index a document without a name field will result in this response:

<result status="1">org.apache.solr.core.SolrException: missing required fields: name
(and then, of course, the stack trace)
</result>

The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

Updated: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Greg Ludington updated SOLR-181:
--------------------------------

    Attachment: solr-181-required-fields.patch

This patch allows a per-field "required" property that can be used to require clients to include certain fields to submit to the index.

With this in the index:
  <field name="name" type="nametext" indexed="true" stored="true" required="true"/>

A document submitted without a name field will throw a SolrException in DocumentBuilder.getDoc() resulting in this sort of message being sent back to the client:

<result status="1">org.apache.solr.core.SolrException: missing required fields:
name

Issues with the patch as submitted:

1) The rule is enforced, and the Exception thrown, from DocumentBuilder.getDoc().  Since this area may change in SOLR-139, this patch may need to be adjusted depending on the final result of SOLR-139.
2) Fields with defaultValues are implicitly required, though currently this patch does *not* automatically make the uniqueKey field required.  It may make sense to do this; however, there seems to be some debate on the mailing lists about this, so It is commented out with a //TODO for now.  See SOLR-172.
3) The RequiredFieldsTest case uses its own schema file, as otherwise all other tests would have to be retrofitted to add these required fields in their submissions, and all future test writers would have to keep this in mind, as well.

> Support for "Required" field Property
> -------------------------------------
>
>                 Key: SOLR-181
>                 URL: https://issues.apache.org/jira/browse/SOLR-181
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Greg Ludington
>            Priority: Minor
>         Attachments: solr-181-required-fields.patch
>
>
> In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:
>    <field name="name" type="nametext" indexed="true" stored="true" required="true"/>
> A request to index a document without a name field will result in this response:
> <result status="1">org.apache.solr.core.SolrException: missing required fields: name
> (and then, of course, the stack trace)
> </result>
> The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)
In reply to this post by Michael Gibney (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan McKinley updated SOLR-181:
-------------------------------

    Attachment: solr-181-required-fields.patch

Finally got a chance to look at this.  It looks good.  I made a few modifications:

1. changed tabs to spaces
2. Added javadoc comments to make it clear that RequiredFields must contain all fieldsWithDefaultValues
3. The error now contains the documents uniqueKey
4. moved the test to o.a.s.schema
5. I added a non-final flag to SchemaField to say if the field is required.
6. Modified IndexSchema.java to set the uniqueKey as required *unless* it is specified as required="false" in the schema
7. Added required="true" to the example schema.xml
8. Added required="false" to the test schema.xml (one test does not include it)

As a note to anyone else looking at the change log, Greg's patch also modifies AbstractSolrTestCase and TestHarness to be able to check what status is expected from checkUpdateU


I think this offers a good solution to the (mis)feature that you could have a null uniqueKey.  This patch lets you have a null uniqueKey, but you have to configure it.



> Support for "Required" field Property
> -------------------------------------
>
>                 Key: SOLR-181
>                 URL: https://issues.apache.org/jira/browse/SOLR-181
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Greg Ludington
>            Priority: Minor
>         Attachments: solr-181-required-fields.patch, solr-181-required-fields.patch
>
>
> In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:
>    <field name="name" type="nametext" indexed="true" stored="true" required="true"/>
> A request to index a document without a name field will result in this response:
> <result status="1">org.apache.solr.core.SolrException: missing required fields: name
> (and then, of course, the stack trace)
> </result>
> The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)
In reply to this post by Michael Gibney (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan McKinley reassigned SOLR-181:
----------------------------------

    Assignee: Ryan McKinley

> Support for "Required" field Property
> -------------------------------------
>
>                 Key: SOLR-181
>                 URL: https://issues.apache.org/jira/browse/SOLR-181
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Greg Ludington
>         Assigned To: Ryan McKinley
>            Priority: Minor
>         Attachments: solr-181-required-fields.patch, solr-181-required-fields.patch
>
>
> In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:
>    <field name="name" type="nametext" indexed="true" stored="true" required="true"/>
> A request to index a document without a name field will result in this response:
> <result status="1">org.apache.solr.core.SolrException: missing required fields: name
> (and then, of course, the stack trace)
> </result>
> The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)
In reply to this post by Michael Gibney (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492532 ]

Yonik Seeley commented on SOLR-181:
-----------------------------------

Haven't looked at the code,  but the description looks fine.

+1

> Support for "Required" field Property
> -------------------------------------
>
>                 Key: SOLR-181
>                 URL: https://issues.apache.org/jira/browse/SOLR-181
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Greg Ludington
>         Assigned To: Ryan McKinley
>            Priority: Minor
>         Attachments: solr-181-required-fields.patch, solr-181-required-fields.patch
>
>
> In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:
>    <field name="name" type="nametext" indexed="true" stored="true" required="true"/>
> A request to index a document without a name field will result in this response:
> <result status="1">org.apache.solr.core.SolrException: missing required fields: name
> (and then, of course, the stack trace)
> </result>
> The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Closed: (SOLR-181) Support for "Required" field Property

Michael Gibney (Jira)
In reply to this post by Michael Gibney (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ryan McKinley closed SOLR-181.
------------------------------

    Resolution: Fixed

commited

> Support for "Required" field Property
> -------------------------------------
>
>                 Key: SOLR-181
>                 URL: https://issues.apache.org/jira/browse/SOLR-181
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Greg Ludington
>         Assigned To: Ryan McKinley
>            Priority: Minor
>         Attachments: solr-181-required-fields.patch, solr-181-required-fields.patch
>
>
> In certain situations, it can be helpful to require every document in your index has a value for a given field.  While ideally the indexing client(s) should be responsible enough to add all necessary fields, this patch allows it to be enforced in the Solr schema, by adding a required property to a field entry.  For example, with this in the schema:
>    <field name="name" type="nametext" indexed="true" stored="true" required="true"/>
> A request to index a document without a name field will result in this response:
> <result status="1">org.apache.solr.core.SolrException: missing required fields: name
> (and then, of course, the stack trace)
> </result>
> The meat of this patch is that DocumentBuilder.getDoc() throws a SolrException if not all required fields have values; this may not work well as is with SOLR-139, Support updateable/modifiable documents, and may have to be changed depending on that issue's final disposition.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.