[jira] [Created] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org
DIH status: successful zero-document delta-import missing "" field
------------------------------------------------------------------

                 Key: SOLR-2729
                 URL: https://issues.apache.org/jira/browse/SOLR-2729
             Project: Solr
          Issue Type: Bug
          Components: contrib - DataImportHandler
    Affects Versions: 3.2
         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

            Reporter: Shawn Heisey
            Priority: Minor
             Fix For: 3.4, 4.0


If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.

A full-import that processes zero documents has the field present as expected:

<str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246562#comment-13246562 ]

Shawn Heisey commented on SOLR-2729:
------------------------------------

Found it.  In solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/DocBuilder.java:

{code}
// Do not commit unnecessarily if this is a delta-import and no documents were created or deleted
if (!requestParameters.clean) {
  if (importStatistics.docCount.get() > 0 || importStatistics.deletedDocCount.get() > 0) {
    finish(lastIndexTimeProps);
  }
} else {
  // Finished operation normally, commit now
  finish(lastIndexTimeProps);
}
{code}

The method named finish is where the status message gets updated with the status that says how many documents were added/updated.

A fix that would take care of the immediate problem is to move the code that populates the "" part of statusMessages into its own method that is called by finish, then add an else clause to the inner if statement above which calls that method.  Does that sound at all reasonable?

               

> DIH status: successful zero-document delta-import missing "" field
> ------------------------------------------------------------------
>
>                 Key: SOLR-2729
>                 URL: https://issues.apache.org/jira/browse/SOLR-2729
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.2
>         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>
> If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.
> A full-import that processes zero documents has the field present as expected:
> <str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246569#comment-13246569 ]

Shawn Heisey commented on SOLR-2729:
------------------------------------

Something that will require a separate issue, perhaps two: I really think that "" is not a good name for the place where this stuff goes, and that "Time Taken " should also be fixed so it has no trailing space.  Perhaps the entire status response needs some TLC.  Making these changes will break a lot of user code, but it specifically says in the status output that the format is experimental and may change.

               

> DIH status: successful zero-document delta-import missing "" field
> ------------------------------------------------------------------
>
>                 Key: SOLR-2729
>                 URL: https://issues.apache.org/jira/browse/SOLR-2729
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.2
>         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>
> If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.
> A full-import that processes zero documents has the field present as expected:
> <str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246587#comment-13246587 ]

James Dyer commented on SOLR-2729:
----------------------------------

I agree the status messages should be better (Fix typos, no blank names, etc).  I don't think we should worry too much about breaking people's code (mine included).  Really, there should be a better way for automated schedulers to be able to check DIH status (JMX maybe?).  This is probably more of a long-term wish though.  In any case, I think the focus on the existing status page should be human-readability.
               

> DIH status: successful zero-document delta-import missing "" field
> ------------------------------------------------------------------
>
>                 Key: SOLR-2729
>                 URL: https://issues.apache.org/jira/browse/SOLR-2729
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.2
>         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>
> If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.
> A full-import that processes zero documents has the field present as expected:
> <str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/SOLR-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shawn Heisey updated SOLR-2729:
-------------------------------

    Attachment: SOLR-2729.patch

I created a method named putFinishStatus().  If this doesn't sound reasonable, or if this is not a good way to go, let me know and I'll fix it if I can.  When it's all ironed out, I'll make a patch for 3x as well.

All solr tests pass with this applied.
               

> DIH status: successful zero-document delta-import missing "" field
> ------------------------------------------------------------------
>
>                 Key: SOLR-2729
>                 URL: https://issues.apache.org/jira/browse/SOLR-2729
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.2
>         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2729.patch
>
>
> If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.
> A full-import that processes zero documents has the field present as expected:
> <str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Assigned] (SOLR-2729) DIH status: successful zero-document delta-import missing "" field

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/SOLR-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man reassigned SOLR-2729:
------------------------------

    Assignee: James Dyer

James: Could you please triage this for 4.0? commit if you think it's ready, otherwise remove the fix version?
               

> DIH status: successful zero-document delta-import missing "" field
> ------------------------------------------------------------------
>
>                 Key: SOLR-2729
>                 URL: https://issues.apache.org/jira/browse/SOLR-2729
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.2
>         Environment: Linux idxst0-a 2.6.18-238.12.1.el5.centos.plusxen #1 SMP Wed Jun 1 11:57:54 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
>            Reporter: Shawn Heisey
>            Assignee: James Dyer
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2729.patch
>
>
> If you have a successful delta-import that happens to process zero documents, the <str name=""> field is not present in the status.  I've run into this situation when the SQL query results in an empty set.  A workaround for the problem is to instead look for the "Time taken " field ... but if you don't happen to notice that this field has an extraneous space in the name, that won't work either.
> A full-import that processes zero documents has the field present as expected:
> <str name="">Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.</str>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]