[jira] [Created] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
CSV ResponseWriter returns fields defined as stored=false in schema
-------------------------------------------------------------------

                 Key: SOLR-2970
                 URL: https://issues.apache.org/jira/browse/SOLR-2970
             Project: Solr
          Issue Type: Bug
          Components: Response Writers
            Reporter: Jan Høydahl
            Priority: Minor


1. Add &wt=csv to a query
2. You get columns for non-stored fields


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Høydahl updated SOLR-2970:
------------------------------

    Attachment: SOLR-2970.patch

This simple patch (for trunk) fixes it
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Høydahl updated SOLR-2970:
------------------------------

    Fix Version/s: 4.0
   

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170302#comment-13170302 ]

Yonik Seeley commented on SOLR-2970:
------------------------------------

The other side of the coin is that if a client asks for x,y,z they may be expecting those columns in that order.

IMO, the problem is more about CSV not having a native representation for null/missing (i.e. if you ask for a sparse field, you will get zero length strings for the missing values too).
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170307#comment-13170307 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

The same solution seems to be valid for 3.x.
I don't see any side effects of always skipping fields which are not stored?
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170315#comment-13170315 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

Take the example:
{noformat}
Query: http://localhost:8983/solr/ac/select/?q=*%3A*&wt=csv
id,subtext,textphon,textng,score,action,value,textnge,type,textsuggest,popularity
a1,"Born 1898, author of Narnia",,,1023.9588,search,author_facet,,author,C.S. Lewis,256
a2,Swedish fake author,,,799.97003,search,author_facet,,author,Carl Larsson,200
a3,Norwegian famous author,,,359.992,search,author_facet,,author,Petter Dass,90
{noformat}

Fields textphon, textng and textnge are not stored. After the patch, we get:
{noformat}
Query: http://localhost:8983/solr/ac/select/?q=*%3A*&wt=csv
id,subtext,score,action,value,type,textsuggest,popularity
a1,"Born 1898, author of Narnia",1023.9588,search,author_facet,author,C.S. Lewis,256
a2,Swedish fake author,799.97003,search,author_facet,author,Carl Larsson,200
a3,Norwegian famous author,359.992,search,author_facet,author,Petter Dass,90
{noformat}

You can still ask for specific fields in any order:

{noformat}
Query: http://localhost:8983/solr/ac/select?q=*:*&wt=csv&fl=type,subtext,id
type,subtext,id
author,"Born 1898, author of Narnia",a1
author,Swedish fake author,a2
author,Norwegian famous author,a3
{noformat}

Is it ever correct to return non-stored fields?
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170357#comment-13170357 ]

Chris A. Mattmann commented on SOLR-2970:
-----------------------------------------

I ran into this when I was putting together the original patch for SOLR-1925, and believe that indeed the order of a field requested is important. And yes, as Yonik put it, CSV doesn't seem to have a native representation for null.

What about a configuration parameter (perhaps attached to the request) to identify a "placeholder" for missing or non-stored values? We see this a lot in science data, when the data quality flag specifies that we should put in a fill value.
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170377#comment-13170377 ]

Yonik Seeley commented on SOLR-2970:
------------------------------------

bq. Query: http://localhost:8983/solr/ac/select/?q=*%3A*&wt=csv

Ah, thanks for the example Jan.  I didn't realize those non-stored fields were returned by default!

bq. Is it ever correct to return non-stored fields?

If the user explicitly asks for the non-stored field, it can be OK to still return it.  It's less a matter of correctness and more a matter of what's most useful.


               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170440#comment-13170440 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

So there are two distinct issues here
A) CSV attempts to return fields which are not stored - which is bound to always return blank value
B) When returning a stored field which has no value (null) for a document, there is no way to distinguish that from the empty string case

I think B) should have its own JIRA.

For A), I still cannot see any usecase where you want to ask for a non-stored field in the CSV output, even if you ask for it explicitly - since it will always be null/empty. If you ask the XML or JSON writer for a non-stored field, you never get it no matter what - it formally does not exist in this context.
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170876#comment-13170876 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

Created SOLR-2974 for B)

I see that today you can explicitly specify non-existing fields in field-list and they will all end up in the response, e.g. &wt=csv&fl=id,type,foo,nonexisting

If this is useful in some cases then let's continue to support it - coupled with the improvement in SOLR-2974
But the default case when "fl" is not specified, or specified as "*", CSV writer should output only the stored fields.
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Høydahl updated SOLR-2970:
------------------------------

    Attachment: SOLR-2970.patch

Updated patch which will respect explicit "fl" but output only stored fields for the default or "*" case
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170890#comment-13170890 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

Hmm, seems as the CSV writer does not support returning functions as fields either.
http://localhost:8983/solr/ac/select/?q=*%3A*&wt=xml&fl=log(popularity) works but
http://localhost:8983/solr/ac/select/?q=*%3A*&wt=csv&fl=log(popularity) does not
But I can spin that off as a separate issue as well, and focus on fixing this one bug
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Insight 49, LLC
In reply to this post by Sebastian Nagel (Jira)
Another useful addition would be to add a csv sort parameter if
requesting all (*) fields -- a way to control the field order on export.

E.g.

&wt=csv&fl=*&csv.fl.sort=alpha asc

I've had cases where:

a) I'm returning too many fields to explicitly define fl=field_J,field_Z,field_A,etc.

b) WANTED all the fields returned (a great way to dump data to a spreadsheet)

but wanted a more orderly csv field structure than the one returned.

Is this possible?

Thanks,

Dan



On 12/16/11 3:02 AM, Jan Høydahl (Commented) (JIRA) wrote:

>      [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170876#comment-13170876 ]
>
> Jan Høydahl commented on SOLR-2970:
> -----------------------------------
>
> Created SOLR-2974 for B)
>
> I see that today you can explicitly specify non-existing fields in field-list and they will all end up in the response, e.g.&wt=csv&fl=id,type,foo,nonexisting
>
> If this is useful in some cases then let's continue to support it - coupled with the improvement in SOLR-2974
> But the default case when "fl" is not specified, or specified as "*", CSV writer should output only the stored fields.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Jan Høydahl / Cominvent
Dan,

I'm sure it is possible. Perhaps it should even be possible to combine fixed column positions and wildcard, by fl=fieldA,fieldB,* to "lock" the named fields as first columns followed by a (sorted) list of the rest of the fields.

Anyway, the first step is for you to create a JIRA issue and next step is to contribute a patch. See http://wiki.apache.org/solr/HowToContribute

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com

On 16. des. 2011, at 22:27, [hidden email] wrote:

> Another useful addition would be to add a csv sort parameter if requesting all (*) fields -- a way to control the field order on export.
>
> E.g.
>
> &wt=csv&fl=*&csv.fl.sort=alpha asc
>
> I've had cases where:
>
> a) I'm returning too many fields to explicitly define fl=field_J,field_Z,field_A,etc.
>
> b) WANTED all the fields returned (a great way to dump data to a spreadsheet)
>
> but wanted a more orderly csv field structure than the one returned.
>
> Is this possible?
>
> Thanks,
>
> Dan
>
>
>
> On 12/16/11 3:02 AM, Jan Høydahl (Commented) (JIRA) wrote:
>>     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170876#comment-13170876 ]
>>
>> Jan Høydahl commented on SOLR-2970:
>> -----------------------------------
>>
>> Created SOLR-2974 for B)
>>
>> I see that today you can explicitly specify non-existing fields in field-list and they will all end up in the response, e.g.&wt=csv&fl=id,type,foo,nonexisting
>>
>> If this is useful in some cases then let's continue to support it - coupled with the improvement in SOLR-2974
>> But the default case when "fl" is not specified, or specified as "*", CSV writer should output only the stored fields.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Insight 49, LLC
Jan,

Would be glad to open a JIRA issue, but I'm a U/X architect not a Java
developer, so wouldn't be able to patch anything.

Dan

On 12/16/11 6:05 PM, Jan Høydahl wrote:

> Dan,
>
> I'm sure it is possible. Perhaps it should even be possible to combine fixed column positions and wildcard, by fl=fieldA,fieldB,* to "lock" the named fields as first columns followed by a (sorted) list of the rest of the fields.
>
> Anyway, the first step is for you to create a JIRA issue and next step is to contribute a patch. See http://wiki.apache.org/solr/HowToContribute
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> On 16. des. 2011, at 22:27, [hidden email] wrote:
>
>> Another useful addition would be to add a csv sort parameter if requesting all (*) fields -- a way to control the field order on export.
>>
>> E.g.
>>
>> &wt=csv&fl=*&csv.fl.sort=alpha asc
>>
>> I've had cases where:
>>
>> a) I'm returning too many fields to explicitly define fl=field_J,field_Z,field_A,etc.
>>
>> b) WANTED all the fields returned (a great way to dump data to a spreadsheet)
>>
>> but wanted a more orderly csv field structure than the one returned.
>>
>> Is this possible?
>>
>> Thanks,
>>
>> Dan
>>
>>
>>
>> On 12/16/11 3:02 AM, Jan Høydahl (Commented) (JIRA) wrote:
>>>      [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170876#comment-13170876 ]
>>>
>>> Jan Høydahl commented on SOLR-2970:
>>> -----------------------------------
>>>
>>> Created SOLR-2974 for B)
>>>
>>> I see that today you can explicitly specify non-existing fields in field-list and they will all end up in the response, e.g.&wt=csv&fl=id,type,foo,nonexisting
>>>
>>> If this is useful in some cases then let's continue to support it - coupled with the improvement in SOLR-2974
>>> But the default case when "fl" is not specified, or specified as "*", CSV writer should output only the stored fields.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171585#comment-13171585 ]

Dan Hertz commented on SOLR-2970:
---------------------------------

When using fl=*, is it possible to include an option to output all the field data (stored and not stored)? I ask, because the CSV ResponseWriter is very useful to back up and/or transfer data between solr indexes and spreadsheets/databases.

Perhaps default to stored fields, but provide an option to export all fields (without having to specify each field name)?
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171948#comment-13171948 ]

Jan Høydahl commented on SOLR-2970:
-----------------------------------

Dan, the problem is that non-stored fields are not stored :) So they cannot be output. Even if it could be possible to un-invert indexed fields, that is, IMHO, not a task for the CSV writer.
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Jan Høydahl / Cominvent
In reply to this post by Insight 49, LLC
Dan,

JIRA is open for everyone, you don't need to submit a patch. It's really helpful if you just submit the feature request. Then others can step in to do the coding, but you will be registered as the reporter, i.e. the one having a need for the feature.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com

On 17. des. 2011, at 15:01, [hidden email] wrote:

> Jan,
>
> Would be glad to open a JIRA issue, but I'm a U/X architect not a Java developer, so wouldn't be able to patch anything.
>
> Dan
>
> On 12/16/11 6:05 PM, Jan Høydahl wrote:
>> Dan,
>>
>> I'm sure it is possible. Perhaps it should even be possible to combine fixed column positions and wildcard, by fl=fieldA,fieldB,* to "lock" the named fields as first columns followed by a (sorted) list of the rest of the fields.
>>
>> Anyway, the first step is for you to create a JIRA issue and next step is to contribute a patch. See http://wiki.apache.org/solr/HowToContribute
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>>
>> On 16. des. 2011, at 22:27, [hidden email] wrote:
>>
>>> Another useful addition would be to add a csv sort parameter if requesting all (*) fields -- a way to control the field order on export.
>>>
>>> E.g.
>>>
>>> &wt=csv&fl=*&csv.fl.sort=alpha asc
>>>
>>> I've had cases where:
>>>
>>> a) I'm returning too many fields to explicitly define fl=field_J,field_Z,field_A,etc.
>>>
>>> b) WANTED all the fields returned (a great way to dump data to a spreadsheet)
>>>
>>> but wanted a more orderly csv field structure than the one returned.
>>>
>>> Is this possible?
>>>
>>> Thanks,
>>>
>>> Dan
>>>
>>>
>>>
>>> On 12/16/11 3:02 AM, Jan Høydahl (Commented) (JIRA) wrote:
>>>>     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170876#comment-13170876 ]
>>>>
>>>> Jan Høydahl commented on SOLR-2970:
>>>> -----------------------------------
>>>>
>>>> Created SOLR-2974 for B)
>>>>
>>>> I see that today you can explicitly specify non-existing fields in field-list and they will all end up in the response, e.g.&wt=csv&fl=id,type,foo,nonexisting
>>>>
>>>> If this is useful in some cases then let's continue to support it - coupled with the improvement in SOLR-2974
>>>> But the default case when "fl" is not specified, or specified as "*", CSV writer should output only the stored fields.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Høydahl updated SOLR-2970:
------------------------------

    Attachment: SOLR-2970.patch

Updated patch with test case.
Think this approaches ready for commit. Comments?
               

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 3.6, 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2970) CSV ResponseWriter returns fields defined as stored=false in schema

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jan Høydahl updated SOLR-2970:
------------------------------

    Fix Version/s: 3.6
   

> CSV ResponseWriter returns fields defined as stored=false in schema
> -------------------------------------------------------------------
>
>                 Key: SOLR-2970
>                 URL: https://issues.apache.org/jira/browse/SOLR-2970
>             Project: Solr
>          Issue Type: Bug
>          Components: Response Writers
>            Reporter: Jan Høydahl
>            Priority: Minor
>             Fix For: 3.6, 4.0
>
>         Attachments: SOLR-2970.patch, SOLR-2970.patch, SOLR-2970.patch
>
>
> 1. Add &wt=csv to a query
> 2. You get columns for non-stored fields

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12