ContentStreamUpdateRequest no longer closes stream

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ContentStreamUpdateRequest no longer closes stream

Colvin Cowie
Hello, I'm in the process of moving from Solr 6. to Solr 8.
We have a client application that streams CSV files to Solr using ContentStreamUpdateRequest and then deletes the CSV file once the data is indexed. That worked fine in Solr 6, but when using 8, the file is locked and can't be deleted. (This is on Windows)

This seems to be because of the changes made in https://issues.apache.org/jira/browse/SOLR-12142
--------------
@Override
  public RequestWriter.ContentWriter getContentWriter(String expectedType) {
    if (contentStreams == null || contentStreams.isEmpty() || contentStreams.size() > 1) return null;
    ContentStream stream = contentStreams.get(0);
    return new RequestWriter.ContentWriter() {
      @Override
      public void write(OutputStream os) throws IOException {
        IOUtils.copy(stream.getStream(), os);
      }

      @Override
      public String getContentType() {
        return stream.getContentType();
      }
    };
  }
--------------
As far as I know, IOUtils.copy will not close the stream.

Adding a close to it, is enough to "fix" it for me
         try {
            IOUtils.copy(innerStream, os);
          } finally {
            IOUtils.closeQuietly(innerStream);
          }

I've attached a simple test case. It passes with the change above and fails without it.

So, is this a bug, or is there something I'm supposed to be doing elsewhere to close the stream?

Thanks,
Colvin
Reply | Threaded
Open this post in threaded view
|

Re: ContentStreamUpdateRequest no longer closes stream

Colvin Cowie
I realize that attachments might not work on the mailing list, so here is
the test case on Drive
https://drive.google.com/file/d/0B7mypFpwbHptTE5nZE0weURFOExFSHphRFlUV0EyTElaOC0w/view?usp=sharing

On Mon, 10 Jun 2019 at 13:17, Colvin Cowie <[hidden email]>
wrote:

> Hello, I'm in the process of moving from Solr 6. to Solr 8.
> We have a client application that streams CSV files to Solr using
> ContentStreamUpdateRequest and then deletes the CSV file once the data is
> indexed. That worked fine in Solr 6, but when using 8, the file is locked
> and can't be deleted. (This is on Windows)
>
> This seems to be because of the changes made in
> https://issues.apache.org/jira/browse/SOLR-12142
> --------------
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *@Override  public RequestWriter.ContentWriter getContentWriter(String
> expectedType) {    if (contentStreams == null || contentStreams.isEmpty()
> || contentStreams.size() > 1) return null;    ContentStream stream =
> contentStreams.get(0);    return new RequestWriter.ContentWriter() {
> @Override      public void write(OutputStream os) throws IOException {
>   IOUtils.copy(stream.getStream(), os);      }      @Override      public
> String getContentType() {        return stream.getContentType();      }
> };  }*
> --------------
> As far as I know, IOUtils.copy will not close the stream.
>
> Adding a close to it, is enough to "fix" it for me
>
>
>
>
> *         try {            IOUtils.copy(innerStream, os);          }
> finally {            IOUtils.closeQuietly(innerStream);          }*
>
> I've attached a simple test case. It passes with the change above and
> fails without it.
>
> So, is this a bug, or is there something I'm supposed to be doing
> elsewhere to close the stream?
>
> Thanks,
> Colvin
>
Reply | Threaded
Open this post in threaded view
|

Re: ContentStreamUpdateRequest no longer closes stream

Mikhail Khludnev-2
FWIW, fixed in 8.2.
Thanks, Colvin!

On Wed, Jun 12, 2019 at 5:30 PM Colvin Cowie <[hidden email]>
wrote:

> I realize that attachments might not work on the mailing list, so here is
> the test case on Drive
>
> https://drive.google.com/file/d/0B7mypFpwbHptTE5nZE0weURFOExFSHphRFlUV0EyTElaOC0w/view?usp=sharing
>
> On Mon, 10 Jun 2019 at 13:17, Colvin Cowie <[hidden email]>
> wrote:
>
> > Hello, I'm in the process of moving from Solr 6. to Solr 8.
> > We have a client application that streams CSV files to Solr using
> > ContentStreamUpdateRequest and then deletes the CSV file once the data is
> > indexed. That worked fine in Solr 6, but when using 8, the file is locked
> > and can't be deleted. (This is on Windows)
> >
> > This seems to be because of the changes made in
> > https://issues.apache.org/jira/browse/SOLR-12142
> > --------------
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *@Override  public RequestWriter.ContentWriter getContentWriter(String
> > expectedType) {    if (contentStreams == null || contentStreams.isEmpty()
> > || contentStreams.size() > 1) return null;    ContentStream stream =
> > contentStreams.get(0);    return new RequestWriter.ContentWriter() {
> > @Override      public void write(OutputStream os) throws IOException {
> >   IOUtils.copy(stream.getStream(), os);      }      @Override      public
> > String getContentType() {        return stream.getContentType();      }
> > };  }*
> > --------------
> > As far as I know, IOUtils.copy will not close the stream.
> >
> > Adding a close to it, is enough to "fix" it for me
> >
> >
> >
> >
> > *         try {            IOUtils.copy(innerStream, os);          }
> > finally {            IOUtils.closeQuietly(innerStream);          }*
> >
> > I've attached a simple test case. It passes with the change above and
> > fails without it.
> >
> > So, is this a bug, or is there something I'm supposed to be doing
> > elsewhere to close the stream?
> >
> > Thanks,
> > Colvin
> >
>


--
Sincerely yours
Mikhail Khludnev