fetch failed error 500

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

fetch failed error 500

宫照
Hi All,

When I am using nutch to crawl url like this http://*******.com/cases/tcsg2html.pl?2321543

It get the error like
fetch of http://*******.com/cases/046418 failed with: Http code=50
0, url=http://*******.com/cases/046418


Do you know the reason of  this ?

Regards,

Gong Zhao
Reply | Threaded
Open this post in threaded view
|

Re: fetch failed error 500

Alex McLintock
Gong,

Have you eliminated the possibility that the cgi script is doing a redirect?

2009/8/11 宫照 <[hidden email]>:

> Hi All,
>
> When I am using nutch to crawl url like this
> http://*******.com/cases/tcsg2html.pl?2321543
>
> It get the error like
> fetch of http://*******.com/cases/046418 failed with: Http code=50
> 0, url=http://*******.com/cases/046418
>
> Do you know the reason of  this ?
>
> Regards,
>
> Gong Zhao
>
Reply | Threaded
Open this post in threaded view
|

Re: fetch failed error 500

宫照
Hi Alex,

Thank you for your reply!

what can i do if it was redirect in cgi script, because I can't get script on this server so i don't know it exactly.

I try to crawl it again today and get the output like this

fetching http://*******.com/cases/007495
Error parsing: http://*******.com/cases/007495: failed(2,200): org.apache.nutch.parse.ParseException: parser not found for contentType=application/octet-stream url=http://*******.com/cases/007495

It seems nutch don't know which parser to use.

Regards,

Gong Zhao




2009/8/11 Alex McLintock <[hidden email]>
Gong,

Have you eliminated the possibility that the cgi script is doing a redirect?

2009/8/11 宫照 <[hidden email]>:
> Hi All,
>
> When I am using nutch to crawl url like this
> http://*******.com/cases/tcsg2html.pl?2321543
>
> It get the error like
> fetch of http://*******.com/cases/046418 failed with: Http code=50
> 0, url=http://*******.com/cases/046418
>
> Do you know the reason of  this ?
>
> Regards,
>
> Gong Zhao
>