SIGSEGV

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

SIGSEGV

Brian Whitman
Got this segfault + crash when fetching in the middle of a large  
fetch. Seems to be in looking up a hostname?

--





#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x55651ed2, pid=8251, tid=2763877264
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_10-b03 mixed mode)
# Problematic frame:
# C  [libc.so.6+0xb8ed2]
#

---------------  T H R E A D  ---------------

Current thread (0xa2a376d0):  JavaThread "FetcherThread" daemon  
[_thread_in_native, id=8288]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000010

Registers:
EAX=0xa4bd5364, EBX=0x556d7ff4, ECX=0xa4bd5940, EDX=0x00000000
ESP=0xa4bd523c, EBP=0xa4bd53e4, ESI=0xa4bd5364, EDI=0x00000000
EIP=0x55651ed2, CR2=0x00000010, EFLAGS=0x00010292

Top of Stack: (sp=0xa4bd523c)
0xa4bd523c:   00000000 a4bd5260 00454002 000081a4
0xa4bd524c:   00000001 00000000 00000000 00000000
0xa4bd525c:   a4bd5948 a4bd52c0 01000044 a4bd5364
0xa4bd526c:   00001000 00000000 0853aef8 0853aec0
0xa4bd527c:   00000000 45b16bc9 00000000 45b16bc9
0xa4bd528c:   0000000e 0000000e 00000000 7fffffff
0xa4bd529c:   a4bd6df4 a4bd5318 5567a30d 00000003
0xa4bd52ac:   556bf0a0 a4bd535c 00000801 a4bd58bc
Instructions: (pc=0x55651ed2)
0x55651ec2:   89 bd 6c fe ff ff 89 fa 89 c6 89 85 84 fe ff ff
0x55651ed2:   8b 4a 10 83 f9 07 76 2b 8b bd cc fe ff ff 0f b6

Stack: [0xa4b57000,0xa4bd7000),  sp=0xa4bd523c,  free space=504k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,  
C=native code)
C  [libc.so.6+0xb8ed2]
C  [libc.so.6+0x2be57]
C  [libc.so.6+0x2bdff]
C  [libc.so.6+0x2bde5]
C  [libc.so.6+0x2c06a]  qsort+0x12a
C  [libc.so.6+0xb8372]  getaddrinfo+0x5c2
C  [libnet.so+0x57e8]  
Java_java_net_Inet6AddressImpl_lookupAllHostAddr+0x198
j  java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[[B+0
j  java.net.InetAddress$1.lookupAllHostAddr(Ljava/lang/String;)[[B+4
j  java.net.InetAddress.getAddressFromNameService(Ljava/lang/String;)
Ljava/lang/Object;+17
j  java.net.InetAddress.getAllByName0(Ljava/lang/String;Z)[Ljava/net/
InetAddress;+35
j  java.net.InetAddress.getAllByName0(Ljava/lang/String;)[Ljava/net/
InetAddress;+2
j  java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/
InetAddress;+319
j  java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/
InetAddress;+1
j  java.net.InetSocketAddress.<init>(Ljava/lang/String;I)V+67
j  org.apache.nutch.protocol.http.HttpResponse.<init>(Lorg/apache/
nutch/protocol/http/api/HttpBa
se;Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDatum;)V+259
j  org.apache.nutch.protocol.http.Http.getResponse(Ljava/net/URL;Lorg/
apache/nutch/crawl/CrawlDa
tum;Z)Lorg/apache/nutch/net/protocols/Response;+7
j  
org.apache.nutch.protocol.http.api.RobotRulesParser.getRobotRulesSet
(Lorg/apache/nutch/protoc
ol/http/api/HttpBase;Ljava/net/URL;)Lorg/apache/nutch/protocol/http/
api/RobotRulesParser$RobotRu
leSet;+85
j  org.apache.nutch.protocol.http.api.RobotRulesParser.isAllowed(Lorg/
apache/nutch/protocol/http
/api/HttpBase;Ljava/net/URL;)Z+24
j  org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(Lorg/
apache/hadoop/io/Text;Lorg
/apache/nutch/crawl/CrawlDatum;)Lorg/apache/nutch/protocol/
ProtocolOutput;+29
j  org.apache.nutch.fetcher.Fetcher$FetcherThread.run()V+262
v  ~StubRoutines::call_stub
V  [libjvm.so+0x26777c]
V  [libjvm.so+0x42e598]
V  [libjvm.so+0x266fd5]
V  [libjvm.so+0x26706e]
V  [libjvm.so+0x2de925]
V  [libjvm.so+0x4ce593]
V  [libjvm.so+0x42f1a8]
C  [libpthread.so.0+0x5183]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[[B+0
j  java.net.InetAddress$1.lookupAllHostAddr(Ljava/lang/String;)[[B+4
j  java.net.InetAddress.getAddressFromNameService(Ljava/lang/String;)
Ljava/lang/Object;+17
j  java.net.InetAddress.getAllByName0(Ljava/lang/String;Z)[Ljava/net/
InetAddress;+35
j  java.net.InetAddress.getAllByName0(Ljava/lang/String;)[Ljava/net/
InetAddress;+2
j  java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/
InetAddress;+319
j  java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/
InetAddress;+1
j  java.net.InetSocketAddress.<init>(Ljava/lang/String;I)V+67
j  org.apache.nutch.protocol.http.HttpResponse.<init>(Lorg/apache/
nutch/protocol/http/api/HttpBa
se;Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDatum;)V+259
j  org.apache.nutch.protocol.http.Http.getResponse(Ljava/net/URL;Lorg/
apache/nutch/crawl/CrawlDa
tum;Z)Lorg/apache/nutch/net/protocols/Response;+7
j  
org.apache.nutch.protocol.http.api.RobotRulesParser.getRobotRulesSet
(Lorg/apache/nutch/protoc
ol/http/api/HttpBase;Ljava/net/URL;)Lorg/apache/nutch/protocol/http/
api/RobotRulesParser$RobotRu
leSet;+85
j  org.apache.nutch.protocol.http.api.RobotRulesParser.isAllowed(Lorg/
apache/nutch/protocol/http
/api/HttpBase;Ljava/net/URL;)Z+24
j  org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(Lorg/
apache/hadoop/io/Text;Lorg
/apache/nutch/crawl/CrawlDatum;)Lorg/apache/nutch/protocol/
ProtocolOutput;+29
j  org.apache.nutch.fetcher.Fetcher$FetcherThread.run()V+262
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
   0x08196028 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8303]
   0x081950d8 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8302]
   0x08194188 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8301]
   0x08193238 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8300]
   0x081922e8 JavaThread "FetcherThread" daemon [_thread_blocked,  
id=8299]
   0x08191398 JavaThread "FetcherThread" daemon [_thread_blocked,  
id=8298]
   0x08190448 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8297]
   0x0818f4f8 JavaThread "FetcherThread" daemon [_thread_blocked,  
id=8296]
   0x085e8ef0 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8295]
   0x085e80c8 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8294]
   0xa2c88c80 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8293]
   0xa2a9fc80 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8292]
   0xa2a9eed0 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8291]
   0xa2a9e168 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8290]
   0xa2a37ca8 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8289]
=>0xa2a376d0 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8288]
   0xa2a370f8 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8287]
   0xa2ba36d0 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8286]
   0x9f0aa888 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8285]
   0x9f0aaf38 JavaThread "FetcherThread" daemon [_thread_in_native,  
id=8284]
   0x0868b1b0 JavaThread "Sort progress reporter for task map_xt69kc"  
daemon [_thread_blocked, id
=8283]
   0x0849b188 JavaThread "Thread-0" [_thread_blocked, id=8282]
   0x9f002a88 JavaThread "Low Memory Detector" daemon  
[_thread_blocked, id=8280]
   0x9f001650 JavaThread "CompilerThread1" daemon [_thread_blocked,  
id=8279]
   0x9f000678 JavaThread "CompilerThread0" daemon [_thread_in_native,  
id=8278]
   0x08120ee0 JavaThread "AdapterThread" daemon [_thread_blocked,  
id=8277]
   0x0811ffa0 JavaThread "Signal Dispatcher" daemon [_thread_blocked,  
id=8276]
   0x08116818 JavaThread "Finalizer" daemon [_thread_blocked, id=8275]
   0x08115138 JavaThread "Reference Handler" daemon [_thread_blocked,  
id=8274]
   0x0805d9b8 JavaThread "main" [_thread_blocked, id=8251]

Other Threads:
   0x08112c00 VMThread [id=8273]
   0x9f004040 WatcherThread [id=8281]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
PSYoungGen      total 12928K, used 1702K [0x97aa0000, 0x98800000,  
0x9e9b0000)
   eden space 12160K, 10% used [0x97aa0000,0x97bd92b8,0x98680000)
   from space 768K, 58% used [0x98740000,0x987b05a0,0x98800000)
   to   space 768K, 0% used [0x98680000,0x98680000,0x98740000)
PSOldGen        total 1408K, used 640K [0x601b0000, 0x60310000,  
0x97aa0000)
   object space 1408K, 45% used [0x601b0000,0x60250070,0x60310000)
PSPermGen       total 16384K, used 5821K [0x5c1b0000, 0x5d1b0000,  
0x601b0000)
   object space 16384K, 35% used [0x5c1b0000,0x5c75f640,0x5d1b0000)

Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Dennis Kubes
Sigsev usually is the result of Hardware errors.  At least that is what
I have found in the past.  I would run memtest on the machine to check
for bad memory.

Dennis Kubes

Brian Whitman wrote:

> Got this segfault + crash when fetching in the middle of a large fetch.
> Seems to be in looking up a hostname?
>
> --
>
>
>
>
>
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> #  SIGSEGV (0xb) at pc=0x55651ed2, pid=8251, tid=2763877264
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_10-b03 mixed mode)
> # Problematic frame:
> # C  [libc.so.6+0xb8ed2]
> #
>
> ---------------  T H R E A D  ---------------
>
> Current thread (0xa2a376d0):  JavaThread "FetcherThread" daemon
> [_thread_in_native, id=8288]
>
> siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00000010
>
> Registers:
> EAX=0xa4bd5364, EBX=0x556d7ff4, ECX=0xa4bd5940, EDX=0x00000000
> ESP=0xa4bd523c, EBP=0xa4bd53e4, ESI=0xa4bd5364, EDI=0x00000000
> EIP=0x55651ed2, CR2=0x00000010, EFLAGS=0x00010292
>
> Top of Stack: (sp=0xa4bd523c)
> 0xa4bd523c:   00000000 a4bd5260 00454002 000081a4
> 0xa4bd524c:   00000001 00000000 00000000 00000000
> 0xa4bd525c:   a4bd5948 a4bd52c0 01000044 a4bd5364
> 0xa4bd526c:   00001000 00000000 0853aef8 0853aec0
> 0xa4bd527c:   00000000 45b16bc9 00000000 45b16bc9
> 0xa4bd528c:   0000000e 0000000e 00000000 7fffffff
> 0xa4bd529c:   a4bd6df4 a4bd5318 5567a30d 00000003
> 0xa4bd52ac:   556bf0a0 a4bd535c 00000801 a4bd58bc
> Instructions: (pc=0x55651ed2)
> 0x55651ec2:   89 bd 6c fe ff ff 89 fa 89 c6 89 85 84 fe ff ff
> 0x55651ed2:   8b 4a 10 83 f9 07 76 2b 8b bd cc fe ff ff 0f b6
>
> Stack: [0xa4b57000,0xa4bd7000),  sp=0xa4bd523c,  free space=504k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> C=native code)
> C  [libc.so.6+0xb8ed2]
> C  [libc.so.6+0x2be57]
> C  [libc.so.6+0x2bdff]
> C  [libc.so.6+0x2bde5]
> C  [libc.so.6+0x2c06a]  qsort+0x12a
> C  [libc.so.6+0xb8372]  getaddrinfo+0x5c2
> C  [libnet.so+0x57e8]  
> Java_java_net_Inet6AddressImpl_lookupAllHostAddr+0x198
> j  java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[[B+0
> j  java.net.InetAddress$1.lookupAllHostAddr(Ljava/lang/String;)[[B+4
> j  
> java.net.InetAddress.getAddressFromNameService(Ljava/lang/String;)Ljava/lang/Object;+17
>
> j  
> java.net.InetAddress.getAllByName0(Ljava/lang/String;Z)[Ljava/net/InetAddress;+35
>
> j  
> java.net.InetAddress.getAllByName0(Ljava/lang/String;)[Ljava/net/InetAddress;+2
>
> j  
> java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/InetAddress;+319
>
> j  
> java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/InetAddress;+1
> j  java.net.InetSocketAddress.<init>(Ljava/lang/String;I)V+67
> j  
> org.apache.nutch.protocol.http.HttpResponse.<init>(Lorg/apache/nutch/protocol/http/api/HttpBa
>
> se;Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDatum;)V+259
> j  
> org.apache.nutch.protocol.http.Http.getResponse(Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDa
>
> tum;Z)Lorg/apache/nutch/net/protocols/Response;+7
> j  
> org.apache.nutch.protocol.http.api.RobotRulesParser.getRobotRulesSet(Lorg/apache/nutch/protoc
>
> ol/http/api/HttpBase;Ljava/net/URL;)Lorg/apache/nutch/protocol/http/api/RobotRulesParser$RobotRu
>
> leSet;+85
> j  
> org.apache.nutch.protocol.http.api.RobotRulesParser.isAllowed(Lorg/apache/nutch/protocol/http
>
> /api/HttpBase;Ljava/net/URL;)Z+24
> j  
> org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(Lorg/apache/hadoop/io/Text;Lorg
>
> /apache/nutch/crawl/CrawlDatum;)Lorg/apache/nutch/protocol/ProtocolOutput;+29
>
> j  org.apache.nutch.fetcher.Fetcher$FetcherThread.run()V+262
> v  ~StubRoutines::call_stub
> V  [libjvm.so+0x26777c]
> V  [libjvm.so+0x42e598]
> V  [libjvm.so+0x266fd5]
> V  [libjvm.so+0x26706e]
> V  [libjvm.so+0x2de925]
> V  [libjvm.so+0x4ce593]
> V  [libjvm.so+0x42f1a8]
> C  [libpthread.so.0+0x5183]
>
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> j  java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[[B+0
> j  java.net.InetAddress$1.lookupAllHostAddr(Ljava/lang/String;)[[B+4
> j  
> java.net.InetAddress.getAddressFromNameService(Ljava/lang/String;)Ljava/lang/Object;+17
>
> j  
> java.net.InetAddress.getAllByName0(Ljava/lang/String;Z)[Ljava/net/InetAddress;+35
>
> j  
> java.net.InetAddress.getAllByName0(Ljava/lang/String;)[Ljava/net/InetAddress;+2
>
> j  
> java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/InetAddress;+319
>
> j  
> java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/InetAddress;+1
> j  java.net.InetSocketAddress.<init>(Ljava/lang/String;I)V+67
> j  
> org.apache.nutch.protocol.http.HttpResponse.<init>(Lorg/apache/nutch/protocol/http/api/HttpBa
>
> se;Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDatum;)V+259
> j  
> org.apache.nutch.protocol.http.Http.getResponse(Ljava/net/URL;Lorg/apache/nutch/crawl/CrawlDa
>
> tum;Z)Lorg/apache/nutch/net/protocols/Response;+7
> j  
> org.apache.nutch.protocol.http.api.RobotRulesParser.getRobotRulesSet(Lorg/apache/nutch/protoc
>
> ol/http/api/HttpBase;Ljava/net/URL;)Lorg/apache/nutch/protocol/http/api/RobotRulesParser$RobotRu
>
> leSet;+85
> j  
> org.apache.nutch.protocol.http.api.RobotRulesParser.isAllowed(Lorg/apache/nutch/protocol/http
>
> /api/HttpBase;Ljava/net/URL;)Z+24
> j  
> org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(Lorg/apache/hadoop/io/Text;Lorg
>
> /apache/nutch/crawl/CrawlDatum;)Lorg/apache/nutch/protocol/ProtocolOutput;+29
>
> j  org.apache.nutch.fetcher.Fetcher$FetcherThread.run()V+262
> v  ~StubRoutines::call_stub
>
> ---------------  P R O C E S S  ---------------
>
> Java Threads: ( => current thread )
>   0x08196028 JavaThread "FetcherThread" daemon [_thread_in_native, id=8303]
>   0x081950d8 JavaThread "FetcherThread" daemon [_thread_in_native, id=8302]
>   0x08194188 JavaThread "FetcherThread" daemon [_thread_in_native, id=8301]
>   0x08193238 JavaThread "FetcherThread" daemon [_thread_in_native, id=8300]
>   0x081922e8 JavaThread "FetcherThread" daemon [_thread_blocked, id=8299]
>   0x08191398 JavaThread "FetcherThread" daemon [_thread_blocked, id=8298]
>   0x08190448 JavaThread "FetcherThread" daemon [_thread_in_native, id=8297]
>   0x0818f4f8 JavaThread "FetcherThread" daemon [_thread_blocked, id=8296]
>   0x085e8ef0 JavaThread "FetcherThread" daemon [_thread_in_native, id=8295]
>   0x085e80c8 JavaThread "FetcherThread" daemon [_thread_in_native, id=8294]
>   0xa2c88c80 JavaThread "FetcherThread" daemon [_thread_in_native, id=8293]
>   0xa2a9fc80 JavaThread "FetcherThread" daemon [_thread_in_native, id=8292]
>   0xa2a9eed0 JavaThread "FetcherThread" daemon [_thread_in_native, id=8291]
>   0xa2a9e168 JavaThread "FetcherThread" daemon [_thread_in_native, id=8290]
>   0xa2a37ca8 JavaThread "FetcherThread" daemon [_thread_in_native, id=8289]
> =>0xa2a376d0 JavaThread "FetcherThread" daemon [_thread_in_native, id=8288]
>   0xa2a370f8 JavaThread "FetcherThread" daemon [_thread_in_native, id=8287]
>   0xa2ba36d0 JavaThread "FetcherThread" daemon [_thread_in_native, id=8286]
>   0x9f0aa888 JavaThread "FetcherThread" daemon [_thread_in_native, id=8285]
>   0x9f0aaf38 JavaThread "FetcherThread" daemon [_thread_in_native, id=8284]
>   0x0868b1b0 JavaThread "Sort progress reporter for task map_xt69kc"
> daemon [_thread_blocked, id
> =8283]
>   0x0849b188 JavaThread "Thread-0" [_thread_blocked, id=8282]
>   0x9f002a88 JavaThread "Low Memory Detector" daemon [_thread_blocked,
> id=8280]
>   0x9f001650 JavaThread "CompilerThread1" daemon [_thread_blocked, id=8279]
>   0x9f000678 JavaThread "CompilerThread0" daemon [_thread_in_native,
> id=8278]
>   0x08120ee0 JavaThread "AdapterThread" daemon [_thread_blocked, id=8277]
>   0x0811ffa0 JavaThread "Signal Dispatcher" daemon [_thread_blocked,
> id=8276]
>   0x08116818 JavaThread "Finalizer" daemon [_thread_blocked, id=8275]
>   0x08115138 JavaThread "Reference Handler" daemon [_thread_blocked,
> id=8274]
>   0x0805d9b8 JavaThread "main" [_thread_blocked, id=8251]
>
> Other Threads:
>   0x08112c00 VMThread [id=8273]
>   0x9f004040 WatcherThread [id=8281]
>
> VM state:not at safepoint (normal execution)
>
> VM Mutex/Monitor currently owned by a thread: None
>
> Heap
> PSYoungGen      total 12928K, used 1702K [0x97aa0000, 0x98800000,
> 0x9e9b0000)
>   eden space 12160K, 10% used [0x97aa0000,0x97bd92b8,0x98680000)
>   from space 768K, 58% used [0x98740000,0x987b05a0,0x98800000)
>   to   space 768K, 0% used [0x98680000,0x98680000,0x98740000)
> PSOldGen        total 1408K, used 640K [0x601b0000, 0x60310000, 0x97aa0000)
>   object space 1408K, 45% used [0x601b0000,0x60250070,0x60310000)
> PSPermGen       total 16384K, used 5821K [0x5c1b0000, 0x5d1b0000,
> 0x601b0000)
>   object space 16384K, 35% used [0x5c1b0000,0x5c75f640,0x5d1b0000)
>
Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Andrzej Białecki-2
In reply to this post by Brian Whitman
Brian Whitman wrote:
> Got this segfault + crash when fetching in the middle of a large fetch.
> Seems to be in looking up a hostname?

Is this by any chance a FreeBSD machine of 4.x or 5.x vintage? There was
a bug in FreeBSD's getaddrinfo, which would manifest in a very similar
way when running multithreaded apps linked to libc_r or libpthread.

--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Brian Whitman
Hi all,
I looked into this a bit more after it crashed for the third time in  
a row.

every time it has segfaulted it's had this url as one of the past few  
fetches:

fetching http://www.c bs.nu/cgi-bin/ac/adcycle.cgi?
gid=4&layout=multi&id=125

Note the space in there. This URL is not in my initial fetchlist so  
it was found somewhere. Not sure if the space is actually a space or  
an encoding -> terminal issue, either way I think this has something  
to do with it. Does anyone know what happens when java/nutch gets a  
hostname that is obviously malformed?

-Brian




On May 6, 2007, at 11:00 AM, Andrzej Bialecki wrote:

> Brian Whitman wrote:
>> Got this segfault + crash when fetching in the middle of a large  
>> fetch. Seems to be in looking up a hostname?
>
> Is this by any chance a FreeBSD machine of 4.x or 5.x vintage?  
> There was a bug in FreeBSD's getaddrinfo, which would manifest in a  
> very similar way when running multithreaded apps linked to libc_r  
> or libpthread.
>
> --
> Best regards,
> Andrzej Bialecki     <><
>  ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>

Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Dennis Kubes


Brian Whitman wrote:

> Hi all,
> I looked into this a bit more after it crashed for the third time in a row.
>
> every time it has segfaulted it's had this url as one of the past few
> fetches:
>
> fetching http://www.c 
> bs.nu/cgi-bin/ac/adcycle.cgi?gid=4&layout=multi&id=125
>
> Note the space in there. This URL is not in my initial fetchlist so it
> was found somewhere. Not sure if the space is actually a space or an
> encoding -> terminal issue, either way I think this has something to do
> with it. Does anyone know what happens when java/nutch gets a hostname
> that is obviously malformed?

I believe is should throw a malformed url exception.

Dennis Kubes

>
> -Brian
>
>
>
>
> On May 6, 2007, at 11:00 AM, Andrzej Bialecki wrote:
>
>> Brian Whitman wrote:
>>> Got this segfault + crash when fetching in the middle of a large
>>> fetch. Seems to be in looking up a hostname?
>>
>> Is this by any chance a FreeBSD machine of 4.x or 5.x vintage? There
>> was a bug in FreeBSD's getaddrinfo, which would manifest in a very
>> similar way when running multithreaded apps linked to libc_r or
>> libpthread.
>>
>> --
>> Best regards,
>> Andrzej Bialecki     <><
>>  ___. ___ ___ ___ _ _   __________________________________
>> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
>> ___|||__||  \|  ||  |  Embedded Unix, System Integration
>> http://www.sigram.com  Contact: info at sigram dot com
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Brian Whitman
On May 7, 2007, at 9:07 AM, Dennis Kubes wrote:

> Brian Whitman wrote:
>> Hi all,
>> I looked into this a bit more after it crashed for the third time  
>> in a row.
>> every time it has segfaulted it's had this url as one of the past  
>> few fetches:
>> fetching http://www.c bs.nu/cgi-bin/ac/adcycle.cgi?
>> gid=4&layout=multi&id=125
>> Note the space in there. This URL is not in my initial fetchlist  
>> so it was found somewhere. Not sure if the space is actually a  
>> space or an encoding -> terminal issue, either way I think this  
>> has something to do with it. Does anyone know what happens when  
>> java/nutch gets a hostname that is obviously malformed?
>
> I believe is should throw a malformed url exception.

OK. I got the crash again today on different urls. It's strange  
because I've been crawling quite regularly with the same nutch setup  
for a while. It's possible that a recent system-level change is  
getting in the way (I'm running debian with a recent full upgrade.)

After googling the culprit for a while I found this trick:

-Djava.net.preferIPv4Stack=true

I'm running a large crawl with it now and will let you know if I  
don't see it in a while!

-Brian


Reply | Threaded
Open this post in threaded view
|

Re: SIGSEGV

Brian Whitman
On May 7, 2007, at 6:34 PM, Brian Whitman wrote:

> OK. I got the crash again today on different urls. It's strange  
> because I've been crawling quite regularly with the same nutch  
> setup for a while. It's possible that a recent system-level change  
> is getting in the way (I'm running debian with a recent full upgrade.)
>
> After googling the culprit for a while I found this trick:
>
> -Djava.net.preferIPv4Stack=true
>
> I'm running a large crawl with it now and will let you know if I  
> don't see it in a while!

Just a note I've crawled 500K pages over a couple of days on the same  
start URL set that has been crashing it without a problem after  
adding that flag in bin/nutch.

So if anyone else gets the segfault it might be that.

-Brian