[jira] [Commented] (TIKA-3441) tika server stuck in loop trying to bind

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (TIKA-3441) tika server stuck in loop trying to bind

Lewis John McGibbney (Jira)

    [ https://issues.apache.org/jira/browse/TIKA-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361069#comment-17361069 ]

Tim Allison commented on TIKA-3441:
-----------------------------------

With the exception of this gem with Java 14:
{noformat}
WARN  [qtp1256893889-32] 12:27:19,572 org.eclipse.jetty.server.HttpChannel /rmeta
java.lang.NoClassDefFoundError: Could not initialize class java.io.DeleteOnExitHook
        at java.io.File.deleteOnExit(File.java:1094) ~[?:?]
        at org.apache.cxf.helpers.FileUtils.createTmpDir(FileUtils.java:161) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.helpers.FileUtils.getDefaultTempDir(FileUtils.java:88) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.helpers.FileUtils.createTempFile(FileUtils.java:269) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.helpers.FileUtils.createTempFile(FileUtils.java:263) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.io.CachedOutputStream.createFileOutputStream(CachedOutputStream.java:475) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.io.CachedOutputStream.enforceLimits(CachedOutputStream.java:437) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.io.CachedOutputStream.write(CachedOutputStream.java:445) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:193) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:146) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.io.DelegatingInputStream.cacheInput(DelegatingInputStream.java:54) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination$1.cacheInput(AbstractHTTPDestination.java:321) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination.cacheInput(AbstractHTTPDestination.java:600) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination.flushHeaders(AbstractHTTPDestination.java:622) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.flushHeaders(JettyHTTPDestination.java:272) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination$WrappedOutputStream.close(AbstractHTTPDestination.java:784) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:722) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:90) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135) ~[tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882) [tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036) [tika-server-standard-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:832) [?:?]
{noformat}

> tika server stuck in loop trying to bind
> ----------------------------------------
>
>                 Key: TIKA-3441
>                 URL: https://issues.apache.org/jira/browse/TIKA-3441
>             Project: Tika
>          Issue Type: Bug
>          Components: docker, server
>    Affects Versions: 1.25
>         Environment: {{PID TTY STAT TIME COMMAND}}
> {{ 1 ? Ssl 2:54 java -jar /tika-server-1.25.jar -h 0.0.0.0 -spawnChild TIKA_CHILD_JVM_OPTS=-JXmx3g -JXX:+ExitOnOutOfMemoryError -status}}
> {{ 127 ? Z 1:15 [tesseract] <defunct>}}
> {{ 131 ? Z 1:31 [tesseract] <defunct>}}
> {{ 139 ? Z 0:16 [tesseract] <defunct>}}
> {{ 219 ? Z 40:20 [tesseract] <defunct>}}
> {{ 324 ? Z 2:32 [tesseract] <defunct>}}
> {{ 342 ? Z 0:09 [tesseract] <defunct>}}
> {{ 343 ? Z 0:09 [tesseract] <defunct>}}
> {{ 380 ? Z 0:00 [tesseract] <defunct>}}
> {{ 430 ? Z 9:18 [tesseract] <defunct>}}
> {{ 435 ? Z 4:52 [tesseract] <defunct>}}
> {{ 446 ? Z 18:34 [tesseract] <defunct>}}
> {{ 453 ? Z 22:33 [tesseract] <defunct>}}
> {{ 461 ? Z 0:25 [tesseract] <defunct>}}
> {{ 517 ? Z 4:35 [tesseract] <defunct>}}
> {{ 526 ? Z 0:04 [tesseract] <defunct>}}
> {{ 536 ? S 0:00 36:39}}
> {{ 881708 pts/0 Ss 0:00 bash}}
> {{ 883782 ? Sl 0:00 java -XX:+ExitOnOutOfMemoryError -Djava.awt.headless=true -cp /tika-server-1.25.jar org.apache.tika.server.TikaServerCli -h 0.0.0.0 TIKA_CHILD_JVM_OPTS=-JXmx3g -status --id ab854166-0a4b-4ecc-93a5-e4504d11e4e8}}
> {{ 883801 pts/0 R+ 0:00 ps ax}}
>            Reporter: Cristian Zamfir
>            Priority: Major
>         Attachments: bind_issue.txt, logs_before.txt
>
>
> Tika seems to be stuck in a loop trying to bind. Please see the attached logs.
>  It is also using a lot of CPU while doing that, so in an environment with horizontal scaling it causes other containers to spin up.
> This seems to happen from time to time to one of the multiple containers running, normally everything runs smoothly.  
> Not sure why it fails to bind in the first place (it's running in docker).
> I suspect that it may be related to the fact that it is trying to bind very quickly.
> Ideas on how to debug this issue further are welcome.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)