[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_25) - Build # 6859 - Failure!

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_25) - Build # 6859 - Failure!

Policeman Jenkins Server-2
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6859/
Java: 32bit/jdk1.7.0_25 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Server refused connection at: http://127.0.0.1:34842/xbhli/mt

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:34842/xbhli/mt
        at __randomizedtesting.SeedInfo.seed([B5C97EC598F7BBC1:342FF0DDEFA8DBFD]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:428)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCustomCollectionsAPI(CollectionsAPIDistributedZkTest.java:764)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:159)
        at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:34842 refused
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        ... 44 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        ... 51 more




Build Log:
[...truncated 9821 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 797286 T3209 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /xbhli/mt
   [junit4]   2> 797289 T3209 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1375525420788
   [junit4]   2> 797290 T3209 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 797290 T3210 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 797390 T3209 oasc.ZkTestServer.run start zk server on port:47617
   [junit4]   2> 797391 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797469 T3216 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c21fab name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797469 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797469 T3209 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 797473 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797474 T3218 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f8101 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797474 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797474 T3209 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 797476 T3209 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 797478 T3209 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 797479 T3209 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 797482 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 797482 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 797485 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 797485 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 797488 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 797488 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 797490 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 797491 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 797492 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 797492 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 797494 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 797495 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 797496 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 797497 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 797498 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 797499 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 797500 T3209 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 797501 T3209 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 797566 T3209 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 797569 T3209 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41741
   [junit4]   2> 797569 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 797569 T3209 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 797570 T3209 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001
   [junit4]   2> 797570 T3209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/'
   [junit4]   2> 797590 T3209 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/solr.xml
   [junit4]   2> 797627 T3209 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 797628 T3209 oasc.CoreContainer.<init> New CoreContainer 20794386
   [junit4]   2> 797628 T3209 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/]
   [junit4]   2> 797629 T3209 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 797629 T3209 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 797630 T3209 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 797630 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 797630 T3209 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 797631 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 797631 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 797631 T3209 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 797631 T3209 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 797632 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 797635 T3209 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 797635 T3209 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 797636 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797637 T3229 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d4fe5a name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797637 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797638 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 797641 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797642 T3231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c536e name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797642 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797644 T3209 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 797646 T3209 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 797648 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 797649 T3209 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41741_xbhli%2Fmt
   [junit4]   2> 797650 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41741_xbhli%2Fmt
   [junit4]   2> 797653 T3209 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 797657 T3209 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 797658 T3209 oasc.Overseer.start Overseer (id=90146433976893443-127.0.0.1:41741_xbhli%2Fmt-n_0000000000) starting
   [junit4]   2> 797660 T3209 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 797663 T3233 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 797663 T3209 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 797665 T3209 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 797666 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 797668 T3232 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 797669 T3209 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001
   [junit4]   2> 797698 T3209 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/conf/
   [junit4]   2> 797698 T3209 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 797699 T3209 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 797699 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 797699 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 797702 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:
   [junit4]   2> 797703 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797705 T3236 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1739c56 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797705 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797706 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 797708 T3209 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
   [junit4]   2> 797769 T3209 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 797771 T3209 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34842
   [junit4]   2> 797771 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 797772 T3209 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 797772 T3209 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206
   [junit4]   2> 797772 T3209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/'
   [junit4]   2> 797784 T3209 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/solr.xml
   [junit4]   2> 797825 T3209 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 797826 T3209 oasc.CoreContainer.<init> New CoreContainer 27019522
   [junit4]   2> 797826 T3209 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/]
   [junit4]   2> 797827 T3209 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 797828 T3209 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 797828 T3209 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 797828 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 797828 T3209 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 797829 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 797829 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 797829 T3209 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 797830 T3209 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 797830 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 797833 T3209 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 797833 T3209 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 797834 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797835 T3247 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@662714 name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797835 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797837 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 797840 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797841 T3249 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a43179 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797841 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797844 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 798846 T3209 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34842_xbhli%2Fmt
   [junit4]   2> 798848 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34842_xbhli%2Fmt
   [junit4]   2> 798851 T3236 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798851 T3249 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798851 T3231 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798857 T3209 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206
   [junit4]   2> 798882 T3209 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/conf/
   [junit4]   2> 798882 T3209 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 798883 T3209 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 798883 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 798883 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 798961 T3209 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 798963 T3209 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48120
   [junit4]   2> 798964 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 798964 T3209 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 798964 T3209 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384
   [junit4]   2> 798965 T3209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384/'
   [junit4]   2> 798977 T3209 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384/solr.xml
   [junit4]   2> 799022 T3209 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 799023 T3209 oasc.CoreContainer.<init> New CoreContainer 28320784
   [junit4]   2> 799023 T3209 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384/]
   [junit4]   2> 799024 T3209 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 799024 T3209 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 799025 T3209 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 799025 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 799025 T3209 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 799026 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 799026 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 799026 T3209 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 799027 T3209 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 799027 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 799030 T3209 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 799030 T3209 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 799031 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 799032 T3261 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d51528 name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 799032 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 799034 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 799037 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 799038 T3263 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11cee5c name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 799038 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 799041 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 800043 T3209 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48120_xbhli%2Fmt
   [junit4]   2> 800044 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48120_xbhli%2Fmt
   [junit4]   2> 800047 T3249 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800047 T3231 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800047 T3263 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800048 T3236 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800052 T3209 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384
   [junit4]   2> 800079 T3209 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375525422384/conf/
   [junit4]   2> 800080 T3209 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 800080 T3209 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 800081 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 800081 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 800150 T3209 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 800152 T3209 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50033
   [junit4]   2> 800152 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 800153 T3209 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 800153 T3209 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582
   [junit4]   2> 800153 T3209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/'
   [junit4]   2> 800165 T3209 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/solr.xml
   [junit4]   2> 800209 T3209 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 800210 T3209 oasc.CoreContainer.<init> New CoreContainer 9249564
   [junit4]   2> 800210 T3209 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/]
   [junit4]   2> 800211 T3209 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 800211 T3209 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 800212 T3209 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 800212 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 800212 T3209 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 800213 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 800213 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 800213 T3209 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 800213 T3209 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 800214 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 800217 T3209 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 800217 T3209 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 800218 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 800219 T3275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d622ce name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 800219 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 800220 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 800223 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 800224 T3277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43d61 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 800224 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 800226 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 801228 T3209 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50033_xbhli%2Fmt
   [junit4]   2> 801230 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50033_xbhli%2Fmt
   [junit4]   2> 801233 T3249 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801234 T3236 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801234 T3263 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801234 T3231 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801234 T3277 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801238 T3209 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582
   [junit4]   2> 801264 T3209 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/conf/
   [junit4]   2> 801265 T3209 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 801266 T3209 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 801266 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 801266 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 801333 T3209 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 801335 T3209 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34487
   [junit4]   2> 801336 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 801336 T3209 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 801336 T3209 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767
   [junit4]   2> 801337 T3209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/'
   [junit4]   2> 801349 T3209 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/solr.xml
   [junit4]   2> 801392 T3209 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 801392 T3209 oasc.CoreContainer.<init> New CoreContainer 21539342
   [junit4]   2> 801393 T3209 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/]
   [junit4]   2> 801394 T3209 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 801394 T3209 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 801394 T3209 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 801395 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 801395 T3209 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 801395 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 801395 T3209 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 801396 T3209 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 801396 T3209 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 801396 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 801399 T3209 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47617/solr
   [junit4]   2> 801400 T3209 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 801400 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801402 T3289 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f65570 name:ZooKeeperConnection Watcher:127.0.0.1:47617 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801402 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801403 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 801406 T3209 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 801407 T3291 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1be89d4 name:ZooKeeperConnection Watcher:127.0.0.1:47617/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 801407 T3209 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 801410 T3209 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 802413 T3209 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34487_xbhli%2Fmt
   [junit4]   2> 802414 T3209 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34487_xbhli%2Fmt
   [junit4]   2> 802417 T3263 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802417 T3236 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802417 T3277 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802417 T3249 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802418 T3231 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802418 T3291 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 802423 T3209 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767
   [junit4]   2> 802448 T3209 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/conf/
   [junit4]   2> 802448 T3209 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 802449 T3209 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 802449 T3209 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 802450 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 802453 T3209 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 802456 T3243 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
   [junit4]   2> 802458 T3231 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 802459 T3233 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2>  "operation":"createcollection",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "name":"nodes_used_collection",
   [junit4]   2>  "replicationFactor":"2"}
   [junit4]   2> 802459 T3233 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2>  "operation":"createcollection",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "name":"nodes_used_collection",
   [junit4]   2>  "replicationFactor":"2"}
   [junit4]   2> 803675 T3232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
   [junit4]   2> 803675 T3232 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: nodes_used_collection
   [junit4]   2> 803675 T3232 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with shards [shard1, shard2]
   [junit4]   2> 803679 T3291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803679 T3277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803679 T3231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803679 T3236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803679 T3263 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803679 T3249 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 803763 T3233 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 803763 T3233 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:34487_xbhli%2Fmt
   [junit4]   2> 803763 T3233 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:41741_xbhli%2Fmt
   [junit4]   2> 803764 T3233 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:34842_xbhli%2Fmt
   [junit4]   2> 803765 T3233 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50033_xbhli%2Fmt
   [junit4]   2> 803766 T3284 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
   [junit4]   2> 803772 T3270 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
   [junit4]   2> 803772 T3225 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
   [junit4]   2> 803766 T3245 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
   [junit4]   2> 803773 T3284 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 803773 T3270 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 803774 T3225 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 803775 T3245 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 805184 T3232 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
   [junit4]   2> 805185 T3232 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 805186 T3232 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node1",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard1",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard1_replica1",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:34487_xbhli%2Fmt",
   [junit4]   2>  "base_url":"http://127.0.0.1:34487/xbhli/mt"}
   [junit4]   2> 805190 T3232 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 805190 T3232 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node2",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard2",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard2_replica2",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:50033_xbhli%2Fmt",
   [junit4]   2>  "base_url":"http://127.0.0.1:50033/xbhli/mt"}
   [junit4]   2> 805194 T3232 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 805194 T3232 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node3",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard1",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard1_replica2",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:41741_xbhli%2Fmt",
   [junit4]   2>  "base_url":"http://127.0.0.1:41741/xbhli/mt"}
   [junit4]   2> 805198 T3232 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 805198 T3232 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node4",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard2",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard2_replica1",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:34842_xbhli%2Fmt",
   [junit4]   2>  "base_url":"http://127.0.0.1:34842/xbhli/mt"}
   [junit4]   2> 805202 T3277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805202 T3263 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805202 T3231 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805202 T3291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805202 T3236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805202 T3249 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 805774 T3284 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard1_replica1
   [junit4]   2> 805775 T3225 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard1_replica2
   [junit4]   2> 805775 T3245 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard2_replica1
   [junit4]   2> 805774 T3270 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard2_replica2
   [junit4]   2> 805776 T3245 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1/core.properties (No such file or directory)
   [junit4]   2> 805776 T3270 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2/core.properties (No such file or directory)
   [junit4]   2> 805776 T3225 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2/core.properties (No such file or directory)
   [junit4]   2> 805775 T3284 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1/core.properties (No such file or directory)
   [junit4]   2> 805777 T3225 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2
   [junit4]   2> 805777 T3270 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2
   [junit4]   2> 805778 T3270 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 805777 T3245 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1
   [junit4]   2> 805779 T3270 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 805779 T3245 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 805778 T3225 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 805778 T3284 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1
   [junit4]   2> 805780 T3284 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 805780 T3245 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 805779 T3270 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 805781 T3245 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 805781 T3284 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 805782 T3284 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 805782 T3245 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 805781 T3225 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 805783 T3284 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 805783 T3245 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 805782 T3270 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 805784 T3284 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 805783 T3225 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 805784 T3270 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 805786 T3245 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 805786 T3225 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 805787 T3284 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 805787 T3245 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1/'
   [junit4]   2> 805787 T3270 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 805788 T3284 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1/'
   [junit4]   2> 805789 T3225 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 805789 T3270 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2/'
   [junit4]   2> 805792 T3225 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2/'
   [junit4]   2> 805844 T3284 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 805853 T3270 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 805854 T3245 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 805858 T3225 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 805922 T3284 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 805924 T3284 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 805932 T3284 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
   [junit4]   2> 805941 T3225 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 805943 T3225 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 805944 T3270 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 805945 T3245 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 805945 T3270 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 805946 T3245 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 805956 T3225 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
   [junit4]   2> 805957 T3245 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
   [junit4]   2> 805959 T3270 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
   [junit4]   2> 806641 T3284 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 806648 T3284 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 806651 T3284 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 806660 T3245 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 806667 T3284 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806669 T3245 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 806672 T3284 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806674 T3245 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 806677 T3284 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806678 T3284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806678 T3284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806679 T3284 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806680 T3284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806681 T3284 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806681 T3284 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 806682 T3284 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767//nodes_used_collection_shard1_replica1/data/
   [junit4]   2> 806682 T3284 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75ec7c
   [junit4]   2> 806684 T3284 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767//nodes_used_collection_shard1_replica1/data
   [junit4]   2> 806684 T3284 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767//nodes_used_collection_shard1_replica1/data/index/
   [junit4]   2> 806684 T3225 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 806685 T3284 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806686 T3284 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375525424767//nodes_used_collection_shard1_replica1/data/index
   [junit4]   2> 806686 T3245 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806688 T3284 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@14883a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ea5789),segFN=segments_1,generation=1}
   [junit4]   2> 806688 T3284 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 806691 T3245 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806698 T3284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806698 T3284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806699 T3225 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 806699 T3284 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806700 T3284 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 806701 T3284 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 806701 T3284 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 806702 T3284 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 806703 T3284 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 806703 T3225 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 806703 T3284 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 806703 T3245 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806706 T3245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806707 T3245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806707 T3245 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806707 T3284 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 806709 T3245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806709 T3245 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806709 T3245 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 806710 T3245 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206//nodes_used_collection_shard2_replica1/data/
   [junit4]   2> 806710 T3245 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75ec7c
   [junit4]   2> 806711 T3284 oass.SolrIndexSearcher.<init> Opening Searcher@f7607a main
   [junit4]   2> 806711 T3245 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206//nodes_used_collection_shard2_replica1/data
   [junit4]   2> 806711 T3245 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206//nodes_used_collection_shard2_replica1/data/index/
   [junit4]   2> 806712 T3284 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 806712 T3245 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806712 T3284 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 806713 T3245 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375525421206//nodes_used_collection_shard2_replica1/data/index
   [junit4]   2> 806715 T3245 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5f5b32 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f1607),segFN=segments_1,generation=1}
   [junit4]   2> 806716 T3245 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 806720 T3225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806721 T3297 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@f7607a main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 806723 T3270 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 806724 T3284 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
   [junit4]   2> 806724 T3284 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:34487/xbhli/mt collection:nodes_used_collection shard:shard1
   [junit4]   2> 806758 T3284 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 806759 T3245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806759 T3225 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806759 T3245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806761 T3245 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806762 T3245 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 806762 T3245 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 806763 T3245 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 806763 T3270 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 806763 T3245 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 806763 T3284 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
   [junit4]   2> 806764 T3245 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 806764 T3225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806765 T3245 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 806766 T3270 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 806767 T3225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806767 T3225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806768 T3225 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806770 T3225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806770 T3225 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806770 T3245 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 806771 T3284 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 806770 T3225 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 806772 T3225 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001//nodes_used_collection_shard1_replica2/data/
   [junit4]   2> 806772 T3225 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75ec7c
   [junit4]   2> 806773 T3245 oass.SolrIndexSearcher.<init> Opening Searcher@1b0cfb9 main
   [junit4]   2> 806774 T3284 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=179999
   [junit4]   2> 806774 T3245 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 806773 T3225 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001//nodes_used_collection_shard1_replica2/data
   [junit4]   2> 806774 T3245 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 806775 T3225 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001//nodes_used_collection_shard1_replica2/data/index/
   [junit4]   2> 806775 T3225 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806776 T3225 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375525421001//nodes_used_collection_shard1_replica2/data/index
   [junit4]   2> 806778 T3225 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ff1c2c lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c885ba),segFN=segments_1,generation=1}
   [junit4]   2> 806779 T3225 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 806779 T3270 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806779 T3298 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@1b0cfb9 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 806782 T3245 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
   [junit4]   2> 806782 T3245 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:34842/xbhli/mt collection:nodes_used_collection shard:shard2
   [junit4]   2> 806783 T3225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806783 T3245 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 806783 T3225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806789 T3270 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 806789 T3225 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806790 T3225 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 806791 T3225 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 806791 T3225 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 806792 T3225 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 806792 T3270 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806793 T3225 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 806793 T3245 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
   [junit4]   2> 806793 T3225 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 806793 T3270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806794 T3270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806795 T3270 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 806796 T3225 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 806797 T3270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 806797 T3270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 806798 T3270 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 806798 T3225 oass.SolrIndexSearcher.<init> Opening Searcher@15c0132 main
   [junit4]   2> 806798 T3245 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 806798 T3270 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582//nodes_used_collection_shard2_replica2/data/
   [junit4]   2> 806799 T3225 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 806799 T3270 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75ec7c
   [junit4]   2> 806800 T3225 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 806801 T3270 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582//nodes_used_collection_shard2_replica2/data
   [junit4]   2> 806802 T3245 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=179999
   [junit4]   2> 806802 T3270 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582//nodes_used_collection_shard2_replica2/data/index/
   [junit4]   2> 806802 T3270 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806803 T3270 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375525423582//nodes_used_collection_shard2_replica2/data/index
   [junit4]   2> 806804 T3299 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@15c0132 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 806804 T3270 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aafd84 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d8b16),segFN=segments_1,generation=1}
   [junit4]   2> 806805 T3270 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 806806 T3225 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
   [junit4]   2> 806806 T3225 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:41741/xbhli/mt collection:nodes_used_collection shard:shard1
   [junit4]   2> 806806 T3225 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 806808 T3270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806808 T3270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806809 T3270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806809 T3270 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 806810 T3270 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 806810 T3270 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 806810 T3270 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 806811 T3270 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 806811 T3270 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 806814 T3270 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 806816 T3270 oass.SolrIndexSearcher.<init> Opening Searcher@213366 main
   [junit4]   2> 806817 T3270 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 806817 T3270 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 806820 T3300 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@213366 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 806821 T3270 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
   [junit4]   2> 806822 T3270 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:50033/xbhli/mt collection:nodes_used_collection shard:shard2
   [junit4]   2> 806822 T3270 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 807275 T3284 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 807276 T3284 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 807276 T3284 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34487/xbhli/mt/nodes_used_collection_shard1_replica1/
   [junit4]   2> 807277 T3284 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:34487/xbhli/mt START replicas=[http://127.0.0.1:41741/xbhli/mt/nodes_used_collection_shard1_replica2/] nUpdates=100
   [junit4]   2> 807278 T3284 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:34487/xbhli/mt DONE.  We have no versions.  sync failed.
   [junit4]   2> 807278 T3284 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 807279 T3284 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 807279 T3284 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34487/xbhli/mt/nodes_used_collection_shard1_replica1/ shard1
   [junit4]   2> 807280 T3284 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
   [junit4]   2> ASYNC  NEW_CORE C5077 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@13dc534 url=http://127.0.0.1:41741/xbhli/mt/nodes_used_collection_shard1_replica2 node=127.0.0.1:41741_xbhli%2Fmt C5077_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{state=down, core=nodes_used_collection_shard1_replica2, node_name=127.0.0.1:41741_xbhli%2Fmt, base_url=http://127.0.0.1:41741/xbhli/mt}
   [junit4]   2> 807287 T3227 C5077 P41741 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/xbhli/mt path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
   [junit4]   2> 807303 T3245 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 807303 T3245 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 807303 T3245 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34842/xbhli/mt/nodes_used_collection_shard2_replica1/
   [junit4]   2> 807303 T3245 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:34842/xbhli/mt START replicas=[http://127.0.0.1:50033/xbhli/mt/nodes_used_collection_shard2_replica2/] nUpdates=100
   [junit4]   2> 807304 T3245 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:34842/xbhli/mt DONE.  We have no versions.  sync failed.
   [junit4]   2> 807304 T3245 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 807305 T3245 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 807305 T3245 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34842/xbhli/mt/nodes_used_collection_shard2_replica1/ shard2
   [junit4]   2> 807305 T3245 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
   [junit4]   2> ASYNC  NEW_CORE C5078 name=nodes_used_collection_shard2_replica2 org.apach

[...truncated too long message...]

st -Dtests.method=testDistribSearch -Dtests.seed=B5C97EC598F7BBC1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ca -Dtests.timezone=Europe/Malta -Dtests.file.encoding=US-ASCII
   [junit4] ERROR    124s J0 | CollectionsAPIDistributedZkTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:34842/xbhli/mt
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B5C97EC598F7BBC1:342FF0DDEFA8DBFD]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:428)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
   [junit4]    > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCustomCollectionsAPI(CollectionsAPIDistributedZkTest.java:764)
   [junit4]    > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:159)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > at java.lang.Thread.run(Thread.java:724)
   [junit4]    > Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:34842 refused
   [junit4]    > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
   [junit4]    > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
   [junit4]    > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
   [junit4]    > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
   [junit4]    > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
   [junit4]    > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
   [junit4]    > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
   [junit4]    > ... 44 more
   [junit4]    > Caused by: java.net.ConnectException: Connection refused
   [junit4]    > at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    > at java.net.Socket.connect(Socket.java:579)
   [junit4]    > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
   [junit4]    > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
   [junit4]    > ... 51 more
   [junit4]   2> 921651 T3209 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 124371 T3208 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 922166 T3291 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/new_implicit_coll_0/leader_elect/c/election
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2>
   [junit4]   2> 922167 T3291 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 922167 T3291 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 922167 T3291 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 922168 T3291 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/new_implicit_coll_0/leader_elect/b/election
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2>
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ca, timezone=Europe/Malta
   [junit4]   2> NOTE: Linux 3.2.0-51-generic i386/Oracle Corporation 1.7.0_25 (32-bit)/cpus=8,threads=11,free=48447192,total=202874880
   [junit4]   2> NOTE: All tests run in this JVM: [TestSort, FileBasedSpellCheckerTest, AnalysisAfterCoreReloadTest, SolrIndexSplitterTest, ChaosMonkeyNothingIsSafeTest, TestSolrIndexConfig, TestStressRecovery, SoftAutoCommitTest, CacheHeaderTest, SolrInfoMBeanTest, TestDocumentBuilder, BasicFunctionalityTest, ZkCLITest, TestFaceting, SolrTestCaseJ4Test, TestFuzzyAnalyzedSuggestions, LoggingHandlerTest, TestFunctionQuery, BadComponentTest, SliceStateTest, TestReversedWildcardFilterFactory, DistributedTermsComponentTest, TestConfig, TestTrie, MinimalSchemaTest, TestDynamicFieldCollectionResource, TestValueSourceCache, LegacyHTMLStripCharFilterTest, TestStressVersions, TestCloudManagedSchemaAddField, BinaryUpdateRequestHandlerTest, PeerSyncTest, ResourceLoaderTest, IndexSchemaTest, TestSurroundQueryParser, TestRemoteStreaming, LeaderElectionTest, ShowFileRequestHandlerTest, TestSweetSpotSimilarityFactory, TestReplicationHandler, AutoCommitTest, CoreMergeIndexesAdminHandlerTest, PluginInfoTest, WordBreakSolrSpellCheckerTest, ScriptEngineTest, TestSolrDeletionPolicy1, TestWriterPerf, DistributedQueryElevationComponentTest, DistributedSpellCheckComponentTest, SliceStateUpdateTest, UUIDFieldTest, TestPseudoReturnFields, ExternalFileFieldSortTest, AddSchemaFieldsUpdateProcessorFactoryTest, StatelessScriptUpdateProcessorFactoryTest, TestSolrDeletionPolicy2, TestPerFieldSimilarity, SuggesterFSTTest, TestGroupingSearch, CircularListTest, SystemInfoHandlerTest, OverseerCollectionProcessorTest, DocumentBuilderTest, TestSolrXmlPersistence, TestXIncludeConfig, TestCoreContainer, CoreAdminHandlerTest, DateFieldTest, BasicDistributedZkTest, PreAnalyzedUpdateProcessorTest, TestQuerySenderListener, RegexBoostProcessorTest, TestFieldResource, TestSolrQueryParserDefaultOperatorResource, HdfsChaosMonkeySafeLeaderTest, MoreLikeThisHandlerTest, ShardRoutingTest, SolrCoreTest, InfoHandlerTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J0 in 125.15s, 1 test, 1 error <<< FAILURES!

[...truncated 493 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1347 tests, 1 error, 34 ignored (6 assumptions)

Total time: 43 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_25 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_25) - Build # 6860 - Still Failing!

Policeman Jenkins Server-2
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6860/
Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Server refused connection at: http://127.0.0.1:54651

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:54651
        at __randomizedtesting.SeedInfo.seed([7A5E553EAAB3C8:819CD04D49F5D3F4]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:428)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1535)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCustomCollectionsAPI(CollectionsAPIDistributedZkTest.java:677)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:159)
        at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:54651 refused
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        ... 45 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        ... 52 more




Build Log:
[...truncated 9793 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 717671 T3109 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 717675 T3109 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1375530521665
   [junit4]   2> 717675 T3109 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 717676 T3110 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 717776 T3109 oasc.ZkTestServer.run start zk server on port:51610
   [junit4]   2> 717777 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718017 T3116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@724a1c71 name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718017 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718018 T3109 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 718022 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718023 T3118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@348cf9c2 name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718023 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718023 T3109 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 718026 T3109 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 718028 T3109 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 718030 T3109 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 718034 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 718035 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 718039 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 718040 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 718043 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 718044 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 718047 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 718047 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 718050 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 718050 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 718053 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 718053 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 718055 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 718056 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 718058 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 718059 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 718061 T3109 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 718062 T3109 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 718124 T3109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 718126 T3109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53511
   [junit4]   2> 718126 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 718127 T3109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 718127 T3109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055
   [junit4]   2> 718127 T3109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/'
   [junit4]   2> 718138 T3109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/solr.xml
   [junit4]   2> 718170 T3109 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 718171 T3109 oasc.CoreContainer.<init> New CoreContainer 1523106792
   [junit4]   2> 718172 T3109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/]
   [junit4]   2> 718173 T3109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 718173 T3109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 718173 T3109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 718174 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 718174 T3109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 718174 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 718174 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 718175 T3109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 718175 T3109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 718175 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 718178 T3109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51610/solr
   [junit4]   2> 718179 T3109 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 718180 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718181 T3129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@273ec836 name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718181 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718182 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 718185 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718186 T3131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd7e648 name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718186 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718188 T3109 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 718190 T3109 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 718193 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 718194 T3109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53511_
   [junit4]   2> 718195 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53511_
   [junit4]   2> 718197 T3109 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 718202 T3109 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 718205 T3109 oasc.Overseer.start Overseer (id=90146768267968515-127.0.0.1:53511_-n_0000000000) starting
   [junit4]   2> 718207 T3109 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 718210 T3133 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 718210 T3109 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 718213 T3109 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 718214 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 718217 T3132 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 718221 T3109 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055
   [junit4]   2> 718257 T3109 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/conf/
   [junit4]   2> 718258 T3109 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 718258 T3109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 718258 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 718259 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 718262 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:
   [junit4]   2> 718263 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718264 T3136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a828bcb name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718264 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718265 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 718267 T3109 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
   [junit4]   2> 718327 T3109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 718331 T3109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60725
   [junit4]   2> 718331 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 718332 T3109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 718332 T3109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257
   [junit4]   2> 718332 T3109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/'
   [junit4]   2> 718343 T3109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/solr.xml
   [junit4]   2> 718375 T3109 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 718375 T3109 oasc.CoreContainer.<init> New CoreContainer 839284776
   [junit4]   2> 718376 T3109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/]
   [junit4]   2> 718377 T3109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 718377 T3109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 718377 T3109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 718377 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 718378 T3109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 718378 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 718378 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 718378 T3109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 718379 T3109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 718379 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 718382 T3109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51610/solr
   [junit4]   2> 718382 T3109 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 718383 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718385 T3147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a9b5cd3 name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718385 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718387 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 718390 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 718392 T3149 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51b0a76 name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 718392 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 718395 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 719398 T3109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60725_
   [junit4]   2> 719399 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60725_
   [junit4]   2> 719401 T3131 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 719401 T3149 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 719401 T3136 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 719432 T3109 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257
   [junit4]   2> 719454 T3109 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/conf/
   [junit4]   2> 719455 T3109 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 719455 T3109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 719455 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 719456 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 719530 T3109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 719532 T3109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:54651
   [junit4]   2> 719533 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 719534 T3109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 719534 T3109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449
   [junit4]   2> 719534 T3109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/'
   [junit4]   2> 719549 T3109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/solr.xml
   [junit4]   2> 719582 T3109 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 719582 T3109 oasc.CoreContainer.<init> New CoreContainer 149992036
   [junit4]   2> 719583 T3109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/]
   [junit4]   2> 719584 T3109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 719584 T3109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 719584 T3109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 719584 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 719585 T3109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 719585 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 719585 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 719586 T3109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 719586 T3109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 719586 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 719589 T3109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51610/solr
   [junit4]   2> 719590 T3109 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 719590 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 719592 T3161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@581fc0c0 name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 719592 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 719593 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 719596 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 719598 T3163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f98284d name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 719598 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 719601 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 720604 T3109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54651_
   [junit4]   2> 720605 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54651_
   [junit4]   2> 720607 T3131 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 720607 T3136 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 720607 T3163 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 720607 T3149 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 720611 T3109 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449
   [junit4]   2> 720637 T3109 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/conf/
   [junit4]   2> 720638 T3109 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 720638 T3109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 720639 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 720639 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 720702 T3109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 720704 T3109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55178
   [junit4]   2> 720705 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 720705 T3109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 720705 T3109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632
   [junit4]   2> 720706 T3109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/'
   [junit4]   2> 720716 T3109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/solr.xml
   [junit4]   2> 720749 T3109 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 720749 T3109 oasc.CoreContainer.<init> New CoreContainer 850354064
   [junit4]   2> 720749 T3109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/]
   [junit4]   2> 720750 T3109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 720751 T3109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 720751 T3109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 720751 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 720752 T3109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 720752 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 720752 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 720752 T3109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 720753 T3109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 720753 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 720756 T3109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51610/solr
   [junit4]   2> 720756 T3109 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 720757 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 720759 T3175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@411903e8 name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 720759 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 720760 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 720763 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 720764 T3177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58132326 name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 720764 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 720767 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 721770 T3109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55178_
   [junit4]   2> 721771 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55178_
   [junit4]   2> 721774 T3163 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 721774 T3131 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 721774 T3149 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 721774 T3136 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 721774 T3177 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 721779 T3109 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632
   [junit4]   2> 721805 T3109 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/conf/
   [junit4]   2> 721806 T3109 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 721807 T3109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 721807 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 721807 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 721875 T3109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 721921 T3109 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46904
   [junit4]   2> 721922 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 721923 T3109 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 721923 T3109 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800
   [junit4]   2> 721924 T3109 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800/'
   [junit4]   2> 721941 T3109 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800/solr.xml
   [junit4]   2> 721975 T3109 oasc.ConfigSolrXml.<init> Config-defined core root directory:
   [junit4]   2> 721975 T3109 oasc.CoreContainer.<init> New CoreContainer 261137057
   [junit4]   2> 721975 T3109 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800/]
   [junit4]   2> 721976 T3109 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 721977 T3109 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 721977 T3109 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 721977 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 721978 T3109 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 721978 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 721978 T3109 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 721978 T3109 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 721979 T3109 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 721979 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 721982 T3109 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51610/solr
   [junit4]   2> 721982 T3109 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 721983 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 721985 T3189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bb7d63f name:ZooKeeperConnection Watcher:127.0.0.1:51610 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721985 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 721986 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 721989 T3109 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 721991 T3191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19450efd name:ZooKeeperConnection Watcher:127.0.0.1:51610/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 721991 T3109 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 721994 T3109 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
   [junit4]   2> 722997 T3109 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46904_
   [junit4]   2> 722998 T3109 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46904_
   [junit4]   2> 723000 T3163 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723000 T3177 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723000 T3149 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723000 T3131 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723000 T3136 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723001 T3191 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 723005 T3109 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800
   [junit4]   2> 723029 T3109 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1375530525800/conf/
   [junit4]   2> 723029 T3109 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 723030 T3109 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 723030 T3109 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 723030 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 723035 T3109 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 723039 T3142 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
   [junit4]   2> 723041 T3131 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 723042 T3133 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2>  "operation":"createcollection",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "name":"nodes_used_collection",
   [junit4]   2>  "replicationFactor":"2"}
   [junit4]   2> 723042 T3133 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2>  "operation":"createcollection",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "name":"nodes_used_collection",
   [junit4]   2>  "replicationFactor":"2"}
   [junit4]   2> 724225 T3132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
   [junit4]   2> 724226 T3132 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: nodes_used_collection
   [junit4]   2> 724226 T3132 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with shards [shard1, shard2]
   [junit4]   2> 724230 T3149 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724230 T3131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724230 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724230 T3177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724230 T3191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724230 T3136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 724245 T3133 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 724245 T3133 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:55178_
   [junit4]   2> 724245 T3133 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:54651_
   [junit4]   2> 724246 T3133 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:60725_
   [junit4]   2> 724247 T3133 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:53511_
   [junit4]   2> 724249 T3158 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
   [junit4]   2> 724249 T3172 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
   [junit4]   2> 724249 T3145 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
   [junit4]   2> 724250 T3125 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
   [junit4]   2> 724250 T3158 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 724251 T3145 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 724251 T3172 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 724251 T3125 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 725735 T3132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
   [junit4]   2> 725736 T3132 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 725736 T3132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node1",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard1",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard1_replica2",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:54651_",
   [junit4]   2>  "base_url":"http://127.0.0.1:54651"}
   [junit4]   2> 725739 T3132 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 725739 T3132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node2",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard2",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard2_replica1",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:60725_",
   [junit4]   2>  "base_url":"http://127.0.0.1:60725"}
   [junit4]   2> 725742 T3132 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 725742 T3132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node3",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard1",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard1_replica1",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:55178_",
   [junit4]   2>  "base_url":"http://127.0.0.1:55178"}
   [junit4]   2> 725745 T3132 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for nodes_used_collection, skipping publish state
   [junit4]   2> 725745 T3132 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2>  "operation":"state",
   [junit4]   2>  "core_node_name":"core_node4",
   [junit4]   2>  "numShards":"2",
   [junit4]   2>  "shard":"shard2",
   [junit4]   2>  "roles":null,
   [junit4]   2>  "state":"down",
   [junit4]   2>  "shard_state":"active",
   [junit4]   2>  "core":"nodes_used_collection_shard2_replica2",
   [junit4]   2>  "collection":"nodes_used_collection",
   [junit4]   2>  "shard_range":null,
   [junit4]   2>  "node_name":"127.0.0.1:53511_",
   [junit4]   2>  "base_url":"http://127.0.0.1:53511"}
   [junit4]   2> 725750 T3191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 725750 T3136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 725750 T3177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 725750 T3131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 725750 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 725750 T3149 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 726251 T3158 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard1_replica2
   [junit4]   2> 726251 T3172 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard1_replica1
   [junit4]   2> 726252 T3158 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2/core.properties (No such file or directory)
   [junit4]   2> 726252 T3158 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2
   [junit4]   2> 726253 T3158 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 726251 T3145 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard2_replica1
   [junit4]   2> 726252 T3172 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1/core.properties (No such file or directory)
   [junit4]   2> 726254 T3172 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1
   [junit4]   2> 726254 T3172 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 726252 T3125 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for nodes_used_collection_shard2_replica2
   [junit4]   2> 726254 T3158 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 726254 T3145 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1/core.properties (No such file or directory)
   [junit4]   2> 726255 T3158 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 726255 T3125 oasc.CorePropertiesLocator.writePropertiesFile ERROR Couldn't persist core properties to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2/core.properties: java.io.FileNotFoundException: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2/core.properties (No such file or directory)
   [junit4]   2> 726255 T3172 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 726256 T3125 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2
   [junit4]   2> 726257 T3125 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 726255 T3145 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1
   [junit4]   2> 726257 T3145 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
   [junit4]   2> 726256 T3158 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 726256 T3172 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 726258 T3158 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 726258 T3145 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 726259 T3145 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 726257 T3125 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
   [junit4]   2> 726259 T3125 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 726259 T3172 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 726260 T3172 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 726260 T3145 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 726261 T3145 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 726262 T3125 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 726262 T3125 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
   [junit4]   2> 726262 T3158 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 726262 T3172 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 726263 T3145 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 726264 T3158 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2/'
   [junit4]   2> 726264 T3145 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1/'
   [junit4]   2> 726264 T3125 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
   [junit4]   2> 726264 T3172 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1/'
   [junit4]   2> 726269 T3125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2/'
   [junit4]   2> 726304 T3125 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 726314 T3158 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 726316 T3145 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 726321 T3172 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 726340 T3125 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 726341 T3125 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 726349 T3125 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
   [junit4]   2> 726367 T3158 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 726369 T3158 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 726370 T3145 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 726372 T3145 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 726374 T3172 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 726376 T3172 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 726380 T3158 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
   [junit4]   2> 726383 T3145 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
   [junit4]   2> 726387 T3172 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
   [junit4]   2> 726834 T3145 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 726840 T3145 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 726843 T3145 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 726855 T3145 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726858 T3145 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726858 T3125 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 726862 T3145 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726863 T3145 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726863 T3145 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726863 T3145 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726864 T3125 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 726864 T3145 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726865 T3145 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726865 T3145 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 726865 T3145 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257//nodes_used_collection_shard2_replica1/data/
   [junit4]   2> 726866 T3145 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25d2845a
   [junit4]   2> 726866 T3125 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 726866 T3145 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257//nodes_used_collection_shard2_replica1/data
   [junit4]   2> 726867 T3145 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257//nodes_used_collection_shard2_replica1/data/index/
   [junit4]   2> 726867 T3145 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 726867 T3145 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1375530522257//nodes_used_collection_shard2_replica1/data/index
   [junit4]   2> 726869 T3145 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12bce2d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@205e49ba),segFN=segments_1,generation=1}
   [junit4]   2> 726870 T3145 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 726874 T3145 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 726874 T3145 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 726875 T3145 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 726875 T3145 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 726876 T3145 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 726876 T3145 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 726876 T3145 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 726877 T3145 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 726877 T3145 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 726877 T3125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726880 T3145 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 726882 T3125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726882 T3145 oass.SolrIndexSearcher.<init> Opening Searcher@2a52e96f main
   [junit4]   2> 726883 T3145 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 726883 T3145 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 726885 T3125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726886 T3125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726887 T3125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726887 T3125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726888 T3125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726888 T3125 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726889 T3125 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 726889 T3125 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055//nodes_used_collection_shard2_replica2/data/
   [junit4]   2> 726889 T3125 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25d2845a
   [junit4]   2> 726890 T3125 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055//nodes_used_collection_shard2_replica2/data
   [junit4]   2> 726890 T3125 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055//nodes_used_collection_shard2_replica2/data/index/
   [junit4]   2> 726891 T3125 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 726891 T3125 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1375530522055//nodes_used_collection_shard2_replica2/data/index
   [junit4]   2> 726893 T3125 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32e9ef3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@476c8824),segFN=segments_1,generation=1}
   [junit4]   2> 726893 T3125 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 726894 T3197 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@2a52e96f main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 726896 T3145 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
   [junit4]   2> 726896 T3145 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:60725 collection:nodes_used_collection shard:shard2
   [junit4]   2> 726896 T3145 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 726897 T3125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 726898 T3125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 726898 T3125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 726899 T3125 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 726899 T3125 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 726900 T3125 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 726900 T3145 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
   [junit4]   2> 726900 T3125 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 726901 T3125 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 726901 T3125 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 726904 T3125 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 726905 T3145 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 726906 T3125 oass.SolrIndexSearcher.<init> Opening Searcher@2e32d4be main
   [junit4]   2> 726907 T3125 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 726907 T3125 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 726908 T3145 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=179999
   [junit4]   2> 726910 T3198 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@2e32d4be main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 726911 T3125 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
   [junit4]   2> 726911 T3125 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:53511 collection:nodes_used_collection shard:shard2
   [junit4]   2> 726912 T3125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 726950 T3158 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 726955 T3158 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 726958 T3158 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 726975 T3158 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726982 T3158 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 726985 T3172 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 726986 T3158 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726988 T3158 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726988 T3158 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726988 T3158 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 726989 T3158 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 726990 T3158 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 726990 T3172 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 726990 T3158 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 726991 T3158 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449//nodes_used_collection_shard1_replica2/data/
   [junit4]   2> 726991 T3158 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25d2845a
   [junit4]   2> 726992 T3158 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449//nodes_used_collection_shard1_replica2/data
   [junit4]   2> 726993 T3172 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 726993 T3158 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449//nodes_used_collection_shard1_replica2/data/index/
   [junit4]   2> 726993 T3158 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 726994 T3158 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1375530523449//nodes_used_collection_shard1_replica2/data/index
   [junit4]   2> 726996 T3158 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@72467391 lockFactory=org.apache.lucene.store.NativeFSLockFactory@65d33a0),segFN=segments_1,generation=1}
   [junit4]   2> 726997 T3158 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 727002 T3158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 727002 T3158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 727003 T3158 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 727004 T3172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 727004 T3158 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 727005 T3158 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 727005 T3158 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 727005 T3158 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 727006 T3158 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 727007 T3158 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 727008 T3172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 727010 T3158 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 727011 T3172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 727012 T3172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 727013 T3172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 727013 T3172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 727013 T3158 oass.SolrIndexSearcher.<init> Opening Searcher@67d89859 main
   [junit4]   2> 727014 T3172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 727014 T3172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 727014 T3158 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 727014 T3172 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 727015 T3158 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 727015 T3172 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632//nodes_used_collection_shard1_replica1/data/
   [junit4]   2> 727016 T3172 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25d2845a
   [junit4]   2> 727017 T3172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632//nodes_used_collection_shard1_replica1/data
   [junit4]   2> 727017 T3172 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632//nodes_used_collection_shard1_replica1/data/index/
   [junit4]   2> 727017 T3172 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 727017 T3172 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1375530524632//nodes_used_collection_shard1_replica1/data/index
   [junit4]   2> 727019 T3199 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@67d89859 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 727019 T3172 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@24adb779 lockFactory=org.apache.lucene.store.NativeFSLockFactory@71bbbd0f),segFN=segments_1,generation=1}
   [junit4]   2> 727019 T3172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 727021 T3158 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
   [junit4]   2> 727021 T3158 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:54651 collection:nodes_used_collection shard:shard1
   [junit4]   2> 727021 T3158 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 727022 T3172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 727023 T3172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 727023 T3172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 727024 T3172 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 727024 T3172 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 727025 T3172 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 727025 T3172 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 727025 T3172 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 727026 T3172 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 727027 T3158 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
   [junit4]   2> 727028 T3172 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 727031 T3172 oass.SolrIndexSearcher.<init> Opening Searcher@1b331bf1 main
   [junit4]   2> 727031 T3172 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 727032 T3158 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 727032 T3172 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 727035 T3158 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=179999
   [junit4]   2> 727035 T3200 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@1b331bf1 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 727036 T3172 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
   [junit4]   2> 727036 T3172 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:55178 collection:nodes_used_collection shard:shard1
   [junit4]   2> 727036 T3172 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 727258 T3132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
   [junit4]   2> 727263 T3131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727263 T3177 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727263 T3191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727263 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727263 T3136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727263 T3149 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 727409 T3145 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 727409 T3145 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 727409 T3145 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60725/nodes_used_collection_shard2_replica1/
   [junit4]   2> 727410 T3145 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:60725 START replicas=[http://127.0.0.1:53511/nodes_used_collection_shard2_replica2/] nUpdates=100
   [junit4]   2> 727410 T3145 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:60725 DONE.  We have no versions.  sync failed.
   [junit4]   2> 727410 T3145 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 727411 T3145 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 727411 T3145 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60725/nodes_used_collection_shard2_replica1/ shard2
   [junit4]   2> 727411 T3145 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
   [junit4]   2> ASYNC  NEW_CORE C823 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1f7ef39c url=http://127.0.0.1:53511/nodes_used_collection_shard2_replica2 node=127.0.0.1:53511_ C823_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{state=down, core=nodes_used_collection_shard2_replica2, node_name=127.0.0.1:53511_, base_url=http://127.0.0.1:53511}
   [junit4]   2> 727417 T3126 C823 P53511 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
   [junit4]   2> 727536 T3158 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 727536 T3158 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   

[...truncated too long message...]

ient.execute(AbstractHttpClient.java:805)
   [junit4]    > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
   [junit4]    > ... 45 more
   [junit4]    > Caused by: java.net.ConnectException: Connection refused
   [junit4]    > at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    > at java.net.Socket.connect(Socket.java:579)
   [junit4]    > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
   [junit4]    > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
   [junit4]    > ... 52 more
   [junit4]   2> 818461 T3109 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 100795 T3108 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 819293 T3191 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 819294 T3191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 819294 T3191 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 819294 T3191 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/new_implicit_coll_1/leader_elect/d/election
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2>
   [junit4]   2> 819295 T3191 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/new_implicit_coll_1/leader_elect/c/election
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2>
   [junit4]   2> 819295 T3191 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/new_implicit_coll_1/leader_elect/a/election
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2>
   [junit4]   2> NOTE: test params are: codec=Lucene42: {timestamp=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), id=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), a_t=PostingsFormat(name=Direct), text=PostingsFormat(name=Lucene41WithOrds), range_facet_si=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=Direct), other_tl1=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=Asserting), a_si=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=Direct)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=DefaultSimilarity, locale=sk, timezone=Etc/GMT+4
   [junit4]   2> NOTE: Linux 3.2.0-51-generic amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=8,threads=1,free=256544016,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SchemaVersionSpecificBehaviorTest, TestSearchPerf, TestPropInjectDefaults, HdfsBasicDistributedZkTest, ScriptEngineTest, TestStressRecovery, RAMDirectoryFactoryTest, WordBreakSolrSpellCheckerTest, AnalysisAfterCoreReloadTest, TestFieldTypeResource, AddSchemaFieldsUpdateProcessorFactoryTest, TestFieldResource, TestCharFilters, TestStressVersions, MoreLikeThisHandlerTest, OverseerTest, TestSolrIndexConfig, SuggesterFSTTest, CoreAdminHandlerTest, PrimitiveFieldTypeTest, TestSolrDeletionPolicy2, TestRandomDVFaceting, TestFaceting, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, PreAnalyzedUpdateProcessorTest, TestReplicationHandler, TestSolrDeletionPolicy1, SpatialFilterTest, TestSurroundQueryParser, UUIDFieldTest, TestSolrXmlPersistence, OverseerCollectionProcessorTest, CircularListTest, SuggesterTest, TestWriterPerf, TestPHPSerializedResponseWriter, AutoCommitTest, TestSolrQueryParser, ShowFileRequestHandlerTest, TestPseudoReturnFields, BinaryUpdateRequestHandlerTest, QueryParsingTest, TestGroupingSearch, TestLMDirichletSimilarityFactory, SearchHandlerTest, QueryEqualityTest, FastVectorHighlighterTest, TestDefaultSearchFieldResource, ShardRoutingTest, PreAnalyzedFieldTest, LukeRequestHandlerTest, DirectSolrConnectionTest, TestCloudManagedSchemaAddField, ExternalFileFieldSortTest, TestXIncludeConfig, LegacyHTMLStripCharFilterTest, PeerSyncTest, IndexSchemaTest, TestLFUCache, DocumentBuilderTest, TestCoreContainer, SimpleFacetsTest, TestSchemaNameResource, TestDynamicFieldResource, SolrTestCaseJ4Test, TestSolrXml, FileBasedSpellCheckerTest, ParsingFieldUpdateProcessorsTest, TestValueSourceCache, TestIBSimilarityFactory, TestMaxScoreQueryParser, UnloadDistributedZkTest, RegexBoostProcessorTest, TestDynamicFieldCollectionResource, ChaosMonkeyNothingIsSafeTest, TestFuzzyAnalyzedSuggestions, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J0 in 101.84s, 1 test, 1 error <<< FAILURES!

[...truncated 497 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1347 tests, 1 error, 34 ignored (6 assumptions)

Total time: 41 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b99) - Build # 6861 - Still Failing!

Policeman Jenkins Server-2
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6861/
Java: 32bit/jdk1.8.0-ea-b99 -server -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Sun Aug 04 01:09:17 PETT 2013

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to exceed: Sun Aug 04 01:09:17 PETT 2013
        at __randomizedtesting.SeedInfo.seed([E717B7C2137E9F2A:3CBCB7041656F699]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1507)
        at org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:811)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:491)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
        at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 9673 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> 257641 T965 oas.SolrTestCaseJ4.setUp ###Starting doTestSnapPullWithMasterUrl
   [junit4]   2> 257644 T965 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 257649 T965 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48505
   [junit4]   2> 257650 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 257650 T965 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 257651 T965 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master
   [junit4]   2> 257651 T965 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/'
   [junit4]   2> 257662 T965 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/solr.xml
   [junit4]   2> 257663 T965 oasc.ConfigSolr.fromFile /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/solr.xml does not exist, using default configuration
   [junit4]   2> 257669 T965 oasc.CoreContainer.<init> New CoreContainer 19479277
   [junit4]   2> 257670 T965 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/]
   [junit4]   2> 257673 T965 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 257673 T965 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 257673 T965 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 257674 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 257674 T965 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 257674 T965 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 257674 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 257675 T965 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 257675 T965 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 257675 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 257679 T975 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1
   [junit4]   2> 257680 T975 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/'
   [junit4]   2> 257696 T975 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 257707 T975 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 257708 T975 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 257710 T975 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 257712 T975 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 257713 T975 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 257713 T975 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/data/
   [junit4]   2> 257713 T975 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 257713 T975 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/data
   [junit4]   2> 257713 T975 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/data/index/
   [junit4]   2> 257714 T975 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 257714 T975 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338119/master/collection1/data/index
   [junit4]   2> 257715 T975 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@192e09f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b1a7ab),segFN=segments_1,generation=1}
   [junit4]   2> 257715 T975 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 257716 T975 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 257716 T975 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 257716 T975 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 257716 T975 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 257716 T975 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 257717 T975 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 257718 T975 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 257719 T975 oass.SolrIndexSearcher.<init> Opening Searcher@166631e main
   [junit4]   2> 257719 T975 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 257719 T975 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 257720 T975 oash.ReplicationHandler.inform Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 257720 T975 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 257720 T976 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@166631e main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 257720 T975 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 257721 T965 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 257721 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 257721 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 257726 T965 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 257728 T965 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59984
   [junit4]   2> 257729 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 257729 T965 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 257730 T965 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave
   [junit4]   2> 257730 T965 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/'
   [junit4]   2> 257743 T965 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/solr.xml
   [junit4]   2> 257744 T965 oasc.ConfigSolr.fromFile /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/solr.xml does not exist, using default configuration
   [junit4]   2> 257749 T965 oasc.CoreContainer.<init> New CoreContainer 11542164
   [junit4]   2> 257750 T965 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/]
   [junit4]   2> 257750 T965 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 257751 T965 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 257751 T965 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 257751 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 257751 T965 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 257752 T965 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 257752 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 257752 T965 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 257752 T965 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 257753 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 257759 T987 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1
   [junit4]   2> 257760 T987 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/'
   [junit4]   2> 257775 T987 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 257784 T987 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 257785 T987 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 257787 T987 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 257789 T987 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 257790 T987 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 257790 T987 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/
   [junit4]   2> 257790 T987 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 257791 T987 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data
   [junit4]   2> 257791 T987 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index/
   [junit4]   2> 257791 T987 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 257791 T987 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index
   [junit4]   2> 257792 T987 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b00810 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f5a6f1),segFN=segments_1,generation=1}
   [junit4]   2> 257792 T987 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 257793 T987 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 257793 T987 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 257793 T987 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 257793 T987 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 257794 T987 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 257794 T987 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 257795 T987 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 257796 T987 oass.SolrIndexSearcher.<init> Opening Searcher@1444f45 main
   [junit4]   2> 257796 T987 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 257796 T987 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 257798 T987 oash.SnapPuller.startExecutorService Poll Scheduled at an interval of 1000ms
   [junit4]   2> 257798 T988 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1444f45 main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 257798 T987 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 257799 T965 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 257799 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 257799 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 257824 T965 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=11542164
   [junit4]   2> 257825 T965 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@9e1e57
   [junit4]   2> 257825 T965 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 257826 T965 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 257826 T965 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 257827 T965 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 257827 T965 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 257827 T965 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 257828 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index;done=false>>]
   [junit4]   2> 257828 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index
   [junit4]   2> 257828 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data;done=false>>]
   [junit4]   2> 257829 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data
   [junit4]   2> 257829 T965 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 257880 T965 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 257882 T965 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60728
   [junit4]   2> 257883 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 257883 T965 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 257884 T965 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave
   [junit4]   2> 257884 T965 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/'
   [junit4]   2> 257891 T965 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/solr.xml
   [junit4]   2> 257892 T965 oasc.ConfigSolr.fromFile /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/solr.xml does not exist, using default configuration
   [junit4]   2> 257903 T965 oasc.CoreContainer.<init> New CoreContainer 9434289
   [junit4]   2> 257903 T965 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/]
   [junit4]   2> 257904 T965 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 257904 T965 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 257904 T965 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 257905 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 257905 T965 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 257905 T965 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 257905 T965 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 257906 T965 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 257906 T965 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 257906 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 257909 T1000 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1
   [junit4]   2> 257910 T1000 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/'
   [junit4]   2> 257927 T1000 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 257934 T1000 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 257935 T1000 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 257937 T1000 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 257939 T1000 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 257939 T1000 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 257939 T1000 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/
   [junit4]   2> 257940 T1000 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 257940 T1000 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data
   [junit4]   2> 257940 T1000 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index/
   [junit4]   2> 257940 T1000 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 257941 T1000 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535338202/slave/collection1/data/index
   [junit4]   2> 257941 T1000 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13b8e09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36691f),segFN=segments_1,generation=1}
   [junit4]   2> 257942 T1000 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 257942 T1000 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 257943 T1000 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 257943 T1000 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 257943 T1000 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 257943 T1000 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 257943 T1000 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 257944 T1000 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 257945 T1000 oass.SolrIndexSearcher.<init> Opening Searcher@1c6207f main
   [junit4]   2> 257945 T1000 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 257946 T1000 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 257946 T1000 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 257946 T1001 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c6207f main{StandardDirectoryReader(segments_1:1)}
   [junit4]   2> 257946 T1000 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 257947 T965 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 257947 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 257947 T965 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> ASYNC  NEW_CORE C2603 name=collection1 org.apache.solr.core.SolrCore@13092db
   [junit4]   2> 257951 T971 C2603 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 257953 T971 C2603 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@192e09f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b1a7ab),segFN=segments_1,generation=1}
   [junit4]   2> 257953 T971 C2603 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 257954 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 3
   [junit4]   2> ASYNC  NEW_CORE C2604 name=collection1 org.apache.solr.core.SolrCore@909738
   [junit4]   2> 257956 T996 C2604 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 257957 T996 C2604 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13b8e09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36691f),segFN=segments_1,generation=1}
   [junit4]   2> 257958 T996 C2604 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 257958 T996 C2604 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 2
   [junit4]   2> 257961 T997 C2604 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 257962 T997 C2604 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13b8e09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36691f),segFN=segments_1,generation=1}
   [junit4]   2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@13b8e09 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36691f),segFN=segments_2,generation=2}
   [junit4]   2> 257962 T997 C2604 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 257962 T997 C2604 oass.SolrIndexSearcher.<init> Opening Searcher@242338 main
   [junit4]   2> 257963 T997 C2604 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 257963 T1001 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@242338 main{StandardDirectoryReader(segments_2:2:nrt)}
   [junit4]   2> 257963 T997 C2604 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 2
   [junit4]   2> 257966 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
   [junit4]   2> 257968 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
   [junit4]   2> 257971 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 1
   [junit4]   2> 257973 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 0
   [junit4]   2> 257975 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
   [junit4]   2> 257977 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 0
   [junit4]   2> 257980 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 0
   [junit4]   2> 257982 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 0
   [junit4]   2> 257985 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
   [junit4]   2> 257987 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 0
   [junit4]   2> 257989 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
   [junit4]   2> 257992 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 1
   [junit4]   2> 257994 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
   [junit4]   2> 257996 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 0
   [junit4]   2> 257998 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
   [junit4]   2> 258000 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 0
   [junit4]   2> 258003 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 1
   [junit4]   2> 258005 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 0
   [junit4]   2> 258007 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
   [junit4]   2> 258009 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 0
   [junit4]   2> 258011 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
   [junit4]   2> 258014 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 1
   [junit4]   2> 258016 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
   [junit4]   2> 258018 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 0
   [junit4]   2> 258020 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
   [junit4]   2> 258022 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 0
   [junit4]   2> 258025 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
   [junit4]   2> 258027 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 0
   [junit4]   2> 258029 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0
   [junit4]   2> 258032 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 1
   [junit4]   2> 258034 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0
   [junit4]   2> 258036 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 0
   [junit4]   2> 258038 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0
   [junit4]   2> 258041 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 0
   [junit4]   2> 258043 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0
   [junit4]   2> 258045 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 0
   [junit4]   2> 258048 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 1
   [junit4]   2> 258050 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 0
   [junit4]   2> 258052 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0
   [junit4]   2> 258054 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 0
   [junit4]   2> 258057 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 1
   [junit4]   2> 258059 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 0
   [junit4]   2> 258061 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0
   [junit4]   2> 258063 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 0
   [junit4]   2> 258066 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0
   [junit4]   2> 258068 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 0
   [junit4]   2> 258071 T972 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0
   [junit4]   2> 258073 T973 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 0
   [junit4]   2> 258076 T974 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0
   [junit4]   2> 258079 T971 C2603 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 1
   [junit4]   2> ASYNC  NEW_CORE C2605 name=collection1 org.apache.solr.core.SolrCore@13092db
   [junit4]   2> 258085 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 4
   [junit4]   2> 258088 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 0
   [junit4]   2> 258094 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 4
   [junit4]   2> 258097 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 0
   [junit4]   2> 258100 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 1
   [junit4]   2> 258102 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 0
   [junit4]   2> 258105 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0
   [junit4]   2> 258108 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[57]} 0 1
   [junit4]   2> 258111 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[58]} 0 1
   [junit4]   2> 258113 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59]} 0 0
   [junit4]   2> 258115 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[60]} 0 0
   [junit4]   2> 258118 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[61]} 0 0
   [junit4]   2> 258120 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[62]} 0 0
   [junit4]   2> 258123 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[63]} 0 0
   [junit4]   2> 258125 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64]} 0 0
   [junit4]   2> 258127 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65]} 0 0
   [junit4]   2> 258129 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66]} 0 0
   [junit4]   2> 258132 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[67]} 0 0
   [junit4]   2> 258134 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68]} 0 0
   [junit4]   2> 258137 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[69]} 0 1
   [junit4]   2> 258139 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[70]} 0 0
   [junit4]   2> 258142 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71]} 0 1
   [junit4]   2> 258144 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[72]} 0 0
   [junit4]   2> 258147 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[73]} 0 0
   [junit4]   2> 258150 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[74]} 0 0
   [junit4]   2> 258153 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[75]} 0 1
   [junit4]   2> 258155 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[76]} 0 0
   [junit4]   2> 258158 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77]} 0 0
   [junit4]   2> 258160 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[78]} 0 0
   [junit4]   2> 258163 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[79]} 0 0
   [junit4]   2> 258165 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[80]} 0 0
   [junit4]   2> 258168 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81]} 0 0
   [junit4]   2> 258170 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[82]} 0 0
   [junit4]   2> 258173 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83]} 0 0
   [junit4]   2> 258176 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[84]} 0 1
   [junit4]   2> 258178 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85]} 0 0
   [junit4]   2> 258183 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[86]} 0 1
   [junit4]   2> 258186 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[87]} 0 0
   [junit4]   2> 258189 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[88]} 0 0
   [junit4]   2> 258192 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[89]} 0 1
   [junit4]   2> 258195 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90]} 0 1
   [junit4]   2> 258197 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[91]} 0 0
   [junit4]   2> 258201 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[92]} 0 0
   [junit4]   2> 258204 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[93]} 0 1
   [junit4]   2> 258206 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[94]} 0 0
   [junit4]   2> 258209 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[95]} 0 1
   [junit4]   2> 258211 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[96]} 0 0
   [junit4]   2> 258214 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[97]} 0 1
   [junit4]   2> 258217 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[98]} 0 0
   [junit4]   2> 258220 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[99]} 0 0
   [junit4]   2> 258223 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[100]} 0 0
   [junit4]   2> 258226 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[101]} 0 0
   [junit4]   2> 258230 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[102]} 0 0
   [junit4]   2> 258233 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[103]} 0 0
   [junit4]   2> 258236 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[104]} 0 1
   [junit4]   2> 258239 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[105]} 0 1
   [junit4]   2> 258242 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[106]} 0 0
   [junit4]   2> 258245 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[107]} 0 1
   [junit4]   2> 258257 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108]} 0 0
   [junit4]   2> 258260 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[109]} 0 0
   [junit4]   2> 258263 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[110]} 0 0
   [junit4]   2> 258266 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[111]} 0 0
   [junit4]   2> 258270 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[112]} 0 1
   [junit4]   2> 258273 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[113]} 0 0
   [junit4]   2> 258280 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[114]} 0 1
   [junit4]   2> 258283 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[115]} 0 1
   [junit4]   2> 258286 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116]} 0 1
   [junit4]   2> 258289 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117]} 0 0
   [junit4]   2> 258292 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[118]} 0 0
   [junit4]   2> 258296 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[119]} 0 1
   [junit4]   2> 258300 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[120]} 0 0
   [junit4]   2> 258303 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[121]} 0 0
   [junit4]   2> 258306 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[122]} 0 0
   [junit4]   2> 258309 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[123]} 0 0
   [junit4]   2> 258313 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[124]} 0 1
   [junit4]   2> 258316 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[125]} 0 1
   [junit4]   2> 258326 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[126]} 0 0
   [junit4]   2> 258330 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[127]} 0 1
   [junit4]   2> 258332 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[128]} 0 0
   [junit4]   2> 258335 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[129]} 0 0
   [junit4]   2> 258338 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[130]} 0 0
   [junit4]   2> 258341 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[131]} 0 1
   [junit4]   2> 258343 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[132]} 0 0
   [junit4]   2> 258346 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[133]} 0 1
   [junit4]   2> 258348 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[134]} 0 0
   [junit4]   2> 258350 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[135]} 0 0
   [junit4]   2> 258353 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[136]} 0 0
   [junit4]   2> 258355 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[137]} 0 0
   [junit4]   2> 258358 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[138]} 0 0
   [junit4]   2> 258361 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[139]} 0 0
   [junit4]   2> 258363 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[140]} 0 0
   [junit4]   2> 258366 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[141]} 0 1
   [junit4]   2> 258368 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[142]} 0 0
   [junit4]   2> 258371 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[143]} 0 0
   [junit4]   2> 258373 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[144]} 0 0
   [junit4]   2> 258376 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[145]} 0 1
   [junit4]   2> 258378 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[146]} 0 0
   [junit4]   2> 258381 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[147]} 0 1
   [junit4]   2> 258383 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[148]} 0 0
   [junit4]   2> 258386 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[149]} 0 0
   [junit4]   2> 258389 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[150]} 0 0
   [junit4]   2> 258393 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[151]} 0 1
   [junit4]   2> 258396 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[152]} 0 1
   [junit4]   2> 258398 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[153]} 0 0
   [junit4]   2> 258401 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[154]} 0 0
   [junit4]   2> 258403 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[155]} 0 0
   [junit4]   2> 258406 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[156]} 0 0
   [junit4]   2> 258408 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[157]} 0 0
   [junit4]   2> 258411 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[158]} 0 0
   [junit4]   2> 258413 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[159]} 0 0
   [junit4]   2> 258416 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[160]} 0 1
   [junit4]   2> 258419 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[161]} 0 0
   [junit4]   2> 258422 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[162]} 0 0
   [junit4]   2> 258425 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[163]} 0 1
   [junit4]   2> 258427 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[164]} 0 0
   [junit4]   2> 258429 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[165]} 0 0
   [junit4]   2> 258432 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[166]} 0 0
   [junit4]   2> 258434 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[167]} 0 0
   [junit4]   2> 258436 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[168]} 0 0
   [junit4]   2> 258439 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[169]} 0 1
   [junit4]   2> 258441 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[170]} 0 0
   [junit4]   2> 258443 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[171]} 0 0
   [junit4]   2> 258446 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[172]} 0 0
   [junit4]   2> 258448 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[173]} 0 0
   [junit4]   2> 258451 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[174]} 0 1
   [junit4]   2> 258453 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[175]} 0 0
   [junit4]   2> 258458 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[176]} 0 0
   [junit4]   2> 258461 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[177]} 0 0
   [junit4]   2> 258463 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[178]} 0 0
   [junit4]   2> 258465 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[179]} 0 0
   [junit4]   2> 258468 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[180]} 0 0
   [junit4]   2> 258470 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[181]} 0 0
   [junit4]   2> 258472 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[182]} 0 0
   [junit4]   2> 258475 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[183]} 0 0
   [junit4]   2> 258477 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[184]} 0 0
   [junit4]   2> 258479 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[185]} 0 0
   [junit4]   2> 258486 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[186]} 0 0
   [junit4]   2> 258489 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[187]} 0 1
   [junit4]   2> 258492 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[188]} 0 0
   [junit4]   2> 258500 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[189]} 0 0
   [junit4]   2> 258503 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[190]} 0 0
   [junit4]   2> 258506 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[191]} 0 0
   [junit4]   2> 258509 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[192]} 0 0
   [junit4]   2> 258513 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[193]} 0 1
   [junit4]   2> 258515 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[194]} 0 0
   [junit4]   2> 258518 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[195]} 0 0
   [junit4]   2> 258521 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[196]} 0 0
   [junit4]   2> 258525 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[197]} 0 0
   [junit4]   2> 258528 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[198]} 0 0
   [junit4]   2> 258531 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[199]} 0 0
   [junit4]   2> 258534 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[200]} 0 1
   [junit4]   2> 258537 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[201]} 0 1
   [junit4]   2> 258539 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[202]} 0 0
   [junit4]   2> 258542 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[203]} 0 0
   [junit4]   2> 258545 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[204]} 0 0
   [junit4]   2> 258548 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[205]} 0 1
   [junit4]   2> 258552 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[206]} 0 0
   [junit4]   2> 258555 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[207]} 0 0
   [junit4]   2> 258558 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[208]} 0 1
   [junit4]   2> 258560 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[209]} 0 0
   [junit4]   2> 258563 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[210]} 0 0
   [junit4]   2> 258566 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[211]} 0 0
   [junit4]   2> 258569 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[212]} 0 0
   [junit4]   2> 258572 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[213]} 0 0
   [junit4]   2> 258575 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[214]} 0 0
   [junit4]   2> 258577 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[215]} 0 0
   [junit4]   2> 258580 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[216]} 0 1
   [junit4]   2> 258582 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[217]} 0 0
   [junit4]   2> 258585 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218]} 0 0
   [junit4]   2> 258587 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[219]} 0 0
   [junit4]   2> 258591 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[220]} 0 0
   [junit4]   2> 258594 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[221]} 0 0
   [junit4]   2> 258596 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[222]} 0 0
   [junit4]   2> 258599 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[223]} 0 0
   [junit4]   2> 258601 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[224]} 0 0
   [junit4]   2> 258604 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[225]} 0 0
   [junit4]   2> 258607 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[226]} 0 0
   [junit4]   2> 258610 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[227]} 0 0
   [junit4]   2> 258613 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[228]} 0 1
   [junit4]   2> 258615 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[229]} 0 0
   [junit4]   2> 258618 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[230]} 0 0
   [junit4]   2> 258621 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[231]} 0 0
   [junit4]   2> 258624 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[232]} 0 0
   [junit4]   2> 258627 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[233]} 0 1
   [junit4]   2> 258629 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[234]} 0 0
   [junit4]   2> 258631 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[235]} 0 0
   [junit4]   2> 258634 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[236]} 0 0
   [junit4]   2> 258637 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[237]} 0 0
   [junit4]   2> 258640 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[238]} 0 1
   [junit4]   2> 258642 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[239]} 0 0
   [junit4]   2> 258645 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[240]} 0 1
   [junit4]   2> 258647 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[241]} 0 0
   [junit4]   2> 258649 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[242]} 0 0
   [junit4]   2> 258652 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[243]} 0 1
   [junit4]   2> 258654 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[244]} 0 0
   [junit4]   2> 258656 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[245]} 0 0
   [junit4]   2> 258659 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[246]} 0 1
   [junit4]   2> 258661 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[247]} 0 0
   [junit4]   2> 258663 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[248]} 0 0
   [junit4]   2> 258665 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[249]} 0 0
   [junit4]   2> 258668 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[250]} 0 0
   [junit4]   2> 258670 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[251]} 0 0
   [junit4]   2> 258672 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252]} 0 0
   [junit4]   2> 258675 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[253]} 0 1
   [junit4]   2> 258677 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[254]} 0 0
   [junit4]   2> 258679 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[255]} 0 0
   [junit4]   2> 258682 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[256]} 0 1
   [junit4]   2> 258684 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[257]} 0 0
   [junit4]   2> 258686 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[258]} 0 0
   [junit4]   2> 258689 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[259]} 0 1
   [junit4]   2> 258691 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[260]} 0 0
   [junit4]   2> 258693 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[261]} 0 0
   [junit4]   2> 258696 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[262]} 0 0
   [junit4]   2> 258698 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[263]} 0 0
   [junit4]   2> 258700 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[264]} 0 0
   [junit4]   2> 258703 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[265]} 0 1
   [junit4]   2> 258705 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[266]} 0 0
   [junit4]   2> 258707 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[267]} 0 0
   [junit4]   2> 258709 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[268]} 0 0
   [junit4]   2> 258712 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[269]} 0 1
   [junit4]   2> 258714 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[270]} 0 0
   [junit4]   2> 258716 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[271]} 0 0
   [junit4]   2> 258718 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[272]} 0 0
   [junit4]   2> 258721 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[273]} 0 1
   [junit4]   2> 258723 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[274]} 0 0
   [junit4]   2> 258725 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[275]} 0 0
   [junit4]   2> 258728 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[276]} 0 0
   [junit4]   2> 258730 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[277]} 0 0
   [junit4]   2> 258733 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[278]} 0 0
   [junit4]   2> 258736 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[279]} 0 0
   [junit4]   2> 258739 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[280]} 0 0
   [junit4]   2> 258742 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[281]} 0 1
   [junit4]   2> 258744 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[282]} 0 0
   [junit4]   2> 258747 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[283]} 0 1
   [junit4]   2> 258749 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[284]} 0 0
   [junit4]   2> 258752 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[285]} 0 0
   [junit4]   2> 258755 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[286]} 0 1
   [junit4]   2> 258757 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287]} 0 0
   [junit4]   2> 258760 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[288]} 0 1
   [junit4]   2> 258762 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[289]} 0 0
   [junit4]   2> 258764 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[290]} 0 0
   [junit4]   2> 258767 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[291]} 0 0
   [junit4]   2> 258769 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[292]} 0 0
   [junit4]   2> 258772 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[293]} 0 0
   [junit4]   2> 258774 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[294]} 0 0
   [junit4]   2> 258777 T974 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[295]} 0 0
   [junit4]   2> 258781 T972 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[296]} 0 0
   [junit4]   2> 258783 T971 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[297]} 0 0
   [junit4]   2> 258786 T973 C2605 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[298]} 0 0
   [junit4]   2> ASYNC  NEW_CORE C2606 name=collection1 org.apache.solr.core.SolrCore@13092db
   [junit4]   2> 258792 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[299]} 0 0
   [junit4]   2> 258795 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[300]} 0 1
   [junit4]   2> 258797 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[301]} 0 0
   [junit4]   2> 258799 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[302]} 0 0
   [junit4]   2> 258802 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[303]} 0 0
   [junit4]   2> 258806 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[304]} 0 2
   [junit4]   2> 258809 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[305]} 0 0
   [junit4]   2> 258811 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[306]} 0 0
   [junit4]   2> 258819 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[307]} 0 0
   [junit4]   2> 258822 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[308]} 0 0
   [junit4]   2> 258829 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[309]} 0 1
   [junit4]   2> 258832 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[310]} 0 1
   [junit4]   2> 258841 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[311]} 0 0
   [junit4]   2> 258847 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[312]} 0 1
   [junit4]   2> 258850 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[313]} 0 1
   [junit4]   2> 258853 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[314]} 0 1
   [junit4]   2> 258856 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[315]} 0 0
   [junit4]   2> 258860 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[316]} 0 0
   [junit4]   2> 258863 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[317]} 0 0
   [junit4]   2> 258867 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[318]} 0 1
   [junit4]   2> 258870 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[319]} 0 1
   [junit4]   2> 258873 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[320]} 0 1
   [junit4]   2> 258877 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[321]} 0 0
   [junit4]   2> 258880 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[322]} 0 0
   [junit4]   2> 258884 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[323]} 0 1
   [junit4]   2> 258887 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[324]} 0 0
   [junit4]   2> 258892 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[325]} 0 0
   [junit4]   2> 258895 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[326]} 0 0
   [junit4]   2> 258899 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[327]} 0 1
   [junit4]   2> 258901 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[328]} 0 0
   [junit4]   2> 258904 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[329]} 0 0
   [junit4]   2> 258907 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[330]} 0 1
   [junit4]   2> 258912 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[331]} 0 1
   [junit4]   2> 258915 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[332]} 0 0
   [junit4]   2> 258918 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[333]} 0 1
   [junit4]   2> 258921 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[334]} 0 1
   [junit4]   2> 258923 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[335]} 0 0
   [junit4]   2> 258926 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[336]} 0 0
   [junit4]   2> 258929 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[337]} 0 0
   [junit4]   2> 258932 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[338]} 0 0
   [junit4]   2> 258935 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[339]} 0 0
   [junit4]   2> 258937 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[340]} 0 0
   [junit4]   2> 258940 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[341]} 0 0
   [junit4]   2> 258943 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[342]} 0 1
   [junit4]   2> 258945 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[343]} 0 0
   [junit4]   2> 258948 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[344]} 0 0
   [junit4]   2> 258951 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[345]} 0 0
   [junit4]   2> 258953 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[346]} 0 0
   [junit4]   2> 258956 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[347]} 0 0
   [junit4]   2> 258959 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[348]} 0 0
   [junit4]   2> 258962 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[349]} 0 0
   [junit4]   2> 258964 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[350]} 0 0
   [junit4]   2> 258967 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[351]} 0 0
   [junit4]   2> 258969 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[352]} 0 0
   [junit4]   2> 258972 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[353]} 0 0
   [junit4]   2> 258974 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[354]} 0 0
   [junit4]   2> 258977 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[355]} 0 0
   [junit4]   2> 258979 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[356]} 0 0
   [junit4]   2> 258982 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[357]} 0 1
   [junit4]   2> 258984 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[358]} 0 0
   [junit4]   2> 258987 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[359]} 0 0
   [junit4]   2> 258990 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[360]} 0 1
   [junit4]   2> 258992 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[361]} 0 0
   [junit4]   2> 258995 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[362]} 0 0
   [junit4]   2> 258998 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[363]} 0 1
   [junit4]   2> 259000 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[364]} 0 0
   [junit4]   2> 259003 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[365]} 0 1
   [junit4]   2> 259005 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[366]} 0 0
   [junit4]   2> 259008 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[367]} 0 0
   [junit4]   2> 259011 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[368]} 0 0
   [junit4]   2> 259013 T973 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[369]} 0 0
   [junit4]   2> 259016 T971 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[370]} 0 1
   [junit4]   2> 259018 T972 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[371]} 0 0
   [junit4]   2> 259021 T974 C2606 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr pa

[...truncated too long message...]

gments_2:3 _0(5.0):C1)}
   [junit4]   2> 324588 T1648 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 324588 T965 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 324589 T965 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 324592 T1652 oash.SnapPuller.<init> WARN 'masterUrl' must be specified without the /replication suffix
   [junit4]   2> 324592 T1652 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
   [junit4]   2> ASYNC  NEW_CORE C2764 name=collection1 org.apache.solr.core.SolrCore@1c5abeb
   [junit4]   2> 324595 T1633 C2764 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> ASYNC  NEW_CORE C2765 name=collection1 org.apache.solr.core.SolrCore@1432e78
   [junit4]   2> 324596 T1644 C2765 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={masterUrl=http://127.0.0.1:57812/solr/replication&command=fetchindex&wait=true} status=0 QTime=4
   [junit4]   2> 324597 T965 oas.SolrTestCaseJ4.tearDown ###Ending testNoWriter
   [junit4]   2> 324618 T965 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=22804058
   [junit4]   2> 324619 T965 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c5abeb
   [junit4]   2> 324619 T965 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 324620 T965 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 324620 T965 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 324621 T965 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 324621 T965 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 324622 T965 oasc.CachingDirectoryFactory.close Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 324622 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data;done=false>>]
   [junit4]   2> 324622 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data
   [junit4]   2> 324623 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data/index;done=false>>]
   [junit4]   2> 324623 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404273/master/collection1/data/index
   [junit4]   2> 324624 T965 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 324696 T965 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=28245518
   [junit4]   2> 324697 T965 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1432e78
   [junit4]   2> 324698 T965 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 324699 T965 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 324699 T965 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 324699 T965 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 324700 T965 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 324701 T965 oasc.CachingDirectoryFactory.close Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 324701 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data/index;done=false>>]
   [junit4]   2> 324701 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data/index
   [junit4]   2> 324702 T965 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data;done=false>>]
   [junit4]   2> 324702 T965 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.handler.TestReplicationHandler$SolrInstance-1375535404368/slave/collection1/data
   [junit4]   2> 324703 T965 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 324762 T965 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Lucene42: {newname=Pulsing41(freqCutoff=5 minBlockSize=44 maxBlockSize=182), id=Pulsing41(freqCutoff=5 minBlockSize=44 maxBlockSize=182), name=Pulsing41(freqCutoff=5 minBlockSize=44 maxBlockSize=182)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=it_CH, timezone=Asia/Kamchatka
   [junit4]   2> NOTE: Linux 3.2.0-51-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=31537392,total=121315328
   [junit4]   2> NOTE: All tests run in this JVM: [TestSchemaVersionResource, TestFieldTypeCollectionResource, TestShardHandlerFactory, TestGroupingSearch, TestSolrQueryParser, SynonymTokenizerTest, TestDynamicFieldResource, UnloadDistributedZkTest, BinaryUpdateRequestHandlerTest, TestPhraseSuggestions, TestWriterPerf, DateFieldTest, SchemaVersionSpecificBehaviorTest, TestFastWriter, PreAnalyzedUpdateProcessorTest, AnalysisAfterCoreReloadTest, SpatialFilterTest, TestCharFilters, CSVRequestHandlerTest, CoreAdminHandlerTest, DirectUpdateHandlerOptimizeTest, TestSerializedLuceneMatchVersion, TestBadConfig, TestTrie, TestDefaultSearchFieldResource, TestSearchPerf, LegacyHTMLStripCharFilterTest, DirectSolrConnectionTest, InfoHandlerTest, MBeansHandlerTest, TestFuzzyAnalyzedSuggestions, WordBreakSolrSpellCheckerTest, FileBasedSpellCheckerTest, SolrIndexSplitterTest, DistributedSpellCheckComponentTest, SimpleFacetsTest, TestSort, ExternalFileFieldSortTest, ZkControllerTest, DocValuesTest, QueryElevationComponentTest, SuggesterFSTTest, RecoveryZkTest, ReturnFieldsTest, AssignTest, TestFastLRUCache, BasicFunctionalityTest, SolrCoreTest, TestStressRecovery, TestQuerySenderListener, SliceStateUpdateTest, TestRandomDVFaceting, StressHdfsTest, JsonLoaderTest, TestSchemaNameResource, AliasIntegrationTest, TestSolrDeletionPolicy1, TestReplicationHandler]
   [junit4] Completed on J0 in 67.15s, 14 tests, 1 failure <<< FAILURES!

[...truncated 606 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1347 tests, 1 failure, 37 ignored (6 assumptions)

Total time: 39 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b99 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]