Poor Solr Cloud Query Performance against a Small Dataset

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Poor Solr Cloud Query Performance against a Small Dataset

Dave Seltzer
Hello!

I'm trying to utilize Solr Cloud to help with a hash search problem. The
record set has only 4,300 documents.

When I run my search against a single core I get results on the order of
10ms. When I run the same search against Solr Cloud results take about
5,000 ms.

Is there something about this particular query which makes it perform
poorly in a Cloud environment? The query looks like this (linebreaks added
for readability):

{!frange+l%3D5+u%3D25}sum(
    termfreq(hashTable_0,'225706351'),
    termfreq(hashTable_1,'17664000'),
    termfreq(hashTable_2,'86447642'),
    termfreq(hashTable_3,'134816033'),
    termfreq(hashTable_4,'1061820218'),
    termfreq(hashTable_5,'543627850'),
    termfreq(hashTable_6,'-1828379348'),
    termfreq(hashTable_7,'423236759'),
    termfreq(hashTable_8,'522192943'),
    termfreq(hashTable_9,'572537937'),
    termfreq(hashTable_10,'286991887'),
    termfreq(hashTable_11,'789711386'),
    termfreq(hashTable_12,'235801909'),
    termfreq(hashTable_13,'67109911'),
    termfreq(hashTable_14,'609628285'),
    termfreq(hashTable_15,'1796472850'),
    termfreq(hashTable_16,'202312085'),
    termfreq(hashTable_17,'306200840'),
    termfreq(hashTable_18,'85657669'),
    termfreq(hashTable_19,'671548727'),
    termfreq(hashTable_20,'71309060'),
    termfreq(hashTable_21,'1125848323'),
    termfreq(hashTable_22,'1077548043'),
    termfreq(hashTable_23,'117638159'),
    termfreq(hashTable_24,'-1408039642'))

The schema looks like this:

   <field name="subFingerprintId" type="string" indexed="true"
stored="true" required="true" multiValued="false"
omitTermFreqAndPositions="true" omitNorms="true" />
   <field name="trackId" type="string" indexed="true" stored="true"
required="true" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
   <dynamicField name="hashTable_*" type="long" indexed="true"
stored="true" omitTermFreqAndPositions="true" omitNorms="true" />
   <field name="sequenceAt" type="double" indexed="false" stored="true"
required="false" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
   <field name="sequenceNumber" type="int" indexed="false" stored="true"
required="false" multiValued="false" omitTermFreqAndPositions="true"
omitNorms="true" />
   <uniqueKey>subFingerprintId</uniqueKey>

I've included some sample output below. I wasn't sure if this was a matter
of changing the routing key in the collections system, or if this is a more
fundamental problem with the way Term Frequencies are counted in a Solr
Cloud environment.

Many thanks!

-Dave

---------------------- Single Core Example Query:
{
  "responseHeader":{
    "status":0,
    "QTime":13,
    "params":{
      "q":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
      "indent":"on",
      "wt":"json",
      "debugQuery":"on",
      "_":"1478024378680"}},
  "response":{"numFound":1,"start":0,"docs":[
      {
        "subFingerprintId":"f6c9093e-e8e9-4c0f-aa2a-387b46e7ef2a",
        "trackId":"5207095a-0126-4c41-8787-16d41165158a",
        "sequenceNumber":136,
        "sequenceAt":12.5399129172714,
        "hashTable_0":354749018,
        "hashTable_1":287779841,
        "hashTable_2":1797994010,
        "hashTable_3":151854851,
        "hashTable_4":375260422,
        "hashTable_5":441911360,
        "hashTable_6":68120837,
        "hashTable_7":420158535,
        "hashTable_8":16979004,
        "hashTable_9":1443304209,
        "hashTable_10":1732468239,
        "hashTable_11":455215642,
        "hashTable_12":135990559,
        "hashTable_13":123093271,
        "hashTable_14":1444029969,
        "hashTable_15":788988675,
        "hashTable_16":53480196,
        "hashTable_17":488255035,
        "hashTable_18":505809973,
        "hashTable_19":201814293,
        "hashTable_20":70208520,
        "hashTable_21":805503541,
        "hashTable_22":658713904,
        "hashTable_23":302387775,
        "hashTable_24":194970957,
        "_version_":1549818240561053696}]
  },
  "debug":{
    "rawquerystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
    "querystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",

"parsedquery":"FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]))",

"parsedquery_toString":"ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25])",
    "explain":{
      "f6c9093e-e8e9-4c0f-aa2a-387b46e7ef2a":"\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n  1.0 = boost\n  1.0 = queryNorm\n"},
    "QParser":"",
    "timing":{
      "time":13.0,
      "prepare":{
        "time":0.0,
        "query":{
          "time":0.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "debug":{
          "time":0.0}},
      "process":{
        "time":12.0,
        "query":{
          "time":10.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "debug":{
          "time":2.0}}}}}


---------------------- Solr Cloud Example Query:
{
  "responseHeader":{
    "zkConnected":true,
    "status":0,
    "QTime":5023,
    "params":{
      "q":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
      "indent":"on",
      "wt":"json",
      "debugQuery":"on",
      "_":"1478024068169"}},
  "response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[
      {
        "subFingerprintId":"98839fbe-7986-440d-89aa-559a24219d87",
        "trackId":"3652cc99-767e-4210-b242-d49730957c31",
        "sequenceNumber":136,
        "sequenceAt":12.5399129172714,
        "hashTable_0":354749018,
        "hashTable_1":287779841,
        "hashTable_2":1797994010,
        "hashTable_3":151854851,
        "hashTable_4":375260422,
        "hashTable_5":441911360,
        "hashTable_6":68120837,
        "hashTable_7":420158535,
        "hashTable_8":16979004,
        "hashTable_9":1443304209,
        "hashTable_10":1732468239,
        "hashTable_11":455215642,
        "hashTable_12":135990559,
        "hashTable_13":123093271,
        "hashTable_14":1444029969,
        "hashTable_15":788988675,
        "hashTable_16":53480196,
        "hashTable_17":488255035,
        "hashTable_18":505809973,
        "hashTable_19":201814293,
        "hashTable_20":70208520,
        "hashTable_21":805503541,
        "hashTable_22":658713904,
        "hashTable_23":302387775,
        "hashTable_24":194970957,
        "_version_":1549806667971952640}]
  },
  "debug":{
    "track":{
      "rid":"crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3",
      "EXECUTE_QUERY":{
        "
http://crompcore02:8983/solr/sf_fingerprints_shard1_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard1_replica1/
":{
          "QTime":"4",
          "ElapsedTime":"5012",
          "RequestPurpose":"GET_TOP_IDS",
          "NumFound":"0",

"Response":"{responseHeader={zkConnected=true,status=0,QTime=4,params={df=text,distrib=false,debug=[false,
timing, track],fl=[subFingerprintId,
score],shards.purpose=4,start=0,fsv=true,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard1_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard1_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_TOP_IDS,NOW=1478024295225,isShard=true,wt=javabin,debugQuery=false,_=1478024068169}},response={numFound=0,start=0,maxScore=0.0,docs=[]},sort_values={},debug={timing={time=4.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=3.0,query={time=3.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}}}}}"},
        "
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/
":{
          "QTime":"4",
          "ElapsedTime":"5013",
          "RequestPurpose":"GET_TOP_IDS",
          "NumFound":"1",

"Response":"{responseHeader={zkConnected=true,status=0,QTime=4,params={df=text,distrib=false,debug=[false,
timing, track],fl=[subFingerprintId,
score],shards.purpose=4,start=0,fsv=true,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_TOP_IDS,NOW=1478024295225,isShard=true,wt=javabin,debugQuery=false,_=1478024068169}},response={numFound=1,start=0,maxScore=1.0,docs=[SolrDocument{subFingerprintId=98839fbe-7986-440d-89aa-559a24219d87,
score=1.0}]},sort_values={},debug={timing={time=4.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=3.0,query={time=3.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}}}}}"}},
      "GET_FIELDS":{
        "
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/
":{
          "QTime":"2",
          "ElapsedTime":"6",
          "RequestPurpose":"GET_FIELDS,GET_DEBUG",
          "NumFound":"1",

"Response":"{responseHeader={zkConnected=true,status=0,QTime=2,params={df=text,distrib=false,debug=[timing,
track],shards.purpose=320,shard.url=
http://crompcore02:8983/solr/sf_fingerprints_shard2_replica2/|http://crompcore01:8983/solr/sf_fingerprints_shard2_replica1/,rows=10,rid=crompcore01-sf_fingerprints_shard2_replica1-1478024295225-3,version=2,q={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),requestPurpose=GET_FIELDS,GET_DEBUG,NOW=1478024295225,ids=98839fbe-7986-440d-89aa-559a24219d87,isShard=true,wt=javabin,debugQuery=true,_=1478024068169}},response={numFound=1,start=0,docs=[SolrDocument{subFingerprintId=98839fbe-7986-440d-89aa-559a24219d87,
trackId=3652cc99-767e-4210-b242-d49730957c31, sequenceNumber=136,
sequenceAt=12.5399129172714, hashTable_0=354749018, hashTable_1=287779841,
hashTable_2=1797994010, hashTable_3=151854851, hashTable_4=375260422,
hashTable_5=441911360, hashTable_6=68120837, hashTable_7=420158535,
hashTable_8=16979004, hashTable_9=1443304209, hashTable_10=1732468239,
hashTable_11=455215642, hashTable_12=135990559, hashTable_13=123093271,
hashTable_14=1444029969, hashTable_15=788988675, hashTable_16=53480196,
hashTable_17=488255035, hashTable_18=505809973, hashTable_19=201814293,
hashTable_20=70208520, hashTable_21=805503541, hashTable_22=658713904,
hashTable_23=302387775, hashTable_24=194970957,
_version_=1549806667971952640}]},debug={rawquerystring={!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),querystring={!frange
l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957')),parsedquery=FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO
25])),parsedquery_toString=ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]),explain={98839fbe-7986-440d-89aa-559a24219d87=\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n  1.0 = boost\n  1.0 =
queryNorm\n},QParser=,timing={time=2.0,prepare={time=0.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=0.0}},process={time=1.0,query={time=0.0},facet={time=0.0},facet_module={time=0.0},mlt={time=0.0},highlight={time=0.0},stats={time=0.0},expand={time=0.0},terms={time=0.0},debug={time=1.0}}}}}"}}},
    "timing":{
      "time":10.0,
      "prepare":{
        "time":0.0,
        "query":{
          "time":0.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "debug":{
          "time":0.0}},
      "process":{
        "time":7.0,
        "query":{
          "time":6.0},
        "facet":{
          "time":0.0},
        "facet_module":{
          "time":0.0},
        "mlt":{
          "time":0.0},
        "highlight":{
          "time":0.0},
        "stats":{
          "time":0.0},
        "expand":{
          "time":0.0},
        "terms":{
          "time":0.0},
        "debug":{
          "time":1.0}}},
    "rawquerystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",
    "querystring":"{!frange l=5
u=25}sum(termfreq(hashTable_0,'354749018'),termfreq(hashTable_1,'286534657'),termfreq(hashTable_2,'1798007322'),termfreq(hashTable_3,'151854851'),termfreq(hashTable_4,'142869766'),termfreq(hashTable_5,'240584768'),termfreq(hashTable_6,'68120837'),termfreq(hashTable_7,'134945863'),termfreq(hashTable_8,'688067644'),termfreq(hashTable_9,'621220625'),termfreq(hashTable_10,'1732446991'),termfreq(hashTable_11,'505547282'),termfreq(hashTable_12,'135990559'),termfreq(hashTable_13,'123097623'),termfreq(hashTable_14,'454174225'),termfreq(hashTable_15,'788988675'),termfreq(hashTable_16,'53480196'),termfreq(hashTable_17,'487550779'),termfreq(hashTable_18,'455477045'),termfreq(hashTable_19,'1141310997'),termfreq(hashTable_20,'71322652'),termfreq(hashTable_21,'805503533'),termfreq(hashTable_22,'656158000'),termfreq(hashTable_23,'302410303'),termfreq(hashTable_24,'194970957'))",

"parsedquery":"FunctionRangeQuery(ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]))",

"parsedquery_toString":"ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25])",
    "QParser":"",
    "explain":{
      "98839fbe-7986-440d-89aa-559a24219d87":"\n1.0 =
ConstantScore(frange(sum(termfreq(hashTable_0,354749018),termfreq(hashTable_1,286534657),termfreq(hashTable_2,1798007322),termfreq(hashTable_3,151854851),termfreq(hashTable_4,142869766),termfreq(hashTable_5,240584768),termfreq(hashTable_6,68120837),termfreq(hashTable_7,134945863),termfreq(hashTable_8,688067644),termfreq(hashTable_9,621220625),termfreq(hashTable_10,1732446991),termfreq(hashTable_11,505547282),termfreq(hashTable_12,135990559),termfreq(hashTable_13,123097623),termfreq(hashTable_14,454174225),termfreq(hashTable_15,788988675),termfreq(hashTable_16,53480196),termfreq(hashTable_17,487550779),termfreq(hashTable_18,455477045),termfreq(hashTable_19,1141310997),termfreq(hashTable_20,71322652),termfreq(hashTable_21,805503533),termfreq(hashTable_22,656158000),termfreq(hashTable_23,302410303),termfreq(hashTable_24,194970957))):[5
TO 25]), product of:\n  1.0 = boost\n  1.0 = queryNorm\n"}}}
Reply | Threaded
Open this post in threaded view
|

Re: Poor Solr Cloud Query Performance against a Small Dataset

Rick Leir-2
Here is a wild guess. Whenever I see a 5 second delay in networking, I
think DNS timeouts. YMMV, good luck.

cheers -- Rick

On 2016-11-01 04:18 PM, Dave Seltzer wrote:

> Hello!
>
> I'm trying to utilize Solr Cloud to help with a hash search problem. The
> record set has only 4,300 documents.
>
> When I run my search against a single core I get results on the order of
> 10ms. When I run the same search against Solr Cloud results take about
> 5,000 ms.
>
> Is there something about this particular query which makes it perform
> poorly in a Cloud environment? The query looks like this (linebreaks added
> for readability):
>
> {!frange+l%3D5+u%3D25}sum(
>      termfreq(hashTable_0,'225706351'),
>      termfreq(hashTable_1,'17664000'),
>      termfreq(hashTable_2,'86447642'),
>      termfreq(hashTable_3,'134816033'),

Reply | Threaded
Open this post in threaded view
|

Re: Poor Solr Cloud Query Performance against a Small Dataset

Dave Seltzer
Good tip Rick,

I'll dig in and make sure everything is set up correctly.

Thanks!

-D

Dave Seltzer <[hidden email]>
Chief Systems Architect
TVEyes
(203) 254-3600 x222

On Wed, Nov 2, 2016 at 9:05 PM, Rick Leir <[hidden email]> wrote:

> Here is a wild guess. Whenever I see a 5 second delay in networking, I
> think DNS timeouts. YMMV, good luck.
>
> cheers -- Rick
>
> On 2016-11-01 04:18 PM, Dave Seltzer wrote:
>
>> Hello!
>>
>> I'm trying to utilize Solr Cloud to help with a hash search problem. The
>> record set has only 4,300 documents.
>>
>> When I run my search against a single core I get results on the order of
>> 10ms. When I run the same search against Solr Cloud results take about
>> 5,000 ms.
>>
>> Is there something about this particular query which makes it perform
>> poorly in a Cloud environment? The query looks like this (linebreaks added
>> for readability):
>>
>> {!frange+l%3D5+u%3D25}sum(
>>      termfreq(hashTable_0,'225706351'),
>>      termfreq(hashTable_1,'17664000'),
>>      termfreq(hashTable_2,'86447642'),
>>      termfreq(hashTable_3,'134816033'),
>>
>
>