Solr query Slow in Solr 6.1.0

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr query Slow in Solr 6.1.0

vishal patel-2
I am using solr 6.1.0. We have 2 shards and each has one replica. Our index size is very large.
I find out that position of field in query will impact of performance.

If I made below query I got slow response

(doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) )) AND project_id:(2104616) AND is_active:true AND ((isLatest:(true) AND isFolderActive:true AND isXref:false AND -document_type_id:(3 7) AND ((is_public:true OR distribution_list:11111 OR folderadmin_list:11111 OR author_user_id:11111) AND (((allowedUsers:(11111) OR allowedRoles:(2222 6440215 6368478) OR combinationUsers:(11111)) AND -blockedUsers:(11111)) OR (defaultAccess:(true) AND -blockedUsers:(11111) AND -blockedRoles:(2222 6440215 6368478))))) OR (isLatestRevPrivate:(true) AND allowedUsersForPvtRev:(11111) AND -folderadmin_list:(11111)))

If I changed (doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) )) part in last then got fast response compare to above.

project_id:(2104616) AND is_active:true AND ((isLatest:(true) AND isFolderActive:true AND isXref:false AND -document_type_id:(3 7) AND ((is_public:true OR distribution_list:11111 OR folderadmin_list:11111 OR author_user_id:11111) AND (((allowedUsers:(11111) OR allowedRoles:(2222 6440215 6368478) OR combinationUsers:(11111)) AND -blockedUsers:(11111)) OR (defaultAccess:(true) AND -blockedUsers:(11111) AND -blockedRoles:(2222 6440215 6368478))))) OR (isLatestRevPrivate:(true) AND allowedUsersForPvtRev:(11111) AND -folderadmin_list:(11111))) AND (doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) ))

Is it possible? How does Solr execute this query? field sequence is matter for performance?
I want to know the step by step Solr query execution same like database query because I will arrange field for better performance.

Regards,
Vishal

Sent from Outlook<http://aka.ms/weboutlook>
Reply | Threaded
Open this post in threaded view
|

Re: Solr query Slow in Solr 6.1.0

Erick Erickson
First of all, if you’re really using pre-and-postfix wildcards and those asterisks are not just bold formatting, those are very expensive operations. I’d suggest you investigate alternatives (like ngramming) or other alternate ways of analyzing your input (both at indexing and query time) before trying to understand the internals of query processing. If for no other reason than there’s no guarantee that the internal processing will remain the same between versions of Solr.

Second, I suspect that you’re seeing spurious speedups. There’s a lot of work that has to be done to gather terms like *cdrl*, so you may be getting caching. Unless you run a rigorous test (i.e. use many different queries with similar form), your timing is unreliable.

Third, try adding &debug=query to some of these and looking at the output, that’ll give you some clue how the queries are parsed.

Best,
Erick

> On Mar 19, 2020, at 9:08 AM, vishal patel <[hidden email]> wrote:
>
> I am using solr 6.1.0. We have 2 shards and each has one replica. Our index size is very large.
> I find out that position of field in query will impact of performance.
>
> If I made below query I got slow response
>
> (doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) )) AND project_id:(2104616) AND is_active:true AND ((isLatest:(true) AND isFolderActive:true AND isXref:false AND -document_type_id:(3 7) AND ((is_public:true OR distribution_list:11111 OR folderadmin_list:11111 OR author_user_id:11111) AND (((allowedUsers:(11111) OR allowedRoles:(2222 6440215 6368478) OR combinationUsers:(11111)) AND -blockedUsers:(11111)) OR (defaultAccess:(true) AND -blockedUsers:(11111) AND -blockedRoles:(2222 6440215 6368478))))) OR (isLatestRevPrivate:(true) AND allowedUsersForPvtRev:(11111) AND -folderadmin_list:(11111)))
>
> If I changed (doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) )) part in last then got fast response compare to above.
>
> project_id:(2104616) AND is_active:true AND ((isLatest:(true) AND isFolderActive:true AND isXref:false AND -document_type_id:(3 7) AND ((is_public:true OR distribution_list:11111 OR folderadmin_list:11111 OR author_user_id:11111) AND (((allowedUsers:(11111) OR allowedRoles:(2222 6440215 6368478) OR combinationUsers:(11111)) AND -blockedUsers:(11111)) OR (defaultAccess:(true) AND -blockedUsers:(11111) AND -blockedRoles:(2222 6440215 6368478))))) OR (isLatestRevPrivate:(true) AND allowedUsersForPvtRev:(11111) AND -folderadmin_list:(11111))) AND (doc_ref:((*KON\-N2*) )) AND (title:((*cdrl*) ))
>
> Is it possible? How does Solr execute this query? field sequence is matter for performance?
> I want to know the step by step Solr query execution same like database query because I will arrange field for better performance.
>
> Regards,
> Vishal
>
> Sent from Outlook<http://aka.ms/weboutlook>