Search Analytics Help

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Search Analytics Help

ennio
Hello,

I'm setting up SOLR on an internal website for my company and I would like to know if anyone can recommend an analytics that I can see what the users are searching for? Does the log in SOLR give me that information?

Thank you,
Ennio Bozzetti

Reply | Threaded
Open this post in threaded view
|

RE: Search Analytics Help

Hanjan, Harinder
This seems promising

https://github.com/lucidworks/banana


-----Original Message-----
From: Ennio Bozzetti [mailto:[hidden email]]
Sent: Thursday, April 26, 2018 1:39 PM
To: [hidden email]
Subject: [EXT] Search Analytics Help

Hello,

I'm setting up SOLR on an internal website for my company and I would like to know if anyone can recommend an analytics that I can see what the users are searching for? Does the log in SOLR give me that information?

Thank you,
Ennio Bozzetti


________________________________
NOTICE -
This communication is intended ONLY for the use of the person or entity named above and may contain information that is confidential or legally privileged. If you are not the intended recipient named above or a person responsible for delivering messages or communications to the intended recipient, YOU ARE HEREBY NOTIFIED that any use, distribution, or copying of this communication or any of the information contained in it is strictly prohibited. If you have received this communication in error, please notify us immediately by telephone and then destroy or delete this communication, or return it to us by mail if requested by us. The City of Calgary thanks you for your attention and co-operation.
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Doug Turnbull
In reply to this post by ennio
Honestly I haven’t seen anything satisfactory (yet). It’s a huge need in
the open source community

On Thu, Apr 26, 2018 at 3:38 PM Ennio Bozzetti <[hidden email]>
wrote:

> Hello,
>
> I'm setting up SOLR on an internal website for my company and I would like
> to know if anyone can recommend an analytics that I can see what the users
> are searching for? Does the log in SOLR give me that information?
>
> Thank you,
> Ennio Bozzetti
>
> --
CTO, OpenSource Connections
Author, Relevant Search
http://o19s.com/doug
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Michal Hlavac-2
Hi,

you have plenty options. Without any special effort there is ELK. Parse solr logs with logstash, feed elasticsearch with data, then analyze in kibana.

Another option is to send every relevant search request to kafka, then you can do more sophisticated data analytic using kafka-stream API. Then use ELK to feed elasticsearch with logstash kafka input plugin. For this scenario you need to do some programming. I`ve already created this component but I hadn't time to publish it.

Another option is use only logstash to feed e.g. graphite database and show results with grafana or combine all these options.

You can also monitor SOLR instances by JMX logstash input plugin.

Really don't understand what do you mean by saying that there is nothing satisfactory.

m.

On štvrtok, 26. apríla 2018 22:23:30 CEST Doug Turnbull wrote:

> Honestly I haven’t seen anything satisfactory (yet). It’s a huge need in
> the open source community
>
> On Thu, Apr 26, 2018 at 3:38 PM Ennio Bozzetti <[hidden email]>
> wrote:
>
> > Hello,
> >
> > I'm setting up SOLR on an internal website for my company and I would like
> > to know if anyone can recommend an analytics that I can see what the users
> > are searching for? Does the log in SOLR give me that information?
> >
> > Thank you,
> > Ennio Bozzetti
> >
> > --
> CTO, OpenSource Connections
> Author, Relevant Search
> http://o19s.com/doug


Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Alessandro Benedetti
Michal,
Doug was referring to an open source solution ready out of the box and just
pluggable ( a sort of plug and play).
Of course you can implement your own solution and using ELK or kafka is
absolutely a valid option.

Cheers


--------------------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
www.sease.io

On Fri, Apr 27, 2018 at 10:21 AM, Michal Hlavac <[hidden email]> wrote:

> Hi,
>
> you have plenty options. Without any special effort there is ELK. Parse
> solr logs with logstash, feed elasticsearch with data, then analyze in
> kibana.
>
> Another option is to send every relevant search request to kafka, then you
> can do more sophisticated data analytic using kafka-stream API. Then use
> ELK to feed elasticsearch with logstash kafka input plugin. For this
> scenario you need to do some programming. I`ve already created this
> component but I hadn't time to publish it.
>
> Another option is use only logstash to feed e.g. graphite database and
> show results with grafana or combine all these options.
>
> You can also monitor SOLR instances by JMX logstash input plugin.
>
> Really don't understand what do you mean by saying that there is nothing
> satisfactory.
>
> m.
>
> On štvrtok, 26. apríla 2018 22:23:30 CEST Doug Turnbull wrote:
> > Honestly I haven’t seen anything satisfactory (yet). It’s a huge need in
> > the open source community
> >
> > On Thu, Apr 26, 2018 at 3:38 PM Ennio Bozzetti <[hidden email]>
> > wrote:
> >
> > > Hello,
> > >
> > > I'm setting up SOLR on an internal website for my company and I would
> like
> > > to know if anyone can recommend an analytics that I can see what the
> users
> > > are searching for? Does the log in SOLR give me that information?
> > >
> > > Thank you,
> > > Ennio Bozzetti
> > >
> > > --
> > CTO, OpenSource Connections
> > Author, Relevant Search
> > http://o19s.com/doug
>
>
>
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Doug Turnbull
Exactly Alessandro -

I can totally build something, but there's not a good open source solution
solution for:

- Gathering queries / user / session metadata at search time from your app
- Gathering the returned result set and their display posn (just doc ids
would be fine)
- Gathering the clicks/conversions/reformulations/etc that result from that
search

Then with that data

- Show search relevance performance in dashboards to business stakeholders
- Allow computation over this data to create training sets for LTR, etc

I see companies reinvent this wheel over and over and over...  Surprised
Elastic hasn't built 'SearchBeat' that does all this :)

-Doug



On Fri, Apr 27, 2018 at 9:47 AM Alessandro Benedetti <[hidden email]>
wrote:

> Michal,
> Doug was referring to an open source solution ready out of the box and just
> pluggable ( a sort of plug and play).
> Of course you can implement your own solution and using ELK or kafka is
> absolutely a valid option.
>
> Cheers
>
>
> --------------------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> www.sease.io
>
> On Fri, Apr 27, 2018 at 10:21 AM, Michal Hlavac <[hidden email]> wrote:
>
> > Hi,
> >
> > you have plenty options. Without any special effort there is ELK. Parse
> > solr logs with logstash, feed elasticsearch with data, then analyze in
> > kibana.
> >
> > Another option is to send every relevant search request to kafka, then
> you
> > can do more sophisticated data analytic using kafka-stream API. Then use
> > ELK to feed elasticsearch with logstash kafka input plugin. For this
> > scenario you need to do some programming. I`ve already created this
> > component but I hadn't time to publish it.
> >
> > Another option is use only logstash to feed e.g. graphite database and
> > show results with grafana or combine all these options.
> >
> > You can also monitor SOLR instances by JMX logstash input plugin.
> >
> > Really don't understand what do you mean by saying that there is nothing
> > satisfactory.
> >
> > m.
> >
> > On štvrtok, 26. apríla 2018 22:23:30 CEST Doug Turnbull wrote:
> > > Honestly I haven’t seen anything satisfactory (yet). It’s a huge need
> in
> > > the open source community
> > >
> > > On Thu, Apr 26, 2018 at 3:38 PM Ennio Bozzetti <[hidden email]
> >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I'm setting up SOLR on an internal website for my company and I would
> > like
> > > > to know if anyone can recommend an analytics that I can see what the
> > users
> > > > are searching for? Does the log in SOLR give me that information?
> > > >
> > > > Thank you,
> > > > Ennio Bozzetti
> > > >
> > > > --
> > > CTO, OpenSource Connections
> > > Author, Relevant Search
> > > http://o19s.com/doug
> >
> >
> >
>
--
CTO, OpenSource Connections
Author, Relevant Search
http://o19s.com/doug
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Walter Underwood
In the past, I’ve recommended seaurchin.io, a great tool. But, they were acquired by Algolia and the service will be shut down this month.

As far as I know, there is nothing close to SeaUrchin.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Apr 27, 2018, at 6:54 AM, Doug Turnbull <[hidden email]> wrote:
>
> Exactly Alessandro -
>
> I can totally build something, but there's not a good open source solution
> solution for:
>
> - Gathering queries / user / session metadata at search time from your app
> - Gathering the returned result set and their display posn (just doc ids
> would be fine)
> - Gathering the clicks/conversions/reformulations/etc that result from that
> search
>
> Then with that data
>
> - Show search relevance performance in dashboards to business stakeholders
> - Allow computation over this data to create training sets for LTR, etc
>
> I see companies reinvent this wheel over and over and over...  Surprised
> Elastic hasn't built 'SearchBeat' that does all this :)
>
> -Doug
>
>
>
> On Fri, Apr 27, 2018 at 9:47 AM Alessandro Benedetti <[hidden email]>
> wrote:
>
>> Michal,
>> Doug was referring to an open source solution ready out of the box and just
>> pluggable ( a sort of plug and play).
>> Of course you can implement your own solution and using ELK or kafka is
>> absolutely a valid option.
>>
>> Cheers
>>
>>
>> --------------------------
>> Alessandro Benedetti
>> Search Consultant, R&D Software Engineer, Director
>> www.sease.io
>>
>> On Fri, Apr 27, 2018 at 10:21 AM, Michal Hlavac <[hidden email]> wrote:
>>
>>> Hi,
>>>
>>> you have plenty options. Without any special effort there is ELK. Parse
>>> solr logs with logstash, feed elasticsearch with data, then analyze in
>>> kibana.
>>>
>>> Another option is to send every relevant search request to kafka, then
>> you
>>> can do more sophisticated data analytic using kafka-stream API. Then use
>>> ELK to feed elasticsearch with logstash kafka input plugin. For this
>>> scenario you need to do some programming. I`ve already created this
>>> component but I hadn't time to publish it.
>>>
>>> Another option is use only logstash to feed e.g. graphite database and
>>> show results with grafana or combine all these options.
>>>
>>> You can also monitor SOLR instances by JMX logstash input plugin.
>>>
>>> Really don't understand what do you mean by saying that there is nothing
>>> satisfactory.
>>>
>>> m.
>>>
>>> On štvrtok, 26. apríla 2018 22:23:30 CEST Doug Turnbull wrote:
>>>> Honestly I haven’t seen anything satisfactory (yet). It’s a huge need
>> in
>>>> the open source community
>>>>
>>>> On Thu, Apr 26, 2018 at 3:38 PM Ennio Bozzetti <[hidden email]
>>>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm setting up SOLR on an internal website for my company and I would
>>> like
>>>>> to know if anyone can recommend an analytics that I can see what the
>>> users
>>>>> are searching for? Does the log in SOLR give me that information?
>>>>>
>>>>> Thank you,
>>>>> Ennio Bozzetti
>>>>>
>>>>> --
>>>> CTO, OpenSource Connections
>>>> Author, Relevant Search
>>>> http://o19s.com/doug
>>>
>>>
>>>
>>
> --
> CTO, OpenSource Connections
> Author, Relevant Search
> http://o19s.com/doug

Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

ennio
In reply to this post by Michal Hlavac-2
Thanks all for the comments. I'm looking at the ELK option here.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

Sameer Maggon-4
Ennio,

Have you taken a look at SearchStax Analytics?

https://www.searchstax.com/docs/search-analytics-start/

Thanks,




On Wed, May 23, 2018 at 11:34 AM, ennio <[hidden email]> wrote:

> Thanks all for the comments. I'm looking at the ELK option here.
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>



--
Sameer Maggon
https://www.searchstax.com
Reply | Threaded
Open this post in threaded view
|

Re: Search Analytics Help

ennio
I have not. I will take a look at it.

Thanks for sharing the link.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html