Mobile phone shop + Solr

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Mobile phone shop + Solr

climbingrose
Hi all,

I've been watching the development of Solr in the last few months. I start
building a mobile phone shop with faceted browsing to allow users to filter
the catalogue in a friendly manner. Since the site will probably expand in a
year or two, I need some advice regarding the design and implementation of
the system.

At the moment, I have a sql table holding mobile phone information (id,
brand, name, size, screen, weight...). The total number of fields are around
41. I want to be able to facetedly browse the catalogue based on, for
example, the weight of the phone (100-150g, >150g) or color (red, black...).
Another feature of the system is that it allows users to sell their own
mobile phones on the website. For example, if they browse Nokia 6600 page,
there'll be a link for them to sell their own 6600 phones. The user can then
provide some more information about the phone he/she is selling such as
condition, description... Obviously, I need to publish the mobile phone
catalogue to  Solr. How about the information that users submit when they
sell the phone? Should I publish them to Solr as well?

Can Solr do facetted browsing based on content of a field? For example, our
mobile phone table has a field named color, which might look like this:

red
chocolate
red
black
red

Can Solr produce something like:

red(3)
chocolate(1)
black(1)


Because the website is currently being developed in Java, I'm a little bit
worried about the use of curl. I want to be able to programmatically
submit/delete document from Java code rather than through command line. I
read JavaSolr page but it seems that the code isn't stable enough for
production. Can anyone who has successfully developed a Java client for Solr
give me some suggestion regarding this matter?

Thanks.

--
Regards,

Cuong Hoang
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

Yonik Seeley-2
On 9/12/06, climbingrose <[hidden email]> wrote:
> Obviously, I need to publish the mobile phone
> catalogue to  Solr. How about the information that users submit when they
> sell the phone? Should I publish them to Solr as well?

It can often make a system simpler if the web front-end only has one
data source to worry about.

> Can Solr produce something like:
>
> red(3)
> chocolate(1)
> black(1)

Yes, it was always able to (with custom Java code), but it just got a
lot easier for simple things like this.  It's now supported
out-of-the-box:
http://wiki.apache.org/solr/SimpleFacetParameters

> Because the website is currently being developed in Java, I'm a little bit
> worried about the use of curl.

Curl is just a tool that talks HTTP.. it's not "part" of Solr at all -
just a convenient way of testing things.

> I want to be able to programmatically
> submit/delete document from Java code rather than through command line. I
> read JavaSolr page but it seems that the code isn't stable enough for
> production.

CNET's in-house client was too tied up with other stuff to cleanly
separate out and open-source.  As you can see, an open source client
is being worked on: http://issues.apache.org/jira/browse/SOLR-20


-Yonik
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

climbingrose
Because the mobile phone info has many fields (>40), I don't want to
repeatedly submit it to Solr.

On 9/13/06, Yonik Seeley <[hidden email]> wrote:

>
> On 9/12/06, climbingrose <[hidden email]> wrote:
> > Obviously, I need to publish the mobile phone
> > catalogue to  Solr. How about the information that users submit when
> they
> > sell the phone? Should I publish them to Solr as well?
>
> It can often make a system simpler if the web front-end only has one
> data source to worry about.
>
> > Can Solr produce something like:
> >
> > red(3)
> > chocolate(1)
> > black(1)
>
> Yes, it was always able to (with custom Java code), but it just got a
> lot easier for simple things like this.  It's now supported
> out-of-the-box:
> http://wiki.apache.org/solr/SimpleFacetParameters
>
> > Because the website is currently being developed in Java, I'm a little
> bit
> > worried about the use of curl.
>
> Curl is just a tool that talks HTTP.. it's not "part" of Solr at all -
> just a convenient way of testing things.
>
> > I want to be able to programmatically
> > submit/delete document from Java code rather than through command line.
> I
> > read JavaSolr page but it seems that the code isn't stable enough for
> > production.
>
> CNET's in-house client was too tied up with other stuff to cleanly
> separate out and open-source.  As you can see, an open source client
> is being worked on: http://issues.apache.org/jira/browse/SOLR-20
>
>
> -Yonik
>



--
Regards,

Cuong Hoang
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

Chris Hostetter-3

: Because the mobile phone info has many fields (>40), I don't want to
: repeatedly submit it to Solr.

i'm not really sure what you mean by "repeatedly submit to Solr" or how it
relates to haveing more then 40 fields.  40 fields really isn't that many.

To give you a basis of comparison: the last Solr index i built from
scratch had 47 <field> declarations, and 4 <dynamicField> declarations
...those 4 dynamic fields result in approximately 1200 'fields' in the
index -- not every document has a value for every field, but the average
is above 200 fields per document.



-Hoss

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

jrodenburg
In reply to this post by Yonik Seeley-2
On 9/12/06, Yonik Seeley <[hidden email]> wrote:

>
> On 9/12/06, climbingrose <[hidden email]> wrote:
> > I want to be able to programmatically
> > submit/delete document from Java code rather than through command line.
> I
> > read JavaSolr page but it seems that the code isn't stable enough for
> > production.
>
> CNET's in-house client was too tied up with other stuff to cleanly
> separate out and open-source.  As you can see, an open source client
> is being worked on: http://issues.apache.org/jira/browse/SOLR-20
>
> -Yonik
>


It doesn't meet the Java requirement (it's in C#, so it's close) but we have
an internal tool that handles individual updates to Solr.  It's called Flare
and is a command-line app that updates a Solr index.

-- j
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

climbingrose
In reply to this post by Chris Hostetter-3
I probably need to visualise my models:

MobileInfo (1)------------------------------------(1...*) SellingItem

MobileInfo has many fields to describe the characteristics of a mobile phone
model (color, size..). SellingItem is an "instance" of MobileInfo that is
currently sold by a user. So in the ERD terms, SellingItem will probably
have foreign key call MobileInfoId that references the primary key of
MobileInfo. Now obviously, I need to index MobileInfo to support faceted
browsing. How should I index SellingItem? The simplest way probably is to
combile mobile phone specs in MobileInfo and and fields in SellingItem, and
then index all of them. In this case, if I have 1000 SellingItems
referencing a particular MobileInfo, I have to repeat the fields in
MobileInfo a thousand times.

On 9/13/06, Chris Hostetter <[hidden email]> wrote:

>
>
> : Because the mobile phone info has many fields (>40), I don't want to
> : repeatedly submit it to Solr.
>
> i'm not really sure what you mean by "repeatedly submit to Solr" or how it
> relates to haveing more then 40 fields.  40 fields really isn't that many.
>
> To give you a basis of comparison: the last Solr index i built from
> scratch had 47 <field> declarations, and 4 <dynamicField> declarations
> ...those 4 dynamic fields result in approximately 1200 'fields' in the
> index -- not every document has a value for every field, but the average
> is above 200 fields per document.
>
>
>
> -Hoss
>
>


--
Regards,

Cuong Hoang
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Mobile phone shop + Solr

Chris Hostetter-3

: MobileInfo. Now obviously, I need to index MobileInfo to support faceted
: browsing. How should I index SellingItem? The simplest way probably is to
: combile mobile phone specs in MobileInfo and and fields in SellingItem, and
: then index all of them. In this case, if I have 1000 SellingItems
: referencing a particular MobileInfo, I have to repeat the fields in
: MobileInfo a thousand times.

it depends on what you want ... hwat is the primary *thing* you want to
return from a search? .. is it a MobileInfo instance, from which people
can look at the list of mapped SellingItem isntances, or is it a
SellingItem instance, from which people can look at the relevent
MobileInfo? ... whatever that case *that* should be your document -- how
many extra fields you have hanging off of those documents is largely
irrelevant (particularly if those fields are Stored by not indexed, which
is what it seems like you would do if you made MobileInfo the documents,
and just wanted to be able to return the list of SellingItem instances for
each one)

Personally: I'd probably use two indexes: let people browse/search
"models" using an index with MobileInfo based documents, and once they
pick a model they like, search for it's identifier in a seperate
"SellingItems" based index.



-Hoss

Loading...