[jira] Commented: (MAHOUT-6) Need a matrix implementation

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (MAHOUT-6) Need a matrix implementation

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/MAHOUT-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577192#action_12577192 ]

Ted Dunning commented on MAHOUT-6:
----------------------------------


Amazing.

Were you able to measure garbage production?






> Need a matrix implementation
> ----------------------------
>
>                 Key: MAHOUT-6
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-6
>             Project: Mahout
>          Issue Type: New Feature
>            Reporter: Ted Dunning
>            Assignee: Grant Ingersoll
>         Attachments: MAHOUT-6a.diff, MAHOUT-6b.diff, MAHOUT-6c.diff, MAHOUT-6d.diff, MAHOUT-6e.diff, MAHOUT-6f.diff, MAHOUT-6g.diff, MAHOUT-6h.patch, MAHOUT-6i.diff, MAHOUT-6j.diff, MAHOUT-6k.diff, MAHOUT-6l.patch
>
>
> We need matrices for Mahout.
> An initial set of basic requirements includes:
> a) sparse and dense support are required
> b) row and column labels are important
> c) serialization for hadoop use is required
> d) reasonable floating point performance is required, but awesome FP is not
> e) the API should be simple enough to understand
> f) it should be easy to carve out sub-matrices for sending to different reducers
> g) a reasonable set of matrix operations should be supported, these should eventually include:
>     simple matrix-matrix and matrix-vector and matrix-scalar linear algebra operations, A B, A + B, A v, A + x, v + x, u + v, dot(u, v)
>     row and column sums  
>     generalized level 2 and 3 BLAS primitives, alpha A B + beta C and A u + beta v
> h) easy and efficient iteration constructs, especially for sparse matrices
> i) easy to extend with new implementations

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Jason Rennie-2
On Mon, Mar 10, 2008 at 5:10 PM, Ted Dunning (JIRA) <[hidden email]> wrote:

> Amazing.


Ditto!


> Were you able to measure garbage production?
>

How would I do that?

Jason
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Ted Dunning-3


Try turning on verbose gc.

In particular, if you mark the start of a particular timing run by printing
a line to output and then collect the GC output between those marks, you can
determine total garbage collected.  A bit of averaging later and you have
the number you seek.  Mis-sizing the newspace could be done to get more
frequent collections and thus more resolution, but I doubt you need that.

There are more elaborate methods using JMX, but this should do.

http://java.sun.com/docs/hotspot/gc1.4.2/example.html

On 3/10/08 6:02 PM, "Jason Rennie" <[hidden email]> wrote:

>> Were you able to measure garbage production?
>>
>
> How would I do that?
>
> Jason

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Dawid Weiss

> Try turning on verbose gc.

Benchmarking is a difficult stuff, really. Much harder than folks usually
consider it. Looking at GC, for example, you have to be aware not only of the GC
type used (and its settings), but also of the machine the benchmarking is run on.

In Jason's case, if he's running 64-bit server JVM on a multi-core machine, then
the JVM is using a parallel GC and is effectively running behind the scenes :)

You can also try using YouKit profiler -- it's does really good job at showing
heap/ GC usage.

I am far from being an expert in JVM architecture, but I think boxed primitive
types do have very special optimization at low-level and since so produce very
little overhead. Also, the collections inside JDK have been optimized heavily
since their initial releases. Much like reg-exes: we recently compared a code
that used Pattern class and hand-crafted parsing code. The hand-crafted code was
100% faster, but much longer and more difficult to read.

Dawid
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Grant Ingersoll-2
Not to mention, we are very young here, and the usual quote about  
optimization applies, so I'd say we not worry too much about  
performance just yet.

Do note, however, that committers can get a free Open Source license  
for YourKit just by asking them.  Same goes for other tools, too, like  
IntelliJ, etc.

-Grant

On Mar 11, 2008, at 4:15 AM, Dawid Weiss wrote:

>
>> Try turning on verbose gc.
>
> Benchmarking is a difficult stuff, really. Much harder than folks  
> usually consider it. Looking at GC, for example, you have to be  
> aware not only of the GC type used (and its settings), but also of  
> the machine the benchmarking is run on.
>
> In Jason's case, if he's running 64-bit server JVM on a multi-core  
> machine, then the JVM is using a parallel GC and is effectively  
> running behind the scenes :)
>
> You can also try using YouKit profiler -- it's does really good job  
> at showing heap/ GC usage.
>
> I am far from being an expert in JVM architecture, but I think boxed  
> primitive types do have very special optimization at low-level and  
> since so produce very little overhead. Also, the collections inside  
> JDK have been optimized heavily since their initial releases. Much  
> like reg-exes: we recently compared a code that used Pattern class  
> and hand-crafted parsing code. The hand-crafted code was 100%  
> faster, but much longer and more difficult to read.
>
> Dawid


Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Isabel Drost-3
On Wednesday 12 March 2008, Grant Ingersoll wrote:
> Not to mention, we are very young here, and the usual quote about
> optimization applies, so I'd say we not worry too much about
> performance just yet.

Although it does pay to think about performance early if the goal is a library
that is to be used with large amounts of data. At the moment it is still easy
to try out different versions. I would guess it won't be that easy once a lot
code has been created...

Cheers,
Isabel

--
I am not afraid of tomorrow, for I have seen yesterday and I love today. --
William Allen White
  |\      _,,,---,,_       Web:   <http://www.isabel-drost.de>
  /,`.-'`'    -.  ;-;;,_
 |,4-  ) )-,_..;\ (  `'-'
'---''(_/--'  `-'\_) (fL)  IM:  <xmpp://[hidden email]>

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Jason Rennie-2
On Thu, Mar 13, 2008 at 3:15 AM, Isabel Drost <[hidden email]>
wrote:

> Although it does pay to think about performance early if the goal is a
> library
> that is to be used with large amounts of data. At the moment it is still
> easy
> to try out different versions. I would guess it won't be that easy once a
> lot
> code has been created...


+1

Optimizing the core aspects of a matrix library intended to scale across
hundreds of machines is not what I'd call premature.  It is well worth
understanding the tradeoffs here, as, yes, important decisions made now
could be difficult to change later.

Btw, I yesterday learned how to determine the size of Java Objects and was
quite surprised by the result:

http://javaquirks.blogspot.com/

Marginal storage cost of Integer/Float (16-24 bytes) is 4-6x larger than
int/float (4 bytes).  Double is 2-3x larger than double.  Probably worth
sticking to primitives if possible...

Jason
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Ted Dunning-3

But the question that is raised by the performance numbers is whether the
compiling is optimizing away this storage overhead.

These numbers are exactly why I *assumed* we would need primitive
implementations, but the speed numbers make me question that assumption.  It
used to be true, but may not be any more.

On 3/14/08 8:15 AM, "Jason Rennie" <[hidden email]> wrote:

> Btw, I yesterday learned how to determine the size of Java Objects and was
> quite surprised by the result:
>
> http://javaquirks.blogspot.com/
>
> Marginal storage cost of Integer/Float (16-24 bytes) is 4-6x larger than
> int/float (4 bytes).  Double is 2-3x larger than double.  Probably worth
> sticking to primitives if possible...
>
> Jason

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Grant Ingersoll-2
In reply to this post by Jason Rennie-2
Don't get me wrong, I do think they are needed, it's just been my  
experience that it is fraught with traps.  In Lucene, until we came up  
w/ a more or less standard way of sharing algorithms (see contrib/
benchmark) it was always a bit hard to draw any real conclusion, b/c a  
lot of it can depend on usage patterns,  different JVMs on different  
machines, etc.

Was just trying to say let's not worry too much about, but you both  
are right, we should worry some :-)

-Grant

On Mar 14, 2008, at 11:15 AM, Jason Rennie wrote:

> On Thu, Mar 13, 2008 at 3:15 AM, Isabel Drost <[hidden email]
> >
> wrote:
>
>> Although it does pay to think about performance early if the goal  
>> is a
>> library
>> that is to be used with large amounts of data. At the moment it is  
>> still
>> easy
>> to try out different versions. I would guess it won't be that easy  
>> once a
>> lot
>> code has been created...
>
>
> +1
>
> Optimizing the core aspects of a matrix library intended to scale  
> across
> hundreds of machines is not what I'd call premature.  It is well worth
> understanding the tradeoffs here, as, yes, important decisions made  
> now
> could be difficult to change later.
>
> Btw, I yesterday learned how to determine the size of Java Objects  
> and was
> quite surprised by the result:
>
> http://javaquirks.blogspot.com/
>
> Marginal storage cost of Integer/Float (16-24 bytes) is 4-6x larger  
> than
> int/float (4 bytes).  Double is 2-3x larger than double.  Probably  
> worth
> sticking to primitives if possible...
>
> Jason


Reply | Threaded
Open this post in threaded view
|

RE: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Jeff Eastman-2-2
The good news in all of this is the Vector and Matrix interfaces, plus their
dense implementations all use primitive types. It is only the sparse
implementations which are impacted by boxing issues. Not to diminish their
importance, the impact is well localized and subject to optimization in the
future if we can get empirical data that indicates they need it. So far, I
think the results have been somewhat surprising and counter-intuitive.

Jeff

> -----Original Message-----
> From: Grant Ingersoll [mailto:[hidden email]]
> Sent: Friday, March 14, 2008 10:20 AM
> To: [hidden email]
> Subject: Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation
>
> Don't get me wrong, I do think they are needed, it's just been my
> experience that it is fraught with traps.  In Lucene, until we came up
> w/ a more or less standard way of sharing algorithms (see contrib/
> benchmark) it was always a bit hard to draw any real conclusion, b/c a
> lot of it can depend on usage patterns,  different JVMs on different
> machines, etc.
>
> Was just trying to say let's not worry too much about, but you both
> are right, we should worry some :-)
>
> -Grant
>
> On Mar 14, 2008, at 11:15 AM, Jason Rennie wrote:
>
> > On Thu, Mar 13, 2008 at 3:15 AM, Isabel Drost <apache_mahout@isabel-
> drost.de
> > >
> > wrote:
> >
> >> Although it does pay to think about performance early if the goal
> >> is a
> >> library
> >> that is to be used with large amounts of data. At the moment it is
> >> still
> >> easy
> >> to try out different versions. I would guess it won't be that easy
> >> once a
> >> lot
> >> code has been created...
> >
> >
> > +1
> >
> > Optimizing the core aspects of a matrix library intended to scale
> > across
> > hundreds of machines is not what I'd call premature.  It is well worth
> > understanding the tradeoffs here, as, yes, important decisions made
> > now
> > could be difficult to change later.
> >
> > Btw, I yesterday learned how to determine the size of Java Objects
> > and was
> > quite surprised by the result:
> >
> > http://javaquirks.blogspot.com/
> >
> > Marginal storage cost of Integer/Float (16-24 bytes) is 4-6x larger
> > than
> > int/float (4 bytes).  Double is 2-3x larger than double.  Probably
> > worth
> > sticking to primitives if possible...
> >
> > Jason
>



Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (MAHOUT-6) Need a matrix implementation

Ted Dunning-3

Exactly correct.  Until the need for optimization shows up, this is all
about curiosity, not implementation need.


On 3/14/08 11:31 AM, "Jeff Eastman" <[hidden email]> wrote:

> Not to diminish their
> importance, the impact is well localized and subject to optimization in the
> future if we can get empirical data that indicates they need it. So far, I
> think the results have been somewhat surprising and counter-intuitive.