ReentrantReadWriteLock

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

ReentrantReadWriteLock

jason rutherglen-2
Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?

Reply | Threaded
Open this post in threaded view
|

Re: ReentrantReadWriteLock

Yonik Seeley
On 6/8/06, jason rutherglen <[hidden email]> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?

Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.

In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.

ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap.  If you want to develop one feel free :-)  In
profiling though, cache lookups aren't that significant.  Are you
seeing something that indicates otherwise?

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server
Reply | Threaded
Open this post in threaded view
|

Re: ReentrantReadWriteLock

jason rutherglen-2
I don't see any place in particular, though since there is a lot of locking in Solr, and a lot I don't fully understand, thought I'd ask.  I don't see it used much in projects, but it is a nice piece of work.  Yeah LinkedConcurrentHashMap would be nice, that could be done.  Is it needed for anything right now?

----- Original Message ----
From: Yonik Seeley <[hidden email]>
To: [hidden email]; jason rutherglen <[hidden email]>
Sent: Thursday, June 8, 2006 1:50:45 PM
Subject: Re: ReentrantReadWriteLock

On 6/8/06, jason rutherglen <[hidden email]> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?

Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.

In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.

ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap.  If you want to develop one feel free :-)  In
profiling though, cache lookups aren't that significant.  Are you
seeing something that indicates otherwise?

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server




Reply | Threaded
Open this post in threaded view
|

Re: ReentrantReadWriteLock

Yonik Seeley
On 6/8/06, jason rutherglen <[hidden email]> wrote:
>  Yeah
> LinkedConcurrentHashMap would be nice, that could be done.  Is it needed for
> anything right now?

I don't think so... if it existed, I would have tried it out against
the standard synchronized LinkedHashMap.  For most uses I doubt it
would be an improvement though... at least for our hardware with 2-4
CPUs per box.

-Yonik
Reply | Threaded
Open this post in threaded view
|

ReentrantLock

jason rutherglen-2
In reply to this post by Yonik Seeley
What about ReentrantLock?  I like it because it cleans up sync blocks.  Adds a few nice methods.  Makes threading more OO.  

----- Original Message ----
From: Yonik Seeley <[hidden email]>
To: [hidden email]; jason rutherglen <[hidden email]>
Sent: Thursday, June 8, 2006 1:50:45 PM
Subject: Re: ReentrantReadWriteLock

On 6/8/06, jason rutherglen <[hidden email]> wrote:
> Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?

Not really... ReentrantReadWriteLocks beat normal synchronization when
there are many readers, fewer writers, and heavy contention for the
locks.

In the case of our LRUCache, there aren't any pure readers... It's
implemented with LinkedHashMap, and every lookup changes the ordering
of the entries.

ConcurrentHashMap is a cool class, but there is no
LinkedConcurrentHashMap.  If you want to develop one feel free :-)  In
profiling though, cache lookups aren't that significant.  Are you
seeing something that indicates otherwise?

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server




Reply | Threaded
Open this post in threaded view
|

Re: ReentrantLock

Yonik Seeley
On 6/8/06, jason rutherglen <[hidden email]> wrote:
> What about ReentrantLock?  I like it because it cleans up sync blocks.  Adds a few nice methods.  Makes threading more OO.

I don't know about cleaning up.... the syntax is slightly messier
because you always need a finally block.

It could have it's place if we needed lock polling or timed lock
waits.  It's really only faster under heavy contention AFAIK.  So if
you are running Solr on one of those monster 128 or 256 CPU machines,
then it might matter more :-)  Of course, at that point, Lucene might
still be the weakest link in the concurrency chain too.  Only way to
tell is to have someone with a high CPU count do some
testing/profiling.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: ReentrantLock

Yonik Seeley
In reply to this post by jason rutherglen-2
Of course it might take fewer CPUs to show a difference:
http://www-128.ibm.com/developerworks/java/library/j-jtp10264/

But that's in the inner loop.... and I can't think of anything in Solr
that synchronizes at that low of a level.  Where this could really pay
off is Lucene's IndexReader.isDeleted() call that Scorers make... but
Lucene is Java1.4 compatible for now.

-Yonik

On 6/8/06, jason rutherglen <[hidden email]> wrote:

> What about ReentrantLock?  I like it because it cleans up sync blocks.  Adds a few nice methods.  Makes threading more OO.
>
> ----- Original Message ----
> From: Yonik Seeley <[hidden email]>
> To: [hidden email]; jason rutherglen <[hidden email]>
> Sent: Thursday, June 8, 2006 1:50:45 PM
> Subject: Re: ReentrantReadWriteLock
>
> On 6/8/06, jason rutherglen <[hidden email]> wrote:
> > Have the Solr team looked at using ReentrantReadWriteLock for things like LRUCache and in other places where read write locks are used?
>
> Not really... ReentrantReadWriteLocks beat normal synchronization when
> there are many readers, fewer writers, and heavy contention for the
> locks.
>
> In the case of our LRUCache, there aren't any pure readers... It's
> implemented with LinkedHashMap, and every lookup changes the ordering
> of the entries.
>
> ConcurrentHashMap is a cool class, but there is no
> LinkedConcurrentHashMap.  If you want to develop one feel free :-)  In
> profiling though, cache lookups aren't that significant.  Are you
> seeing something that indicates otherwise?
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene search server
>
>
>
>
>
>


--
-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server