[VOTE] merge lucene/solr development (take 3)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
126 messages Options
1234567
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Grant Ingersoll-2

On Mar 9, 2010, at 11:28 AM, Grant Ingersoll wrote:

>
> On Mar 9, 2010, at 10:59 AM, Dennis Kubes wrote:
>
>> I agree.  Most of those things can/should be moved into Lucene.  That doesn't necessitate merging.  Separate responsibilities.
>>
>>> For that matter, why do we even need to have this discussion at all?  > Most of us Solr committers are Lucene committers.  We can simply start
>>> committing Solr code to Lucene such that in 6 months the whole
>>> discussion is moot and the three committers on Solr who aren't Lucene
>>> committers can earn their Lucene merit very quickly by patching the
>>> "Solr" portion of Lucene.  We can move all the code to it's
>>> appropriate place, add a contrib module for the WAR stuff and the
>>> response writers and voila, Solr is in Lucene, the dev mailing lists
>>> have merged by the fact that Solr dev would be defunct and all of the
>>> proposals in this vote are implemented simply by employing our commit
>>> privileges in a concerted way.
>>
>> Am I reading you right.  Are you are proposing a hostile takeover of the Lucene project?  Even being committers there needs to be discussion with the community about the best path.  Or are you suggesting we bypass discussion?  I am now even more concerned that merging is not the right way to go.
>>
>
> No.  Would you please re-read it and not quote me out of context.  You left the next sentence off, which is of course the vital one.

Namely:
"Yet, somehow, me thinks that isn't a good solution either, right?  Yet it is perfectly "legal" and is just as valid a solution as the "poaching" solution and in a lot of ways seems to be what Chris is proposing."

My point is, if people would just step back for a minute and remove the labels of Solr and Lucene and look at the code, the discussion would be a whole lot different.  Furthermore, if they stepped back and looked at the actual people who do the actual work on Lucene and Solr, you would see that these separations are slowing down both Lucene and Solr and hurting them more than helping.   This has been expressed time and time again by committers in both Lucene and in Solr and even in Nutch, including many committers who do not even work on Solr.  

-Grant
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Michael Busch
In reply to this post by Robert Muir
On 3/9/10 7:35 AM, Robert Muir wrote:

>> 2. duplicate the Lucene code in Solr, address any issues there, and then
>> contribute it back
>>
>>      
> Not that I can stop anything, but -1 to any further analysis code
> duplication. There has to be a better way.
>
> Its stupid to duplicate the code. Its also stupid to just move the code.
>
>    

I agree with that.  No matter if this dev-merge vote passes or not, we
still want a separate analysis module, right?  Gathering all the
analyzers from the different projects and putting them into one place
was Mike's first proposal and I believe everyone was excited about it.

So how about we start with that, considering that we want to do it
anyway?  It seems like this solves the immediate problem Robert and
others are having with working on the analyzer patches and will help us
learn about what such a re-structuring entails.  And then, maybe in a
few weeks or months, we can discuss again if more modules should be
factored out or if we still see the need for merging the Solr and Lucene
devs.
It seems like the analysis house is currently the only one on fire.

  Michael
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Grant Ingersoll-2
In reply to this post by Mattmann, Chris A (3010)

On Mar 9, 2010, at 11:29 AM, Mattmann, Chris A (388J) wrote:

> Hey Yonik,
>
>>> However, like I said it seems to be like
>>> the discussion of the real issues is only happening recently over the past
>>> few days.
>>
>> This certainly isn't new territory for lucene/solr devs though - the
>> issue of what belongs in Solr and what belongs in Lucene, and problems
>> around pulling out schema and faceting and putting it in Lucene have
>> come up before (also in lengthy threads).
>
> I hear ya. I've seen some on solr-{user|dev}@ within the past months.

I know it dates me, but:  s/months/years/  :-)

-Grant
Reply | Threaded
Open this post in threaded view
|

RE: [VOTE] merge lucene/solr development (take 3)

Granroth, Neal V.
In reply to this post by Mattmann, Chris A (3010)

On Tue, Mar 9, 2010 at 10:01 AM, Mattmann, Chris A (388J)
<[hidden email]> wrote:

> I think the point that you and some others are
> missing is that JARs are not the only artifact of a system.


This is the critical point.  The Solr and Lucene sources must be kept seprate for those of us that reference the Lucene source but have no use whatsoever for the JARs or the Solr source.


- Neal
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Mattmann, Chris A (3010)
In reply to this post by Grant Ingersoll-2
Haha it dates me too but that's OK I know I'm a young-un! :)

Cheers,
Chris


On 3/9/10 8:35 AM, "Grant Ingersoll" <[hidden email]> wrote:



On Mar 9, 2010, at 11:29 AM, Mattmann, Chris A (388J) wrote:

> Hey Yonik,
>
>>> However, like I said it seems to be like
>>> the discussion of the real issues is only happening recently over the past
>>> few days.
>>
>> This certainly isn't new territory for lucene/solr devs though - the
>> issue of what belongs in Solr and what belongs in Lucene, and problems
>> around pulling out schema and faceting and putting it in Lucene have
>> come up before (also in lengthy threads).
>
> I hear ya. I've seen some on solr-{user|dev}@ within the past months.

I know it dates me, but:  s/months/years/  :-)

-Grant



++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [hidden email]
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Yonik Seeley
In reply to this post by Michael Busch
On Tue, Mar 9, 2010 at 11:35 AM, Michael Busch <[hidden email]> wrote:
> No matter if this dev-merge vote passes or not, we still
> want a separate analysis module, right?

No.  That's the point of the dev merge - to allow free movement of
source code that benefits both projects.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Michael Busch
On 3/9/10 8:41 AM, Yonik Seeley wrote:

> On Tue, Mar 9, 2010 at 11:35 AM, Michael Busch<[hidden email]>  wrote:
>    
>> No matter if this dev-merge vote passes or not, we still
>> want a separate analysis module, right?
>>      
> No.  That's the point of the dev merge - to allow free movement of
> source code that benefits both projects.
>
> -Yonik
>
>    

Now I'm confused. This is part of the proposal:

>  Modulariize the sources: pull things out of Lucene's core (break
>      out query parser, move all core queries&  analyzers under their
>      contrib counterparts), pull things out of Solr's core (analyzers,
>      queries).


  Michael
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Michael McCandless-2
In reply to this post by Grant Ingersoll-2
I'm still +1 for merging Solr/Lucene dev.

I think poaching, when we have so much that needs to be shared, is
going to cause far more problems than it'll solve.  It's not the right
tool for [this] job.

I do think poaching is good & legitimate tool when it's less code (eg
the CommonGrams case), so, we should do both ;)

Mike

On Tue, Mar 9, 2010 at 8:49 AM, Grant Ingersoll <[hidden email]> wrote:

>
> On Mar 9, 2010, at 8:21 AM, Michael McCandless wrote:
>
>> On Tue, Mar 9, 2010 at 7:21 AM, Grant Ingersoll <[hidden email]> wrote:
>>
>>>> If we had that freedom ("poaching is perfectly fine"), then,
>>>> interested devs could freely "refactor" across sub projects.
>>>
>>> As someone who works on both, I don't think it is fine.  Just look at the function query mess.  Just look at the version mess.  It's very frustrating as a developer and it makes me choose between two projects that I happen to like equally, but for different reasons.  If I worked on Nutch, I would feel the same way.
>>
>> But... Lucene should poach from external (eg non-Apache) projects, if
>> the license works?
>>
>> Ie if some great analyzer is out there, and Robert spots it, and the
>> license works, we should poach it?  (In fact he just did this w/
>> Andrzej's Polish stemmer ;) ).
>
> I'd prefer "donate" to poach, but, realize that isn't always the case.
>
>
>>
>> So we have something of a double standard...
>>
>> And, ironically, I think it's the fact that there's so much committer
>> overlap between Solr and Lucene that is causing this antagonism
>> towards poaching.
>>
>> When in fact I think poaching, at a wider scale (across unrelated
>> projects) is a very useful means for any healthy open source software
>> to evolve.
>>
>> Why should Lucene be prevented from having a useful feature just
>> because Solr happened to create it first?
>
> But why should I be forced to maintain two versions due to some arbitrary code separation?  And why should you force a good chunk of us to do a whole lot of extra work simply because of some arbitrary code separation?  Here, it is the Lucene PMC that releases code and it is just silly that with all of this overlap at the committer level we still have this duplication.   I can't speak for the external projects (I don't believe any of them have even responded here other than Jackrabbit), but if they don't like it, they should get more involved in the community and work to be committers.
>
> At any rate, this is exactly why merging makes sense.  You would no longer have this issue of "first".  I would no longer have to choose where to add my spatial work based on some arbitrary line that someone drew in the sand that isn't all that pertinent anymore given the desires of most in the community to blur that line.  It would be available to everyone.
>
> For that matter, why do we even need to have this discussion at all?  Most of us Solr committers are Lucene committers.  We can simply start committing Solr code to Lucene such that in 6 months the whole discussion is moot and the three committers on Solr who aren't Lucene committers can earn their Lucene merit very quickly by patching the "Solr" portion of Lucene.  We can move all the code to it's appropriate place, add a contrib module for the WAR stuff and the response writers and voila, Solr is in Lucene, the dev mailing lists have merged by the fact that Solr dev would be defunct and all of the proposals in this vote are implemented simply by employing our commit privileges in a concerted way.  Yet, somehow, me thinks that isn't a good solution either, right?  Yet it is perfectly "legal" and is just as valid a solution as the "poaching" solution and in a lot of ways seems to be what Chris is proposing.
>
> -Grant
>
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Dennis Kubes-2
In reply to this post by Yonik Seeley
 From what I see I don't think we are going to agree on this.  Some of
us  believe there should be clear separations, some of us don't.  Let's
move on.

My vote is still a -1 on this.  If that means I get overruled, so be it.
  I have brought up what I believe to be important points in this
discussion.  I think we are just looping now.

Dennis

Yonik Seeley wrote:
> On Tue, Mar 9, 2010 at 11:35 AM, Michael Busch <[hidden email]> wrote:
>> No matter if this dev-merge vote passes or not, we still
>> want a separate analysis module, right?
>
> No.  That's the point of the dev merge - to allow free movement of
> source code that benefits both projects.
>
> -Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Robert Muir
In reply to this post by Mattmann, Chris A (3010)
>
> Can you provide more detail on how it's failed? Did it fail because Solr
> wasn't able to upgrade to a newer Lucene that would fix the deprecations? If
> so, what were the reasons?
>

Its just more work. Besides the duplication functionality itself, in
Lucene we (mostly Uwe) developed code to assist with tasks like this.
For example, we have utility code that makes it easier to maintain
backwards compatibility, test harnesses, etc.

The job still isn't done, but to do it right, it only makes sense to
re-use these resources so its done safely and more effectively. But
this would require even more duplication!!!

>
> With which "hat" on? Where are you converting them? I'm guessing you mean
> one for Solr and one for Lucene(-java), right?

Both Solr and Lucene. I'm not wearing a hat in this case: its just
something that needs to be done and we are in it together.


--
Robert Muir
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Otis Gospodnetic-2
In reply to this post by Yonik Seeley
Hello,

(just using Yonik's email to reply, but my comments are more general)


----- Original Message ----

> From: Yonik Seeley <[hidden email]>
> To: [hidden email]
> Sent: Tue, March 9, 2010 10:04:20 AM
> Subject: Re: [VOTE] merge lucene/solr development (take 3)
>
> On Tue, Mar 9, 2010 at 9:48 AM, Mattmann, Chris A (388J)
> wrote:
> > I have built 10s of projects that
> > have simply used Lucene as an API and had no need for Solr, and I've built
> > 10s of projects where Solr made perfect sense. So, I appreciate their
> > separation.
>
> As does everyone - which is why there will always be separate
> downloads.  As a user, the only side affect you should see is an
> improved Lucene and Solr.
>
> Saying that Solr should move some stuff to Lucene for Lucene's
> benefit, without regard to if it's actually benefitial to Solr, is a
> non-starter.  The lucene/solr committers have been down that road
> before.  The solution that most committers agreed would improve the
> development of both projects is to merge development.

* I'd completely understand the "non-starter" part if Lucene and Solr had disjoint sets of committers.  But that's not the case.

* Which is why I (like a few others) don't see why this whole thing cannot be solved by "better discussion of what to develop where from the get-go"

* Whenever people listed features built in Solr that really should have been in Lucene, I wondered "so why were not they developed in Lucene in the first place?"  Again, this should be possible because the same person can commit to both projects.

* I hear Grant's explanation on wanting something in Solr ASAP and not wanting to commit that something to Lucene (even though it logically belongs there) because Solr is not on Lucene trunk, but isn't this just a matter of getting "Lucene trunk nightly -> Solr trunk lib in svn" process going?

* Ian is 100% right.  This stuff clearly requires more discussion and a proper VOTE should wait a week or so.

Otis
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Otis Gospodnetic-2
* Re poaching (aka cross-project refactoring) - I think this is the way to go.  I think this is normal evolution of OSS projects.  I think this should be done if the functionality was not committed to the best (lowest common denominator?) project from the beginning, as in all the Solr/Lucene examples brought up

* I think Grant may be right.  We don't need this discussion.  Because the Solr/Lucene developer overlap is excellent, why not just start moving selected Solr code to new Lucene modules, just like Mike proposed we move Analysis from Lucene core to a new Lucene module?

* What do people think about doing what I wrote above as step 1 in this whole process?  When that is done in N months, we can see if we can improve on it?  This would also fit "progress, not perfection" mantra.

Otis




----- Original Message ----

> From: Otis Gospodnetic <[hidden email]>
> To: [hidden email]
> Sent: Tue, March 9, 2010 12:23:59 PM
> Subject: Re: [VOTE] merge lucene/solr development (take 3)
>
> Hello,
>
> (just using Yonik's email to reply, but my comments are more general)
>
>
> ----- Original Message ----
> > From: Yonik Seeley
> > To: [hidden email]
> > Sent: Tue, March 9, 2010 10:04:20 AM
> > Subject: Re: [VOTE] merge lucene/solr development (take 3)
> >
> > On Tue, Mar 9, 2010 at 9:48 AM, Mattmann, Chris A (388J)
> > wrote:
> > > I have built 10s of projects that
> > > have simply used Lucene as an API and had no need for Solr, and I've built
> > > 10s of projects where Solr made perfect sense. So, I appreciate their
> > > separation.
> >
> > As does everyone - which is why there will always be separate
> > downloads.  As a user, the only side affect you should see is an
> > improved Lucene and Solr.
> >
> > Saying that Solr should move some stuff to Lucene for Lucene's
> > benefit, without regard to if it's actually benefitial to Solr, is a
> > non-starter.  The lucene/solr committers have been down that road
> > before.  The solution that most committers agreed would improve the
> > development of both projects is to merge development.
>
> * I'd completely understand the "non-starter" part if Lucene and Solr had
> disjoint sets of committers.  But that's not the case.
>
> * Which is why I (like a few others) don't see why this whole thing cannot be
> solved by "better discussion of what to develop where from the get-go"
>
> * Whenever people listed features built in Solr that really should have been in
> Lucene, I wondered "so why were not they developed in Lucene in the first
> place?"  Again, this should be possible because the same person can commit to
> both projects.
>
> * I hear Grant's explanation on wanting something in Solr ASAP and not wanting
> to commit that something to Lucene (even though it logically belongs there)
> because Solr is not on Lucene trunk, but isn't this just a matter of getting
> "Lucene trunk nightly -> Solr trunk lib in svn" process going?
>
> * Ian is 100% right.  This stuff clearly requires more discussion and a proper
> VOTE should wait a week or so.
>
> Otis

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Jason Rutherglen
> * I think Grant may be right.  We don't need this discussion.  Because the Solr/Lucene developer overlap is excellent, why not just start moving selected Solr code to new Lucene modules, just like Mike proposed we move Analysis from Lucene core to a new Lucene module?

The underlying hesitation could be that the initial proposal sounds
like a press release.  With all the discussing, a few modules could
already have been moved (given the number of keys pressed to enter the
emails).

On Tue, Mar 9, 2010 at 9:38 AM, Otis Gospodnetic
<[hidden email]> wrote:

> * Re poaching (aka cross-project refactoring) - I think this is the way to go.  I think this is normal evolution of OSS projects.  I think this should be done if the functionality was not committed to the best (lowest common denominator?) project from the beginning, as in all the Solr/Lucene examples brought up
>
> * I think Grant may be right.  We don't need this discussion.  Because the Solr/Lucene developer overlap is excellent, why not just start moving selected Solr code to new Lucene modules, just like Mike proposed we move Analysis from Lucene core to a new Lucene module?
>
> * What do people think about doing what I wrote above as step 1 in this whole process?  When that is done in N months, we can see if we can improve on it?  This would also fit "progress, not perfection" mantra.
>
> Otis
>
>
>
>
> ----- Original Message ----
>> From: Otis Gospodnetic <[hidden email]>
>> To: [hidden email]
>> Sent: Tue, March 9, 2010 12:23:59 PM
>> Subject: Re: [VOTE] merge lucene/solr development (take 3)
>>
>> Hello,
>>
>> (just using Yonik's email to reply, but my comments are more general)
>>
>>
>> ----- Original Message ----
>> > From: Yonik Seeley
>> > To: [hidden email]
>> > Sent: Tue, March 9, 2010 10:04:20 AM
>> > Subject: Re: [VOTE] merge lucene/solr development (take 3)
>> >
>> > On Tue, Mar 9, 2010 at 9:48 AM, Mattmann, Chris A (388J)
>> > wrote:
>> > > I have built 10s of projects that
>> > > have simply used Lucene as an API and had no need for Solr, and I've built
>> > > 10s of projects where Solr made perfect sense. So, I appreciate their
>> > > separation.
>> >
>> > As does everyone - which is why there will always be separate
>> > downloads.  As a user, the only side affect you should see is an
>> > improved Lucene and Solr.
>> >
>> > Saying that Solr should move some stuff to Lucene for Lucene's
>> > benefit, without regard to if it's actually benefitial to Solr, is a
>> > non-starter.  The lucene/solr committers have been down that road
>> > before.  The solution that most committers agreed would improve the
>> > development of both projects is to merge development.
>>
>> * I'd completely understand the "non-starter" part if Lucene and Solr had
>> disjoint sets of committers.  But that's not the case.
>>
>> * Which is why I (like a few others) don't see why this whole thing cannot be
>> solved by "better discussion of what to develop where from the get-go"
>>
>> * Whenever people listed features built in Solr that really should have been in
>> Lucene, I wondered "so why were not they developed in Lucene in the first
>> place?"  Again, this should be possible because the same person can commit to
>> both projects.
>>
>> * I hear Grant's explanation on wanting something in Solr ASAP and not wanting
>> to commit that something to Lucene (even though it logically belongs there)
>> because Solr is not on Lucene trunk, but isn't this just a matter of getting
>> "Lucene trunk nightly -> Solr trunk lib in svn" process going?
>>
>> * Ian is 100% right.  This stuff clearly requires more discussion and a proper
>> VOTE should wait a week or so.
>>
>> Otis
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Grant Ingersoll-2
In reply to this post by Otis Gospodnetic-2

On Mar 9, 2010, at 12:38 PM, Otis Gospodnetic wrote:

>
>
> * I think Grant may be right.  We don't need this discussion.  Because the Solr/Lucene developer overlap is excellent, why not just start moving selected Solr code to new Lucene modules, just like Mike proposed we move Analysis from Lucene core to a new Lucene module?

Note, if you read what I said again you will realize I wasn't actually proposing this.  I was saying actually, that I think it would not be something that people really wanted, even though it is perfectly "legal", just like poaching is perfectly "legal", but isn't, in my mind a good solution.  Sigh.  The problem with email, I guess, especially on long threads.
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

pjaol
Sigh...
http://www.surveymonkey.com/

On Tue, Mar 9, 2010 at 2:00 PM, Grant Ingersoll <[hidden email]> wrote:

>
> On Mar 9, 2010, at 12:38 PM, Otis Gospodnetic wrote:
>
> >
> >
> > * I think Grant may be right.  We don't need this discussion.  Because
> the Solr/Lucene developer overlap is excellent, why not just start moving
> selected Solr code to new Lucene modules, just like Mike proposed we move
> Analysis from Lucene core to a new Lucene module?
>
> Note, if you read what I said again you will realize I wasn't actually
> proposing this.  I was saying actually, that I think it would not be
> something that people really wanted, even though it is perfectly "legal",
> just like poaching is perfectly "legal", but isn't, in my mind a good
> solution.  Sigh.  The problem with email, I guess, especially on long
> threads.
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

hossman
In reply to this post by Mark Miller-3

: from Solr into Lucene don't need to be in there. All of a sudden I'm agreeing
: with Hoss about goals rather than actual steps ;) Because those points are not
: important to this vote at all - they are more examples of what we will be able
        ...
: This is about merging dev so we can put code where it belongs and do things
: that can make sense - its not a vote where specific code refactorings matter

This is the point where i say...

AHHHHHHHHHHHHH!!!!!!! HA HA HA HA HAAAAAAA!!!!!

        HA!

I've had relationships (with live people that i cared deeply about) that
recieved less of my attention this all of this, and i just don't give a
fuck anymore.

I'm -1 to "the proposal" on the grounds that 150+ messages into the
discusison, it still seems like no one has any fucking clue what it is
we're voting on.

If anyone has any patches, releases, or a potential committers they'd like
me to vote on please let me know -- otherwise i'm wiping my hands of the
whole thing.  I'm done with this thread, and i don't plan on reading any
more messages posted to it.  (one of the bueatiful things about Lucene:
the community (and the PMC) are large enough that the wheels will keep
turning and smart people will make smart choices even if close my eyes and
stick my fingers in my ears)

if you'll excuse me: i desperately need to go review some patches, and
answer some questions on the user lists so i can ultimately finish this
day with some sense of accomplishment.


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Mike Klaas
In reply to this post by Grant Ingersoll-2
On Mon, Mar 8, 2010 at 8:24 PM, Grant Ingersoll <[hidden email]> wrote:

> It should also be noted that a good chunk of the Solr committers are already Lucene committers, and of the remaining there are: Bill Au, Mike Klaas, Ryan McKinley, Shalin and Noble.  Mike has been inactive for quite some time (and has elected to go emeritus even though it's not marked on the page) and and Ryan, Shalin and Noble already contribute to Lucene in various parts (AFAICT), so to me it's not a big stretch to say bring them into the fold.  I haven't tracked Bill's involvement, but I also know Bill and trust he knows what it means to be a committer, i.e. he knows as much what not to touch as what to touch.  Of course, we can do a separate vote on that if that helps satisfy Chris' issue on the committers.

I don't expect that it makes much of a difference either way, but feel
free to leave me out of the Lucene auto-committership should that be
an issue.  I don't expect to become an active committer in the near
future.

> In the end, for me anyway, the current separation hurts Lucene a good deal as much as it hurts Solr, if not more.

Agreed.

The central issue is that Solr committers often work on features which
are core to "full-text search" rather than "an HTTP full-text search
server".  The parts of the features related to full-text search would
improve Lucene (the fact that Solr is used by people as a library in
an embedded context provides glaring testament to that).  But they
don't end up there, due to the friction of simultaneously developing a
feature in two projects that aren't synchronized.

Does this happen often?  It does.  I'd say over 50% of my non-trivial
changes to Solr could have been useful in Lucene.  (This is probably
not representative of the entire Solr committerdom, of course.)  In
fact, the *very first patch* I developed for Solr was adding in hit
highlighting, and I ended up copy&pasting a class from contrib
Highlighter to extend it.  Of course, I was a committer for neither
project at the time; I don't want to think about the logistics of
trying to submit patches to both projects which are so inter-dependent
(and would pretty much rely on review by someone who was a committer
on both projects anyway).

I think someone could make an argument that I should have been more
conscientious about submitting patches to Lucene, and they are
probably right.  But, I ultimately wasn't, and many other committers
weren't as well (even those who were committers for both projects),
and so we've ended up in this situation where we really have *three*
projects:  Lucene, the java search library, Lucene+, the set of
improvements and extensions to Lucene which could be in Lucene itself
and developed by the same people as Lucene, and Solr, the HTTP server
around Lucene.

The Lucene Project as a whole would benefit if this situation were
improved.  Auto-syncing Lucene-trunk with Solr would bring an
improvement, but it isn't a fundamental solution and has its own set
of problems.  The current proposal seems reasonable, but I worry about
the level of contention it is receiving.

+0

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Simon Willnauer
In reply to this post by hossman
On Tue, Mar 9, 2010 at 11:54 PM, Chris Hostetter
<[hidden email]> wrote:

>
> : from Solr into Lucene don't need to be in there. All of a sudden I'm agreeing
> : with Hoss about goals rather than actual steps ;) Because those points are not
> : important to this vote at all - they are more examples of what we will be able
>        ...
> : This is about merging dev so we can put code where it belongs and do things
> : that can make sense - its not a vote where specific code refactorings matter
>
> This is the point where i say...
>
> AHHHHHHHHHHHHH!!!!!!! HA HA HA HA HAAAAAAA!!!!!
>
>        HA!
>
> I've had relationships (with live people that i cared deeply about) that
> recieved less of my attention this all of this, and i just don't give a
> fuck anymore.
>
> I'm -1 to "the proposal" on the grounds that 150+ messages into the
> discusison, it still seems like no one has any fucking clue what it is
> we're voting on.

Thanks Chris for pointing this out! I'm somewhat lost and I can not
keep up with the amount of email regarding this "vote".
IMO if we vote on something a reply should be +1, -1 or +-0. If you
want to discuss, vote -1.

simon


>
> If anyone has any patches, releases, or a potential committers they'd like
> me to vote on please let me know -- otherwise i'm wiping my hands of the
> whole thing.  I'm done with this thread, and i don't plan on reading any
> more messages posted to it.  (one of the bueatiful things about Lucene:
> the community (and the PMC) are large enough that the wheels will keep
> turning and smart people will make smart choices even if close my eyes and
> stick my fingers in my ears)
>
> if you'll excuse me: i desperately need to go review some patches, and
> answer some questions on the user lists so i can ultimately finish this
> day with some sense of accomplishment.
>
>
> -Hoss
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Andi Vajda-2
In reply to this post by Yonik Seeley
+1

Andi..

On Mar 9, 2010, at 3:11, Yonik Seeley <[hidden email]> wrote:

> Apoligies in advance for calling yet another vote, but I just wanted
> to make sure this was official.
> Mike's second VOTE thread could probably technically stand on it's own
> (since it included PMC votes), but given that I said in my previous
> VOTE thread that I was just polling Lucene/Solr committers and would
> call a second PMC vote, that may have acted to suppress PMC votes on
> Mike's thread also.
>
> Please vote for the proposal quoted below to merge lucene/solr  
> development.
> Here's my +1
>
> -Yonik
>
> Mike's call for a VOTE (amongst lucene/solr committers +11 to -1):
> http://search.lucidimagination.com/search/document/a400ffe62ae21aca/vote_merge_the_development_of_solr_lucene_take_2#22d7cd086d9c5cf0
>> Subject: Merge the development of Solr/Lucene (take 2)
>> A new vote, that slightly changes proposal from last vote (adding  
>> only
>> that Lucene can cut a release even if Solr doesn't):
>>
>> * Merging the dev lists into a single list.
>>
>> * Merging committers.
>>
>> * When any change is committed (to a module that "belongs to" Solr or
>>   to Lucene), all tests must pass.
>>
>> * Release details will be decided by dev community, but, Lucene may
>>   release without Solr.
>>
>> * Modulariize the sources: pull things out of Lucene's core (break
>>   out query parser, move all core queries & analyzers under their
>>   contrib counterparts), pull things out of Solr's core (analyzers,
>>   queries).
>>
>> These things would not change:
>>
>> * Besides modularizing (above), the source code would remain factored
>>   into separate dirs/modules the way it is now.
>>
>> * Issue tracking remains separate (SOLR-XXX and LUCENE-XXX
>>   issues).
>>
>> * User's lists remain separate.
>>
>> * Web sites remain separate.
>>
>> * Release artifacts/jars remain separate.
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] merge lucene/solr development (take 3)

Ryan McKinley
In reply to this post by Yonik Seeley
+1


On Mon, Mar 8, 2010 at 9:11 PM, Yonik Seeley <[hidden email]> wrote:

> Apoligies in advance for calling yet another vote, but I just wanted
> to make sure this was official.
> Mike's second VOTE thread could probably technically stand on it's own
> (since it included PMC votes), but given that I said in my previous
> VOTE thread that I was just polling Lucene/Solr committers and would
> call a second PMC vote, that may have acted to suppress PMC votes on
> Mike's thread also.
>
> Please vote for the proposal quoted below to merge lucene/solr development.
> Here's my +1
>
> -Yonik
>
> Mike's call for a VOTE (amongst lucene/solr committers +11 to -1):
> http://search.lucidimagination.com/search/document/a400ffe62ae21aca/vote_merge_the_development_of_solr_lucene_take_2#22d7cd086d9c5cf0
>> Subject: Merge the development of Solr/Lucene (take 2)
>> A new vote, that slightly changes proposal from last vote (adding only
>> that Lucene can cut a release even if Solr doesn't):
>>
>>  * Merging the dev lists into a single list.
>>
>>  * Merging committers.
>>
>>  * When any change is committed (to a module that "belongs to" Solr or
>>    to Lucene), all tests must pass.
>>
>>  * Release details will be decided by dev community, but, Lucene may
>>    release without Solr.
>>
>>  * Modulariize the sources: pull things out of Lucene's core (break
>>    out query parser, move all core queries & analyzers under their
>>    contrib counterparts), pull things out of Solr's core (analyzers,
>>    queries).
>>
>> These things would not change:
>>
>>  * Besides modularizing (above), the source code would remain factored
>>    into separate dirs/modules the way it is now.
>>
>>  * Issue tracking remains separate (SOLR-XXX and LUCENE-XXX
>>    issues).
>>
>>  * User's lists remain separate.
>>
>>  * Web sites remain separate.
>>
>>  * Release artifacts/jars remain separate.
>
1234567