Solr/SolrCloud

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr/SolrCloud

Mark Miller-3
Dear Lucene/Solr Community,

I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.

I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me.

So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start.

Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast.

So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work.

I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big.

But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr.

Mark
Reply | Threaded
Open this post in threaded view
|

Re: Solr/SolrCloud

Mark Miller-3
New people to Solr and maybe some old ones :)

This is an old project. There is a lot of stuff in the history. This whole thing is more about me than anyone else. This software is salvageable, I've seen it. I've seen the stuff in the software to know you can do it - a lot of what you need is there, just not thoroughly done, or its a little off, or whatever. You know, its people trying and having good ideas, but a lot of them not taking root.

So don't be scared, this community is good, for some reason there is weird Solr road block, but I'm pretty confident you will get through it now. And you won't have all my code, you don't need all my code, and the code I have, I'm sure you will end up with. I'm entrusting it to good hands.

- mark

On Sat, Nov 9, 2019 at 7:30 PM Mark Miller <[hidden email]> wrote:
Dear Lucene/Solr Community,

I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.

I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me.

So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start.

Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast.

So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work.

I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big.

But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr.

Mark


--
Reply | Threaded
Open this post in threaded view
|

Re: Solr/SolrCloud

Mark Miller-3
And don't worry, there are a lot of people in my wake that don't want to fight with me, I'm not fun to fight with, I don't want fight with you either. I'm disappointed in myself and in what I've accomplished in a decade - pretty much 0 of that is on anyone here but me. I did not intend to take it out on anyone here.

- Mark

On Sun, Nov 10, 2019 at 5:16 PM Mark Miller <[hidden email]> wrote:
New people to Solr and maybe some old ones :)

This is an old project. There is a lot of stuff in the history. This whole thing is more about me than anyone else. This software is salvageable, I've seen it. I've seen the stuff in the software to know you can do it - a lot of what you need is there, just not thoroughly done, or its a little off, or whatever. You know, its people trying and having good ideas, but a lot of them not taking root.

So don't be scared, this community is good, for some reason there is weird Solr road block, but I'm pretty confident you will get through it now. And you won't have all my code, you don't need all my code, and the code I have, I'm sure you will end up with. I'm entrusting it to good hands.

- mark

On Sat, Nov 9, 2019 at 7:30 PM Mark Miller <[hidden email]> wrote:
Dear Lucene/Solr Community,

I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.

I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me.

So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start.

Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast.

So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work.

I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big.

But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr.

Mark


--


--
Reply | Threaded
Open this post in threaded view
|

Re: Solr/SolrCloud

Mark Miller-3
And I don't know you were on Twitter :) Bravo, because I should be able to guess. Like that actually is going haunt me a little and it shouldn't. It should be easy. And like I have a guess and then doesn't seem to quite fit...I mean I didn't read everything towards the end, but good job. It's like 3 of you where doing that.

- Mark

On Sun, Nov 10, 2019 at 5:35 PM Mark Miller <[hidden email]> wrote:
And don't worry, there are a lot of people in my wake that don't want to fight with me, I'm not fun to fight with, I don't want fight with you either. I'm disappointed in myself and in what I've accomplished in a decade - pretty much 0 of that is on anyone here but me. I did not intend to take it out on anyone here.

- Mark

On Sun, Nov 10, 2019 at 5:16 PM Mark Miller <[hidden email]> wrote:
New people to Solr and maybe some old ones :)

This is an old project. There is a lot of stuff in the history. This whole thing is more about me than anyone else. This software is salvageable, I've seen it. I've seen the stuff in the software to know you can do it - a lot of what you need is there, just not thoroughly done, or its a little off, or whatever. You know, its people trying and having good ideas, but a lot of them not taking root.

So don't be scared, this community is good, for some reason there is weird Solr road block, but I'm pretty confident you will get through it now. And you won't have all my code, you don't need all my code, and the code I have, I'm sure you will end up with. I'm entrusting it to good hands.

- mark

On Sat, Nov 9, 2019 at 7:30 PM Mark Miller <[hidden email]> wrote:
Dear Lucene/Solr Community,

I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.

I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me.

So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start.

Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast.

So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work.

I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big.

But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr.

Mark


--


--


--
Reply | Threaded
Open this post in threaded view
|

Re: Solr/SolrCloud

Mark Miller-3
Finally. You will not lose my knowledge. I am and will share as much as I can with my teammates. And if you want to talk to me about Solr and Lucene I will talk your ear off. I’m not against anyone here. 

Mark 

On Sun, Nov 10, 2019 at 6:04 PM Mark Miller <[hidden email]> wrote:
And I don't know you were on Twitter :) Bravo, because I should be able to guess. Like that actually is going haunt me a little and it shouldn't. It should be easy. And like I have a guess and then doesn't seem to quite fit...I mean I didn't read everything towards the end, but good job. It's like 3 of you where doing that.

- Mark

On Sun, Nov 10, 2019 at 5:35 PM Mark Miller <[hidden email]> wrote:
And don't worry, there are a lot of people in my wake that don't want to fight with me, I'm not fun to fight with, I don't want fight with you either. I'm disappointed in myself and in what I've accomplished in a decade - pretty much 0 of that is on anyone here but me. I did not intend to take it out on anyone here.

- Mark

On Sun, Nov 10, 2019 at 5:16 PM Mark Miller <[hidden email]> wrote:
New people to Solr and maybe some old ones :)

This is an old project. There is a lot of stuff in the history. This whole thing is more about me than anyone else. This software is salvageable, I've seen it. I've seen the stuff in the software to know you can do it - a lot of what you need is there, just not thoroughly done, or its a little off, or whatever. You know, its people trying and having good ideas, but a lot of them not taking root.

So don't be scared, this community is good, for some reason there is weird Solr road block, but I'm pretty confident you will get through it now. And you won't have all my code, you don't need all my code, and the code I have, I'm sure you will end up with. I'm entrusting it to good hands.

- mark

On Sat, Nov 9, 2019 at 7:30 PM Mark Miller <[hidden email]> wrote:
Dear Lucene/Solr Community,

I have been searching for an answer for Solr and SolrCloud for a long time. I feel like I landed in a tornado and I don’t know where the time went. I forget even why I’m here. Because I didn’t come here to work for silicon valley companies, or make a lot of money, or impress people I don’t know. I came here for Lucene. I love Lucene. I love developing. I love Lucene tests. I don’t do much Lucene anymore. I was needed more in Solr, and someone started acting like a dictator.

I still love Lucene. I’ve tried to love to Solr. But I don’t. And so I’ve been searching for an answer, when not being depressed about it, and as often happens, it was right in front of me.

So yeah, a couple times when I got sick of you guys - which is no one and everyone - I went off on my own and started chasing one of my own itches, which leads to things, which leads to things, which leads things. I love that I have no idea at the start.

Anyway, after time and some learning I kind of got to the point where I knew enough about the stupid technologies and the whole system - it’s like a lot of code, a lot of debt, blah blah. But I’m banging my head against this - intuition guy - like, just bang bang bang, starts to make sense and I don’t even do any work. So starts to makes sense. I start to address this. And that. I make some progress. I find some things. I say screw working on making this work anymore, it’s impossible, I’m sick of it, I’m finally gonna do the thing I love. Make it fast.

So I start making it fast here and there, sometimes. Most efforts are in like 3-4-5 different huge sprints or something - but always efforts around that. You know the lost work story. Lot of lost work.

I usually don’t duplicate all the work when I make another attempt. I have enough memories that that is not the important part. The importance is that I learned that none of you you know anything about this system or the components that make it up. I didn’t either. I knew more than a lot of you, but not early enough. And you guys have worked on the very edges on some great necessary stuff and tools - and I take heavy goddamn advantage of those things. Thank you. And I add things. And I track things. And I turn on enforcers. And pluck away. And I strip out all our darn randomization or craziness test hierarchy (or start to try and control it), and I start adding logging that's useful, and debug logging, and I use a good profiler, and I start limiting resources and minimizing shit, until I have a system that I can start to understand and work through. And I spend almost just as much on making myself efficient, cause it’s big.

But. All basic stuff. Maybe I’m smart somewhere, maybe I’m not. I’m lazy. I don’t think. I’m a math minor and most can probably attest I will not do a 1 dollar tip in my head. So I’m just learning about the system, the components, plucking away, cleaning up, finding bugs, adding stuff that will allow me to understand. Starting with basic tests, and like shooting for high goals. I want to be able to start 500 solrcores in 10-15 seconds in a single corecontainer. Thats what I want. So sometimes I work towards. Brings out a lot of great stuff. But the solution is neither fancy or some huge credit to me. We dont know anything, we have no good enforcement really, and we make it too crazy and wild when it's already crazy and wild and the it’s all way more than any human can realistically do anything with. Now I wrote a lot of this foundation. It’s not easy for people to take me seriously when I say its cause we are shit software developers. “Haha, you say cocreator, your software, please tell me how I am the one that sucks”. Even I had no confidence this could work so well compared to what was happening. I had to basically get there. Get there again cause then I didn't care, and then get close again. Like, I don’t trust myself or brain. So I didn’t need everything - god my knowledge and code is so spread around - but it’s not important. The design not important. I’d like you to have whatever design you want. But I know this one can work good enough to get you to the next one, and you need to conquer these demons before you can do anything on Solr.

Mark


--


--


--
--