Concerned about Solr's V2 API synchronized with V1

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Concerned about Solr's V2 API synchronized with V1

david.w.smiley@gmail.com
I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer
Reply | Threaded
Open this post in threaded view
|

Re: Concerned about Solr's V2 API synchronized with V1

Jan Høydahl / Cominvent
+1

Perhaps it’s possible to use forbiddenapis to block new test classes explicitly calling v1 apis? (Look for “/solr/admin/.*”)? And of course make sure that SolrJ commands map to v2...

Jan Høydahl

16. mai 2019 kl. 19:33 skrev David Smiley <[hidden email]>:

I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer
Reply | Threaded
Open this post in threaded view
|

Re: Concerned about Solr's V2 API synchronized with V1

Noble Paul നോബിള്‍  नोब्ळ्
In reply to this post by david.w.smiley@gmail.com
This is a problem. V2 APIs need a lot more metadata and nobody is doing it. This leads to a lot of technical debt

On Fri, May 17, 2019, 3:42 AM David Smiley <[hidden email]> wrote:
I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer
Reply | Threaded
Open this post in threaded view
|

Re: Concerned about Solr's V2 API synchronized with V1

Gus Heck
Also the Collections API docs are almost devoid of v2 examples. Just fixing this would provide a really good reminder to those implementing features to check that it works in v2. (unless they add features without documenting them... which usually doesn't happen)

On Sun, Jul 7, 2019 at 9:51 PM Noble Paul <[hidden email]> wrote:
This is a problem. V2 APIs need a lot more metadata and nobody is doing it. This leads to a lot of technical debt

On Fri, May 17, 2019, 3:42 AM David Smiley <[hidden email]> wrote:
I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer


--
Reply | Threaded
Open this post in threaded view
|

Re: Concerned about Solr's V2 API synchronized with V1

Gus Heck
We have places where there are curl/solrj alternatives in the examples. Maybe a similar widget could be used for V1/V2 examples? or even better v1/v2/solrj examples for collections api :)

On Mon, Jul 8, 2019 at 2:02 PM Gus Heck <[hidden email]> wrote:
Also the Collections API docs are almost devoid of v2 examples. Just fixing this would provide a really good reminder to those implementing features to check that it works in v2. (unless they add features without documenting them... which usually doesn't happen)

On Sun, Jul 7, 2019 at 9:51 PM Noble Paul <[hidden email]> wrote:
This is a problem. V2 APIs need a lot more metadata and nobody is doing it. This leads to a lot of technical debt

On Fri, May 17, 2019, 3:42 AM David Smiley <[hidden email]> wrote:
I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer


--


--
Reply | Threaded
Open this post in threaded view
|

Re: Concerned about Solr's V2 API synchronized with V1

Cassandra Targett
The v2 examples that are in the Ref Guide already use the widget you mention. See something like https://lucene.apache.org/solr/guide/8_1/adding-custom-plugins-in-solrcloud-mode.html#config-api-commands-to-use-jars-as-runtime-libraries for an example.

SOLR-11646 tracks the effort to add v2 examples and lists which pages have been updated and which remain to be done. I left the Collections APIs (the hardest) for last, but haven’t had time to get back to it recently. Whenever I make examples I make real examples I’ve actually tested, so I need to have enough time to actually run through each one of them for the alternate syntaxes.

Cassandra
On Jul 8, 2019, 1:09 PM -0500, Gus Heck <[hidden email]>, wrote:
We have places where there are curl/solrj alternatives in the examples. Maybe a similar widget could be used for V1/V2 examples? or even better v1/v2/solrj examples for collections api :)

On Mon, Jul 8, 2019 at 2:02 PM Gus Heck <[hidden email]> wrote:
Also the Collections API docs are almost devoid of v2 examples. Just fixing this would provide a really good reminder to those implementing features to check that it works in v2. (unless they add features without documenting them... which usually doesn't happen)

On Sun, Jul 7, 2019 at 9:51 PM Noble Paul <[hidden email]> wrote:
This is a problem. V2 APIs need a lot more metadata and nobody is doing it. This leads to a lot of technical debt

On Fri, May 17, 2019, 3:42 AM David Smiley <[hidden email]> wrote:
I'm concerned about Solr's V2 API and the maintenance burden of attempting to maintain consistency with V1.  For example upon looking through the release notes and seeing a new exciting REINDEXCOLLECTION command (a V1 reference), I see no corresponding adjustments in V2 -- lucene-solr/solr/solrj/src/resources/apispec/*   It's so easy for this to fall out of sync.  When working on a feature affecting admin API stuff I need to somehow just remember/know and then ask myself if I want to test a new feature with just one API or both. Ugh.  Additionally, the vast majority of our documentation is in V1, and our help in solr-user and elsewhere often uses a one-liner URL to the V1 API as well.

As if Solr needed more maintenance challenges than it has already (e.g. tests).   :-(

I mainly want to point out this problem right now to see if others also see the problem and if anyone else has thought about it.  While working on Time Routed Aliases, I saw it but didn't call it out.  I thought maybe somehow our implementation of the admin functionality could be done differently so as to nearly require a V2 adjustment, and thus we don't forget.  For example if the V2 API was basically primary, and if it had metadata that described how a virtual V1 API could work based off metadata in the V2 apispec there that does mapping.  In this way, everything would work in V2 and V1 by default, or at least the majority of the time.  V2 requires more information than V1, so if we continue to have V1 primary (i.e. do nothing), V2 will always be falling behind.

~ David Smiley
Apache Lucene/Solr Search Developer


--


--