[lucy-user] Question about query parsing API

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[lucy-user] Question about query parsing API

Andrew S. Townley
Hi Folks,

Me again.

Another issue I just hit with Ferret (actually, the same root problem, but manifested in a different way) made me wonder something else about the design of lucy: the query parsing API.

Is lucy's QueryParser API effectively this one?

http://www.rectangular.com/kinosearch/docs/devel/KinoSearch/Search/QueryParser.html

If so, is there a way to traverse the query parse tree (or get a callback notification when it matches terms)?  What I need to be able to do is take an arbitrary fulltext search query and then determine whether any search fields have been requested explicitly.  I suppose a more ideal solution would be to have a #target_fields method or something on the query object, but that would just be convenience.

The problem I'm having with Ferret is that the query objects are effectively opaque to Ruby, and there's no bubbling of either the terms or the fields to the top-level query object.  Is this sort of thing possible in lucy?

I know there's no Ruby bindings yet, but if I'm planning on migrating to lucy anytime soon after the immediate crisis is over, I want to at least design my interfaces with that in mind.

Also, I mentioned SWIG in passing the other day in a previous message.  Would it not be possible to just generate the bindings for Ruby with SWIG?

I did a couple of Ruby/GTK+ bindings (GtkHTML3 and WebKit), and the issue there is that it was all hand-coded C.  Trying to track a fast moving target like I was with WebKit proved to be nearly impossible, and I eventually gave up.  In this case, there wasn't any other way to do it because the rest of the bindings for GLib/GTK+/GNOME were all hand-coded, but I wouldn't ever want to go down that path again. ;)

Thanks in advance,

ast
--
Andrew S. Townley <[hidden email]>
http://atownley.org

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Question about query parsing API

Jens Wille
hi andrew!

i can't comment on lucy in particular as i just started following
its development only recently. but...

Andrew S. Townley [2011-02-25 13:41]:
> If so, is there a way to traverse the query parse tree (or get a
> callback notification when it matches terms)?  What I need to be
> able to do is take an arbitrary fulltext search query and then
> determine whether any search fields have been requested
> explicitly.  I suppose a more ideal solution would be to have a
> #target_fields method or something on the query object, but that
> would just be convenience.
i understand that problem has been solved in the meantime
(Ferret::Search::Query#terms, basically)? ;-)

> Also, I mentioned SWIG in passing the other day in a previous
> message.  Would it not be possible to just generate the bindings
> for Ruby with SWIG?
re SWIG, there was a clarifying statement by marvin humphrey on
lucy-dev recently:

<http://mail-archives.apache.org/mod_mbox/incubator-lucy-dev/201102.mbox/%3C20110221215214.GA27653@...%3E>

cheers
jens

--
Jens Wille, Dipl.-Bibl. (FH)
prometheus - Das verteilte digitale Bildarchiv für Forschung & Lehre
Kunsthistorisches Institut der Universität zu Köln
Albertus-Magnus-Platz, D-50923 Köln
Tel.: +49 (0)221 470-6219, E-Mail: [hidden email]
http://www.prometheus-bildarchiv.de/
Reply | Threaded
Open this post in threaded view
|

[lucy-user] Mostly re: the SWIG part (was Re: [lucy-user] Question about query parsing API

Andrew S. Townley
Hi Jens,

On 25 Feb 2011, at 1:41 PM, Jens Wille wrote:

>
> Andrew S. Townley [2011-02-25 13:41]:
>> If so, is there a way to traverse the query parse tree (or get a
>> callback notification when it matches terms)?  What I need to be
>> able to do is take an arbitrary fulltext search query and then
>> determine whether any search fields have been requested
>> explicitly.  I suppose a more ideal solution would be to have a
>> #target_fields method or something on the query object, but that
>> would just be convenience.
> i understand that problem has been solved in the meantime
> (Ferret::Search::Query#terms, basically)? ;-)

Yes, it seems like you did that as a byproduct of your change to Ferret (https://github.com/blackwinter/ferret).  Doubly awesome.  Thanks!

>> Also, I mentioned SWIG in passing the other day in a previous
>> message.  Would it not be possible to just generate the bindings
>> for Ruby with SWIG?
> re SWIG, there was a clarifying statement by marvin humphrey on
> lucy-dev recently:
>
> <http://mail-archives.apache.org/mod_mbox/incubator-lucy-dev/201102.mbox/%3C20110221215214.GA27653@...%3E>

Thanks for the pointer.  Archive failure on my part.  However, after reading that and this http://s.apache.org/pSd (referenced in the above), I understand the rationale, and think it's cool, but am slightly concerned about how easy it'll be to get it to generate Ruby.

To the larger Lucy community:

I can't find any documentation about Clownfish other than the above JIRA entry.  Is there some discussion/project/docs somewhere?

The advantages of SWIG are that it's pretty widely used, and anyone with experience with both the source and the target binding language can usually get something working in hours, not days.  It seems to me that for lucy to get the adoption it needs to grow communities in other circles besides Perl, you need to start doing some marketing about how much easier it is for Joe-random-developer to take a Clownfish-enabled project and automatically generate the language bindings of their choice.

As it stands, it seems that anyone interested in using lucy outside of Perl (or actually, just as a C library) needs to:

a) Learn Clownfish
b) Implement Clownfish -> Lang conversion routines (in Perl?)
c) Run it on the lucy source
d) write binding tests
e) tweak bindings and/or conversion, e.g. goto b
f) rinse & repeat

I really like the mechanisms you're using in Clownfish--I really do, but it isn't going to matter much (except for KinoSearch/Perl users) if there aren't people able to easily create language bindings--no matter how much more powerful the plumbing is than SWIG or other approaches.

Please understand I'm not trying to be negative here.  I really want Lucy to succeed in its goals.  I'm just trying to get a realistic picture of where things are and what's actually involved.  I think finishing the incubation process and getting more user and host language developer documentation in place (even a few example blog posts) would go a long way to attracting some support.  Right now, except for the mailing list, lucy's just a black box that sounds like it'll be cool--eventually (at least for non-Perl users). ;)

If these documentation resources, Clownfish intro materials or HOW-TO guides exist, then please point me to them.  Google wasn't much help here, and I've been through the SVN browser several times now with no luck.

It seems like the core architecture of Lucy and Clownfish are first-rate efforts.  Now, you need to think about the right way to market it so that you can build the community.  Again, it looks very much like Lucy is the long-term fit for what I'm working on, so I'm very interested in how things develop and what I might have to bite off in terms of additional work to integrate it at any level (C++ or Ruby) into what I have.

Thanks for your time,

ast
--
Andrew S. Townley <[hidden email]>
http://atownley.org

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Question about query parsing API

Marvin Humphrey
In reply to this post by Andrew S. Townley
On Fri, Feb 25, 2011 at 12:41:02PM +0000, Andrew S. Townley wrote:
> Another issue I just hit with Ferret (actually, the same root problem, but
> manifested in a different way) made me wonder something else about the
> design of lucy: the query parsing API.

I think there's a fundamental challenge with the proposed design
(Lucy/Ferret/Lucene doesn't preserve metadata during scoring) regardless of
which engine you choose, and I'll address it in a reply on the earlier thread.
 
> Is lucy's QueryParser API effectively this one?
>
> http://www.rectangular.com/kinosearch/docs/devel/KinoSearch/Search/QueryParser.html

Yes.
 
> If so, is there a way to traverse the query parse tree

It's not public API yet, but it can be done, and perhaps we should consider
making the API public.

ANDQuery, ORQuery, RequiredOptionalQuery and NOTQuery are all subclasses of
PolyQuery.  PolyQuery provides a PolyQuery_Get_Children() method (which would
be spelled get_children() in the Perl or Ruby-vaporware bindings).  Using
that, you can traverse the hierarchy.  Indeed, that's what QueryParser does
internally.

> Also, I mentioned SWIG in passing the other day in a previous message.
> Would it not be possible to just generate the bindings for Ruby with SWIG?

SWIG won't do:

    http://lucy.markmail.org/thread/5uxmc655dvzzdpvx

There are portions of Lucy that have been intentionally left unimplemented by
the core.  The Perl implementation code is located in trunk/perl/xs/ and
trunk/perl/lib/Lucy.pm.  This code will have to be ported for each new host
language regardless.

Once that's done, it *might* be theoretically possible to generate SWIG
bindings as a short-term experiment, but there would be a lot of problems.
Lucy's autogenerated header files won't map well.  There will be some quirks
that would need to be worked out regarding Lucy's object model.  Lots of
features will be missing -- subclassing, automated refcount management,
default parameter values, etc.  It will also be quite unwieldy, because
hashes, arrays, and strings won't get automatically converted at the binding
barrier -- you'll have to do crazy stuff like creating Lucy::Object::CharBuf
objects every time you want to pass a string into the Lucy core.

What is planned instead is to adapt the materials under
trunk/clownfish/lib/Clownfish/Binding/Perl to generate Ruby C API code instead
of Perl C API code.  There's actually not a lot there:

    $ wc -l lib/Clownfish/Binding/Perl.pm lib/Clownfish/Binding/Perl/*
         528 lib/Clownfish/Binding/Perl.pm
         475 lib/Clownfish/Binding/Perl/Class.pm
         150 lib/Clownfish/Binding/Perl/Constructor.pm
         277 lib/Clownfish/Binding/Perl/Method.pm
         269 lib/Clownfish/Binding/Perl/Subroutine.pm
         298 lib/Clownfish/Binding/Perl/TypeMap.pm
        1997 total
    $

Most of the work the Clownfish compiler does involves parsing the Lucy header
files and building a model of the Lucy object hierarchy in memory.  That work
is done.  What's left is to port the code that walks that object hierarchy and
generates binding code.  We have such code for Perl; we need to adapt it for
Ruby.

Once that work is done, it's done.  Changes within Lucy's core don't require
changes to Clownfish.

I'm actively working on the Clownfish code now -- adding host languages is a
force-multiplier for the project, so it's a high priority.  See the roadmap at
<http://markmail.org/thread/nfqfphjigqcl2svc>.

I've been vacillating between Python and Ruby as far as which bindings to work
on next, but I tend to go where there are active collaborators.  If you're
interested in contributing, you'll have company. :)

At the least, I intend to finish porting the bulk of the Clownfish compiler
from Perl to C so that it's easier for non-Perl people to grok.

> I did a couple of Ruby/GTK+ bindings (GtkHTML3 and WebKit), and the issue
> there is that it was all hand-coded C.  Trying to track a fast moving target
> like I was with WebKit proved to be nearly impossible, and I eventually gave
> up.  

Right, absolutely.  We've been through the same slog, and now we autogenerate
the vast majority of our binding code.

Marvin Humphrey

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Mostly re: the SWIG part (was Re: [lucy-user] Question about query parsing API

Nathan Kurz
In reply to this post by Andrew S. Townley
On Fri, Feb 25, 2011 at 6:56 AM, Andrew S. Townley <[hidden email]> wrote:
> The advantages of SWIG are that it's pretty widely used, and anyone with experience with both the source and the target binding language can usually get something working in hours, not days.  It seems to me that for lucy to get the adoption it needs to grow communities in other circles besides Perl, you need to start doing some marketing about how much easier it is for Joe-random-developer to take a Clownfish-enabled project and automatically generate the language bindings of their choice.

As Marvin's lost post claimed, SWIG is not really a good fit here.

> a) Learn Clownfish
> b) Implement Clownfish -> Lang conversion routines (in Perl?)
> c) Run it on the lucy source
> d) write binding tests
> e) tweak bindings and/or conversion, e.g. goto b
> f) rinse & repeat

Essentially correct, although it will soon be possible to do this with
only C and the target language (no Perl required).  Also on the bright
side, most of the heavy lifting only has to be once for Clownfish,
rather than once per project.  Unfortunately, this lifting is heavy
and has yet to be done, and will initially require working closely
with Marvin.

Realistically, your hypothetical Joe-random-developer isn't going to
be able to do this.  While turnkey documentation would help, that time
would probably be better spent just making/polishing the bindings
themselves for the most likely languages, which I'd guess would be
Perl, Ruby, and Python.  The full documentation can follow after these
are solid.

> I really like the mechanisms you're using in Clownfish--I really do, but it isn't going to matter much (except for KinoSearch/Perl users) if there aren't people able to easily create language bindings--no matter how much more powerful the plumbing is than SWIG or other approaches.

I think you're on the right path, but the bar is actually a bit
higher.  Lucy is not going to get significant traction in any language
until the bindings are sufficiently solid that Clownfish doesn't even
come up.  I think it's a great tool for the purpose, but it's a core
developers tool rather than an end-user tool.  It's more akin to
porting SWIG to a new language than merely adding a SWIG interface.

What's needed is a few more core developers (such as yourself) with a
strong language-specific itch and a willingness to dive into the C.
Once that works done (once), Lucy should start to spread pretty
easily.

Nathan Kurz
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Question about query parsing API

Andrew S. Townley
In reply to this post by Marvin Humphrey
Hi Marvin,

On 25 Feb 2011, at 4:05 PM, Marvin Humphrey wrote:

> On Fri, Feb 25, 2011 at 12:41:02PM +0000, Andrew S. Townley wrote:
>> Another issue I just hit with Ferret (actually, the same root problem, but
>> manifested in a different way) made me wonder something else about the
>> design of lucy: the query parsing API.
>
> I think there's a fundamental challenge with the proposed design
> (Lucy/Ferret/Lucene doesn't preserve metadata during scoring) regardless of
> which engine you choose, and I'll address it in a reply on the earlier thread.

Will address this in separate reply.

>> Is lucy's QueryParser API effectively this one?
>>
>> http://www.rectangular.com/kinosearch/docs/devel/KinoSearch/Search/QueryParser.html
>
> Yes.
>
>> If so, is there a way to traverse the query parse tree
>
> It's not public API yet, but it can be done, and perhaps we should consider
> making the API public.
>
> ANDQuery, ORQuery, RequiredOptionalQuery and NOTQuery are all subclasses of
> PolyQuery.  PolyQuery provides a PolyQuery_Get_Children() method (which would
> be spelled get_children() in the Perl or Ruby-vaporware bindings).  Using
> that, you can traverse the hierarchy.  Indeed, that's what QueryParser does
> internally.

At the very least, I need to be able to walk the query tree for any string input query (or any query object) with a consistent API.  What you have here is pretty similar to my own implementation of the Query Object pattern for my system, so that would be a start.  For any compound query term, I'm also "bubbling" up references to the property names as well as the query terms themselves.  This means that I can retrieve these easily and do some analysis on the query before actually executing it.

Anything that will support me doing the same type of thing with Lucy will work.

>
>> Also, I mentioned SWIG in passing the other day in a previous message.
>> Would it not be possible to just generate the bindings for Ruby with SWIG?
>
> SWIG won't do:
>
>    http://lucy.markmail.org/thread/5uxmc655dvzzdpvx

Yeah, I got that and the rationale from Jens' reference.

> There are portions of Lucy that have been intentionally left unimplemented by
> the core.  The Perl implementation code is located in trunk/perl/xs/ and
> trunk/perl/lib/Lucy.pm.  This code will have to be ported for each new host
> language regardless.

Interesting approach.  Is there some docs/rationale on which parts and why somewhere?  Sounds worth understanding in more detail.

> Once that's done, it *might* be theoretically possible to generate SWIG
> bindings as a short-term experiment, but there would be a lot of problems.
> Lucy's autogenerated header files won't map well.  There will be some quirks
> that would need to be worked out regarding Lucy's object model.  Lots of
> features will be missing -- subclassing, automated refcount management,
> default parameter values, etc.  It will also be quite unwieldy, because
> hashes, arrays, and strings won't get automatically converted at the binding
> barrier -- you'll have to do crazy stuff like creating Lucy::Object::CharBuf
> objects every time you want to pass a string into the Lucy core.

Yeah, ugh.

> What is planned instead is to adapt the materials under
> trunk/clownfish/lib/Clownfish/Binding/Perl to generate Ruby C API code instead
> of Perl C API code.  There's actually not a lot there:
>
>    $ wc -l lib/Clownfish/Binding/Perl.pm lib/Clownfish/Binding/Perl/*
>         528 lib/Clownfish/Binding/Perl.pm
>         475 lib/Clownfish/Binding/Perl/Class.pm
>         150 lib/Clownfish/Binding/Perl/Constructor.pm
>         277 lib/Clownfish/Binding/Perl/Method.pm
>         269 lib/Clownfish/Binding/Perl/Subroutine.pm
>         298 lib/Clownfish/Binding/Perl/TypeMap.pm
>        1997 total
>    $
>
> Most of the work the Clownfish compiler does involves parsing the Lucy header
> files and building a model of the Lucy object hierarchy in memory.  That work
> is done.  What's left is to port the code that walks that object hierarchy and
> generates binding code.  We have such code for Perl; we need to adapt it for
> Ruby.
>
> Once that work is done, it's done.  Changes within Lucy's core don't require
> changes to Clownfish.
>
> I'm actively working on the Clownfish code now -- adding host languages is a
> force-multiplier for the project, so it's a high priority.  See the roadmap at
> <http://markmail.org/thread/nfqfphjigqcl2svc>.
>
> I've been vacillating between Python and Ruby as far as which bindings to work
> on next, but I tend to go where there are active collaborators.  If you're
> interested in contributing, you'll have company. :)
>
> At the least, I intend to finish porting the bulk of the Clownfish compiler
> from Perl to C so that it's easier for non-Perl people to grok.

That's sounds good, and the roadmap makes sense.  Have since subscribed to lucy-devel as well since a lot of the traffic there seems to address the kinds of things I'm interested in.

Thanks for the explanation.

ast
--
Andrew S. Townley <[hidden email]>
http://atownley.org

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Mostly re: the SWIG part (was Re: [lucy-user] Question about query parsing API

Andrew S. Townley
In reply to this post by Nathan Kurz

On 25 Feb 2011, at 7:45 PM, Nathan Kurz wrote:

> On Fri, Feb 25, 2011 at 6:56 AM, Andrew S. Townley <[hidden email]> wrote:
>> The advantages of SWIG are that it's pretty widely used, and anyone with experience with both the source and the target binding language can usually get something working in hours, not days.  It seems to me that for lucy to get the adoption it needs to grow communities in other circles besides Perl, you need to start doing some marketing about how much easier it is for Joe-random-developer to take a Clownfish-enabled project and automatically generate the language bindings of their choice.
>
> As Marvin's lost post claimed, SWIG is not really a good fit here.

I wasn't really advocating SWIG as a solution with the above; more that it's a known beast and people can create bindings for something they don't know the internals for relatively easily.  I get why you've gone with something like Clownfish.

>> a) Learn Clownfish
>> b) Implement Clownfish -> Lang conversion routines (in Perl?)
>> c) Run it on the lucy source
>> d) write binding tests
>> e) tweak bindings and/or conversion, e.g. goto b
>> f) rinse & repeat
>
> Essentially correct, although it will soon be possible to do this with
> only C and the target language (no Perl required).  Also on the bright
> side, most of the heavy lifting only has to be once for Clownfish,
> rather than once per project.  Unfortunately, this lifting is heavy
> and has yet to be done, and will initially require working closely
> with Marvin.
>
> Realistically, your hypothetical Joe-random-developer isn't going to
> be able to do this.  While turnkey documentation would help, that time
> would probably be better spent just making/polishing the bindings
> themselves for the most likely languages, which I'd guess would be
> Perl, Ruby, and Python.  The full documentation can follow after these
> are solid.
>
>> I really like the mechanisms you're using in Clownfish--I really do, but it isn't going to matter much (except for KinoSearch/Perl users) if there aren't people able to easily create language bindings--no matter how much more powerful the plumbing is than SWIG or other approaches.
>
> I think you're on the right path, but the bar is actually a bit
> higher.  Lucy is not going to get significant traction in any language
> until the bindings are sufficiently solid that Clownfish doesn't even
> come up.  I think it's a great tool for the purpose, but it's a core
> developers tool rather than an end-user tool.  It's more akin to
> porting SWIG to a new language than merely adding a SWIG interface.
>
> What's needed is a few more core developers (such as yourself) with a
> strong language-specific itch and a willingness to dive into the C.
> Once that works done (once), Lucy should start to spread pretty
> easily.


I get what you're saying, but here's the rub for me:  I'm not really a developer anymore--I just play one on TV. ;)  At the moment, I'm wearing about 10 different hats, and I'm trying to stabilize my own platform so I can divest myself of as many development responsibilities as possible.  If I don't, I won't be able to do a lot of the other work that I need to be doing.

I'm confident that I *could* do the Ruby bindings, I'm just not sure that taking on yet another development task is very smart right now.  Puts me in a rather unpleasant dilemma, actually, because I want to help, and I suspect that I'm going to hit more issues with Ferret as I go along since it doesn't seem to be designed exactly for how I want/need to use it.  Unfortunately, I can't make any more promises at the moment. :(

ast
--
Andrew S. Townley <[hidden email]>
http://atownley.org

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Mostly re: the SWIG part (was Re: [lucy-user] Question about query parsing API

Peter Karman
Andrew S. Townley wrote on 2/26/11 7:26 AM:

> I get what you're saying, but here's the rub for me:  I'm not really a
> developer anymore--I just play one on TV. ;)  At the moment, I'm wearing
> about 10 different hats, and I'm trying to stabilize my own platform so I can
> divest myself of as many development responsibilities as possible.  If I
> don't, I won't be able to do a lot of the other work that I need to be
> doing.
>
> I'm confident that I *could* do the Ruby bindings, I'm just not sure that
> taking on yet another development task is very smart right now.  Puts me in a
> rather unpleasant dilemma, actually, because I want to help, and I suspect
> that I'm going to hit more issues with Ferret as I go along since it doesn't
> seem to be designed exactly for how I want/need to use it.  Unfortunately, I
> can't make any more promises at the moment. :(
>

Completely fair and I appreciate your self-knowledge on this score.

Please do continue to lurk, at least, and consider reviewing the Ruby binding
RFCs when they emerge. One of our primary goals is idiomatically consistent APIs
for each host language, and it helps immensely to have "native speakers"
reviewing the idioms. :)

pek

--
Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Question about query parsing API

Peter Karman
In reply to this post by Andrew S. Townley
Andrew S. Townley wrote on 2/26/11 6:55 AM:

>
> At the very least, I need to be able to walk the query tree for any string
> input query (or any query object) with a consistent API.  What you have here
> is pretty similar to my own implementation of the Query Object pattern for my
> system, so that would be a start.  For any compound query term, I'm also
> "bubbling" up references to the property names as well as the query terms
> themselves.  This means that I can retrieve these easily and do some analysis
> on the query before actually executing it.
>
> Anything that will support me doing the same type of thing with Lucy will
> work.
>

It's not core to Lucy, but Search::Query::Dialect has a KSx (KinoSearch
extension) implementation that makes it easy to walk the tree:

http://search.cpan.org/~karman/Search-Query-0.18/lib/Search/Query/Dialect.pm#walk(_CODE_)

>
>> There are portions of Lucy that have been intentionally left unimplemented
>> by the core.  The Perl implementation code is located in trunk/perl/xs/
>> and trunk/perl/lib/Lucy.pm.  This code will have to be ported for each new
>> host language regardless.
>
> Interesting approach.  Is there some docs/rationale on which parts and why
> somewhere?  Sounds worth understanding in more detail.

Marvin can answer as to whether there are docs on this; my understanding of the
rationale is that since our goal is idiomatic language implementations on top of
the underlying C, each host language must do *some* work.

Lucy isn't a C library in the traditional sense; it's more like a
some-assembly-required C framework for writing an IR library in a dynamic
language. The C code handles the heavy-lifting bits that are too
resource-intensive to be practical in the host language. Then each host language
must glue it all together. Clownfish provides a way of generating (most of) that
glue.

--
Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Question about query parsing API

Marvin Humphrey
On Mon, Feb 28, 2011 at 09:20:46PM -0600, Peter Karman wrote:
> > Interesting approach.  Is there some docs/rationale on which parts and why
> > somewhere?  Sounds worth understanding in more detail.
>
> Marvin can answer as to whether there are docs on this; my understanding of the
> rationale is that since our goal is idiomatic language implementations on top of
> the underlying C, each host language must do *some* work.

I've replied to this on lucy-dev, under the subject heading "Per-host abstract
elements".

Marvin Humphrey