[lucy-user] Installation location weirdness under daemon for Lucy libs

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[lucy-user] Installation location weirdness under daemon for Lucy libs

Ashley Pond V
I am in the process of porting an old (four years) KinoSearch
application to Lucy; from scratch, not drop in, but starting from
small parts and building back up.

I have a stubbed indexer script working appropriately *until* I
daemonize it with Proc::Daemon::Init(), When I do that, a single index
pass/commit or an index pass with truncate => 1 works still but if it
has to build on the current index instead, it bombs out with a
memory/paging/bad-file error from a peculiar location for the Lucy
code; I think it's the build dir.

FATAL - mmap of offset 0 and length 2044 (page size 4096) against
'/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/cf.dat' failed: Bad
file descriptor
(/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/lexicon-28.ix)
        S_fill at /root/Lucy-0.3.3/core/Lucy/Store/InStream.c line 206
        at /usr/local/apv_4045_2/hss/lib/StudyShare/Lucy/Indexer.pm line 152

It says Lucy code is in /root/ (where it was built and installed per
the INSTALL doc that comes with the dist). But the code was installed
in the regular site_perl tree-

  perldoc -l Lucy
  /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/Lucy.pod

If it's relevant the actual call that is throwing the error is-

    $self->delete_by_term( field => "id", term => $case->id );

If it is commented out, it just moves the error to the ->commit call
but is still related to looking in the wrong place for the Lucy code-

FATAL - mmap of offset 4096 and length 656 (page size 4096) against
'/usr/local/apv_4045_2/hss/var/lucy/cases/seg_7/cf.dat' failed:
Permission denied
(/usr/local/apv_4045_2/hss/var/lucy/cases/seg_7/lexicon-2.ixix)
        S_fill at /root/Lucy-0.3.3/core/Lucy/Store/InStream.c line 206

The self gives the clue that I'm subclassing-
   use base "Lucy::Index::Indexer";

Not sure if that's become a no-no. It worked with KinoSearch.

Any ideas? Why is it looking in /root/? The script is not run as root.
What is it about daemonization is making it wonky?

Thank you!
-Ashley
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Peter Karman
On 12/5/13 1:37 PM, Ashley Pond V wrote:
> I am in the process of porting an old (four years) KinoSearch
> application to Lucy; from scratch, not drop in, but starting from
> small parts and building back up.
>

What version of KS?

I expect you're seeing the results of Lucy being most C now, and so the
.c -based messages are porting the compile-time directory. Has nothing
to do with daemonization.

Are you by any chance running more than one indexer at a time?


--
Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Ashley Pond V
No, just one indexer. The locks and such are working fine when I have
accidentally launched two. And I'm not using any KinoSearch code at
all in the rebuild so I think it's not relevant? It was 0.315 if it
somehow is.

The code I'm working with is 100% Lucy (so far, I haven't added in any
custom parts or my new Analyzer yet). Perl 5.8.8 (thanks CentOS), but
will be moving to a more reasonable/newer one soon.

Thanks, Peter!

On Thu, Dec 5, 2013 at 11:46 AM, Peter Karman <[hidden email]> wrote:

> On 12/5/13 1:37 PM, Ashley Pond V wrote:
>>
>> I am in the process of porting an old (four years) KinoSearch
>> application to Lucy; from scratch, not drop in, but starting from
>> small parts and building back up.
>>
>
> What version of KS?
>
> I expect you're seeing the results of Lucy being most C now, and so the .c
> -based messages are porting the compile-time directory. Has nothing to do
> with daemonization.
>
> Are you by any chance running more than one indexer at a time?
>
>
> --
> Peter Karman  .  http://peknet.com/  .  [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Nick Wellnhofer
In reply to this post by Ashley Pond V
On Dec 5, 2013, at 20:37 , Ashley Pond V <[hidden email]> wrote:

> FATAL - mmap of offset 0 and length 2044 (page size 4096) against
> '/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/cf.dat' failed: Bad
> file descriptor
> (/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/lexicon-28.ix)
>        S_fill at /root/Lucy-0.3.3/core/Lucy/Store/InStream.c line 206
>        at /usr/local/apv_4045_2/hss/lib/StudyShare/Lucy/Indexer.pm line 152
>
> It says Lucy code is in /root/ (where it was built and installed per
> the INSTALL doc that comes with the dist). But the code was installed
> in the regular site_perl tree-

That’s normal. The location of the C source files for stack traces is recorded at compile time.

Nick

Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Ashley Pond V
The script is not compiling with that path at all though. The @INC has
no entry for it, I double checked when this first happened. ???


On Thu, Dec 5, 2013 at 12:14 PM, Nick Wellnhofer <[hidden email]> wrote:

> On Dec 5, 2013, at 20:37 , Ashley Pond V <[hidden email]> wrote:
>
>> FATAL - mmap of offset 0 and length 2044 (page size 4096) against
>> '/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/cf.dat' failed: Bad
>> file descriptor
>> (/usr/local/apv_4045_2/hss/var/lucy/cases/seg_3/lexicon-28.ix)
>>        S_fill at /root/Lucy-0.3.3/core/Lucy/Store/InStream.c line 206
>>        at /usr/local/apv_4045_2/hss/lib/StudyShare/Lucy/Indexer.pm line 152
>>
>> It says Lucy code is in /root/ (where it was built and installed per
>> the INSTALL doc that comes with the dist). But the code was installed
>> in the regular site_perl tree-
>
> That’s normal. The location of the C source files for stack traces is recorded at compile time.
>
> Nick
>
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Marvin Humphrey
In reply to this post by Ashley Pond V
On Thu, Dec 5, 2013 at 11:37 AM, Ashley Pond V <[hidden email]> wrote:
> FATAL - mmap of offset 4096 and length 656 (page size 4096) against
> '/usr/local/apv_4045_2/hss/var/lucy/cases/seg_7/cf.dat' failed:
> Permission denied
  ^^^^^^^^^^ ^^^^^^

> What is it about daemonization is making it wonky?

Underneath all the stack traces, it seems like there's a permissions issue.

I don't recall whether we've tightened up permissions since the version of
KinoSearch you're using, but I'd look at owner/group for the index and the
what user you're running under as a daemon.

Marvin Humphrey
Reply | Threaded
Open this post in threaded view
|

Re: [lucy-user] Installation location weirdness under daemon for Lucy libs

Ashley Pond V
Thanks all. It seems that I'm running an older version of Proc::Daemon
that is not well behaved in combination with Lucy. A reduced bug test
shows it working right with an updated Proc::Daemon so that's probably
good enough. Thanks everyone for looking.

On Thu, Dec 5, 2013 at 1:05 PM, Marvin Humphrey <[hidden email]> wrote:

> On Thu, Dec 5, 2013 at 11:37 AM, Ashley Pond V <[hidden email]> wrote:
>> FATAL - mmap of offset 4096 and length 656 (page size 4096) against
>> '/usr/local/apv_4045_2/hss/var/lucy/cases/seg_7/cf.dat' failed:
>> Permission denied
>   ^^^^^^^^^^ ^^^^^^
>
>> What is it about daemonization is making it wonky?
>
> Underneath all the stack traces, it seems like there's a permissions issue.
>
> I don't recall whether we've tightened up permissions since the version of
> KinoSearch you're using, but I'd look at owner/group for the index and the
> what user you're running under as a daemon.
>
> Marvin Humphrey