snapshooter no go

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

snapshooter no go

Otis Gospodnetic-2
Hi,

Here is a puzzling one.  I can't get Solr to invoke snaphooter properly.  Solr claims my snapshooter is not where I said it is:

SEVERE: java.io.IOException: Cannot run program "snapshooter" (in
directory "solr/bin"): java.io.IOException: error=2, No such file or
directory


So I added some logging to RunExecutableListener:

INFO: CMD: snapshooter
INFO: ENV: MYVAR=val1
INFO: DIR: /foobar/jetty-6.1.3/solr/bin

This looks correct, and I can see snapshooter is there:

$ cd /foo/jetty-6.1.3
$ ls -al solr/bin/snapshooter
-rwxr-xr-x 1 otis otis 2624 Jun 20 02:53 solr/bin/snapshooter

So snapshooter *does* exist under /foobar/jetty-6.1.3/solr/bin and is executable by everyone.  I've tried moving solr home dir outside the jetty dir, but that made no difference.  I tried specifying the absolute path to the "dir" parameter in the config for snapshooter in solrconfig.xml, again no luck.

Huh?  Is this some new bug?  Has anyone seen this before?  I'm using the HEAD.

Thanks,
Otis


Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

James liu-2
first try it? which system u use?

if u use freebsd, just give up trying. it not fit for freebsd.


2007/6/27, Otis Gospodnetic <[hidden email]>:

>
> Hi,
>
> Here is a puzzling one.  I can't get Solr to invoke snaphooter
> properly.  Solr claims my snapshooter is not where I said it is:
>
> SEVERE: java.io.IOException: Cannot run program "snapshooter" (in
> directory "solr/bin"): java.io.IOException: error=2, No such file or
> directory
>
>
> So I added some logging to RunExecutableListener:
>
> INFO: CMD: snapshooter
> INFO: ENV: MYVAR=val1
> INFO: DIR: /foobar/jetty-6.1.3/solr/bin
>
> This looks correct, and I can see snapshooter is there:
>
> $ cd /foo/jetty-6.1.3
> $ ls -al solr/bin/snapshooter
> -rwxr-xr-x 1 otis otis 2624 Jun 20 02:53 solr/bin/snapshooter
>
> So snapshooter *does* exist under /foobar/jetty-6.1.3/solr/bin and is
> executable by everyone.  I've tried moving solr home dir outside the jetty
> dir, but that made no difference.  I tried specifying the absolute path to
> the "dir" parameter in the config for snapshooter in solrconfig.xml, again
> no luck.
>
> Huh?  Is this some new bug?  Has anyone seen this before?  I'm using the
> HEAD.
>
> Thanks,
> Otis
>
>
>


--
regards
jl
Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Chris Hostetter-3
In reply to this post by Otis Gospodnetic-2

: Here is a puzzling one.  I can't get Solr to invoke snaphooter properly.
: Solr claims my snapshooter is not where I said it is:
:
: SEVERE: java.io.IOException: Cannot run program "snapshooter" (in
: directory "solr/bin"): java.io.IOException: error=2, No such file or
: directory

off the top of my head, i suspect the problem may be that there is not
PATH env var so it can't find "snapshooter" but it might be able to find
"./snapshooter"

either that or...

: $ cd /foo/jetty-6.1.3
: $ ls -al solr/bin/snapshooter
: -rwxr-xr-x 1 otis otis 2624 Jun 20 02:53 solr/bin/snapshooter

...what does "ls -ald solr solr/bin" tell you?  are the directories world
executable (assuming jetty isn't running as otis)




-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Bill Au
I would also check

ls -ld /foo/jetty-6.1.3

too.

Bill

On 6/27/07, Chris Hostetter <[hidden email]> wrote:

>
>
> : Here is a puzzling one.  I can't get Solr to invoke snaphooter properly.
> : Solr claims my snapshooter is not where I said it is:
> :
> : SEVERE: java.io.IOException: Cannot run program "snapshooter" (in
> : directory "solr/bin"): java.io.IOException: error=2, No such file or
> : directory
>
> off the top of my head, i suspect the problem may be that there is not
> PATH env var so it can't find "snapshooter" but it might be able to find
> "./snapshooter"
>
> either that or...
>
> : $ cd /foo/jetty-6.1.3
> : $ ls -al solr/bin/snapshooter
> : -rwxr-xr-x 1 otis otis 2624 Jun 20 02:53 solr/bin/snapshooter
>
> ...what does "ls -ald solr solr/bin" tell you?  are the directories world
> executable (assuming jetty isn't running as otis)
>
>
>
>
> -Hoss
>
>
Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Otis Gospodnetic-2
In reply to this post by Otis Gospodnetic-2
PATH - bingo!
The config + the log message are a bit misleading there, IMHO.

The "dir" is documented as:
  dir - dir to use as the current working directory. default="."

And its default value is defined as:
  <str name="dir">solr/bin</str>

The error was:
   Cannot run program "snapshooter" (in directory "/foo/jetty-6.1.3/solr/bin"): java.io.IOException: error=2, No such file or directory

All of this makes you (me) think that Solr uses the dir + exe name to call snapshooter with the full path, a la "/foo/jetty-6.1.3/solr/bin/snapshooter", which must not be the case.

Moreover, it looks like even though I set Solr home to be outside Jetty directory, one has to put the dir that is the Solr home dir under Jetty, or at least link <jetty>/solr/bin to <solr home>/bin, OR change that "dir" to be ../solr/bin (but this didn't work without adding /full/path/to/solr/bin to PATH).

Otis



----- Original Message ----
From: Chris Hostetter <[hidden email]>
To: [hidden email]
Sent: Wednesday, June 27, 2007 7:20:28 AM
Subject: Re: snapshooter no go


: Here is a puzzling one.  I can't get Solr to invoke snaphooter properly.
: Solr claims my snapshooter is not where I said it is:
:
: SEVERE: java.io.IOException: Cannot run program "snapshooter" (in
: directory "solr/bin"): java.io.IOException: error=2, No such file or
: directory

off the top of my head, i suspect the problem may be that there is not
PATH env var so it can't find "snapshooter" but it might be able to find
"./snapshooter"

either that or...

: $ cd /foo/jetty-6.1.3
: $ ls -al solr/bin/snapshooter
: -rwxr-xr-x 1 otis otis 2624 Jun 20 02:53 solr/bin/snapshooter

...what does "ls -ald solr solr/bin" tell you?  are the directories world
executable (assuming jetty isn't running as otis)




-Hoss




Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Chris Hostetter-3

: The "dir" is documented as:
:   dir - dir to use as the current working directory. default="."
:
: And its default value is defined as:
:   <str name="dir">solr/bin</str>
:
: The error was:
:    Cannot run program "snapshooter" (in directory "/foo/jetty-6.1.3/solr/bin"): java.io.IOException: error=2, No such file or directory
:
: All of this makes you (me) think that Solr uses the dir + exe name to
: call snapshooter with the full path, a la
: "/foo/jetty-6.1.3/solr/bin/snapshooter", which must not be the case.

no, the RunExecuterListern executes the exe specified usinbg the CWD of
the dir specified ... unless you set some env options, you can mimic
exactly what it's doing by CDing to the directory logged, and running "env
-i [exex name] [your args]

we could probably cahnge the log message to be more explicit about what it
tried to do.

: Moreover, it looks like even though I set Solr home to be outside Jetty
: directory, one has to put the dir that is the Solr home dir under Jetty,
: or at least link <jetty>/solr/bin to <solr home>/bin, OR change that
: "dir" to be ../solr/bin (but this didn't work without adding
: /full/path/to/solr/bin to PATH).

I'm not sure i understand ... part of hte confusion here may be that i
don't think the "dir" directory of the executor is resolved relative
solr.home ... it's relative the CWD of teh servlet container.

ideally we should document that it should be an absolute path whenever
possible ... but we can't do that in the example.



-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
I got similar problems, tried both default setting solr/bin and full
path /export/home/jboss/jboss-4.0.5.GA/bin/solr/bin, neither works. I'm
using 1.2.


2007-06-27 14:10:03,907 ERROR [STDERR] Jun 27, 2007 2:10:03 PM
org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=true,waitFlush=true,waitSearcher=true)
2007-06-27 14:10:03,961 ERROR [STDERR] Jun 27, 2007 2:10:03 PM
org.apache.solr.core.SolrException log
SEVERE: java.io.IOException: snapshooter: not found
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
        at java.lang.ProcessImpl.start(ProcessImpl.java:65)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
        at java.lang.Runtime.exec(Runtime.java:591)
        at
org.apache.solr.core.RunExecutableListener.exec(RunExecutableListener
.java:70)
        at
org.apache.solr.core.RunExecutableListener.postCommit(RunExecutableLi
stener.java:97)
        at
org.apache.solr.update.UpdateHandler.callPostCommitCallbacks(UpdateHa
ndler.java:99)


Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Chris Hostetter-3

: I got similar problems, tried both default setting solr/bin and full
: path /export/home/jboss/jboss-4.0.5.GA/bin/solr/bin, neither works. I'm
: using 1.2.

Did you try setting "exe" to "./snapshooter" ?

for the record, there is no "default" setting of "solr/bin" in either the
code, or in the example solrconfig ... there is only a commented out
example (which is clearly a bad example based on the confusion it seems to
be causing and the fact that if you uncomment it, it doesn't work
properly)


-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
I thought I had to use the full path in dir attribute, later I realized
I should modify the environment variable path. Now it's working, I
didn't append ./ before snapshooter.

Thanks
Xuesong

-----Original Message-----
From: Chris Hostetter [mailto:[hidden email]]
Sent: Wednesday, June 27, 2007 2:49 PM
To: [hidden email]
Subject: RE: snapshooter no go


: I got similar problems, tried both default setting solr/bin and full
: path /export/home/jboss/jboss-4.0.5.GA/bin/solr/bin, neither works.
I'm
: using 1.2.

Did you try setting "exe" to "./snapshooter" ?

for the record, there is no "default" setting of "solr/bin" in either
the
code, or in the example solrconfig ... there is only a commented out
example (which is clearly a bad example based on the confusion it seems
to
be causing and the fact that if you uncomment it, it doesn't work
properly)


-Hoss


Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong


Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
Just found I can use "ln dir1/* dir2" to make hard link for all files in
dir1 to dir2.

-----Original Message-----
From: Xuesong Luo
Sent: Thursday, June 28, 2007 11:17 AM
To: [hidden email]
Subject: RE: snapshooter no go

I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong



Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Otis Gospodnetic-2
In reply to this post by Otis Gospodnetic-2
Hoss, I think the example solrconfig.xml does document a default "dir" value:

dir - dir to use as the current working directory. default="."
 
Otis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Chris Hostetter <[hidden email]>
To: [hidden email]
Sent: Wednesday, June 27, 2007 11:49:05 PM
Subject: RE: snapshooter no go


: I got similar problems, tried both default setting solr/bin and full
: path /export/home/jboss/jboss-4.0.5.GA/bin/solr/bin, neither works. I'm
: using 1.2.

Did you try setting "exe" to "./snapshooter" ?

for the record, there is no "default" setting of "solr/bin" in either the
code, or in the example solrconfig ... there is only a commented out
example (which is clearly a bad example based on the confusion it seems to
be causing and the fact that if you uncomment it, it doesn't work
properly)


-Hoss




Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Otis Gospodnetic-2
In reply to this post by Otis Gospodnetic-2
Maybe you can open an issue in JIRA with your solution, so Bill Au & Co. can fix snappuller.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Xuesong Luo <[hidden email]>
To: [hidden email]
Sent: Thursday, June 28, 2007 8:42:25 PM
Subject: RE: snapshooter no go

Just found I can use "ln dir1/* dir2" to make hard link for all files in
dir1 to dir2.

-----Original Message-----
From: Xuesong Luo
Sent: Thursday, June 28, 2007 11:17 AM
To: [hidden email]
Subject: RE: snapshooter no go

I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong






Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Otis Gospodnetic-2
In reply to this post by Otis Gospodnetic-2
Look at <solr home>/logs directory - you should see snapshooter.log there.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Xuesong Luo <[hidden email]>
To: [hidden email]
Sent: Thursday, June 28, 2007 8:17:14 PM
Subject: RE: snapshooter no go

I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong





Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Otis Gospodnetic-2
In reply to this post by Otis Gospodnetic-2
I'll read your email more carefully next time.  Yes, solr/bin part is commented out in the example solrconfig.xml.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Otis Gospodnetic <[hidden email]>
To: [hidden email]
Sent: Friday, June 29, 2007 1:45:48 AM
Subject: Re: snapshooter no go

Hoss, I think the example solrconfig.xml does document a default "dir" value:

dir - dir to use as the current working directory. default="."
 
Otis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Chris Hostetter <[hidden email]>
To: [hidden email]
Sent: Wednesday, June 27, 2007 11:49:05 PM
Subject: RE: snapshooter no go


: I got similar problems, tried both default setting solr/bin and full
: path /export/home/jboss/jboss-4.0.5.GA/bin/solr/bin, neither works. I'm
: using 1.2.

Did you try setting "exe" to "./snapshooter" ?

for the record, there is no "default" setting of "solr/bin" in either the
code, or in the example solrconfig ... there is only a commented out
example (which is clearly a bad example based on the confusion it seems to
be causing and the fact that if you uncomment it, it doesn't work
properly)


-Hoss







Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
The log didn't have any error message and there is no
snapshot.20070628204821 directory generated.

2007/06/28 20:48:21 started by
2007/06/28 20:48:21 command:
/export/home/jboss/jboss-3.2.7/bin/solr/bin/snapshooter arg1 arg2
2007/06/28 20:48:21 taking snapshot
/export/home/jboss/jboss-3.2.7/bin/solr/data/snapshot.20070628204821
2007/06/28 20:48:21 ended (elapsed time:  sec)

Thanks
Xuesong

-----Original Message-----
From: Otis Gospodnetic [mailto:[hidden email]]
Sent: Thursday, June 28, 2007 4:53 PM
To: [hidden email]
Subject: Re: snapshooter no go

Look at <solr home>/logs directory - you should see snapshooter.log
there.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Xuesong Luo <[hidden email]>
To: [hidden email]
Sent: Thursday, June 28, 2007 8:17:14 PM
Subject: RE: snapshooter no go

I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong






Reply | Threaded
Open this post in threaded view
|

RE: snapshooter no go

Xuesong Luo
In reply to this post by Otis Gospodnetic-2
Indeed there is one more step: mkdir dir2. ln reports error if the
destination dir doesn't not exist.
I'll create an JIRA later.

Thanks
Xuesong

-----Original Message-----
From: Otis Gospodnetic [mailto:[hidden email]]
Sent: Thursday, June 28, 2007 4:52 PM
To: [hidden email]
Subject: Re: snapshooter no go

Maybe you can open an issue in JIRA with your solution, so Bill Au & Co.
can fix snappuller.

Otis
 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Simpy -- http://www.simpy.com/  -  Tag  -  Search  -  Share

----- Original Message ----
From: Xuesong Luo <[hidden email]>
To: [hidden email]
Sent: Thursday, June 28, 2007 8:42:25 PM
Subject: RE: snapshooter no go

Just found I can use "ln dir1/* dir2" to make hard link for all files in
dir1 to dir2.

-----Original Message-----
From: Xuesong Luo
Sent: Thursday, June 28, 2007 11:17 AM
To: [hidden email]
Subject: RE: snapshooter no go

I got another problem: solr is able to find snapshooter but didn't
generate any snapshot files after I updated the index. I checked the
log, everything looks fine, then I run snapshooter from command line. It
failed because solaris doesn't support -l option when using cp command.
I'm trying to find an alternative way but it seems the only option is to
loop through all files in that dir and use ln command to create hard
link for each of them. Does anyone know a better solution?

I'm also curious why there is no error log when solr failed running
snapshooter.

Thanks
Xuesong







Reply | Threaded
Open this post in threaded view
|

Re: snapshooter no go

Chris Hostetter-3
In reply to this post by Otis Gospodnetic-2

: Hoss, I think the example solrconfig.xml does document a default "dir" value:
:
: dir - dir to use as the current working directory. default="."

exactly ... the message i was replying to suggested that the default value
was "solr/bin" ... i just wanted to clarify that is not the default,
that's just a smaple value in the example solrconfig.xml file...

: : I got similar problems, tried both default setting solr/bin and full

: for the record, there is no "default" setting of "solr/bin" in either the
: code, or in the example solrconfig ... there is only a commented out
: example (which is clearly a bad example based on the confusion it seems to
: be causing and the fact that if you uncomment it, it doesn't work
: properly)


-Hoss