Delete then re-add a core

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Delete then re-add a core

Amanda Shuman
Hi all, I'm a bit of a newbie still but have clearly screwed something
up... so I think what I need to do now is to delete a core (saving current
conf files as-is) then re-add/re-create the core and re-index. (It's not a
big site and it's not public yet, so I'm not concerned about taking
anything down during this process.)

So what's the quickest way to do this:

1. Create a new core at command line with different name, move all conf
files into that (?)
2. Delete the current core at command line, but what's the script for doing
that to make sure it's totally gone? I see different responses online...
not sure what's the "best practice" for this...

Thanks!
Amanda



------
Dr. Amanda Shuman
Post-doc researcher, University of Freiburg, The Maoist Legacy Project
<http://www.maoistlegacy.uni-freiburg.de/>
PhD, University of California, Santa Cruz
http://www.amandashuman.net/
http://www.prchistoryresources.org/
Office: +49 (0) 761 203 4925
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Amanda Shuman
Oh, and I also have a related question - how can I change my CWD (current
working directory)? It is set for the /opt/ folder and not /var/ and I
think that's screwing things up...
Thanks!
Amanda

------
Dr. Amanda Shuman
Post-doc researcher, University of Freiburg, The Maoist Legacy Project
<http://www.maoistlegacy.uni-freiburg.de/>
PhD, University of California, Santa Cruz
http://www.amandashuman.net/
http://www.prchistoryresources.org/
Office: +49 (0) 761 203 4925


On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <[hidden email]>
wrote:

> Hi all, I'm a bit of a newbie still but have clearly screwed something
> up... so I think what I need to do now is to delete a core (saving current
> conf files as-is) then re-add/re-create the core and re-index. (It's not a
> big site and it's not public yet, so I'm not concerned about taking
> anything down during this process.)
>
> So what's the quickest way to do this:
>
> 1. Create a new core at command line with different name, move all conf
> files into that (?)
> 2. Delete the current core at command line, but what's the script for
> doing that to make sure it's totally gone? I see different responses
> online... not sure what's the "best practice" for this...
>
> Thanks!
> Amanda
>
>
>
> ------
> Dr. Amanda Shuman
> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> <http://www.maoistlegacy.uni-freiburg.de/>
> PhD, University of California, Santa Cruz
> http://www.amandashuman.net/
> http://www.prchistoryresources.org/
> Office: +49 (0) 761 203 4925
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

BlackIce
I'm not a Solr guru....

I take i that you installed Solr with the install script....
then it installs into a dir where normal users have no right to access the
necessary files...

One way to circumvent this is to un-install Solr and then re-install
without using the default and have it install into a directory where the
solr and login user have access to.

Deleting a Core is a simple as deleting its directory...

Hope this helps - good luck

On Wed, Jun 6, 2018 at 3:59 PM, Amanda Shuman <[hidden email]>
wrote:

> Oh, and I also have a related question - how can I change my CWD (current
> working directory)? It is set for the /opt/ folder and not /var/ and I
> think that's screwing things up...
> Thanks!
> Amanda
>
> ------
> Dr. Amanda Shuman
> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> <http://www.maoistlegacy.uni-freiburg.de/>
> PhD, University of California, Santa Cruz
> http://www.amandashuman.net/
> http://www.prchistoryresources.org/
> Office: +49 (0) 761 203 4925
>
>
> On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <[hidden email]>
> wrote:
>
> > Hi all, I'm a bit of a newbie still but have clearly screwed something
> > up... so I think what I need to do now is to delete a core (saving
> current
> > conf files as-is) then re-add/re-create the core and re-index. (It's not
> a
> > big site and it's not public yet, so I'm not concerned about taking
> > anything down during this process.)
> >
> > So what's the quickest way to do this:
> >
> > 1. Create a new core at command line with different name, move all conf
> > files into that (?)
> > 2. Delete the current core at command line, but what's the script for
> > doing that to make sure it's totally gone? I see different responses
> > online... not sure what's the "best practice" for this...
> >
> > Thanks!
> > Amanda
> >
> >
> >
> > ------
> > Dr. Amanda Shuman
> > Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> > <http://www.maoistlegacy.uni-freiburg.de/>
> > PhD, University of California, Santa Cruz
> > http://www.amandashuman.net/
> > http://www.prchistoryresources.org/
> > Office: +49 (0) 761 203 4925
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Erick Erickson
Assuming this is stand-alone:
> find the data dir for the core (parent of the index dir)
> find the config dir for the core
> shut down Solr
> "rm -rf data"
> make any changes to the configs you want
> start Solr

As BlackIce said, reinstalling works too.

If it's SolrCloud delete and recreate the collection, your configs
will be in ZooKeeper. Of course update your configs with your changes
before creating the new collection.

Best,
Erick


On Wed, Jun 6, 2018 at 7:09 AM, BlackIce <[hidden email]> wrote:

> I'm not a Solr guru....
>
> I take i that you installed Solr with the install script....
> then it installs into a dir where normal users have no right to access the
> necessary files...
>
> One way to circumvent this is to un-install Solr and then re-install
> without using the default and have it install into a directory where the
> solr and login user have access to.
>
> Deleting a Core is a simple as deleting its directory...
>
> Hope this helps - good luck
>
> On Wed, Jun 6, 2018 at 3:59 PM, Amanda Shuman <[hidden email]>
> wrote:
>
>> Oh, and I also have a related question - how can I change my CWD (current
>> working directory)? It is set for the /opt/ folder and not /var/ and I
>> think that's screwing things up...
>> Thanks!
>> Amanda
>>
>> ------
>> Dr. Amanda Shuman
>> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
>> <http://www.maoistlegacy.uni-freiburg.de/>
>> PhD, University of California, Santa Cruz
>> http://www.amandashuman.net/
>> http://www.prchistoryresources.org/
>> Office: +49 (0) 761 203 4925
>>
>>
>> On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <[hidden email]>
>> wrote:
>>
>> > Hi all, I'm a bit of a newbie still but have clearly screwed something
>> > up... so I think what I need to do now is to delete a core (saving
>> current
>> > conf files as-is) then re-add/re-create the core and re-index. (It's not
>> a
>> > big site and it's not public yet, so I'm not concerned about taking
>> > anything down during this process.)
>> >
>> > So what's the quickest way to do this:
>> >
>> > 1. Create a new core at command line with different name, move all conf
>> > files into that (?)
>> > 2. Delete the current core at command line, but what's the script for
>> > doing that to make sure it's totally gone? I see different responses
>> > online... not sure what's the "best practice" for this...
>> >
>> > Thanks!
>> > Amanda
>> >
>> >
>> >
>> > ------
>> > Dr. Amanda Shuman
>> > Post-doc researcher, University of Freiburg, The Maoist Legacy Project
>> > <http://www.maoistlegacy.uni-freiburg.de/>
>> > PhD, University of California, Santa Cruz
>> > http://www.amandashuman.net/
>> > http://www.prchistoryresources.org/
>> > Office: +49 (0) 761 203 4925
>> >
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Amanda Shuman
Thanks, I was able to do most of the but didn't reinstall... Still running
into an issue I think is related to current working directory. I guess
reinstalling might fix that?
Amanda

On Wed, Jun 6, 2018, 17:27 Erick Erickson <[hidden email]> wrote:

> Assuming this is stand-alone:
> > find the data dir for the core (parent of the index dir)
> > find the config dir for the core
> > shut down Solr
> > "rm -rf data"
> > make any changes to the configs you want
> > start Solr
>
> As BlackIce said, reinstalling works too.
>
> If it's SolrCloud delete and recreate the collection, your configs
> will be in ZooKeeper. Of course update your configs with your changes
> before creating the new collection.
>
> Best,
> Erick
>
>
> On Wed, Jun 6, 2018 at 7:09 AM, BlackIce <[hidden email]> wrote:
> > I'm not a Solr guru....
> >
> > I take i that you installed Solr with the install script....
> > then it installs into a dir where normal users have no right to access
> the
> > necessary files...
> >
> > One way to circumvent this is to un-install Solr and then re-install
> > without using the default and have it install into a directory where the
> > solr and login user have access to.
> >
> > Deleting a Core is a simple as deleting its directory...
> >
> > Hope this helps - good luck
> >
> > On Wed, Jun 6, 2018 at 3:59 PM, Amanda Shuman <[hidden email]>
> > wrote:
> >
> >> Oh, and I also have a related question - how can I change my CWD
> (current
> >> working directory)? It is set for the /opt/ folder and not /var/ and I
> >> think that's screwing things up...
> >> Thanks!
> >> Amanda
> >>
> >> ------
> >> Dr. Amanda Shuman
> >> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> >> <http://www.maoistlegacy.uni-freiburg.de/>
> >> PhD, University of California, Santa Cruz
> >> http://www.amandashuman.net/
> >> http://www.prchistoryresources.org/
> >> Office: +49 (0) 761 203 4925
> >>
> >>
> >> On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <[hidden email]>
> >> wrote:
> >>
> >> > Hi all, I'm a bit of a newbie still but have clearly screwed something
> >> > up... so I think what I need to do now is to delete a core (saving
> >> current
> >> > conf files as-is) then re-add/re-create the core and re-index. (It's
> not
> >> a
> >> > big site and it's not public yet, so I'm not concerned about taking
> >> > anything down during this process.)
> >> >
> >> > So what's the quickest way to do this:
> >> >
> >> > 1. Create a new core at command line with different name, move all
> conf
> >> > files into that (?)
> >> > 2. Delete the current core at command line, but what's the script for
> >> > doing that to make sure it's totally gone? I see different responses
> >> > online... not sure what's the "best practice" for this...
> >> >
> >> > Thanks!
> >> > Amanda
> >> >
> >> >
> >> >
> >> > ------
> >> > Dr. Amanda Shuman
> >> > Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> >> > <http://www.maoistlegacy.uni-freiburg.de/>
> >> > PhD, University of California, Santa Cruz
> >> > http://www.amandashuman.net/
> >> > http://www.prchistoryresources.org/
> >> > Office: +49 (0) 761 203 4925
> >> >
> >> >
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

BlackIce
One of the issues with the install script is that when its run by any user
other than "solr" and installed into default directories,
is that one might get ownership/permission problems.

The easiest way to avoid these is by creating the "solr" user BEFORE
installing Solr as a regular "Login-User",
and then install Solr while being logged into this account (Or sudo, etc..)
and then install Solr with NON default values for directories,
have everything installed within the "Solr" users home directory space,
that way everything belongs to the solr user, it is then easily modified,
by just logging into the solr account and one doesn't have to worry
about ownership/permissions.. ad if one makes a mistake it only affects
the "solr" user...

Ayway, just my 2 cents

On Wed, Jun 6, 2018 at 9:41 PM, Amanda Shuman <[hidden email]>
wrote:

> Thanks, I was able to do most of the but didn't reinstall... Still running
> into an issue I think is related to current working directory. I guess
> reinstalling might fix that?
> Amanda
>
> On Wed, Jun 6, 2018, 17:27 Erick Erickson <[hidden email]> wrote:
>
> > Assuming this is stand-alone:
> > > find the data dir for the core (parent of the index dir)
> > > find the config dir for the core
> > > shut down Solr
> > > "rm -rf data"
> > > make any changes to the configs you want
> > > start Solr
> >
> > As BlackIce said, reinstalling works too.
> >
> > If it's SolrCloud delete and recreate the collection, your configs
> > will be in ZooKeeper. Of course update your configs with your changes
> > before creating the new collection.
> >
> > Best,
> > Erick
> >
> >
> > On Wed, Jun 6, 2018 at 7:09 AM, BlackIce <[hidden email]> wrote:
> > > I'm not a Solr guru....
> > >
> > > I take i that you installed Solr with the install script....
> > > then it installs into a dir where normal users have no right to access
> > the
> > > necessary files...
> > >
> > > One way to circumvent this is to un-install Solr and then re-install
> > > without using the default and have it install into a directory where
> the
> > > solr and login user have access to.
> > >
> > > Deleting a Core is a simple as deleting its directory...
> > >
> > > Hope this helps - good luck
> > >
> > > On Wed, Jun 6, 2018 at 3:59 PM, Amanda Shuman <[hidden email]
> >
> > > wrote:
> > >
> > >> Oh, and I also have a related question - how can I change my CWD
> > (current
> > >> working directory)? It is set for the /opt/ folder and not /var/ and I
> > >> think that's screwing things up...
> > >> Thanks!
> > >> Amanda
> > >>
> > >> ------
> > >> Dr. Amanda Shuman
> > >> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> > >> <http://www.maoistlegacy.uni-freiburg.de/>
> > >> PhD, University of California, Santa Cruz
> > >> http://www.amandashuman.net/
> > >> http://www.prchistoryresources.org/
> > >> Office: +49 (0) 761 203 4925
> > >>
> > >>
> > >> On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <
> [hidden email]>
> > >> wrote:
> > >>
> > >> > Hi all, I'm a bit of a newbie still but have clearly screwed
> something
> > >> > up... so I think what I need to do now is to delete a core (saving
> > >> current
> > >> > conf files as-is) then re-add/re-create the core and re-index. (It's
> > not
> > >> a
> > >> > big site and it's not public yet, so I'm not concerned about taking
> > >> > anything down during this process.)
> > >> >
> > >> > So what's the quickest way to do this:
> > >> >
> > >> > 1. Create a new core at command line with different name, move all
> > conf
> > >> > files into that (?)
> > >> > 2. Delete the current core at command line, but what's the script
> for
> > >> > doing that to make sure it's totally gone? I see different responses
> > >> > online... not sure what's the "best practice" for this...
> > >> >
> > >> > Thanks!
> > >> > Amanda
> > >> >
> > >> >
> > >> >
> > >> > ------
> > >> > Dr. Amanda Shuman
> > >> > Post-doc researcher, University of Freiburg, The Maoist Legacy
> Project
> > >> > <http://www.maoistlegacy.uni-freiburg.de/>
> > >> > PhD, University of California, Santa Cruz
> > >> > http://www.amandashuman.net/
> > >> > http://www.prchistoryresources.org/
> > >> > Office: +49 (0) 761 203 4925
> > >> >
> > >> >
> > >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Amanda Shuman
Definitely not a permissions problem - everything is run by the solr user,
which owns everything in the directories. I just can't figure out why the
default working directory is in opt rather than var (which is where it
should be according to a previous chain I was in).

But at this point I'm at a total loss, so maybe a fresh install wouldn't
hurt.


------
Dr. Amanda Shuman
Post-doc researcher, University of Freiburg, The Maoist Legacy Project
<http://www.maoistlegacy.uni-freiburg.de/>
PhD, University of California, Santa Cruz
http://www.amandashuman.net/
http://www.prchistoryresources.org/
Office: +49 (0) 761 203 4925


On Wed, Jun 6, 2018 at 11:09 PM, BlackIce <[hidden email]> wrote:

> One of the issues with the install script is that when its run by any user
> other than "solr" and installed into default directories,
> is that one might get ownership/permission problems.
>
> The easiest way to avoid these is by creating the "solr" user BEFORE
> installing Solr as a regular "Login-User",
> and then install Solr while being logged into this account (Or sudo, etc..)
> and then install Solr with NON default values for directories,
> have everything installed within the "Solr" users home directory space,
> that way everything belongs to the solr user, it is then easily modified,
> by just logging into the solr account and one doesn't have to worry
> about ownership/permissions.. ad if one makes a mistake it only affects
> the "solr" user...
>
> Ayway, just my 2 cents
>
> On Wed, Jun 6, 2018 at 9:41 PM, Amanda Shuman <[hidden email]>
> wrote:
>
> > Thanks, I was able to do most of the but didn't reinstall... Still
> running
> > into an issue I think is related to current working directory. I guess
> > reinstalling might fix that?
> > Amanda
> >
> > On Wed, Jun 6, 2018, 17:27 Erick Erickson <[hidden email]>
> wrote:
> >
> > > Assuming this is stand-alone:
> > > > find the data dir for the core (parent of the index dir)
> > > > find the config dir for the core
> > > > shut down Solr
> > > > "rm -rf data"
> > > > make any changes to the configs you want
> > > > start Solr
> > >
> > > As BlackIce said, reinstalling works too.
> > >
> > > If it's SolrCloud delete and recreate the collection, your configs
> > > will be in ZooKeeper. Of course update your configs with your changes
> > > before creating the new collection.
> > >
> > > Best,
> > > Erick
> > >
> > >
> > > On Wed, Jun 6, 2018 at 7:09 AM, BlackIce <[hidden email]>
> wrote:
> > > > I'm not a Solr guru....
> > > >
> > > > I take i that you installed Solr with the install script....
> > > > then it installs into a dir where normal users have no right to
> access
> > > the
> > > > necessary files...
> > > >
> > > > One way to circumvent this is to un-install Solr and then re-install
> > > > without using the default and have it install into a directory where
> > the
> > > > solr and login user have access to.
> > > >
> > > > Deleting a Core is a simple as deleting its directory...
> > > >
> > > > Hope this helps - good luck
> > > >
> > > > On Wed, Jun 6, 2018 at 3:59 PM, Amanda Shuman <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > >> Oh, and I also have a related question - how can I change my CWD
> > > (current
> > > >> working directory)? It is set for the /opt/ folder and not /var/
> and I
> > > >> think that's screwing things up...
> > > >> Thanks!
> > > >> Amanda
> > > >>
> > > >> ------
> > > >> Dr. Amanda Shuman
> > > >> Post-doc researcher, University of Freiburg, The Maoist Legacy
> Project
> > > >> <http://www.maoistlegacy.uni-freiburg.de/>
> > > >> PhD, University of California, Santa Cruz
> > > >> http://www.amandashuman.net/
> > > >> http://www.prchistoryresources.org/
> > > >> Office: +49 (0) 761 203 4925
> > > >>
> > > >>
> > > >> On Wed, Jun 6, 2018 at 3:35 PM, Amanda Shuman <
> > [hidden email]>
> > > >> wrote:
> > > >>
> > > >> > Hi all, I'm a bit of a newbie still but have clearly screwed
> > something
> > > >> > up... so I think what I need to do now is to delete a core (saving
> > > >> current
> > > >> > conf files as-is) then re-add/re-create the core and re-index.
> (It's
> > > not
> > > >> a
> > > >> > big site and it's not public yet, so I'm not concerned about
> taking
> > > >> > anything down during this process.)
> > > >> >
> > > >> > So what's the quickest way to do this:
> > > >> >
> > > >> > 1. Create a new core at command line with different name, move all
> > > conf
> > > >> > files into that (?)
> > > >> > 2. Delete the current core at command line, but what's the script
> > for
> > > >> > doing that to make sure it's totally gone? I see different
> responses
> > > >> > online... not sure what's the "best practice" for this...
> > > >> >
> > > >> > Thanks!
> > > >> > Amanda
> > > >> >
> > > >> >
> > > >> >
> > > >> > ------
> > > >> > Dr. Amanda Shuman
> > > >> > Post-doc researcher, University of Freiburg, The Maoist Legacy
> > Project
> > > >> > <http://www.maoistlegacy.uni-freiburg.de/>
> > > >> > PhD, University of California, Santa Cruz
> > > >> > http://www.amandashuman.net/
> > > >> > http://www.prchistoryresources.org/
> > > >> > Office: +49 (0) 761 203 4925
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Shawn Heisey-2
On 6/7/2018 4:12 AM, Amanda Shuman wrote:
> Definitely not a permissions problem - everything is run by the solr user,
> which owns everything in the directories. I just can't figure out why the
> default working directory is in opt rather than var (which is where it
> should be according to a previous chain I was in).
>
> But at this point I'm at a total loss, so maybe a fresh install wouldn't
> hurt.

The "bin/solr" script, which is ultimately how Solr is started even when
it is installed as a service, initially sets the current working
directory to a directory that it knows as SOLR_TIP.  This is the
directory containing bin, server, and others.  It defaults to /opt/solr
when Solr is installed as a service.

Then just before Solr is started, the script will change the current
working directory to the server directory, which is a subdirectory of
SOLR_TIP.

So when Solr starts, the current working directory is $SOLR_TIP/server.

The service installer sets the owner of everything in SOLR_TIP to root. 
The solr user has absolutely no reason to write to that directory at
all.  Everything that Solr writes will be to an absolute path under the
"var dir" given during service install, which defaults to /var/solr. 
THAT directory and all its contents will be owned by the user specified
during install, which defaults to solr.

The current working directory is where the developers want it, and will
not be in the "var dir".  Its location is critical for correct Jetty
operation.  When Solr is configured in the expected way for a service
install, it does not use the current working directory.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Amanda Shuman
Thanks, Shawn, that is a remarkably clear description.

I am able to create the core and all appears fine, but when I go to index I
am unfortunately running into a new problem. I am indexing from the same
site content as before (it's just an Omeka install with a solr plug-in that
reindexes the sitE), but now it only indexes 3 (!) records out of 3000+ and
then stops. I have no idea why. The old core - with a different name -
still works, even I choose to reindex it. Now I have to figure out which
error logs to check -- Solr or Omeka.

Amanda

------
Dr. Amanda Shuman
Post-doc researcher, University of Freiburg, The Maoist Legacy Project
<http://www.maoistlegacy.uni-freiburg.de/>
PhD, University of California, Santa Cruz
http://www.amandashuman.net/
http://www.prchistoryresources.org/
Office: +49 (0) 761 203 4925


On Thu, Jun 7, 2018 at 3:08 PM, Shawn Heisey <[hidden email]> wrote:

> On 6/7/2018 4:12 AM, Amanda Shuman wrote:
>
>> Definitely not a permissions problem - everything is run by the solr user,
>> which owns everything in the directories. I just can't figure out why the
>> default working directory is in opt rather than var (which is where it
>> should be according to a previous chain I was in).
>>
>> But at this point I'm at a total loss, so maybe a fresh install wouldn't
>> hurt.
>>
>
> The "bin/solr" script, which is ultimately how Solr is started even when
> it is installed as a service, initially sets the current working directory
> to a directory that it knows as SOLR_TIP.  This is the directory containing
> bin, server, and others.  It defaults to /opt/solr when Solr is installed
> as a service.
>
> Then just before Solr is started, the script will change the current
> working directory to the server directory, which is a subdirectory of
> SOLR_TIP.
>
> So when Solr starts, the current working directory is $SOLR_TIP/server.
>
> The service installer sets the owner of everything in SOLR_TIP to root.
> The solr user has absolutely no reason to write to that directory at all.
> Everything that Solr writes will be to an absolute path under the "var dir"
> given during service install, which defaults to /var/solr.  THAT directory
> and all its contents will be owned by the user specified during install,
> which defaults to solr.
>
> The current working directory is where the developers want it, and will
> not be in the "var dir".  Its location is critical for correct Jetty
> operation.  When Solr is configured in the expected way for a service
> install, it does not use the current working directory.
>
> Thanks,
> Shawn
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Erick Erickson
Amanda:

Your Solr log will record each update that comes through. It's a
little opaque, by default it'll show you the first 10 IDs of each
batch it receives.

Guesses:
- you're somehow having the same ID (<uniqueKey>) assigned to multiple documents
- your schemas are a bit different and the docs can't be indexed
(undefined field for instance).


Best,
Erick


On Thu, Jun 7, 2018 at 7:49 AM, Amanda Shuman <[hidden email]> wrote:

> Thanks, Shawn, that is a remarkably clear description.
>
> I am able to create the core and all appears fine, but when I go to index I
> am unfortunately running into a new problem. I am indexing from the same
> site content as before (it's just an Omeka install with a solr plug-in that
> reindexes the sitE), but now it only indexes 3 (!) records out of 3000+ and
> then stops. I have no idea why. The old core - with a different name -
> still works, even I choose to reindex it. Now I have to figure out which
> error logs to check -- Solr or Omeka.
>
> Amanda
>
> ------
> Dr. Amanda Shuman
> Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> <http://www.maoistlegacy.uni-freiburg.de/>
> PhD, University of California, Santa Cruz
> http://www.amandashuman.net/
> http://www.prchistoryresources.org/
> Office: +49 (0) 761 203 4925
>
>
> On Thu, Jun 7, 2018 at 3:08 PM, Shawn Heisey <[hidden email]> wrote:
>
>> On 6/7/2018 4:12 AM, Amanda Shuman wrote:
>>
>>> Definitely not a permissions problem - everything is run by the solr user,
>>> which owns everything in the directories. I just can't figure out why the
>>> default working directory is in opt rather than var (which is where it
>>> should be according to a previous chain I was in).
>>>
>>> But at this point I'm at a total loss, so maybe a fresh install wouldn't
>>> hurt.
>>>
>>
>> The "bin/solr" script, which is ultimately how Solr is started even when
>> it is installed as a service, initially sets the current working directory
>> to a directory that it knows as SOLR_TIP.  This is the directory containing
>> bin, server, and others.  It defaults to /opt/solr when Solr is installed
>> as a service.
>>
>> Then just before Solr is started, the script will change the current
>> working directory to the server directory, which is a subdirectory of
>> SOLR_TIP.
>>
>> So when Solr starts, the current working directory is $SOLR_TIP/server.
>>
>> The service installer sets the owner of everything in SOLR_TIP to root.
>> The solr user has absolutely no reason to write to that directory at all.
>> Everything that Solr writes will be to an absolute path under the "var dir"
>> given during service install, which defaults to /var/solr.  THAT directory
>> and all its contents will be owned by the user specified during install,
>> which defaults to solr.
>>
>> The current working directory is where the developers want it, and will
>> not be in the "var dir".  Its location is critical for correct Jetty
>> operation.  When Solr is configured in the expected way for a service
>> install, it does not use the current working directory.
>>
>> Thanks,
>> Shawn
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Delete then re-add a core

Amanda Shuman
Erick - thank you, the issue was the second one you mentioned -- I
completely forgot about changes that were made in conf files I never copied
over (including schema.xml). Once I overwrote and reloaded I had no
problems reindexing. I guess I had forgotten to check those since I was
copying various files back and forth... the errors in the solr log clearly
showed that what was happening.

Best,
Amanda


------
Dr. Amanda Shuman
Post-doc researcher, University of Freiburg, The Maoist Legacy Project
<http://www.maoistlegacy.uni-freiburg.de/>
PhD, University of California, Santa Cruz
http://www.amandashuman.net/
http://www.prchistoryresources.org/
Office: +49 (0) 761 203 4925


On Thu, Jun 7, 2018 at 5:39 PM, Erick Erickson <[hidden email]>
wrote:

> Amanda:
>
> Your Solr log will record each update that comes through. It's a
> little opaque, by default it'll show you the first 10 IDs of each
> batch it receives.
>
> Guesses:
> - you're somehow having the same ID (<uniqueKey>) assigned to multiple
> documents
> - your schemas are a bit different and the docs can't be indexed
> (undefined field for instance).
>
>
> Best,
> Erick
>
>
> On Thu, Jun 7, 2018 at 7:49 AM, Amanda Shuman <[hidden email]>
> wrote:
> > Thanks, Shawn, that is a remarkably clear description.
> >
> > I am able to create the core and all appears fine, but when I go to
> index I
> > am unfortunately running into a new problem. I am indexing from the same
> > site content as before (it's just an Omeka install with a solr plug-in
> that
> > reindexes the sitE), but now it only indexes 3 (!) records out of 3000+
> and
> > then stops. I have no idea why. The old core - with a different name -
> > still works, even I choose to reindex it. Now I have to figure out which
> > error logs to check -- Solr or Omeka.
> >
> > Amanda
> >
> > ------
> > Dr. Amanda Shuman
> > Post-doc researcher, University of Freiburg, The Maoist Legacy Project
> > <http://www.maoistlegacy.uni-freiburg.de/>
> > PhD, University of California, Santa Cruz
> > http://www.amandashuman.net/
> > http://www.prchistoryresources.org/
> > Office: +49 (0) 761 203 4925
> >
> >
> > On Thu, Jun 7, 2018 at 3:08 PM, Shawn Heisey <[hidden email]>
> wrote:
> >
> >> On 6/7/2018 4:12 AM, Amanda Shuman wrote:
> >>
> >>> Definitely not a permissions problem - everything is run by the solr
> user,
> >>> which owns everything in the directories. I just can't figure out why
> the
> >>> default working directory is in opt rather than var (which is where it
> >>> should be according to a previous chain I was in).
> >>>
> >>> But at this point I'm at a total loss, so maybe a fresh install
> wouldn't
> >>> hurt.
> >>>
> >>
> >> The "bin/solr" script, which is ultimately how Solr is started even when
> >> it is installed as a service, initially sets the current working
> directory
> >> to a directory that it knows as SOLR_TIP.  This is the directory
> containing
> >> bin, server, and others.  It defaults to /opt/solr when Solr is
> installed
> >> as a service.
> >>
> >> Then just before Solr is started, the script will change the current
> >> working directory to the server directory, which is a subdirectory of
> >> SOLR_TIP.
> >>
> >> So when Solr starts, the current working directory is $SOLR_TIP/server.
> >>
> >> The service installer sets the owner of everything in SOLR_TIP to root.
> >> The solr user has absolutely no reason to write to that directory at
> all.
> >> Everything that Solr writes will be to an absolute path under the "var
> dir"
> >> given during service install, which defaults to /var/solr.  THAT
> directory
> >> and all its contents will be owned by the user specified during install,
> >> which defaults to solr.
> >>
> >> The current working directory is where the developers want it, and will
> >> not be in the "var dir".  Its location is critical for correct Jetty
> >> operation.  When Solr is configured in the expected way for a service
> >> install, it does not use the current working directory.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
>