[Fwd: Re: intranet crawl issue]

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Fwd: Re: intranet crawl issue]

mholt
Any ideas? I've exhausted all the ends I know to take a look at.
  matt

Here is my crawl-urlfilter.txt file.
Matt

# The url filter file used by the crawl command.

# Better for intranet crawling.
# Be sure to change MY.DOMAIN.NAME to your domain name.

# Each non-comment, non-blank line contains a regular expression
# prefixed by '+' or '-'.  The first matching pattern in the file
# determines whether a URL is included or ignored.  If no pattern
# matches, the URL is ignored.

# skip file:, ftp:, & mailto: urls
-^(file|ftp|mailto):

# skip image and other suffixes we can't yet parse
-\.(gif|GIF|jpg|JPG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|MOV|exe|png|PNG)$

# skip URLs containing certain characters as probable queries, etc.
-[?*!@=]

# accept hosts in MY.DOMAIN.NAME
+^http://([a-z0-9]*\.)*corp.mydomain.com/

# skip everything else
-.


Stefan Neufeind wrote:

> Matthew Holt wrote:
>
>> Just fyi,.. both of the sites I am trying to crawl are under the same
>> domain. The sub-domains just differ. Works for one, the other it o
>> nly appears to fetch 6 or so pages then doesn't fetch anymore. Do you
>> need any more information to solve the problem? I've tried everything
>> and havent' had any luck.. Thanks.
>
>
> What does your crawl-urlfilter.txt look like?
>
>  Stefan
>

Reply | Threaded
Open this post in threaded view
|

Re: [Fwd: Re: intranet crawl issue]

mholt
Stefan,
  Thanks a bunch, turns out the ? mark was the cause of the problems.
Able to run the search now and not having any problems with it. Thanks a
ton!
Matt

Stefan Groschupf wrote:

> Hi Matt,
> the first impression I have is that your segment has only 6 pages.
> You generated many empthy pages.
> is the website a CMS? Has it questionmarks in the URLs? You exclude
> all pages with questions marks.
> Check how many pages are after injecting in your web db.
> Check how many pages are in your segment fetch list.
>
> HTH
> Stefan
>
>> -.
>>
>>
>> Stefan Neufeind wrote:
>>
>>> Matthew Holt wrote:
>>>
>>>> Just fyi,.. both of the sites I am trying to crawl are under the
>>>> same domain. The sub-domains just differ. Works for one, the other
>>>> it o nly appears to fetch 6 or so pages then doesn't fetch anymore.
>>>> Do you need any more information to solve the problem? I've tried
>>>> everything and havent' had any luck.. Thanks.
>>>
>>>
>>>
>>> What does your crawl-urlfilter.txt look like?
>>>
>>>  Stefan
>>>
>>
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Fwd: Re: intranet crawl issue]

mholt
In reply to this post by mholt
Only question I have.. I am able to get it to index all pages that begin
with a ? (such as
http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities&toggle=00372).
However, it doesn't seem to index any pages that are linked to from that
page unless they are of the ? form.. (such as
http://intranet.corp.redhat.com/ic/intranet/NewOffice.html is a link
from the page, but it won't index it).. Any ideas? My crawl file and my
crawl-urlfilter.txt files are attached. Thanks.

Matt

Stefan Groschupf wrote:

> Hi Matt,
> the first impression I have is that your segment has only 6 pages.
> You generated many empthy pages.
> is the website a CMS? Has it questionmarks in the URLs? You exclude
> all pages with questions marks.
> Check how many pages are after injecting in your web db.
> Check how many pages are in your segment fetch list.
>
> HTH
> Stefan
>
>
>>
>>
>> Stefan Neufeind wrote:
>>
>>> Matthew Holt wrote:
>>>
>>>> Just fyi,.. both of the sites I am trying to crawl are under the
>>>> same domain. The sub-domains just differ. Works for one, the other
>>>> it o nly appears to fetch 6 or so pages then doesn't fetch anymore.
>>>> Do you need any more information to solve the problem? I've tried
>>>> everything and havent' had any luck.. Thanks.
>>>
>>>
>>>
>>> What does your crawl-urlfilter.txt look like?
>>>
>>>  Stefan
>>>
>>
>>
>>
>

060612 133747 parsing file:/home/rdu/mholt/Desktop/nutch-0.7.2/conf/nutch-default.xml
060612 133747 parsing file:/home/rdu/mholt/Desktop/nutch-0.7.2/conf/crawl-tool.xml
060612 133747 parsing file:/home/rdu/mholt/Desktop/nutch-0.7.2/conf/nutch-site.xml
060612 133747 No FS indicated, using default:local
060612 133747 crawl started in: crawl/
060612 133747 rootUrlFile = urls/redhat
060612 133747 threads = 10
060612 133747 depth = 10
060612 133747 Created webdb at LocalFS,/home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 133747 Starting URL processing
060612 133747 Plugins: looking in: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/clustering-carrot2
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/creativecommons
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/index-basic/plugin.xml
060612 133747 impl: point=org.apache.nutch.indexer.IndexingFilter class=org.apache.nutch.indexer.basic.BasicIndexingFilter
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/index-more
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/language-identifier
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/nutch-extensionpoints/plugin.xml
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/ontology
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-ext
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-html/plugin.xml
060612 133747 impl: point=org.apache.nutch.parse.Parser class=org.apache.nutch.parse.html.HtmlParser
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-js
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-msword
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-pdf
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-rss
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/parse-text/plugin.xml
060612 133747 impl: point=org.apache.nutch.parse.Parser class=org.apache.nutch.parse.text.TextParser
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/protocol-file
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/protocol-ftp
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/protocol-http/plugin.xml
060612 133747 impl: point=org.apache.nutch.protocol.Protocol class=org.apache.nutch.protocol.http.Http
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/protocol-httpclient
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/query-basic/plugin.xml
060612 133747 impl: point=org.apache.nutch.searcher.QueryFilter class=org.apache.nutch.searcher.basic.BasicQueryFilter
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/query-more
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/query-site/plugin.xml
060612 133747 impl: point=org.apache.nutch.searcher.QueryFilter class=org.apache.nutch.searcher.site.SiteQueryFilter
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/query-url/plugin.xml
060612 133747 impl: point=org.apache.nutch.searcher.QueryFilter class=org.apache.nutch.searcher.url.URLQueryFilter
060612 133747 not including: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/urlfilter-prefix
060612 133747 parsing: /home/rdu/mholt/Desktop/nutch-0.7.2/plugins/urlfilter-regex/plugin.xml
060612 133747 impl: point=org.apache.nutch.net.URLFilter class=org.apache.nutch.net.RegexURLFilter
060612 133747 found resource crawl-urlfilter.txt at file:/home/rdu/mholt/Desktop/nutch-0.7.2/conf/crawl-urlfilter.txt
060612 133747 Using URL normalizer: org.apache.nutch.net.BasicUrlNormalizer
060612 133747 Added 1 pages
060612 133747 Processing pagesByURL: Sorted 1 instructions in 0.0050 seconds.
060612 133747 Processing pagesByURL: Sorted 200.0 instructions/second
060612 133747 Processing pagesByURL: Merged to new DB containing 1 records in 0.0010 seconds
060612 133747 Processing pagesByURL: Merged 1000.0 records/second
060612 133747 Processing pagesByMD5: Sorted 1 instructions in 0.0050 seconds.
060612 133747 Processing pagesByMD5: Sorted 200.0 instructions/second
060612 133747 Processing pagesByMD5: Merged to new DB containing 1 records in 0.0010 seconds
060612 133747 Processing pagesByMD5: Merged 1000.0 records/second
060612 133747 Processing linksByMD5: Copied file (4096 bytes) in 0.0090 secs.
060612 133747 Processing linksByURL: Copied file (4096 bytes) in 0.0080 secs.
060612 133748 FetchListTool started
060612 133748 Processing pagesByURL: Sorted 1 instructions in 0.0050 seconds.
060612 133748 Processing pagesByURL: Sorted 200.0 instructions/second
060612 133748 Processing pagesByURL: Merged to new DB containing 1 records in 0.0010 seconds
060612 133748 Processing pagesByURL: Merged 1000.0 records/second
060612 133748 Processing pagesByMD5: Sorted 1 instructions in 0.0040 seconds.
060612 133748 Processing pagesByMD5: Sorted 250.0 instructions/second
060612 133748 Processing pagesByMD5: Merged to new DB containing 1 records in 0.0 seconds
060612 133748 Processing pagesByMD5: Merged Infinity records/second
060612 133748 Processing linksByMD5: Copied file (4096 bytes) in 0.0060 secs.
060612 133748 Processing linksByURL: Copied file (4096 bytes) in 0.012 secs.
060612 133748 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748/fetchlist.unsorted: Sorted 1 entries in 0.0050 seconds.
060612 133748 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748/fetchlist.unsorted: Sorted 200.0 entries/second
060612 133748 Overall processing: Sorted 1 entries in 0.0050 seconds.
060612 133748 Overall processing: Sorted 0.0050 entries/second
060612 133748 FetchListTool completed
060612 133748 logging at INFO
060612 133748 fetching http://intranet.corp.redhat.com/ic/intranet/
060612 133748 http.proxy.host = null
060612 133748 http.proxy.port = 8080
060612 133748 http.timeout = 10000
060612 133748 http.content.limit = 65536
060612 133748 http.agent = NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; [hidden email])
060612 133748 fetcher.server.delay = 1000
060612 133748 http.max.delays = 100
060612 133750 status: segment 20060612133748, 1 pages, 0 errors, 41391 bytes, 2016 ms
060612 133750 status: 0.49603173 pages/s, 160.40039 kb/s, 41391.0 bytes/page
060612 133751 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 133751 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748
060612 133751 Processing document 0
060612 133751 Finishing update
060612 133751 Processing pagesByURL: Sorted 36 instructions in 0.0070 seconds.
060612 133751 Processing pagesByURL: Sorted 5142.857142857143 instructions/second
060612 133751 Processing pagesByURL: Merged to new DB containing 35 records in 0.0030 seconds
060612 133751 Processing pagesByURL: Merged 11666.666666666666 records/second
060612 133751 Processing pagesByMD5: Sorted 36 instructions in 0.0070 seconds.
060612 133751 Processing pagesByMD5: Sorted 5142.857142857143 instructions/second
060612 133751 Processing pagesByMD5: Merged to new DB containing 35 records in 0.0040 seconds
060612 133751 Processing pagesByMD5: Merged 8750.0 records/second
060612 133751 Processing linksByMD5: Sorted 36 instructions in 0.014 seconds.
060612 133751 Processing linksByMD5: Sorted 2571.4285714285716 instructions/second
060612 133751 Processing linksByMD5: Merged to new DB containing 34 records in 0.0040 seconds
060612 133751 Processing linksByMD5: Merged 8500.0 records/second
060612 133751 Processing linksByURL: Sorted 34 instructions in 0.022 seconds.
060612 133751 Processing linksByURL: Sorted 1545.4545454545455 instructions/second
060612 133751 Processing linksByURL: Merged to new DB containing 34 records in 0.0060 seconds
060612 133751 Processing linksByURL: Merged 5666.666666666667 records/second
060612 133751 Processing linksByMD5: Sorted 34 instructions in 0.0060 seconds.
060612 133751 Processing linksByMD5: Sorted 5666.666666666667 instructions/second
060612 133751 Processing linksByMD5: Merged to new DB containing 34 records in 0.0040 seconds
060612 133751 Processing linksByMD5: Merged 8500.0 records/second
060612 133752 Update finished
060612 133752 FetchListTool started
060612 133752 Processing pagesByURL: Sorted 34 instructions in 0.0050 seconds.
060612 133752 Processing pagesByURL: Sorted 6800.0 instructions/second
060612 133752 Processing pagesByURL: Merged to new DB containing 35 records in 0.0030 seconds
060612 133752 Processing pagesByURL: Merged 11666.666666666666 records/second
060612 133752 Processing pagesByMD5: Sorted 34 instructions in 0.0050 seconds.
060612 133752 Processing pagesByMD5: Sorted 6800.0 instructions/second
060612 133752 Processing pagesByMD5: Merged to new DB containing 35 records in 0.0020 seconds
060612 133752 Processing pagesByMD5: Merged 17500.0 records/second
060612 133752 Processing linksByMD5: Copied file (4096 bytes) in 0.0070 secs.
060612 133752 Processing linksByURL: Copied file (4096 bytes) in 0.018 secs.
060612 133752 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133752/fetchlist.unsorted: Sorted 34 entries in 0.021 seconds.
060612 133752 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133752/fetchlist.unsorted: Sorted 1619.047619047619 entries/second
060612 133752 Overall processing: Sorted 34 entries in 0.021 seconds.
060612 133752 Overall processing: Sorted 6.176470588235294E-4 entries/second
060612 133752 FetchListTool completed
060612 133752 logging at INFO
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Lead%20Generation&toggle=01031
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reports%20Portal&toggle=01394
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services&toggle=00611
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs&toggle=01036
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services&toggle=00471
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations&toggle=00980
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications&toggle=01356
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Corporate%20Affairs&toggle=00009
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing&toggle=01046
060612 133752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting&toggle=01345
060612 133754 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Corporate%20Development&toggle=00010
060612 133758 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Online%20Strategy%20%26%20Operations&toggle=01338
060612 133801 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations&toggle=01476
060612 133804 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Open%20Source%20Affairs&toggle=01339
060612 133808 fetching http://intranet.corp.redhat.com/ic/intranet/backlinks.html
060612 133812 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network&toggle=01376
060612 133816 fetching http://intranet.corp.redhat.com/cgi-bin/search.cgi
060612 133817 fetching http://intranet.corp.redhat.com/ic/intranet/login.html
060612 133821 fetching http://intranet.corp.redhat.com/ic/intranet/recentchanges
060612 133824 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services&toggle=00633
060612 133829 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Product%20Marketing&toggle=01343
060612 133832 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities&toggle=00372
060612 133836 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs&toggle=00001
060612 133838 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone&toggle=01395
060612 133840 fetching http://intranet.corp.redhat.com/ic/intranet/index.html
060612 133843 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%20Services&toggle=00044
060612 133847 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Open%20Source%20Now&toggle=01340
060612 133850 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering&toggle=00043
060612 133853 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service&toggle=00011
060612 133856 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital&toggle=00807
060612 133907 fetch of http://intranet.corp.redhat.com/ic/intranet/recentchanges failed with: java.lang.Exception: java.net.SocketTimeoutException: Read timed out
060612 133907 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales&toggle=01407
060612 133912 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance&toggle=00426
060612 133915 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony&toggle=01481
060612 133918 fetching http://intranet.corp.redhat.com/ic/intranet/IntranetHelp
060612 133919 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Corporate%20Affairs&toggle=00009 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 133951 status: segment 20060612133752, 32 pages, 2 errors, 1088830 bytes, 118269 ms
060612 133951 status: 0.27056965 pages/s, 71.92489 kb/s, 34025.938 bytes/page
060612 133952 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 133952 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133752
060612 133952 Processing document 0
060612 133952 Finishing update
060612 133952 Processing pagesByURL: Sorted 1181 instructions in 0.034 seconds.
060612 133952 Processing pagesByURL: Sorted 34735.294117647056 instructions/second
060612 133952 Processing pagesByURL: Merged to new DB containing 217 records in 0.026 seconds
060612 133952 Processing pagesByURL: Merged 8346.153846153846 records/second
060612 133952 Processing pagesByMD5: Sorted 248 instructions in 0.017 seconds.
060612 133952 Processing pagesByMD5: Sorted 14588.235294117647 instructions/second
060612 133952 Processing pagesByMD5: Merged to new DB containing 217 records in 0.0060 seconds
060612 133952 Processing pagesByMD5: Merged 36166.666666666664 records/second
060612 133952 Processing linksByMD5: Sorted 1179 instructions in 0.027 seconds.
060612 133952 Processing linksByMD5: Sorted 43666.666666666664 instructions/second
060612 133952 Processing linksByMD5: Merged to new DB containing 1116 records in 0.035 seconds
060612 133952 Processing linksByMD5: Merged 31885.714285714283 records/second
060612 133952 Processing linksByURL: Sorted 1116 instructions in 0.04 seconds.
060612 133952 Processing linksByURL: Sorted 27900.0 instructions/second
060612 133952 Processing linksByURL: Merged to new DB containing 1116 records in 0.02 seconds
060612 133952 Processing linksByURL: Merged 55800.0 records/second
060612 133953 Processing linksByMD5: Sorted 1116 instructions in 0.026 seconds.
060612 133953 Processing linksByMD5: Sorted 42923.07692307692 instructions/second
060612 133953 Processing linksByMD5: Merged to new DB containing 1116 records in 0.019 seconds
060612 133953 Processing linksByMD5: Merged 58736.84210526316 records/second
060612 133953 Update finished
060612 133953 FetchListTool started
060612 133953 Processing pagesByURL: Sorted 182 instructions in 0.0080 seconds.
060612 133953 Processing pagesByURL: Sorted 22750.0 instructions/second
060612 133953 Processing pagesByURL: Merged to new DB containing 217 records in 0.0050 seconds
060612 133953 Processing pagesByURL: Merged 43400.0 records/second
060612 133953 Processing pagesByMD5: Sorted 182 instructions in 0.0080 seconds.
060612 133953 Processing pagesByMD5: Sorted 22750.0 instructions/second
060612 133953 Processing pagesByMD5: Merged to new DB containing 217 records in 0.0050 seconds
060612 133953 Processing pagesByMD5: Merged 43400.0 records/second
060612 133953 Processing linksByMD5: Copied file (4096 bytes) in 0.023 secs.
060612 133953 Processing linksByURL: Copied file (4096 bytes) in 0.039 secs.
060612 133953 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133953/fetchlist.unsorted: Sorted 182 entries in 0.01 seconds.
060612 133953 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133953/fetchlist.unsorted: Sorted 18200.0 entries/second
060612 133953 Overall processing: Sorted 182 entries in 0.01 seconds.
060612 133953 Overall processing: Sorted 5.4945054945054945E-5 entries/second
060612 133953 FetchListTool completed
060612 133954 logging at INFO
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aEMEA%2fAPAC%20Corporate%20IS&master=00471&toggle=00472
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aTam%20Newsletter&master=00633&toggle=00799
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aDeveloper%20Tools&master=00043&toggle=00061
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aHiring%20Process&master=00807&toggle=00855
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aGlobal%20Partners&master=00980&toggle=01002
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aProject%20Template&master=00043&toggle=00231
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aCalendar&master=01046&toggle=01057
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone%3aFAQ&master=01395&toggle=01400
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aTraining&master=00633&toggle=00806
060612 133954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aNew%20Product%20and%20Program%20Meetings&master=00001&toggle=00005
060612 133956 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aTools&master=01345&toggle=01355
060612 133959 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aProduct%20Marketing&master=01356&toggle=01370
060612 134003 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone%3aOffice%20Administrators&master=01395&toggle=01401
060612 134006 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Open%20Source%20Now%3aVolunteer%20Opportunities&master=01340&toggle=01342
060612 134009 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aProducts&master=01345&toggle=01351
060612 134012 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aPatents&master=01036&toggle=01042
060612 134014 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aBrand%20Communications%20%26%20Design&master=01046&toggle=01056
060612 134017 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aWhom%20Should%20I%20Talk%20To%3f&master=01036&toggle=01045
060612 134020 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aInternal%20Subscriptions&master=00011&toggle=00039
060612 134023 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aInvestor%20Relations&master=00426&toggle=00435
060612 134027 fetching http://intranet.corp.redhat.com/ic/intranet/AlexPinchev.html
060612 134030 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aCorporate%20Presentation&master=01407&toggle=01433
060612 134033 fetching http://intranet.corp.redhat.com/ic/intranet/PeterChilders.html
060612 134036 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aOrganizational%20Chart&master=00426&toggle=00437
060612 134039 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aArchive&master=01356&toggle=01357
060612 134044 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone%3aSample%20Rewards&master=01395&toggle=01402
060612 134047 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aNew%20Hire%20Info%2e&master=00011&toggle=00040
060612 134050 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony%3aAudio%20Teleconferencing&master=01481&toggle=01482
060612 134054 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%20Services%3aAll%20Projects&master=00044&toggle=00045
060612 134057 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aMeetings&master=01376&toggle=01387
060612 134100 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aTreasuryTax&master=00426&toggle=00467
060612 134103 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Open%20Source%20Now%3aEducational%20Initiatives&master=01340&toggle=01341
060612 134106 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aTAM%20Accounts&master=00633&toggle=00797
060612 134109 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aCredit%20%26%20Accounts%20Receivable&master=00426&toggle=00428
060612 134112 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aProjects&master=00471&toggle=00518
060612 134115 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aInvoicing%20Collection&master=00426&toggle=00436
060612 134116 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aCalendar&master=01046&toggle=01057 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134116 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aExport%20Control&master=01036&toggle=01041
060612 134118 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aGoals&master=01046&toggle=01202
060612 134122 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aMarketing%20Communications&master=01046&toggle=01204
060612 134125 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aAPAC%20Marketing&master=01046&toggle=01047
060612 134128 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aPresentations&master=00633&toggle=00663
060612 134131 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aFinancial%20Reporting&master=00426&toggle=00433
060612 134134 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aEMEA%20Marketing&master=01046&toggle=01196
060612 134137 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aUsing%20Red%20Hat%20Trademarks&master=01036&toggle=01044
060612 134139 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aInternational%20Information&master=00807&toggle=00882
060612 134143 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aEmergingTechnologies&master=00043&toggle=00081
060612 134143 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aInternal%20Subscriptions&master=00011&toggle=00039 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134143 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%20Services%3aDeveloper%20Tools&master=00044&toggle=00046
060612 134146 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aRed%20Hat%20Linux&master=00043&toggle=00232
060612 134150 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aInstructor%20Information&master=00611&toggle=00612
060612 134153 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3a2006%20Interns&master=00807&toggle=00808
060612 134156 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aCorporate%20Communications&master=01046&toggle=01078
060612 134159 fetching http://intranet.corp.redhat.com/ic/intranet/GaryJordan.html
060612 134203 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aSELinux&master=00043&toggle=00337
060612 134206 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aAPAC&master=01407&toggle=01408
060612 134210 fetching http://intranet.corp.redhat.com/ic/intranet/KarenClark.html
060612 134213 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aPayables&master=00426&toggle=00438
060612 134216 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations%3aSubscription%20Center%20Training&master=01476&toggle=01480
060612 134220 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aTools&master=00043&toggle=00356
060612 134224 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aEMEA&master=01356&toggle=01362
060612 134225 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aTreasuryTax&master=00426&toggle=00467 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134225 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aBenefits&master=00807&toggle=00809
060612 134227 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aUS%20Corporate%20IS&master=00471&toggle=00519
060612 134230 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aSell%20Red%20Hat%20Training&master=00611&toggle=00627
060612 134233 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aHealthcare%20Vertical&master=01046&toggle=01203
060612 134236 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony%3aHeadsets&master=01481&toggle=01486
060612 134239 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone%3aCriteria&master=01395&toggle=01396
060612 134244 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aTraining%20and%20Development&master=00807&toggle=00889
060612 134247 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aServices&master=00043&toggle=00346
060612 134251 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aRHEA%20International&master=00980&toggle=01021
060612 134254 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aInternal%20Jobs%20Posting&master=00807&toggle=00868
060612 134257 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aMarketing%20Operations&master=01046&toggle=01283
060612 134301 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aTechnical%20Support&master=00633&toggle=00803
060612 134304 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aTrainingSchedule&master=00980&toggle=01024
060612 134308 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aProduct%20Marketing%20%26%20Management&master=00043&toggle=00177
060612 134311 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aProduct%20Management&master=00043&toggle=00122
060612 134314 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aPolicy%20%26%20Procedure&master=00807&toggle=00885
060612 134317 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aContract%20Info&master=01407&toggle=01427
060612 134322 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aProduction%20Support&master=00633&toggle=00665
060612 134325 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aReferral%20Bonus%20Program&master=00807&toggle=00888
060612 134329 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aResearch%20Library&master=01046&toggle=01337
060612 134331 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Product%20Marketing
060612 134334 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations%3aItem%20Request%20Form&master=01476&toggle=01478
060612 134337 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aProduct%20Management&master=01356&toggle=01369
060612 134341 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aSecurity%20Response%20Team&master=00043&toggle=00339
060612 134345 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aAPAC%20South&master=01407&toggle=01413
060612 134348 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aWeb%20Production&master=00471&toggle=00521
060612 134350 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aEMEA&master=01356&toggle=01362 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134350 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aInternationalPartners&master=00980&toggle=01011
060612 134351 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aOrg%20Chart&master=01046&toggle=01284
060612 134354 fetching http://intranet.corp.redhat.com/ic/intranet/NeilFox.html
060612 134357 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities%3aProcedures%20%26%20Policies&master=00372&toggle=00373
060612 134400 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aEngineering&master=01356&toggle=01366
060612 134404 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aGSS%20Test%20Lab&master=00633&toggle=00646
060612 134407 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aCompliance%20and%20Ethics%20Training&master=01036&toggle=01038
060612 134410 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations%3aMedia%20Kit%20Request%20Form&master=01476&toggle=01479
060612 134412 fetching http://intranet.corp.redhat.com/ic/intranet/KateJohnson.html
060612 134415 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aServices&master=00043&toggle=00346 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134415 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aRed%20Hat%20Network&master=00043&toggle=00336
060612 134415 fetching http://intranet.corp.redhat.com/ic/intranet/MikeEvans.html
060612 134419 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aRHEL%20International&master=00980&toggle=01022
060612 134423 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aPUNE&master=00633&toggle=00658
060612 134426 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aCommonQA&master=01407&toggle=01426
060612 134429 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aSolutions&master=00043&toggle=00350
060612 134432 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aPartner%20Development&master=01046&toggle=01285
060612 134435 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Lead%20Generation%3aLead%20Gen%20Reports&master=01031&toggle=01033
060612 134438 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aEngineering&master=01376&toggle=01379
060612 134442 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony%3aCalling%20Cards&master=01481&toggle=01484
060612 134445 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Lead%20Generation%3aArchived%20SD%20Reports&master=01031&toggle=01032
060612 134448 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aSecurityEngineering&master=00043&toggle=00340
060612 134450 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aTools%20Engineering&master=00043&toggle=00357
060612 134454 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aProduct%20Management&master=01376&toggle=01389
060612 134457 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aPeople&master=00980&toggle=01018
060612 134501 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aEMEA&master=00633&toggle=00642
060612 134504 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aAPAC%20North&master=01407&toggle=01410
060612 134507 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aCorporate%20Information&master=01036&toggle=01039
060612 134510 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aOracle%20Production%20Ops&master=00471&toggle=00511
060612 134513 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aPresentationMaterial&master=01046&toggle=01318
060612 134517 status: segment 20060612133953, 100 pages, 5 errors, 2812608 bytes, 323145 ms
060612 134517 status: 0.3094586 pages/s, 67.998886 kb/s, 28126.08 bytes/page
060612 134517 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aCode%20of%20Business%20Conduct%20and%20Ethics&master=01036&toggle=01037
060612 134520 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aVendor%20Contracting&master=00001&toggle=00008
060612 134524 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aDocumentation&master=00043&toggle=00076
060612 134527 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications%3aProfessional%20Services&master=01356&toggle=01373
060612 134529 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aNon%20disclosure%20Agreements&master=00001&toggle=00006
060612 134533 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aTSANetProcedures&master=00633&toggle=00798
060612 134536 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCustomer%20Service%20%2d%20APAC&master=00011&toggle=00013
060612 134540 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aContent%20Services&master=00043&toggle=00047
060612 134543 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCustomer%20Service%20Teams&master=00011&toggle=00038
060612 134546 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aGSS%20Operations&master=00633&toggle=00644
060612 134549 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone%3aTraining&master=01395&toggle=01406
060612 134552 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aDirectory%20%26%20Security&master=00043&toggle=00064
060612 134558 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCustomer%20Service%20Info&master=00011&toggle=00027
060612 134601 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Lead%20Generation%3aLead%20Gen%20Reports&master=01031&toggle=01033 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134601 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aAPAC&master=00633&toggle=00635
060612 134601 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aQuick%20Guide&master=00633&toggle=00679
060612 134604 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aInternal&master=00611&toggle=00613
060612 134608 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aPurchasing%20Group&master=00426&toggle=00439
060612 134610 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aHuman%20Capital%20Contact%20List&master=00807&toggle=00867
060612 134614 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aIntroduction&master=00043&toggle=00121
060612 134618 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aProducts%20and%20Services&master=01046&toggle=01319
060612 134622 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aPlanning&master=01345&toggle=01349
060612 134625 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aProgram%20Marketing&master=01046&toggle=01324
060612 134628 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCustomer%20Service%20EMEA&master=00011&toggle=00019
060612 134631 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aEvents&master=00980&toggle=00984
060612 134633 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aCC%20Engineering&master=01376&toggle=01377
060612 134635 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aGlobal%20Production%20Ops&master=00471&toggle=00473
060612 134639 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony%3aCell%20Phone%20Wireless%20Information&master=01481&toggle=01485
060612 134643 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities%3aWhere%20We%20Are&master=00372&toggle=00381
060612 134646 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony%3aCall%20Center%20Extensions&master=01481&toggle=01483
060612 134648 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aProjects&master=01345&toggle=01352
060612 134652 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aCompensation&master=00807&toggle=00833
060612 134655 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aForm%20Agreements&master=00001&toggle=00004
060612 134700 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aArchive&master=01345&toggle=01346
060612 134702 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aBasic%20Financial%20Terms&master=00426&toggle=00427
060612 134704 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aContent%20Services&master=00043&toggle=00047 failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 134704 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aSupportEscalationsGroup&master=00633&toggle=00794
060612 134705 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aProcess&master=01345&toggle=01350
060612 134708 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aWho%20We%20Are&master=00611&toggle=00629
060612 134712 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aEMEA%20Services&master=00980&toggle=00981
060612 134716 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aInternationalization&master=00043&toggle=00084
060612 134720 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aInternal%20Processes&master=01376&toggle=01384
060612 134722 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aDeveloper%20Nation&master=01046&toggle=01194
060612 134725 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Product%20Marketing%3aJoelsStuff&master=01343&toggle=01344
060612 134729 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCS%20Key%20Projects&master=00011&toggle=00012
060612 134732 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services%3aHelp%20Desk&master=00471&toggle=00474
060612 134735 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aTake%20Red%20Hat%20Training&master=00611&toggle=00628
060612 134739 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aTemplates&master=00011&toggle=00042
060612 134741 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aCopyright%20Guidelines&master=00043&toggle=00060
060612 134744 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital%3aHR%20Online%20Forms&master=00807&toggle=00840
060612 134748 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations%3aBundle%20Request%20Form&master=01476&toggle=01477
060612 134752 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aApproval%20Framework&master=00001&toggle=00002
060612 134756 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aAPAC%20Japan&master=01407&toggle=01409
060612 134759 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aRed%20Hat%20Knowledgebase&master=00633&toggle=00680
060612 134802 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aContract%20Processing&master=00001&toggle=00003
060612 134806 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services%3aOperational%20Metrics&master=00611&toggle=00623
060612 134809 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aPartner%20Marketing&master=01046&toggle=01286
060612 134813 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aDomain%20Names&master=01036&toggle=01040
060612 134817 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aOffices&master=00980&toggle=01012
060612 134820 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Applications
060612 134822 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance%3aRevenue%20Recognition&master=00426&toggle=00466
060612 134825 fetching http://intranet.corp.redhat.com/ic/intranet/CharlesPeters.html
060612 134829 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aPartner%20Complaint%20Form&master=00011&toggle=00041
060612 134832 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs%3aPolicies%20and%20Guidelines&master=01036&toggle=01043
060612 134835 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3aMeet%20Production%20Support&master=00633&toggle=00657
060612 134838 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services%3a24x7customerinfos&master=00633&toggle=00634
060612 134842 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aRHEN%20International&master=00980&toggle=01023
060612 134846 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service%3aCustomer%20Service%20Reports&master=00011&toggle=00035
060612 134850 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs%3aSalesAlert&master=00001&toggle=00007
060612 134853 fetching http://intranet.corp.redhat.com/ic/intranet/PaulCormier.html
060612 134926 status: segment 20060612133953, 175 pages, 7 errors, 4858786 bytes, 572185 ms
060612 134926 status: 0.30584514 pages/s, 66.3409 kb/s, 27764.492 bytes/page
060612 134927 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 134927 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133953
060612 134927 Processing document 0
060612 134928 Finishing update
060612 134928 Processing pagesByURL: Sorted 6424 instructions in 0.215 seconds.
060612 134928 Processing pagesByURL: Sorted 29879.069767441862 instructions/second
060612 134929 Processing pagesByURL: Merged to new DB containing 253 records in 0.03 seconds
060612 134929 Processing pagesByURL: Merged 8433.333333333334 records/second
060612 134929 Processing pagesByMD5: Sorted 393 instructions in 0.052 seconds.
060612 134929 Processing pagesByMD5: Sorted 7557.692307692308 instructions/second
060612 134929 Processing pagesByMD5: Merged to new DB containing 253 records in 0.0070 seconds
060612 134929 Processing pagesByMD5: Merged 36142.857142857145 records/second
060612 134929 Processing linksByMD5: Sorted 6417 instructions in 0.193 seconds.
060612 134929 Processing linksByMD5: Sorted 33248.70466321243 instructions/second
060612 134929 Processing linksByMD5: Merged to new DB containing 7111 records in 0.087 seconds
060612 134929 Processing linksByMD5: Merged 81735.63218390805 records/second
060612 134930 Processing linksByURL: Sorted 6067 instructions in 0.261 seconds.
060612 134930 Processing linksByURL: Sorted 23245.21072796935 instructions/second
060612 134930 Processing linksByURL: Merged to new DB containing 7111 records in 0.104 seconds
060612 134930 Processing linksByURL: Merged 68375.0 records/second
060612 134930 Processing linksByMD5: Sorted 6235 instructions in 0.233 seconds.
060612 134930 Processing linksByMD5: Sorted 26759.656652360514 instructions/second
060612 134930 Processing linksByMD5: Merged to new DB containing 7111 records in 0.087 seconds
060612 134930 Processing linksByMD5: Merged 81735.63218390805 records/second
060612 134931 Update finished
060612 134931 FetchListTool started
060612 134931 Processing pagesByURL: Sorted 36 instructions in 0.0040 seconds.
060612 134931 Processing pagesByURL: Sorted 9000.0 instructions/second
060612 134931 Processing pagesByURL: Merged to new DB containing 253 records in 0.0030 seconds
060612 134931 Processing pagesByURL: Merged 84333.33333333333 records/second
060612 134931 Processing pagesByMD5: Sorted 36 instructions in 0.0050 seconds.
060612 134931 Processing pagesByMD5: Sorted 7200.0 instructions/second
060612 134931 Processing pagesByMD5: Merged to new DB containing 253 records in 0.0050 seconds
060612 134931 Processing pagesByMD5: Merged 50600.0 records/second
060612 134931 Processing linksByMD5: Copied file (4096 bytes) in 0.102 secs.
060612 134932 Processing linksByURL: Copied file (4096 bytes) in 0.111 secs.
060612 134932 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612134931/fetchlist.unsorted: Sorted 36 entries in 0.0040 seconds.
060612 134932 Processing /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612134931/fetchlist.unsorted: Sorted 9000.0 entries/second
060612 134932 Overall processing: Sorted 36 entries in 0.0040 seconds.
060612 134932 Overall processing: Sorted 1.1111111111111112E-4 entries/second
060612 134932 FetchListTool completed
060612 134932 logging at INFO
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aEvents
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Telephony
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities%3aProcedures%20%26%20Policies
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Lead%20Generation
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Finance
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing%3aMarketing%20Communications
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aPeople
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Customer%20Service
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Information%20Services
060612 134932 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Reward%20Zone
060612 134933 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering
060612 134938 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%20Services
060612 134941 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Learning%20Services
060612 134945 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Marketing
060612 134948 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aProduct%20Marketing%20%26%20Management
060612 134951 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aInternal%20Processes
060612 134954 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales
060612 134956 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aCC%20Engineering
060612 135000 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting%3aArchive
060612 135003 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network%3aMeetings
060612 135005 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aPeople%3aGusRobertson&master=00980&toggle=01020
060612 135008 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations
060612 135010 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Human%20Capital
060612 135014 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Legal%20Affairs
060612 135018 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Professional%20Consulting
060612 135021 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Business%20Affairs
060612 135024 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Open%20Source%20Now
060612 135027 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Sales%3aAPAC%20North
060612 135030 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aDocumentation
060612 135034 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Global%20Support%20Services
060612 135037 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aOffices
060612 135039 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Facilities
060612 135042 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=International%20Operations%3aPeople%3aAlexPinchev&master=00980&toggle=01019
060612 135045 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Red%20Hat%20Network
060612 135049 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Subscription%20Operations
060612 135053 fetching http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering%3aDeveloper%20Tools
060612 135101 fetch of http://intranet.corp.redhat.com/ic/intranet/wiki.html?wikicat=Engineering failed with: java.lang.Exception: org.apache.nutch.protocol.RetryLater: Exceeded http.max.delays: retry later.
060612 135115 status: segment 20060612134931, 35 pages, 1 errors, 1046143 bytes, 103210 ms
060612 135115 status: 0.33911443 pages/s, 79.18799 kb/s, 29889.8 bytes/page
060612 135116 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135117 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612134931
060612 135117 Processing document 0
060612 135117 Finishing update
060612 135117 Processing pagesByURL: Sorted 1330 instructions in 0.046 seconds.
060612 135117 Processing pagesByURL: Sorted 28913.04347826087 instructions/second
060612 135117 Processing pagesByURL: Merged to new DB containing 253 records in 0.01 seconds
060612 135117 Processing pagesByURL: Merged 25300.0 records/second
060612 135117 Processing pagesByMD5: Sorted 71 instructions in 0.029 seconds.
060612 135117 Processing pagesByMD5: Sorted 2448.2758620689656 instructions/second
060612 135117 Processing pagesByMD5: Merged to new DB containing 253 records in 0.0050 seconds
060612 135117 Processing pagesByMD5: Merged 50600.0 records/second
060612 135118 Processing linksByMD5: Sorted 1329 instructions in 0.054 seconds.
060612 135118 Processing linksByMD5: Sorted 24611.111111111113 instructions/second
060612 135118 Processing linksByMD5: Merged to new DB containing 7181 records in 0.072 seconds
060612 135118 Processing linksByMD5: Merged 99736.11111111112 records/second
060612 135118 Processing linksByURL: Sorted 1259 instructions in 0.064 seconds.
060612 135118 Processing linksByURL: Sorted 19671.875 instructions/second
060612 135118 Processing linksByURL: Merged to new DB containing 7181 records in 0.082 seconds
060612 135118 Processing linksByURL: Merged 87573.17073170732 records/second
060612 135118 Processing linksByMD5: Sorted 1393 instructions in 0.059 seconds.
060612 135118 Processing linksByMD5: Sorted 23610.169491525427 instructions/second
060612 135119 Processing linksByMD5: Merged to new DB containing 7181 records in 0.07 seconds
060612 135119 Processing linksByMD5: Merged 102585.71428571428 records/second
060612 135120 Update finished
060612 135120 FetchListTool started
060612 135121 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135121 Overall processing: Sorted NaN entries/second
060612 135121 FetchListTool completed
060612 135121 logging at INFO
060612 135122 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135122 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135120
060612 135122 Finishing update
060612 135122 Update finished
060612 135122 FetchListTool started
060612 135123 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135123 Overall processing: Sorted NaN entries/second
060612 135123 FetchListTool completed
060612 135123 logging at INFO
060612 135124 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135124 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135122
060612 135124 Finishing update
060612 135124 Update finished
060612 135124 FetchListTool started
060612 135125 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135125 Overall processing: Sorted NaN entries/second
060612 135125 FetchListTool completed
060612 135125 logging at INFO
060612 135126 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135126 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135124
060612 135126 Finishing update
060612 135126 Update finished
060612 135126 FetchListTool started
060612 135127 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135127 Overall processing: Sorted NaN entries/second
060612 135127 FetchListTool completed
060612 135127 logging at INFO
060612 135128 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135128 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135126
060612 135128 Finishing update
060612 135128 Update finished
060612 135128 FetchListTool started
060612 135129 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135129 Overall processing: Sorted NaN entries/second
060612 135129 FetchListTool completed
060612 135129 logging at INFO
060612 135130 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135130 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135128
060612 135130 Finishing update
060612 135130 Update finished
060612 135130 FetchListTool started
060612 135131 Overall processing: Sorted 0 entries in 0.0 seconds.
060612 135131 Overall processing: Sorted NaN entries/second
060612 135131 FetchListTool completed
060612 135131 logging at INFO
060612 135132 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135132 Updating for /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135130
060612 135132 Finishing update
060612 135132 Update finished
060612 135132 Updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments from /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133752
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133953
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612134931
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135120
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135122
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135124
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135126
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135128
060612 135132  reading /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612135130
060612 135133 Sorting pages by url...
060612 135133 Getting updated scores and anchors from db...
060612 135133 Sorting updates by segment...
060612 135133 Updating segments...
060612 135133  updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748
060612 135133  updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133752
060612 135133  updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133953
060612 135133  updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612134931
060612 135133 Done updating /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments from /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/db
060612 135133 indexing segment: /home/rdu/mholt/Desktop/nutch-0.7.2/crawl/segments/20060612133748
060612 135134 * Opening segment 20060612133748
060612 135134 * Indexing segment 20060612133748
060612 135134 found resource common-terms.utf8 at file:/home/rdu/mholt/Desktop/nutch-0.7.2/conf/common-terms.utf8
060612 135134 * Optimizing index...
060612 135134 * Moving index to NFS if needed...
060612 135134 DONE indexing segment 20060612133748: total 1 records in 0.571 s (Infinity rec/s).
060612 135134 done