[jira] Created: (NUTCH-127) uncorrect values using -du, or ls does not return items

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (NUTCH-127) uncorrect values using -du, or ls does not return items

Steve Loughran (Jira)
uncorrect values using -du, or ls does not return items
-------------------------------------------------------

         Key: NUTCH-127
         URL: http://issues.apache.org/jira/browse/NUTCH-127
     Project: Nutch
        Type: Bug
  Components: ndfs  
    Versions: 0.8-dev, 0.7.2-dev    
    Reporter: Stefan Groschupf
    Priority: Blocker


The ndfs client return uncorrect values by using du or ls does not return items.
It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.


[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 092409 No FS indicated, using default:192.168.200.3:50000
051118 092409 Client connection to 192.168.200.3:50000: starting
Found 1 items
/user/nutch/indexes/20051022033721      974606348

[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 092416 No FS indicated, using default:192.168.200.3:50000
051118 092416 Client connection to 192.168.200.3:50000: starting
Found 0 items


[nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
051118 093332 No FS indicated, using default:192.168.200.3:50000
051118 093332 Client connection to 192.168.200.3:50000: starting
Found 0 items

So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.



--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-127) uncorrect values using -du, or ls does not return items

Steve Loughran (Jira)
    [ http://issues.apache.org/jira/browse/NUTCH-127?page=comments#action_12363613 ]

Mike Cafarella commented on NUTCH-127:
--------------------------------------

Stefan,

I fixed a namespace problem in NDFS just before Christmas.  It sounds like your
issue falls into the set of things I fixed.

You mention the "mv" tool.  The problem I fixed was:  if you tried to move certain
directories, the move would fail and the dir you tried to move would disappear.

If that was your problem, then we can close this bug!


> uncorrect values using -du, or ls does not return items
> -------------------------------------------------------
>
>          Key: NUTCH-127
>          URL: http://issues.apache.org/jira/browse/NUTCH-127
>      Project: Nutch
>         Type: Bug
>   Components: ndfs
>     Versions: 0.8-dev, 0.7.2-dev
>     Reporter: Stefan Groschupf
>     Priority: Blocker

>
> The ndfs client return uncorrect values by using du or ls does not return items.
> It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
> We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092409 No FS indicated, using default:192.168.200.3:50000
> 051118 092409 Client connection to 192.168.200.3:50000: starting
> Found 1 items
> /user/nutch/indexes/20051022033721      974606348
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092416 No FS indicated, using default:192.168.200.3:50000
> 051118 092416 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
> 051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 093332 No FS indicated, using default:192.168.200.3:50000
> 051118 093332 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (NUTCH-127) uncorrect values using -du, or ls does not return items

Steve Loughran (Jira)
In reply to this post by Steve Loughran (Jira)
     [ http://issues.apache.org/jira/browse/NUTCH-127?page=all ]
     
Stefan Groschupf resolved NUTCH-127:
------------------------------------

    Resolution: Fixed

I guess it is solved, thanks. If able to reproduce it again I will just reopen this or a new report.
Thanks!

> uncorrect values using -du, or ls does not return items
> -------------------------------------------------------
>
>          Key: NUTCH-127
>          URL: http://issues.apache.org/jira/browse/NUTCH-127
>      Project: Nutch
>         Type: Bug
>   Components: ndfs
>     Versions: 0.8-dev, 0.7.2-dev
>     Reporter: Stefan Groschupf
>     Priority: Blocker

>
> The ndfs client return uncorrect values by using du or ls does not return items.
> It looks like there is a problem with the virtual file strcuture, since -du only reads the meta data, isn't it?
> We had moved some data from folder to folder and after that we notice that a folder with zero items has a size.
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092409 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092409 No FS indicated, using default:192.168.200.3:50000
> 051118 092409 Client connection to 192.168.200.3:50000: starting
> Found 1 items
> /user/nutch/indexes/20051022033721      974606348
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -du indexes/20051022033721/
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 092416 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 092416 No FS indicated, using default:192.168.200.3:50000
> 051118 092416 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> [nutch@myservernutch-0.8-dev]$ bin/nutch ndfs -ls indexes/20051022033721
> 051118 093331 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-default.xml
> 051118 093332 parsing file:/home/nutch/nutch-0.8-dev/conf/nutch-site.xml
> 051118 093332 No FS indicated, using default:192.168.200.3:50000
> 051118 093332 Client connection to 192.168.200.3:50000: starting
> Found 0 items
> So may the mv tool has a problem, the du or the ls tool. :-O Any ideas where to search for the problem? Dubugging ndfs is tricky.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira