[jira] Created: (NUTCH-256) Cannot open filename ....index.done.crc

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
Cannot open filename ....index.done.crc
---------------------------------------

         Key: NUTCH-256
         URL: http://issues.apache.org/jira/browse/NUTCH-256
     Project: Nutch
        Type: Bug

  Components: indexer  
    Versions: 0.8-dev    
    Reporter: [hidden email]
    Priority: Minor


Trying to copy indices out of DFS I always get:

[bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
060427 160318 No FS indicated, using default:localhost:9001
060427 160318 Client connection to 127.0.0.1:9001: starting
060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
        at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:589)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)



--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/NUTCH-256?page=all ]

[hidden email] updated NUTCH-256:
------------------------------------

    Attachment: index.done.crc.patch

Ensure creation of companion index.done .crc file

> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376839 ]

Doug Cutting commented on NUTCH-256:
------------------------------------

That's not a fatal exception, right?  Everything still works?  It should.  This is just the DFS version of FileNotFound, which we otherwise ignore.  Perhaps we should fix DFS to really throw a FileNotFound in this case, or we should just change this to a LOG.fine()?

> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376840 ]

[hidden email] commented on NUTCH-256:
-----------------------------------------

Correct.  Though it looks fatal, its not.  Nor does it inspire confidence in the copy just made.



> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376993 ]

Doug Cutting commented on NUTCH-256:
------------------------------------

I think this is really a bug in Hadoop's FileSystem.createNewFile() method.  I've just fixed that.  Does that work for you?  If so, I'll close this.

> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/NUTCH-256?page=comments#action_12376999 ]

[hidden email] commented on NUTCH-256:
-----------------------------------------

Works for me.  Thanks.  Please close as fixed.

> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (NUTCH-256) Cannot open filename ....index.done.crc

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/NUTCH-256?page=all ]
     
Doug Cutting resolved NUTCH-256:
--------------------------------

    Resolution: Fixed
     Assign To: Doug Cutting

This is fixed in Hadoop 0.2.

> Cannot open filename ....index.done.crc
> ---------------------------------------
>
>          Key: NUTCH-256
>          URL: http://issues.apache.org/jira/browse/NUTCH-256
>      Project: Nutch
>         Type: Bug

>   Components: indexer
>     Versions: 0.8-dev
>     Reporter: [hidden email]
>     Assignee: Doug Cutting
>     Priority: Minor
>  Attachments: index.done.crc.patch
>
> Trying to copy indices out of DFS I always get:
> [bregeon] workspace > ./hadoop/bin/hadoop dfs -get outputs .
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-default.xml
> 060427 160317 parsing file:/home/stack/workspace/hadoop-local-conf/hadoop-site.xml
> 060427 160318 No FS indicated, using default:localhost:9001
> 060427 160318 Client connection to 127.0.0.1:9001: starting
> 060427 160318 Problem opening checksum file: /user/stack/outputs/indexes/part-00000/index.done.  Ignoring with exception org.apache.hadoop.ipc.RemoteException: java.io.IOException: Cannot open filename /user/stack/outputs/indexes/part-00000/.index.done.crc
>         at org.apache.hadoop.dfs.NameNode.open(NameNode.java:130)
>         at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:589)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:240)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:218)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira