lundi 5 janvier 2015

Hadoop Data Replication error

Hope you all had a wonderful vacation. I am trying to setup Hadoop cluster on Amazon EC2. While copying data file from local disk to hdfs with the command hadoop fs -copyFromLocal d.txt /user/ubuntu/data, I am getting data replication error. The error from the log is following



15/01/06 00:09:50 WARN hdfs.DFSClient: Could not get block locations. Source file "/user/ubuntu/data/d.txt" - Aborting... copyFromLocal: java.io.IOException: File /user/ubuntu/data/d.txt could only be replicate d to 0 nodes, instead of 1


15/01/06 00:09:50 ERROR hdfs.DFSClient: Failed to close file /user/ubuntu/data/d.txt org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/ubuntu/data/d.txt could only be replicated to 0 nodes, instead of 1



Now, I had been checking StackOverFlow and other forums about this problem and I found most of them talk about DataNode, TaskTracker not running as a probable cause & relevant solutions. But these things are running fine in my setup. The screenshot of the JPS command http://ift.tt/1Iik97K


From HadooWiki, the other possible causes are DataNode not able talk to the server, through networking or Hadoop configuration problems or some configuration problem is preventing effective two-way communication.


I have configured hadoop-env.sh, core-site.xml, hdfs-site.xml and mapred-site.xml following the tutorial http://ift.tt/17esAWf . Could anyone tell please me what I am diong wrong ? I badly need this to be dnoe for my project. I will be immensely grateful if anyone help me to resolve the problem.


Thanks,





Aucun commentaire:

Enregistrer un commentaire