java - MapReduce code cleaning staging area error -
i new mapreduce running mapreduce program.its compiling fine , jar file created without error when run final hadoop command shows next error , stops. please help asap... finish log:
[u@h(ipc2-gold) w]$ hadoop jar join.jar runner -dmapred.job.queue.name=score2 /axp/rim/score2/dev/cmc_score/rishabh/inp/file1.txt /axp/rim/score2/dev/cmc_score/rishabh/inp/file2.txt /axp/rim/score2/dev/cmc_score/rishabh/asd 14/06/19 21:28:44 info fs.jobtrackerwatcher: current running jobtracker is: lgpbd1010.gso.aexp.com/10.22.45.20:9001 14/06/19 21:28:44 info mapred.jobclient: cleaning staging area maprfs:/var/mapr/cluster/mapred/jobtracker/staging/rdwiv5/.staging/job_201405310436_72947 exception in thread "main" org.apache.hadoop.mapred.filealreadyexistsexception: output directory /axp/rim/score2/dev/cmc_score/rishabh/inp/file2.txt exists @ org.apache.hadoop.mapreduce.lib.output.fileoutputformat.checkoutputspecs(fileoutputformat.java:132) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:926) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:885) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:396) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1127) @ org.apache.hadoop.mapred.jobclient.submitjobinternal(jobclient.java:885) @ org.apache.hadoop.mapreduce.job.submit(job.java:536) @ org.apache.hadoop.mapreduce.job.waitforcompletion(job.java:566) @ runner.run(runner.java:55) @ org.apache.hadoop.util.toolrunner.run(toolrunner.java:65) @ runner.main(runner.java:64) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.util.runjar.main(runjar.java:197)
i see have 3 arguments (two input files , 1 output dir). think happening programme expecting two, 1 input , 1 output. if want files in directory input, utilize directory input
/axp/rim/score2/dev/cmc_score/rishabh/inp
hadoop grab files directory. what's happening hadoop taking sec argument (which file in same directory) output directory, that's why you're getting exception
note: check directories , see if
/axp/rim/score2/dev/cmc_score/rishabh/inp/file1.txt
directory exists. it's possible accidentily ran programme path output, , file1.txt
directory (not file) created also. when ran 1 time again trying utilize same arguments exception because directory exists, perchance same reason discussed above (the programme expecting 2 args)
java hadoop mapreduce
No comments:
Post a Comment