Wednesday, 15 February 2012

java - MapReduce code cleaning staging area error -



java - MapReduce code cleaning staging area error -

i new mapreduce running mapreduce program.its compiling fine , jar file created without error when run final hadoop command shows next error , stops. please help asap... finish log:

[u@h(ipc2-gold) w]$ hadoop jar join.jar runner -dmapred.job.queue.name=score2 /axp/rim/score2/dev/cmc_score/rishabh/inp/file1.txt /axp/rim/score2/dev/cmc_score/rishabh/inp/file2.txt /axp/rim/score2/dev/cmc_score/rishabh/asd 14/06/19 21:28:44 info fs.jobtrackerwatcher: current running jobtracker is: lgpbd1010.gso.aexp.com/10.22.45.20:9001 14/06/19 21:28:44 info mapred.jobclient: cleaning staging area maprfs:/var/mapr/cluster/mapred/jobtracker/staging/rdwiv5/.staging/job_201405310436_72947 exception in thread "main" org.apache.hadoop.mapred.filealreadyexistsexception: output directory /axp/rim/score2/dev/cmc_score/rishabh/inp/file2.txt exists @ org.apache.hadoop.mapreduce.lib.output.fileoutputformat.checkoutputspecs(fileoutputformat.java:132) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:926) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:885) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:396) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1127) @ org.apache.hadoop.mapred.jobclient.submitjobinternal(jobclient.java:885) @ org.apache.hadoop.mapreduce.job.submit(job.java:536) @ org.apache.hadoop.mapreduce.job.waitforcompletion(job.java:566) @ runner.run(runner.java:55) @ org.apache.hadoop.util.toolrunner.run(toolrunner.java:65) @ runner.main(runner.java:64) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.util.runjar.main(runjar.java:197)

i see have 3 arguments (two input files , 1 output dir). think happening programme expecting two, 1 input , 1 output. if want files in directory input, utilize directory input

/axp/rim/score2/dev/cmc_score/rishabh/inp

hadoop grab files directory. what's happening hadoop taking sec argument (which file in same directory) output directory, that's why you're getting exception

note: check directories , see if

/axp/rim/score2/dev/cmc_score/rishabh/inp/file1.txt

directory exists. it's possible accidentily ran programme path output, , file1.txt directory (not file) created also. when ran 1 time again trying utilize same arguments exception because directory exists, perchance same reason discussed above (the programme expecting 2 args)

java hadoop mapreduce

No comments:

Post a Comment