hadoop2 - Hadoop jobs fail when submitted by users other than yarn (MRv2) or mapred (MRv1) -

hadoop2 - Hadoop jobs fail when submitted by users other than yarn (MRv2) or mapred (MRv1) -

i running test cluster running mrv1 (cdh5) paired localfilesystem, , user able run jobs mapred (as mapred user starting jobtracker/tasktracker daemons). when submitting jobs other user, jobs fail because jobtracker/tasktracker unable find job.jar under .staging directory.

i have exact same issue yarn (mrv2) when paired localfilesystem, i.e. when submitting jobs user other 'yarn', application master unable locate job.jar under .staging directory.

upon inspecting .staging directory of user submitting job found job.jar exists under .staging// directory, permissions on , .staging directories set 700 (drwx------) , hence application master / tasktracker not able access job.jar , supporting files.

we running test cluster localfilesystem since utilize mapreduce part of hadoop project paired ocfs in our production setup.

any assistance in regard immensely helpful.

you need setting staging directory each user in cluster. not complicated sounds.

check next properties:

<property> <name>hadoop.tmp.dir</name> <value>/tmp/hadoop-${user.name}</value> <source>core-default.xml</source> </property>

this setups tmp directory each user.

tie staging directory :

<property> <name>mapreduce.jobtracker.staging.root.dir</name> <value>${hadoop.tmp.dir}/mapred/staging</value> <source>mapred-default.xml</source> </property>

let me know if works or if setup way.

these properties should in yarn-site.xml - if remember correctly.

hadoop hadoop2