Java - MySQL to Hive Import where MySQL Running on Windows and Hive Running on Cent OS (Horton Sandbox) -
before reply , comments. tried several alternative found in stackoverflow end failure. next links -
how can execute sqoop in java? how utilize sqoop in java program? how import table mysql hive using java? how load sql info hortonworks sandbox?i tried in horton sandbox through command line , succeded.
sqoop import --connect jdbc:mysql://192.168.56.101:3316/database_name --username=user --password=pwd --table table_name --hive-import -m 1 -- --schema defaultwhere 192.168.56.101 windows , 192.168.56.102 horton sandbox 2.6.
now want same thing java java code run somewhere else not in horton sandbox.
how loacate hive_home , other sqoop parameters because running in sandbox. parameters have pass. should passes sqoopoptions or sqoop.runtools string array arguments. both failing. i confused while import library (com.cloudera.sqoop , org.apache.sqoop) , thisthe method run(com.cloudera.sqoop.sqoopoptions) in type importtool not applicable arguments (org.apache.sqoop.sqoopoptions) 2 line (option parameter added between 2 lines)
sqoopoptions options = new sqoopoptions(); int ret = new importtool().run(options);
if take cloudera method deprecated if take apace run method doesn't take options argument
i strucked in weeks. please help.
yes can via ssh. horton sandbox comes ssh back upwards pre installed. can execute sqoop command via ssh client on windows. or if want programaticaly (thats have done in java) have follow step.
download sshxcute java library : https://code.google.com/p/sshxcute/ add build path of java project contains next java codeimport net.neoremind.sshxcute.core.sshexec; import net.neoremind.sshxcute.core.connbean; import net.neoremind.sshxcute.task.customtask; import net.neoremind.sshxcute.task.impl.execcommand; public class testssh { public static void main(string args[]) throws exception{ // initialize connbean object, parameter list ip, username, password connbean cb = new connbean("192.168.56.102", "root","hadoop"); // set connbean instance parameter sshexec static method getinstance(connbean) retrieve singleton sshexec instance sshexec ssh = sshexec.getinstance(cb); // connect server ssh.connect(); customtask sampletask1 = new execcommand("echo $ssh_client"); // print client ip connected ssh server on horton sandbox system.out.println(ssh.exec(sampletask1)); customtask sampletask2 = new execcommand("sqoop import --connect jdbc:mysql://192.168.56.101:3316/mysql_db_name --username=mysql_user --password=mysql_pwd --table mysql_table_name --hive-import -m 1 -- --schema default"); ssh.exec(sampletask2); ssh.disconnect(); } }
java mysql hive sqoop hortonworks-data-platform
No comments:
Post a Comment