Quantcast
Channel: JetBrains Developer Community : Thread List - IntelliJ IDEA Users
Viewing all articles
Browse latest Browse all 5661

Cant launch MapReduce job while debugging when hadoop is in pseudo mode

$
0
0

I'm having an issue inIntelliJ when debugging a function that launches a MapReduce job via Hadoop's job.waitForCompletion(), when hadoop is configured to run in pseudo mode.  The Hadoop job tracker gets the job, but quickly throws a ClassNotFoundException on my custom mapper class.  Turns out the Hadoop client API isnt passing the project jar to Hadoop.  It looks like the IntelliJ launched JVM doesnt have the (maven) generated jar file for the project in the class path. 

 

Is there a way I can configure the JVM process (launced by IntelliJ) to have the maven generated jar in the class path, so when I launch the Hadoop jar, it will pass the jar to the Hadoop job tracker?

 

BTW: If the hadoop configuration is configured to run in single mode, everything runs just fine (in process)

 

Thanks,
John


Viewing all articles
Browse latest Browse all 5661

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>