DarkMatter in Cyberspace
  • Home
  • Categories
  • Tags
  • Archives

Build Hadoop Develop Environment


Prerequisites

  1. Download Oracle jdk-6u45-linux-i586.bin from Oracle website. Openjdk is unsuitable here because some package like sun.net.dns.ResolverConfiguration only exists in rt.jar of Oracle jdk;

  2. Install (./jdk.bin) this jdk at ~/apps/jdk with current user (do not install it into /opt with root user, because this will cause access restriction of rt.jar of jdk);

  3. Add $JAVA_HOME and $CLASSPATH into /etc/profile and re-log into OS to validate;

  4. Add this jdk to eclipse (Preferences -> Install JREs);

Method 8-10

Follow Working with Hadoop under Eclipse

  1. make sure your maven version is 3.x (mvn --version), if not:

sudo apt-get remove maven2 sudo apt-get update sudo apt-get install maven

  1. git clone git://git.apache.org/hadoop-common.git ()

  2. mvn install -DskipTests

  3. mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true

Method 8-9

  1. Create a Java Project in Eclipse, modify default output folder to $PROJECT_HOME/out (because "bin" is a script folder in hadoop source pack);

  2. Exclude src from source folder and create "lib", "conf" folder under $PROJECT_HOME;

  3. Import $HADOOP_HOME/src, lib, conf into $PROJECT_HOME;

  4. Add all jar files under lib to build path;

  5. Add ant.jar to build path;

  6. Add mapred, tools, ant, core, hdfs under src folder as source folder;

Verify

  1. modify config files;

  2. run hadoop



Published

Aug 9, 2013

Last Updated

Aug 9, 2013

Category

Tech

Tags

  • Eclipse 40
  • hadoop 7
  • JDK 8

Contact

  • Powered by Pelican. Theme: Elegant by Talha Mansoor