I have a problem running a map-reduce java application I simplified my problem using the tutorial code given from AWS which runs a pre-defined step:
public class Main {
public static void main(String[] args) {
AWSCredentials credentials = getCredentials();
AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(
credentials);
StepFactory stepFactory = new StepFactory();
StepConfig enabledebugging = new StepConfig()
.withName("Enable debugging")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newEnableDebuggingStep());
StepConfig installHive = new StepConfig().withName("Install Hive")
.withActionOnFailure("TERMINATE_JOB_FLOW")
.withHadoopJarStep(stepFactory.newInstallHiveStep());
RunJobFlowRequest request = new RunJobFlowRequest()
.withName("Hive Interactive")
.withAmiVersion("3.3.1")
.withSteps(enabledebugging, installHive)
.withLogUri("s3://tweets-hadoop/")
.withServiceRole("service_role")
.withJobFlowRole("jobflow_role")
.withInstances(
new JobFlowInstancesConfig().withEc2KeyName("hadoop")
.withInstanceCount(5)
.withKeepJobFlowAliveWhenNoSteps(true)
.withMasterInstanceType("m3.xlarge")
.withSlaveInstanceType("m1.large"));
RunJobFlowResult result = emr.runJobFlow(request);
System.out.println(result);
}
private static AWSCredentials getCredentials() {
AWSCredentials credentials = null;
credentials = new BasicAWSCredentials("<KEY>","<VALUE>");
return credentials;
}
}
where , are the secret active key and 'hadoop' is a keypair I created in the EC2 console.
After running I see the Job trying to start in the EMR console, after 1 minute it changes from 'starting' to 'Terminated with errors Validation error'
no other information is given
Any ideas what goes wrong?
Thanks!
Aucun commentaire:
Enregistrer un commentaire