vendredi 4 septembre 2015

AWS EMR - Adding Hive steps

I am trying to add Hive steps when I spawn up EMR clusters so that the tasks are automated. Looking at resources, I came up with the following command line entry:

aws emr create-cluster --name "testing"   --release-label emr-4.0.0  --log-uri "s3://bucket/logs/"  --enable-debugging  --applications Name=Hive  --use-default-roles --ec2-attributes KeyName=testKey  --instance-type m3.xlarge --instance-count 10  --bootstrap-action Path=s3://bucket/script.sh  --steps Type=Hive Args=[-f,s3://bucket/test.hql]

However, I'm getting the following error:

aws: error: The following required parameters are missing for HiveStepConfig: Args.

From what I can tell, it should be working. I have also tried input and output directories in the args, but it seems like the issue is that the Args are just not being read. Any ideas?

Thanks




Aucun commentaire:

Enregistrer un commentaire