vendredi 3 juillet 2015

Running Sqoop in AWS EMR master

I am trying to sqoop data from MySQL data store into S3 in my AWS EMR master node.

sqoop import --fields-terminated-by '\t' --escaped-by '\\' --optionally-enclosed-by '\"' --bindir ./ --connect jdbc:mysql://<remote mysql ip>/<mysql database> --username <user> --password <pass> --table table1 --target-dir=s3n://accessKey:secretKey@MyBucket/table1/ 

I get the following Error:

15/07/03 09:43:31 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/07/03 09:43:33 INFO fs.EmrFileSystem: Consistency disabled, using com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as filesystem implementation
15/07/03 09:43:33 INFO metrics.MetricsSaver: MetricsConfigRecord disabledInCluster: false instanceEngineCycleSec: 60 clusterEngineCycleSec: 60 disableClusterEngine: false maxMemoryMb: 3072 maxInstanceCount: 500 
15/07/03 09:43:33 INFO metrics.MetricsSaver: Created MetricsSaver j-2OE9ME93MKQED:i-735879bf:Sqoop:02084 period:60 /mnt/var/em/raw/i-735879bf_20150703_Sqoop_02084_raw.bin
15/07/03 09:43:35 ERROR tool.ImportTool: Imported Failed: The bucketName parameter must be specified.

I referred to http://ift.tt/1JG6cU2

On further looking into http://ift.tt/1JG6cU4

try {
  if (options.doHiveImport()) {
    hiveImport = new HiveImport(options, manager, options.getConf(), false);
  }

  // Import a single table (or query) the user specified.
  importTable(options, options.getTableName(), hiveImport);
} catch (IllegalArgumentException iea) {
    LOG.error("Imported Failed: " + iea.getMessage()); <------ This is the only line to have that Error Message
    if (System.getProperty(Sqoop.SQOOP_RETHROW_PROPERTY) != null) {
      throw iea;
    }

I am not sure if I understand it right. But, Why is an error under Hive Import thrown while I am trying to import into S3? Am I missing something?

Thanks Karthick




Aucun commentaire:

Enregistrer un commentaire