vendredi 6 février 2015

RedShift COPY Command on Sub-set of Files

I'm trying to run the COPY command to copy a sub-set of my s3 bucket's objects into Redshift.



copy my_table
FROM 's3://my_bucket/1234*'
credentials 'aws_access_key_id=<REMOVED>;aws_secret_access_key=<REMOVED>’
json 'http://s3my_bucket/json_path.json'


However, I'm getting the following error:



Problem reading manifest file - S3ServiceException:The bucket you are attempting to access must be addressed using the specified endpoint



I looked up this error "... using the specified endpoint" - the AWS docs say to put the bucket + redshift in the same region.


My S3 bucket is located in US Standard. According to this answer, "US Standard" equates to us-east-1.


And the Redshift cluster's region is us-east-1a.


Although I'm not sure of the cause of this problem, I'm curious if I need to use a manifest rather than only copy s3 bucket objects (from my_bucket) that begin with 1234.


What's my issue here?


Aucun commentaire:

Enregistrer un commentaire