mercredi 7 octobre 2015

What AWS technologies should I use in order to do light weight processing in real time before storing the data in Redshift?

I want to create a few HTTP points where mobile clients, servers & IoT devices will be posting data. I may need to preprocess the events & act up on them. Eventually I want to access all the raw data & make queries using Domo, Cloud Business Intelligence | Chartio or Tableau .

I need to understand what are the differences & advantages for the following architectures:

  1. AWS API Management + Lambda + Redshift: I can create an HTTP endpoint & a lambda function that will parse the data, compute & store in Redshift
  2. Kinesis Firehose + Redshift (how do I stream the data over HTTP here?)
  3. S3 + Kinesis + Redshift (I can use an HTTP endpoint that writes data to S3)
  4. S3 + Kinesis Firehose + Redshift
  5. S3 + Lambda + Redshift

I feel like 3, 4 & 5 create redundancy because of S3. Will the execution of Lambda functions have a significant cost overhead over using Kinesis?




Aucun commentaire:

Enregistrer un commentaire