How to improve TTFB from Amazon S3?

I'm developing a small web app, and I have some data files stored in a s3 bucket with CORS setup that need to be accessed from the client browser.

For some reason, my ajax call to S3 files has a very high latency (over 4sec for a 7KB file), due to the TTFB (as you can see in this dev tools screenshot http://ift.tt/1JqTV69). Both the client and S3 bucket are in Northern California.

Any ideas about what might be happening here would be appreciated.

thanks

url = 'http://ift.tt/1Kg2qVc';
$.ajax({
  url: url,
  dataType: "json",
  contentType: "application/json; charset=utf-8",
  success: function(response) {
    console.log("received");
  },
  failure: function(response) {
    // TODO: decent error message
    alert('Unable to load');
  }
});