Rails AWS assets on Cloudfront & s3

So I setup paperclip to work with S3 to store images on upload. That is working fine.

I then went to add cloudfront for assets (with the code below)

config.action_controller.asset_host = ENV['CLOUDFRONT_ENDPOINT']

and build the assets, it seems to build all correctly and everything, but whenever I go to the page the links are there

<link rel="stylesheet" media="all" href="http://ift.tt/1cSb78y" />
<script src="http://ift.tt/1JHAoze"></script>

but when I go to that link i get this XML error

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>
assets/application-ef64d41d2d57abb59ffe5bd71a4f727580ef276a6440e70210cf8d0ab22a6dc2.css
</Key>
<RequestId>374DF77BF548DE75</RequestId>
<HostId>
TqrV7id3elsBjugWNkUObG259mU6Vk8MhxcXjrre1qv+XvxGBERDjWoW50iiCyp4
</HostId>
</Error>

I looked at my s3 box and it's not there either..

All my cloudfront settings were default except my origin which was my s3 box