S3 and RTMP access of uploads via Cloudfront

We upload objects via signed cloudfront url. The objects are only accessible via cloudfront Web but NOT via RTMP or S3.

If I browse to the S3 location via the AWS Admin Console, and look at the permission for the objects, it is simply empty.

We have tried adding the header x-amz-content-sha256 with a value of the SHA256 of the content, while the file upload fine, we still cannot access the file by any other means than CF Web again...

Anyone know how to fix this?