Rails - Elastic Beanstalk AWS - sessions not maintained

I just deployed a Rails 3.2.17 app to AWS Elastic Beanstalk.

I was first unable to login to the application (we use Devise) and kept getting WARNING: Can't verify CSRF token authenticity in the logs.

Then when I tried to see the session in the Chrome developer tools, there was no cookies set for the application which is super weird.

Then I tried printing out the session has from the ApplicationController and every time I got different session_id.

{"session_id"=>"e0f2ccf03d07f48def5c4fbb872f2c0c", "last_action_time"=>Wed, 23 Sep 2015 08:40:09 EDT -04:00, "test"=>"test"}
....
 {"session_id"=>"0f9ef3aaf150619ac408da2bac4f51ae", "last_action_time"=>Wed, 23 Sep 2015 08:43:24 EDT -04:00, "test"=>"test"}

This is in my session_store.rb:

MyApp::Application.config.session_store :cookie_store, key: ENV['SESSION_NAME'], domain: :all

Do I need to do anything else? Thanks! This app works fine in a single EC2 instance (staging environment).