How to send email with AWS SES using ActionMailer

I am trying to use html templates to send invitation emails. Currently I'm using aws-ses gem like this:

  ses = AWS::SES::Base.new(
      :access_key_id     => 'XXXXXXXXXXXXXXX',
      :secret_access_key => 'XXXXXXXXXXXXXXX')
  ses.send_email(:to => ..., :source => ..., :subject => ..., :html_body => <p> Hi how are you</p>)

and I send the html code as a string in :html_body. This works fine.

What I want to do is use a template, and store it in a separate file, for example invite_email.html.erb, which will be stored under app/views/user_mailer/.

So I figured I had to use action mailer to use rendered views. I setup action mailer to use AWS::SES gem, and followed the rails guides to setup action mailer with rails g mailer UserMailer. I have a UserMailer, a layout, and I restarted the server. My developement.rb looks like this:

  config.action_mailer.perform_deliveries = true
  config.action_mailer.raise_delivery_errors = true
  config.action_mailer.delivery_method = :ses

and I initialized :ses like this in initializers/action_mailer.rb:

ActionMailer::Base.add_delivery_method :ses, AWS::SES::Base,
                                   access_key_id: 'XXXXX',
                                   secret_access_key: 'XXXXX'

The server responds: UserMailer#invite_email: processed outbound mail in 184.0ms

The problem is, I still don't get any emails. I tried in the test environment with the same settings in environments/test.rb on a different machine, and still no emails. The server shows the layout is being rendered and the email is being processed. Am I missing a setting?