SSH freeze when connecting to AWS

Connecting to ubuntu 14.04 server at AWS, from an ubuntu 14.04 system with the following command:

ssh -i ~/.ssh/key.pem ubuntu@12.121.12.321

Normally it would just connect, but now it times out with this error:

ssh: connect to host 54.171.53.164 port 22: Connection timed out

I can ctrl+C out of the freeze though. I have tried to restart I have tried to sudo apt-get update