Cannot SSH to Amazon EC2 instance from ubuntu 14.04 LTS

I followed everything in exact same order as refered here :Amazon Set Up Documentation. I also read many SO answers but everyone suggested on checking the security group and route tables. I did everything none of them helped.

I have a USB Dongle which I use to surf internet. Its IP changes everytime I disconnect the dongle and reconnect it. But if I stay connected its IP remains same. So I did created a security group my_ip/32 and launched an ec2 instance. So without disconnecting my dongle i.e keeping my IP intact I tried to connect to the instance through SSH with simple

ssh -v -i my-key-pair.pem user@public_dns

command but with no luck.

But the funny thing is I can connect to the ec2 instance if I change my security group to 0.0.0.0/0 which is not suggested by amazon because of security issue.

Does anybody has faced the similar situation. Or Amazon did miss something in their documentation.

NOTE: I also enable firewall in ubuntu for ssh as follows:

sudo ufw allow ssh/tcp
sudo ufw enable

Am I missing something??