How do I remove default ssh host from ssh configuration?

I used to connect to Amazon web services using ssh command and application.pem key. Now when I try to connect to other platforms such as Github my ssh client looks for same application.pem key and tries to connect to AWS. How do I connect to Github or change the default host and key configuration.I am using a Ubuntu 13.10 system and following is my ssh output.

pranav@pranav-SVF15318SNW:~/.ssh$ ssh Warning: Identity file application.pem not accessible: No such file or directory.