Remote Desktop Gateway Authentication Issue

We have two Windows 2008 R2 servers configured with Remote Desktop Gateway role and placed in a HTTPS AWS Elastic Load Balancer.

We have a public CNAME created pointing to the URL of load balancer in order to attain High Availability. Also we have set the DefaultTSGateway parameter in Application Settings of Pages under RDWeb in IIS to point to the load balancer url in both gateway servers and the load balancer will automatically balance the load.

Issue:

Both the gateway servers are allowing to access the resources locally but when we are trying 
to access the remote computers from internet, 
its giving a 'logon attempt failed' error while entering credentials.

Checked the authorization settings in IIS but no luck!

Stuck at this issue, no clue why its not accessible from internet.

Note: Port 443 is opened. And the User , Resource Authorization Policies are fine as its allowing to access resources locally.

Any help will be highly appreciated!