What is the bottleneck on my nginx+php setup?

I am running some siege tests on my nginx server.

I try to do:

sudo siege -t 10s -c 500 server_ip/test.php

The response time goes to 10 seconds, I get errors and siege aborts before completing.

But I if run the above on my server

siege -t 10s -c 500 localhost/test.php

I get:

Transactions:               6555 hits
Availability:              95.14 %
Elapsed time:               9.51 secs
Data transferred:         117.30 MB
Response time:              0.18 secs
Transaction rate:         689.27 trans/sec
Throughput:            12.33 MB/sec
Concurrency:              127.11
Successful transactions:        6555
Failed transactions:             335
Longest transaction:            1.31
Shortest transaction:           0.00

I also noticed for lower concurrent figures, I get vastly improved transaction rate on localhost compared to externally.

But when the above is running on localhost the CPU usage is low, memory usage is low on HTOP. So I'm confused how I can boost performance because I can't see a bottleneck.

ulimit returns 50000 because I've increased it. There are 4 nginx worker processes which is 2 times my cpu cores. Here are my other settings

worker_rlimit_nofile 40000;

events {
        worker_connections 20000;
        # multi_accept on;
}

  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;

The test.php is just a echo phpinfo() script, nothing else. No database connections.

The machine is an AWS m3 large, 2 cpu cores and about 7gb of ram I believe.