Determining where added latency is coming from?

I have a website that I can see in Newrelic and Nginx spends a very small amount of time processing per request, even at the 99% level. Nevertheless, I see a lot of added processing time when actual users hit the site (+100-900ms). I'd trying to figure out lowest hanging fruit to work on at this point, but don't know how to go about isolating what's taking up this additional time. It's being spent somewhere between replies leaving my content boxes (in AWS), going through my DNS provider (Cloudflare), and winding up at the user's laptop. What's a good way to see where the most time is being spent?