Suggestions for finding performance bottleneck on LAMP server

Question

I know this question has been asked a thousand times and I've read through a bunch of the answers, but haven't found one that really helped me.

I've used top and I'm not swapping at all, I've got about 500 MB of free RAM (each Apache process is consuming about 80MB), the CPU load is essentially non-existent... I've set MySQL's slow query log to contain any query taking more than 1 second and it's empty.

I was running a bunch of Drupal sites on my old linux box. I decided to upgrade the machine because I was going to run a Groupon deal and I wanted to make sure I could handle the load spikes. I wasn't experiencing any slowness on my server with the current traffic, but I didn't want to slashdot myself with a Groupon. The new machine has 4 GB of RAM compared to 1 GB on the old machine. It's got a much faster processor, but I don't have the specs on the new vs old on hand.

I've been experiencing HORRIBLE bouts of slowness on the new server, but my ISP refuses to consider the possibility that it's the server or the configuration on the server. They just keep blaming "my software" (even though it's Drupal's software and Drupal is wicked fast).

When I say horrible, I mean about 30% of the time it takes more than 3 minutes to load a single page with barely any content. I turned on profiling inside Drupal and from Drupal's perspective, the entire page transaction takes less than 500ms on the high end (usually closer to 300ms) and MySQL queries take a total of about 30ms.

Whenever I'm browsing to my own site and I get a patch of really REALLY slow requests, I check my connection with pings and traceroutes to other websites and I don't find myself dropping packets. During one of the slow downs I was even able to login to my ISP's admin panel and write an entire Trouble Ticket before the page finally loaded.

Can you give me any suggestions for where I can look for the problem? How can I narrow the field to find where the problem exists?

My code didn't change from the old server to the new server. The only variables are the hardware and and the server configuration, so I feel like the problem has to be in the server configuration.

Here is an example URL on my site:

http://www.inadaydevelopment.com/content/financial-freedom-principles

I tried running a ping to my site, to see if my home FiOS connection was causing me to drop packets and interfere with the connections, but in 1000+ pings I didn't drop a single one. However, during those 1000 pings, I experienced several page loads which took more than 6 MINUTES. The browser spins in the "connecting" state, as opposed to the "connected and downloading resources" state.

--- inadaydevelopment.com ping statistics ---
1010 packets transmitted, 1010 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 80.661/83.935/222.001/8.891 ms