Determining contributors to elapsed time during a network request

0

The situation:

  • There is a remote server.
  • I am running a client program that makes calls to the server and requests data.
  • These requests perform very slowly.
  • The server limits the number of results to 50, and paginates the rest. So if there are 300 total results, I have to make a total of 6 separate requests.
  • (Per James' comment) The data is in JSON text format.
  • (Per James' other comment) The connection is an https (SSL) connection.

The owner of the server has offered to increase the number of results per page so that I can make fewer requests.

While I think this will help, I am worried that it still won't make the transaction fast enough for my purposes. I wonder: What if the problem is not the server's overhead for retrieving data? What if the biggest hit to performance is due to network bandwidth and latency?

How would I figure this out? I don't have access to the server at all. Let's say that a request takes 3 seconds to complete. Then:

  1. What portion of that 3 seconds is spent opening the TCP connection?
  2. What portion is spent sending the request data?
  3. What portion is spent waiting for the server to retrieve the data?
  4. What portion is spent waiting for all of the requested data to arrive at my machine?

I don't really want to have to roll my own TCP/IP code to get the low-level access needed to measure things like this.

I'm sure there have to be tools written for this. As I Google around, I see programs like netstat, ss, netperf, ttcp, etc. But I'm not even sure what words to use in a Google search to look for solutions to this problem.

Any ideas?

Aaron Johnson

Posted 2013-05-02T21:50:20.697

Reputation: 143

1

Suppose it depends on what the data is you are requesting - if returning json text then probably not bandwidth limiting and more processing constraints at server end. Whilst it might not be a complete solution, something like Wireshark (http://www.wireshark.org/) will help you monitor network traffic and log packets in order. You could then compare timestamps on them to see which section of the process is taking most time?

– James – 2013-05-02T21:54:57.963

It is returning JSON text. I'll update the post to reflect that. Packet capture in Wireshark is a good idea. I'll give that a try. – Aaron Johnson – 2013-05-02T21:58:41.077

Oh and I'd also add that negotiating a SSL handshake will undoubtedly cause an overhead with each request. You don't state whether this is http:// or https:// – James – 2013-05-02T22:06:38.770

Good point. https. I'll update the question to reflect that as well. – Aaron Johnson – 2013-05-02T22:08:59.793

In which case reducing the number of requests and increasing result set for each might have a material impact – James – 2013-05-02T22:10:08.953

I think your issue is likely query related, or in the way you are processing the result set (doing it client side, rather than letting the DB do the work). how long does your query take to execute if you call it from a DB management console while remoted into the server? unless your site is under a signifigant load, factors like connection creation time, feeding data onto the wire, https negotiation, etc are negligible, and if you are composing and executing your queries correctly, many small connections should be faster than one big one. your delay is almost certianly in waiting for response. – Frank Thomas – 2013-05-02T22:23:09.397

Aaron: Regarding to your comment on SO: Posting the same question to several Stack Exchange sites at once is against the rules. If you think you started out on the wrong site, you can flag your question and ask a moderator to migrate it to a different site. Sometimes you'll also still be able to delete your question on one site, and post it again on another. Migrating is probably the nicer solution, especially if there are already answers or comments. If you want this question on Stack Overflow (instead of Super User), let me know. – Daniel Beck – 2013-05-03T09:06:13.213

Daniel: Thank you for informative comment and good information here. I didn't notice the link you put in your original comment at SO and didn't read it until this morning. As it stands, I've gotten no responses at SO. Andrew closed the question, but it may as well just be deleted. Thanks for your help. – Aaron Johnson – 2013-05-03T16:41:19.390

No answers

Asked: 2013-05-02T21:50:20.697

Viewed: 125 times

Active: 2019-07-03T06:49:18.457