Performance Evaluation of Web Servers

Httperf

Httperf is an application that allows you to perform an automated performance evaluation of a web server. It allows you to benchmark the static web page behavior, CGI performance and SSL performance. It provides support for HTTP/1.0 and HTTP/1.1 and allows you to simulate sessions that involve think time and several statistical models of user behavior. This somewhat dated paper provides a overview of the design issues involved in creating the performance tool. You should also read the manual page associated with the running version in ~clement/bin/httperf on schizo.

The three distinguishing characteristics of httperf are

A simple example illustrates the use of httperf: httperf --server paintball.cs.byu.edu --port 5556 --uri /test.html --rate 400 --num-conn 8000 --timeout 20 httperf --server paintball.cs.byu.edu Specify the sever --port 5556 Specify the port --uri /test.html The file you want to download --rate 150 The rate in requests/second --num-conn 27000 The total number of TCP Connections --num-call 1 The number of requests for each connection --timeout 5 The request will fail if it takes longer than this The output from this command would look something like the following httperf --server paintball.cs.byu.edu --port 5556 --uri /test.html --rate 400 --num-conn 8000 --timeout 20 Maximum connect burst length: 1 Total: connections 8000 requests 8000 replies 8000 test-duration 20.914 s Connection rate: 382.5 conn/s (2.6 ms/conn, <=263 concurrent connections) Connection time [ms]: min 0.8 avg 125.9 max 3000.5 median 0.5 stddev 594.7 Connection time [ms]: connect 122.8 Connection length [replies/conn]: 1.000 Request rate: 382.5 req/s (2.6 ms/req) Request size [B]: 65.0 Reply rate [replies/s]: min 351.2 avg 395.9 max 432.2 stddev 33.4 (4 samples) Reply time [ms]: response 3.0 transfer 0.0 Reply size [B]: header 64.0 content 49.0 footer 0.0 (total 113.0) Reply status: 1xx=0 2xx=8000 3xx=0 4xx=0 5xx=0 CPU time [s]: user 4.18 system 16.73 (user 20.0% system 80.0% total 100.0%) Net I/O: 66.5 KB/s (0.5*10^6 bps) Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 You can extract the reply rate, %errors and Reply time to create a graph similar to this one. The rate is varied along the X axis.
Performance Graph
Errors will occur when the client connection experiences a timeout. You can reduce the timeout value and increase the file size and rate to see the results: httperf --server apu --port 5556 --verbose --uri /testmid.html --rate 800 --num-conn 8000 --timeout 2 Connection rate: 799.9 conn/s (1.3 ms/conn, <=660 concurrent connections) Reply rate [replies/s]: min 671.8 avg 735.8 max 799.8 stddev 90.4 (2 samples) Errors: total 641 client-timo 641 socket-timo 0 connrefused 0 connreset 0 You can also create sessions that has calls that are spaced out by wait times httperf --hog --ser=www --wsess=10,5,2 --rate 1 --timeout 5 You can also set the inter-arrival rate to be 2 seconds using the "--period" option in the following command: httperf --server apu --port 5556 --uri /test.mid --hog --num-conn 100000 --rate 1000 --timeout 2 --verbose --period=e2 One of the most powerful options in httperf is "--wset". httperf --server apu --port 5556 --uri /Pareto --hog --num-conn 100000 --rate 1000 --timeout 2 --verbose --wset 999,1 --period=e2 The --wset directive indicates that you will access files in the /Pareto directory in a round robin fashion. The URIs generated are of the form prefix/path.html, where prefix is the URI prefix specified by option --wset and path is generated as follows: for the i-th file in the working set, write down i in decimal, prefixing the number with as many zeroes as necessary to get a string that has as many digits as N-1. Then insert a slash character between each digit. For example, the 103rd file in a working set consisting of 999 files would result in a path of ''/Pareto/1/0/3''. In other words, the files on the server need to be organized as a 10ary tree.

Performance analysis of web servers is important for many reasons. Experimental characterization using httperf can help you to understand the limits of the web server. The httperf application also lets you benchmark cookies, ssl connection times and many other important web server performance metrics.