Modern HTTP Benchmarking Tools ready for 2018 – h2load, hey & wrk

The HTTP protocol continues to be the most popular communications protocol on the web. In addition to the classic web browsing with hypertext documents, HTTP is also the de-facto protocol for API communications with RESTful APIs. WebSockets and other protocols exist, but HTTP is everywhere.

The HTTP protocol itself received a significant upgrade in 2015 with the release of HTTP/2, and it already enjoys a significant marketshare. Thanks to the improved infrastructure, free certificates from Let's Encrypt and browser vendors pushing encrypted traffic the web is faster and more secure today.

The transport layer of the web has received significant, but quiet, upgrades in the past few years. Perhaps because the changes have been taken on gradually, some areas have not been discussed much. One of these is HTTP benchmarking tools, which are used to generate load to a HTTP server.

Benchmarking HTTP in 2017

From the days of yore developers have used tools like Apache Bench or Siege to generate simple high concurrency traffic for benchmarks. But by todays standards these are very old; Apache Bench for example is based on the Zeus Technology tool from 1996 - that's over twenty years of history.

With increased performance requirements of contemporary web services, and technologies like Node.js and Golang that enable high HTTP throughput should also receive some care. And there are modern options on the market - so developers should no longer default to Apache Bench or Siege.

Luckily there are many good tools available for developers to use for benchmarking. In this article I discuss three contemporary HTTP benchmarking tools. The three options that we'll have a quick look at are, in alphabetical order:

• h2load
• hey
• wrk

h2load

h2load is benchmarking tool for HTTP/2 and HTTP/1.1. It is written in C and is using the nghttp2 library underneath. Currently h2load is not available in popular Linux distros and needs to be compiled from source for installation. So installation a bit tricky, but see installing h2load on Ubuntu 16.04.

An example command to make 100000 requests to a server at a concurrency of 1000 with h2load:

$ h2load -n100000 -c100 https://localhost:3000/

hey

hey is a HTTP benchmarking tool written in Go (AKA Golang). It supports both HTTP 1.1 and HTTP/2 through the native Go libraries - Go is designed for high concurrency out of the box. As a standard Golang application, installation is straightforward once you've got the go toolkit installed.

An example command to make 100000 requests to a server at a concurrency of 1000 with hey:

$ hey -n100000 -c100 https://localhost:3000/

wrk

wrk is a HTTP benchmarking tool written in Lua. It currently only support HTTP/1.1. There is some work in supporting HTTP/2 in wrk, but expecting results soon is not feasible. Wrk promises to deliver high load on multicore CPUs. Installation requires some knowledge of LuaJIT and build tools.

An example command to load server at a concurrency of 1000 for 60 seconds with wrk:

$ wrk -t12 -c100 -d60s https://localhost:3000/

Conclusion

Modern HTTP load testing tools are more stable than the age old predecessors on high loads and they give better statistics as well. None of tools offers capabilities for generating organic traffic, but rather reference traffic to a single URL - for complex scenarios you should look at specialised tools or SaaS.

Out of these tools h2load and hey continue to be in active development, where as wrk has received little attention over the course of 2017. From the three tools I found hey to be the easiest to install and providing best reporting - so I have settled on using it for my benchmark needs as of December 2017.

-- Jani Tarvainen, 01/12/2017