Profiling your Go Web App

By doing profiling, we can measure how good or how bad our application is based on the non-functional aspects such as performance and memory usage.

Introduction

Project Manager: Functionality?

Lead Developer: check

Product Manager: Suitability with the requirements?

Lead Developer: check

Product Manager: cool, now it's time to deploy the app to the play store.

A few moments later

Product Manager: Oh gosh, our app rating too low. To make it worse, many of our customers complain that our app is too bloated. Can you find the root cause of this?

Lead Developer: Ok lemme profile the app first.

Lead Developer: Hmm…it seems like there are so many redundant SQL queries that made our apps slow. I think we need to refactor this.

That scenario basically represents what profiling looks like in a software engineering context. It measures non-functional aspects of software like the number of requests that can be processed in a second, memory usage, and CPU usage. Contrary to what happened in that dialogue, I think it’s better to do profiling before deploying because prevention is better than cure right? Profiling itself is important because we know that tests such as functional & unit tests can cover functional aspects or in other words,” is our app doing the right thing?”, those two kinds of tests can’t cover non-functional aspects. Meanwhile, profiling can cover the measurement of non-functional aspects in our app. Furthermore, profiling can detect the bottleneck of our app.

How Do We Do Profiling In Go(lang)?

Now I’ll show you how to do profiling in go(lang) since my team uses golang for the backend and the backend process is much heavier than the frontend since it involves many database queries. In this article, I’ll show profiling to measure three aspects that consist of the number of requests, memory usage, and CPU usage. I use permohonan penyitaan list endpoint that will list of permohonan penyitaan because the endpoint is accessed frequently and it returns much data. I use a tool called “bombardier” to do requests spam and pprof for the profiling tools.

Before we jump into the profiling, you need to install bombardier using this command.

go get -u github.com/codesenberg/bombardier

Once you’ve installed the bombardier, then you need to install graphviz. I don’t know how to install it non-ubuntu (you can google it by yourself), but if you’re using Ubuntu, you can install it with this command.

sudo apt install graphviz

Now in your main.go app, put another port for serving the pprof. As shown below, I use port 6060.

Now it’s time to bombard the endpoint (no pun intended lul). To check the number of requests you can use bombardier. I put accessToken header because it’s authenticated endpoint so you must supply the valid token. Otherwise it’ll return 401 bad requests

Here I check the connection with 10000 users & 100 concurrent requests. A number of concurrent requests is basically a maximum number of requests at a time. As shown in the picture, the endpoint can process 208 requests/sec on average and all of the requests return success response which is I think it’s pretty good.

To measure the CPU, run the previous bombardier command and then at another terminal window and fire up this command.

The command will run for about 30 seconds and it’ll open the result in the browser at port 8081. Here is the CPU usage measurement result.

As you can see the entire requests mostly happened in the database query part which takes about 70–80 seconds more or less.

To measure the Memory, run the previous bombardier command and then at another terminal window and fire up this command.

This command will be finished instantly and open up the result on the browser at port 8081. Here is the memory usage result.

Conclusion

I have shown you how to do basic profiling to measure the CPU usage, memory usage, and number requests using bombardier & pprof. Profiling in golang is pretty easy because the profiling tool pprof is a built-in tool so you just need to install graphviz (for visualization) & bombardier. In a large-scale software, I think there are many non-functional aspects that should be considered for profiling (not just three aspects that I have just shown here). Also we can’t rely too much on profiling result because there is external aspect that we can’t control that can bother the assessment result such as connection speed. But profiling is helpful for us as a developer to identify what can be improved or fixed in the non-functional aspects.

References