How to black box test a Go app with RSpec

Automated testing is all the rage in web development these days and goes on across the whole industry. A well-written test dramatically reduces the risk of accidentally breaking an application when you add new features or fix bugs. When you have a complex system that’s built from several components that interact with each other, it’s incredibly hard to test how each component interacts with other components.

Let’s take a look at how to write good automatic tests for developing components in Go and how to do so using the RSpec library in Ruby on Rails.

The part of the project that processes requests is the most important, thus we needed to maximize its reliability and availability.

As part of a monolithic application, there’s a high risk of a bug affecting the request processor, even when there are changes in code in parts of the app not related to it. Likewise, there’s a risk of crashing the request processor when other components are under a heavy load. The number of Ngnix workers for the app is limited, which can cause problems as the load increases. For instance, when a number of resource-intensive pages are opened at once in the admin panel, the processor slows down or even crashes the entire app.

These risks, as well as the maturity of the system in question — we didn’t have to make major changes for months — made this app an ideal candidate for creating a separate service to handle request processing.

We decided to write the separate service in Go, that shared the database access with the Rails application that remained responsible for changes in the table structure. With only two applications, such a scheme with a shared database works fine. Here’s what it looked like:

We wrote and deployed the service in a separate Rails instance. This way, there was no need to worry that the request processor would be affected whenever the Rails app was deployed. The service directly accepts HTTP requests without Ngnix and doesn’t use a lot of memory. You could call it a minimalist app!

We created unit tests for the Go application where all database requests were mocked. In addition to other arguments for this solution, the main Rails application was responsible for the database structure, thus the Go application didn’t actually have the information for creating a test database. Half of processing was business logic, while the other half was database queries, all of which were mocked.

Mocked objects are much less readable in Go than in Ruby. Whenever new functions were added for reading data from the database, we had to add mocked objects during many failed tests that had previously worked. In the end, such unit tests didn’t prove very effective and were extremely fragile.

In order to make up for these drawbacks, we decided to cover the service with functional tests in the Rails application and test the service in Go like a black box. White-box testing wouldn’t work in any case, since it was impossible to use Ruby to get inside the service and see whether a method was being called.

That also means that requests sent through the test service were also impossible to mock, thus we needed another application for managing and writing these tests. Something like RequestBin would work, but it had to work locally. We’d already written a utility that’d do the trick, so we decided to try using it.

This was the resulting setup:

Here’s how we implemented this. As a demonstration, let’s call the test service TheService and create a wrapper:

It’s worth mentioning that autoloading files have to be configured in the support folder when using RSpec:

The start method:

The configuration itself:

The “stop” method simply stops the process. There’s a gotcha though! Ruby runs a “go run” command, which compiles TheService and launches a binary in a child process with an unknown ID. If we just stop the process that’s running in Ruby, the child process doesn’t stop automatically and the port will remain in use. Thus stopping TheService has to go through the Process Group ID:

And now we can look at writing the specs themselves:

TheService can make HTTP requests to external services. We can configure it to redirect requests to the local utility that logs them. For this utility, there’s also a wrapper for starting and stopping it that’s similar to ‘TheServiceControl’, except that this utility can just be started as a binary without compilation.

The Go application was written so that all the logs and debugging information would be sent to STDOUT. On production, this output is sent to a file. When launching from RSpec the log is displayed in the console, which really helps with debugging.

If you specifically run the specs that don’t need TheService, then it won’t start.

In order not to waste time on launching TheService each time whenever a spec changes, during the development process you can launch TheService manually in the terminal and simply not turn it off. Whenever it’s necessary, you can even launch it in an IDE debugging mode. Then the specs prepare everything, send the request to the service, it stops and you can easily debug it. This makes the TDD approach really convenient.

We’ve been using this setup for about a year now and haven’t experienced any failures with it. The specs come out far more readable than unit testing in Go, and they don’t rely on knowing the internal structure of the service. If we, for some reason, need to rewrite the service in another language, then we won’t need to change the specs. Only the wrappers, which are used for launching the test service with a different command would need to be rewritten.

Related posts:

A lot of times data needs to be stored in the cloud for various reasons, but uploading to the cloud and maintaining the storage hasn’t been known as an easy job. Google’s Firebase services are one of…

This AMA session is based completely on questions from the CobinHood community. The CEO at SocialMedia.Market project — Dmitriy Shishov — answers the most popular questions concerning the nature and…

Things might get a little weird but don’t stop reading, I promise I won’t leave you hanging half… hearted “This guy Marina, this fucking guy is driving me crazy. He’s just so…. boring, so…