The Importance of Unit Testing for Smart Contracts

Smart contracts are only as smart as the code. The smallest bug in your smart contract means you need to redeploy your entire contract.

Written by
Bill Brock
on
May 14, 2018
Filed under:

 

Smart contracts are only as smart as the code.

Of course, you can say that about any program—to some extent. But blockchain-based smart contracts are not only unlike regular contracts; they are unlike any other programs.

That’s because—as you probably know by now—finding even a tiny bug means you need to redeploy your entire contract, so the consequences of an error—even a small one—are much greater than in most traditional software systems.

Here’s the problem: Smart contracts don’t generally offer simple upgrade paths, especially when it comes to critical components, such as code that controls the value in a token contract. Once a smart contract is up and running, changing it becomes complicated, if not impossible.

“Smart contract development is different from most other contemporary development as there are rarely any easy upgrade paths,” explains Daniel Zakrisson, editor of ICONOMI. “Deploy it and once the contract is in use you are pretty much stuck with it.”

The only way to be confident your smart contract is solid is to test meticulously. That begins with smart contract unit testing. At its simplest, unit testing means testing the code at the earliest level—at the smallest unit of code—to identify problems early, before they affect the program.

Catch the Bugs Early and Often

Bugs are nothing new. It’s just that the stakes are higher in smart contracts. So you don’t just need to think about writing code; you need to think of everything that could go wrong down the road and stop it from happening.

But in some ways, it’s simpler than it may sound. Consider, for instance, the omelet.

In a piece for Hacker Noon, software engineer Iman Tumorang compares unit testing to making an omelet. He goes through the entire process, but the first two points convey the concept nicely.

  • Before pouring the eggs into the mixing bowl, you will crack the eggs, if you find a putrid egg, then you will choose another egg. This is an expected behavior.
  • When you whisk the eggs less than 1 minute, the whisked eggs will be unevenly stirred. If so, you will whisk again. This is an expected behavior.

It’s “expected behavior” to test and adjust each step of the way, he explains. “When you do all the steps above, you have done a test for each function/step/method/unit.” So, when a simple error occurs—such as too much salt—“we already know how to deal and handle it in our steps/function/method/unit. That’s the use of unit testing.”

Before we go any further, it may be time for a very brief refresher course in smart contracts.

A Refresher: Smart Contracts

You obviously know what a smart contract is, but let’s start at the beginning, anyway.

Smart contracts are computer programs that act once specific conditions are met. A simple protocol executes the terms of a contract—if A is met, then B happens.

Like a conventional contract, a smart contract defines the provisions and penalties around an agreement. Unlike a conventional contract, a smart contract automatically executes those obligations. (And, of course, also unlike traditional contracts, they cannot be amended.)

If you need more information about or examples of smart contracts, from creating one in Solidity to explaining it to your mother, we have several blog posts to help you out.

Making the Case for Unit Testing

Because a single error can spell disaster, a smart contract requires unit testing. That seems so obvious to us that it almost seems absurd to have to explain it. But unit testing gets a bad rap.

We know there are different perspectives on unit testing, and we know it’s not universally embraced. And yes, we get that it’s fashionable in some quarters to rail against test-driven development in general and unit testing in particular. You probably remember a few years ago when James Coplien’s “Why Most Unit Testing is Waste” was attracting so much attention.

All that debate is fine and healthy. We get it. We even enjoy it. But none of it applies to smart contracts built on a blockchain. We cannot say it enough: Blockchain is different. Smart contracts are different. It doesn’t matter what you and your engineers think about unit testing, or if your team is somehow philosophically opposed to the approach. It’s the only way forward.

Regardless of their opinions about unit testing, your programmers and engineers understand it. But what about the CEO and other non-tech people? If unit testing makes things take a bit longer than they expect, how will you explain it?

It can be confusing to the layperson, given that there are so many different types of tests, including unit tests, acceptance tests, integration tests, end-to-end tests, component tests, and service tests. Moreover, as JAX London speaker Gil Tayar, architect and evangelist at Applitools, warns in a Hacker Noon piece, when one person uses one of these terms to describe a test, “it is probable that their definition of that term is different from another person’s.”

The lack of specificity and the seemingly interchangeable names are enough to give us a headache. Imagine how confused your colleagues in operations, marketing, finance, etc. will be.

With that in mind, here’s a little unit testing 101 you can share with the CEO and others who don’t really get the point.

What Unit Testing Really Is

Because unit testing ensures code is performing correctly at the most basic level, it’s your first line of defense. Fortunately, although it can be time consuming, it’s not hard.

Don’t just rely on our experience: Here’s what Tayar has to say: “Regardless of how many unit tests you have, unit tests are the easiest tests to write, and the easiest tests to understand, as they are usually functional in nature—set up a unit with input, make it do its thing, and check the output (where the input could be a function parameter, and the output just the return value).”

Trying to explain it to a novice? Use comparisons. Start with the omelet example. For a slightly more advanced audience, the educational site Guru99 offers a useful breakdown of different kinds of testing for the novice. Perhaps the most useful aspect of the explanation is the comparison between unit and integration testing. 

In unit testing, as we’ve discussed, individual units of source code are tested to determine if they are ready to use. Errors are easier to identify and fix during the early phases of development. It’s conducted by the developer.

Integration testing ensures the communication of the code between and among different components. Individual units of a program are combined and tested as a group. It’s performed by developers and test engineers and, as the name suggests, focuses on how well the individual units work together and how well the code works with external dependencies.

Vulnerable Contracts

If you still need to be convinced of the importance of unit testing, keep in mind that, if your code is buggy, it’s not just your one contract that’s compromised. Bad code has a way of replicating itself.

Ted Mlynar and Ira Schaefer of the Intellectual Property practice at Hogan Lovells call for a “new kind of due diligence for this new kind of contract.” In a 2016 Coin Desk piece, they argue that because smart contracts combine law and computer science, “due diligence on smart contracts should do the same.”

They, of course, cite the Ethereum “hard fork.” But they also cite research showing that among the roughly 19,000 Ethereum smart contracts studied, 44 percent contained vulnerabilities.

You know how that works. A flawed smart contract code was copied repeatedly. As the attorneys put it, “Old, flawed code apparently became the unsteady foundation for towering new smart contracts.”

You’re probably thinking “2016 was eons ago in blockchain time. Things are better now.” Maybe yes on the first part, but newer research suggests the problem isn’t getting any better.

Gaming the Candy Machine

A group of international researchers published a 2018 paper (gloriously titled “Finding The Greedy, Prodigal, and Suicidal Contracts at Scale”) that identified a distressing number of Ethereum smart contracts with vulnerabilities “ripe for exploit,” Motherboard reports.

Essentially, what the researchers did was download a copy of the Ethereum blockchain and create a private “fork” for testing. Ilya Sergey, an assistant professor of computer science at University College London and co-author of the research, told Motherboard:

“Imagine your goal isn’t to interact with the vending machine in a proper way, but rather you want to break it or get it to serve you for free. Assume we put a few coins in the machine, and just start randomly pushing buttons hoping that the inner workings of the vending machine—which we have no knowledge about, springs and whatnot—eventually releases the latch so you can take the candy.”

Of roughly a million smart contracts analyzed, 34,200 were “critically vulnerable.” On a subset of about 3,000 contracts, the team was able to verify and reproduce the flagged vulnerabilities with an 89 percent success rate.

This isn’t just theoretical. Here’s what Sergey told Motherboard about smart contracts: “We’re dealing with applications that have two very unpleasant traits: They manage your money, and they cannot be amended.”

At Least Do This

Of course, unit testing is necessary, but it’s not sufficient. Zakrisson offers an outline for the minimal amount of testing required.

  1. Have a testing plan for both:
    • verification testing [which includes unit tests]
    • validation testing [which ensures that the product meets the user needs, and that the requirements were correct in the first place.]
  2. Do code reviews
  3. Do verification testing with automated unit, functional, and integration tests
  4. Do external security audits
  5. Do validation testing by extensively testing on the testnet
  6. After testnet testing, do more testing with limited risk in proof-of-concept or alpha stage on the main net

Need a simpler overview for your team? One of the simplest comes from the Loom Network's Georgios Konstantopoulos:

  • Don’t write fancy code
  • Use audited and tested code
  • Write as many unit tests as possible

But it all starts with smart contract unit testing.

We’ll give the final word to Brenn Hill, director of business development, Blockchain Advisory Council LLC. He argues that for every function in every smart contract, there needs to be a unit test. He offers the following in a Hacker Noon post:

“People write code, people are imperfect and write imperfect code, imperfect code has bugs, and if all your money relies on perfect code because it’s on a blockchain then you can’t rely on your money which means you won’t rely on the blockchain. And if you can’t rely on the blockchain… what’s the point?”

If you want to ensure meticulous testing for your blockchain project, we can help. Drop us a line. 

innovator's guide to picking the right blockchain