The 4 Key Benefits of Automated Testing

In many ways, I’m your stereotypical software engineer. I don’t do small talk, but I can talk for days about my work and side projects. I’m easily irked by inefficiencies — if it can be done more efficiently, why not do it that way? I also like to “fix” things like it’s my job (to be fair, it is my job from 8 to 5), and I spend what many people would consider “too much” time on /r/dataisbeautiful. You’ve surely heard one engineer or another say in jest, “the best engineers are lazy.” I know I have. What we really mean is those who have an eye for identifying inefficiencies and the drive to fix them generally make good engineers. 

In the case of software engineers, we strive for efficiency and have a tendency to automate things. The process of writing code to automate something tedious sure beats doing the task by hand. We create many products that automate mundane tasks, like starting your coffee maker every morning or turning on your lights when you enter a room. In our field of work, I think many would readily agree that testing is one of the more tedious tasks, so naturally, we’ve come up with ways to automate that too!

Automation at Various Levels of Testing

Automated testing is an incredibly common way to test software, and automation can be introduced at various levels of testing:

1. Unit testing verifies behavior at a functional or modular level. It’s standard practice to automate unit tests, and many tools are available to do so. Imagine trying to verify a function that performs some complex calculation by hand, many times, with different inputs each time. It sounds like a nightmare, doesn’t it?
2. Integration testing verifies that two or more more units behave as expected when integrated together. This level of testing is also typically automated because it can often be difficult to manually re-create and observe the interaction between two units or modules within a program. 
3. System testing verifies the functional behavior of the entire system against requirements. This is the least often automated level of testing. System requirements are observable to end users, so it’s much more feasible, albeit not necessarily more efficient, to test manually. For example, it’s easy enough to send a text message to your phone and verify a notification appears on your smartwatch. However, what if you had to verify this feature along with dozens of other features, multiple times? That would become very tedious, very quickly.

4 Key Benefits

The benefits of test automation spans all levels of testing. Not only does it save time, and therefore money, but it improves software quality and can provide valuable metrics for the project.

1. Test automation can save time and money, if there are enough tests that need to be run and/or tests that need to be run often. 
   
   If you can count the number of tests you need to run on your fingers, and they only need to be run — let’s say — once a month, the upfront cost of setting up the test infrastructure and writing the automated tests will likely outweigh the time savings. However, in most cases, the product you’re developing will be bigger, more complex, and perhaps must meet regulatory requirements. In these cases, the upfront cost pays dividends. System tests may take hours or days to perform manually, meanwhile it might only take minutes or hours to run when automated. Not to mention, manual testing absorbs a significant amount of a tester’s time and attention, whereas automated tests require minimal to no attention while they run. The time spent to analyze test results can also be reduced if the output of automated tests are nicely formatted and presented.

2. Because automated tests are less time-consuming, they can be run more often, and catch bugs sooner, which helps improve software quality. 
   
   The sooner bugs are caught, the easier and quicker they are to identify and fix. Additionally, automated tests can verify constraints that would be difficult to do with manual testing, so performance and stress tests become feasible when automated. Automation also reduces variation during testing, as they will run the same way every time. For example, it may prove difficult for a tester to send a message every 50 milliseconds, but very reasonable for a test script to do so. The ability to write more robust and constrained tests along with reproducibility greatly supports improved software quality.

3. Test automation also contributes to a culture that enables change. 
   
   Far too often, teams hesitate to refactor old code or make design changes to improve a feature due to the fear of breaking code that already works. However, just because something works doesn’t mean it can’t or shouldn’t be improved! Be honest, how many times have you been told not to pursue a change that would very obviously improve the codebase or improve a feature because it might introduce a bug? In contrast to manual testing, automated testing consumes relatively little time and requires no supervision or user intervention during runtime. As such, it becomes easy to run a test suite before and after a software change. Results from these test runs can be compared to increase confidence in the change(s) and decrease the likelihood of bug introduction. The risk reduction that automated testing provides to the team will allow them to target improvements in software maintainability, robustness, and overall quality, resulting in a better product.

4. Being able to obtain metrics for analysis from automated testing is the cherry on top. 
   
   The test results of multiple runs over time can be aggregated to determine the overall health, performance, and quality of the software over time. It can also be used to identify and focus on problem areas during development. It’s not to say that test results from manual testing cannot be analyzed the same way, but because automated tests are less time-consuming, they can be run more frequently to provide a larger data set to be analyzed. 

There are so many benefits to automated testing, of which I’ve only discussed a few. Transitioning to automated testing is not without effort. There’s the upfront cost of standing up the test infrastructure and the additional time that must be spent to plan, develop, and maintain tests through the life of the project — but, for many projects, the cost is well worth the benefit. There’s also definitely a conversation to be had about how automated testing feeds into continuous integration and hardware-in-the-loop testing, but that will have to wait for another blog post. In the meantime, think about automating your tests so you’re working smarter, not harder.