Performance is an important feature for many projects. Unfortunately, it's an all too common situation when a developer accidentally spoils the performance adding some new code. After a series of such incidents, people often start to think about performance regression testing.
As developers, we write unit tests all the time. These tests check that our business logic work as designed and that new features don't break existing code. It looks like a good idea to write some perf tests as well, which will verify that we don't have any performance regressions.
Turns out this is harder than it sounds. A lot of developers don't write perf tests at all. Some teams write perf tests, but almost all of them use their own infrastructure for analysis (which is not a bad thing in general because it's usually designed for specific projects and requirements). There are a lot of books about test-driven development (TDD), but there are no books about performance-driven development (PDD). There are well-known libraries for unit-testing (like xUnit/NUnit/MSTest for .NET), but there are almost no libraries for performance regression testing. Yeah, of course, there are some libraries which you can use. But there are troubles with well-known all recognized libraries, approaches, and tools. Ask your colleagues about it: some of them will give you different answers, the rest of them will start Googling it.
There is no common understanding of what performance testing should look like. This situation exists because it's really hard to develop a solution which solves all problems for all kind of projects. However, it doesn't mean that we shouldn't try. And we should try, we should share our experience and discuss best practices.
Read more