Testing Pyramid and Testing Ice Cream Cone | What is the difference?
In this article, I will give a comparison between pyramid testing and ice cream cone testing. What are pyramid and ice cream cone tests, and which one is better?
What is Testing Pyramid?
The “Test Pyramid” is a metaphor for describing different groups of software tests. It also indicates how many tests should have in each group of granularity. The pyramid provides a starting place for new QA testers to jump off from and indicates how many tests are required at each level. It serves as a single source of truth for established QA testers to refer back to, which meets their needs.
The test pyramid consists of three layers that your test suite should consist of (bottom to top):
- Unit Tests
- Service Tests
- User Interface Tests
It’s all about the cost
‘It costs almost five times more to fix a coding defect once the system is released than it does to fix it in unit testing.’
— Paul Grossman, IT consultant, in ‘Automated testing ROI: fact or fiction?’
Testing Pyramid Component
Unit testing focuses on individual pieces of code. This can give you a detailed view of the code’s quality and performance, enabling you to get fast feedback as to whether your changes will improve things. As such, it makes sense to automate them. Furthermore, automated unit testing makes it easier to pinpoint where problems are so that you can more quickly fix them.
Integration testing checks whether all the parts of your software work together as a unit. It exposes any problems in how your software interacts. For example, if different developers have worked on different components, integration testing can uncover problems that might have gone unnoticed otherwise.
End-to-end testing checks the entire application by using a test environment and data to simulate real-world conditions. This is the most expensive to maintain and has the slowest operating pace, but it’s also the easiest to identify issues because it checks assembled applications.
As with end-to-end testing, it can be expensive. With its extreme setup and overall control and monitoring requirements, this is easily the most time-consuming software quality test method. It requires a staging server, server network, devices pool, or developer kit.
It is expensive to automate high-level tests
As you climb up the testing pyramid, you will encounter issues like:
- Test fragility (tests that break easily and unexpectedly, even when changes ‘shouldn’t’ have influenced the test)
- Longer feedback time
- Increased effort levels
- Higher costs to implementation
- More specialized knowledge required
Benefits of the test automation pyramid
- Time and cost savings
- Elimination of human error for accuracy
- Ability to reuse, repeat, and scale application tests to meet your software requirements
What is Ice Cream Cone Pyramid?
The pyramid becomes a pattern that can be declined in many different ways [Meerts 2019].
Ice Cream Cone Tests happen if the pyramid is reversed.
This phenomenon appears when test automation mainly focuses on E2E testing with fewer IT and even fewer UT. With software testing ice cream cones, the majority of testing is done manually. UI automated tests are a close second, integration tests in the middle, with unit testing lagging completely. This is not scalable.
Why Ice Cream Cones Are Bad
The anti-pattern that followed the pyramid test pattern was the Testing Ice Cream Cone. Ice cream cone is what happens if you put the majority of testing effort into a manual test.
Along with the growth of the software, and its features, the amount of testing labor required grows linearly. If a new feature interacts with other features in a way, it might have to be tested in the context of other features as well. So organizations need to be willing to provide the amount of labor required to continue the manual testing approach.
Connect with me
Krisnawan: Twitter | Linkedin | Medium
MydoQA: Twitter | Blog | Instagram | Facebook
