5.3 Test
5.3.7 Alpha and Beta Testing
Alpha and beta testing have become popular in software development. Today, they are also used for products and services. In these tests, a functional and almost final product is tested by a limited number of users.
An alpha test is held internally: in the developers' organization. This principle is sometimes referred to as "eat your own dog food". Often, this internal test is mainly for functional testing, like finding bugs. But by choosing an internal user group in a smart way, you can get a good impression of usability, utility, and market potential. Several years ago, a well-known Swedish furniture company decided to develop a new kind of products: playful furniture that would enhance children’s fantasy. They organized an alfa test that lasted an afternoon. Employees with young children were invited to a sports hall, were all kinds of prototypes were showcased. The children were allowed to play with whatever they liked, while the parents were drinking a cup of coffee. The children were unaware they were in a kind of test and responded fully naturally as if they were in a playground. Some things were so popular, that children quarrelled about it, whereas some other things were fully ignored. Although not all costly prototypes could stand the rough play, the test was very successful. The company knew exactly what to do.
A beta test is a field test, whereby an often incomplete product is shared with a limited number of users. Although a beta test is time-consuming and costly, it enables you to perform genuine functional -, usability -, utility -and market testing at the same time. It is the golden standard for tech companies. A company like Google permanently has a range of products in beta.
For beta testing, you need test persons who are willing to participate and represent the future user group. They provide valuable data, as they are willing to invest time and effort and try everything. Every bug will pop up; usability issues surface by means of request for help (‘where is the … functionality?’); when some functionalities are not used you learn about the utility as well; and whether or not the group is embracing the product informs you about the market need. The imminent risk is that it typically concerns early adopters: people who embrace new things first. They are not fully representative and you should be careful about extrapolating the results of a beta test.
Beta tests are so common for software that today even the smallest updates are ‘in beta’ for some time. Developers simply distrust what they made and only real-life testing proves whether their designs are good or not. Unfortunately, this is entirely different for e.g., governmental policies or reorganizations. Although new policies are subject to political scrutiny; rarely, if ever, the policies are tested in the field to learn whether or not they have the intended effect.