Falsification

[This blog was originally published as Dutch article in TestNet Nieuws 
(http://nieuws.testnet.org/vak/falsificatie/)]

I had a discussion with a colleague not long ago. He's an information analyst that has knowledge about both the (ancient) Greek language as Latin and who likes to get involved in a nice, bit philosophically substantiated conversation. That particular time we discussed one of my most favorite topics, namely, my profession: Testing.

The discussion started because I illustrated the V-model to another colleague on the whiteboard. I explained that the flow was not only downwards, but that there was also an upward stream so that it could be an iterative process. I illustrated the difference between verification and validation in the various steps. After the colleague had left the room, my information-Latin-colleague turned around and pointed at the Mappa Testi, that hung on the wall behind me. 

Now you have to know that the MappaTesti is a product derived from a meeting called the TestingRetreat, where we (I and the other testers that attended) were inspired after a visit to the MappaMundi in Hereford to make a similar map of how the software testing world would look like from our perspective in 2011. The MappaMundi is the oldest still existing medieval map of the world and as was customary in that time, the map was intended for topographic, religious and mythical display. It was also intend to awe the person who saw it. Hell, or the netherworld is displayed - for instance - at the bottom.

On my MappaTesti that hell is also at the bottom. With, as my colleague pointed out, written in dog-Latin 'Infernum Falsificatum Consequitur'. I meant to translate the phrase 'The hell of falsified results'. In other words: if you make yourself guilty of falsifying your results, you deserve to end up in hell!

My colleague changed that perspective in the light of the V-model discussion. He genuinely asked himself if 'falsification' in the software testing expertise wasn't allowed in relation to 'validation' and 'verification'. He had wondered for some time now, but now the opportunity arose to ask the question in the needed context.

After I explained to him that the phrase in the MappaTesti was meant differently as he had assumed, the fascination for the topic falsification remained though. Even more so after my colleague explained that with many a thesis, falsification is a form of providing evidence (anti-evidence actually). Isn't it the case that If you are a system- and software tester and you practice verification and validation, that you should also be practicing 'falsification'.  

So what is falsification actually? The word derives from the Latin word 'falsificatio' (late Latin) or 'falsificare' (ancient Latin). In short: demonstrating something by proving the opposite. The explanation on Wikipedia (mind you; for this article I used the Dutch one), has a nice explanation. Imagine that the theory states that all swans ar white. The opposite of this statement, also states the possible 'falsificator': there is at least one swan that isn't white' (also"there is at least one black swan"). If it is accepted (or proven) that this black swan exists, than the statement 'all swans are white' can be refuted.

The principle of demonstrating something by presenting the untruth, comes from a man named 'Karl Popper'. He was a philosopher of science and his opinion was that scientific theories could only be tested indirectly by testing their implications performing a crucial test. If the outcome was positive (or actually 'negative'; there isn't a black swan to be found), than that doesn't mean that this is verification; there's always the possiblity that a black swan might show up. Popper describes this as 'corrobation', what is best described as 'the probability of the statement being true is higher'.

The more you dig into the matter of philosophy behind science, the more fascinating it gets. Terms like 'inductive verification', 'deductive falsification' come by. For instance; there's also an assertion that a crucial test isn't possible at all and there's a shifting of paradigms. Also the - for some already known - epistemology came up during my search for knowledge.

What I was wondering particularly in this whole matter was: 'What is the significance of falsification within the software- and system testing craft?'

In practice I regularly get the goals for testing like:  'prove by executing a test that the system is fit for purpose' or 'prove by executing a test that the build functionality is as described'. Let us - for argument sake- leave these statements for what they are and if they are correct as goals for testing. By showing one single error, fault or failure in the system we demonstrate that the statement 'the system is free of faults, failures or errors' (bug free) can be accepted and thus we practice falsification and not validation. Bugs are in basic our falsificators.

Falsifying is an essential part of our craft, next to validation and verification. Though according to the Popperian principles the latter isn't ever possible!