Automatic tests in Python

The statistics examples – which are on GitHub – show three ways to add automatic tests in Python, each of them with a specific scope:

1. Unit Tests (white box)

White box is a method of testing software that tests internal structures or workings of a function/module/application, as opposed to its functionality (i.e. black-box testing, see point #2). Unit testing means to test individual pieces of the source code and is generally associated to test cases written by the developers themselves (who know the internal workings of the piece to be tested), often written before the source code.
In Python there is a unit testing framework available that allows to quickly set unit tests:

  • create a separate file that has a suffix  “_unittest” attached, for example:
  • In this file import unittest (and the module you want to test)
  • define a class inheriting from unittest.TestCase that will contain as methods all the unit tests.

The framework is inspired to JUnit so it provides several assert statements to compare actual and expected results, and to produce a final report.

Here is an example (file is called

import unittest

class statsTest(unittest.TestCase):
  # mean unit tests
  def testEmptyMean(self):
    # passing empty list to mean() must raise exception
    self.assertRaises(stats.StatsError, stats.mean, [])

  def testSingleMean(self):
    # passing only one data point to mean() must return the same value
    self.assertEqual(stats.mean([25]), 25)

assertRaises will test if the function will return the specified exception (in this case I created a specific one in the stats module) when called with the given input.
The parameters to be passed are: the expected exception, the function under test and the list of inputs for that function.

Use like this:

Ran 17 tests in 0.002s

You can see the complete unit test example file for in GitHub and the framework details in the Python documentation.

2. Acceptance Tests (black box)

Acceptance tests are black-box tests and each of them represents some expected result from the system, without leveraging internal structures or workings.
This means to put together a list of tests for as many test cases as possible, especially edge cases.
The test functions can be grouped in a separate file like this:

from datascience import stats

# === helpers ===
def checkValues(text, valueCurrent, valueExpected):
  if valueCurrent == valueExpected:
    print (text, " OK")
    print (text, " WRONG!!, not {0} but {1}".format(valueExpected, valueCurrent))

# === Acceptance tests ===
X = [10.3, 4.1, 12, 15.5, 20.2, 5.5, 15.5, 4.1]
print ("===1. Test Case X1= ", X)
checkValues("mean(X1)", stats.mean(X), 10.9)
checkValues("median(X1)", stats.median(X), 11.15)
mo = stats.mode(X)
if (len(mo) == 2) and (mo[1] == 4.1) and (mo[0] == 15.5):
  print ("mode(X1) OK")
  print ("mode(X1) WRONG, not two: 4.1 and 15.5 but {0}".format(mo))
checkValues("stdDev(X1)", stats.stdDev(X), 5.613)
checkValues("coeffVar(X1)", stats.coeffVar(X), 0.515)

Use like this:

$ python
===1. Test Case X1=  [10.3, 4.1, 12, 15.5, 20.2, 5.5, 15.5, 4.1]
mean(X1) OK
median(X1) OK
mode(X1) OK
stdDev(X1) OK
coeffVar(X1) OK

3. Tests as example of library usage

The acceptance tests can be in a separate file as above or in the same file, especially if it’s a library. In this case they can act as examples how to use the functions.
You can put the test cases inside an if statement that will run them only if the library is called like a main file, using the property __name__ :

if __name__ == "__main__":
  X = [10.3, 4.1, 12, 15.5, 20.2, 5.5, 15.5, 4.1]

 print ("=== Stats library - Type help( to see the available functions.")
 print ("Examples of usage these functions:")
 print ("X= ", X)
 print ("mean(X) = ", mean(X), " and shall be 10.9")
 print ("median(X) = ", median(X), " and shall be 11.15")

Those test case will not be run if you just import the library into another application and call some of its functions, but only if invoked directly:

$ python
=== Stats library - Type help( to see the available functions.
Examples of usage these functions:
X=  [10.3, 4.1, 12, 15.5, 20.2, 5.5, 15.5, 4.1]
mean(X) = 10.9 and shall be 10.9
median(X) = 11.15 and shall be 11.15

All these examples are on GitHub.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s