Welcome, guest | Sign In | My Account | Store | Cart

The main point is that there was no binding between a unit tests and the concrete class. It did happend often that you are indirectly testing a class using it also the concrete test class has missed some concrete test methods. I found this fact sometimes very irritating seeing 100% coverage.

Therefor I now provide this class decorator where you can specify the class that will be tested. If you do not specify test methods for each method of the testable class an exception will be thrown providing a list of missed tests.

Here some examples how it works: You implemented a

  • method "__eq__", then write a "testEqual" method
  • method "__init__", then write a testInit" method
  • method "scalar_product", then write a testScalarProduct" method
  • and so on ...

The way to use this class decorator you can see in the doctest area (see below in the code) The only point to be aware of is that when you use the decorator you have to implement all test to get rid of the thrown execption. Of course you can implement more tests for same class.

New in revision 4 (1st march 2014):

  • Bugfix: the algorithm were not correctly detecting which methods were really overwritten forcing to implement tests for methods which were base class only.
  • Bugfix: decorated classes which contain the attribute "decorated_object" can be handled properly.
  • A new second parameter now allows you to implement several classes in same test class forcing you to include the class name in the test method:

    • @ValidateTestResponsibilityFor(Square, True)
    • @ValidateTestResponsibilityFor(Sin, True)
    • => testSquareInit, testSquareGet, testSinInit, testSinGet, ...
    • This for smaller classes to avoid too many files (at least ... you decided)
Python, 143 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
"""
.. module:: decorators
    :platform: Unix, Windows
    :synopis: some decorator tools

.. moduleauthor:: Thomas Lehmann <thomas.lehmann.private@googlemail.com>

   =======
   License
   =======
   Copyright (c) 2014 Thomas Lehmann

   Permission is hereby granted, free of charge, to any person obtaining a copy of this
   software and associated documentation files (the "Software"), to deal in the Software
   without restriction, including without limitation the rights to use, copy, modify, merge,
   publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
   to whom the Software is furnished to do so, subject to the following conditions:

   The above copyright notice and this permission notice shall be included in all copies
   or substantial portions of the Software.

   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
   IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
   DAMAGES OR OTHER LIABILITY,
   WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"""
import sys
import inspect

if sys.version.startswith("2."):
    from types import NoneType
else:
    NoneType = type(None)

class ValidateTestResponsibilityFor(object):
    """ a class decorator that throws an exception when the test class does not
        implement all tests for all methods of the testable class (unit).
        The next code gives you an example on how it is used and what does happen:

        >>> class Value:
        ...     def __init__(self, value):
        ...         self.value = value
        ...
        >>> try:
        ...     import unittest
        ...     @ValidateTestResponsibilityFor(Value)
        ...     class TestValue(unittest.TestCase):
        ...         pass
        ... except Exception as e:
        ...     print("|%s|" % str(e).strip())
        |...failed to provide test method 'TestValue.testInit' for method 'Value.__init__'|

    """
    def __init__(self, testableClass, includeClassName=False):
        """ stores the class for test and checks all methods of that class """
        if hasattr(testableClass, "decorated_object"):
            testableClass = testableClass.decorated_object

        self.testableClass = testableClass
        self.includeClassName = includeClassName
        self.methodsInTestableClass\
            = self.getEntries(self.testableClass, inspect.isfunction)\
            + self.getEntries(self.testableClass, inspect.ismethod)


    @staticmethod
    def getEntries(the_class, mode):
        """ get all entries by given mode (function or method) but the
            members of the concrete class only; not from its base """

        classes = {}
        for concrete_class in reversed(inspect.getmro(the_class)):
            classes[concrete_class] = {}
            for name, definition in dict(inspect.getmembers(concrete_class, mode)).items():
                object_name = name
                is_base_method_only = False

                for known_class in classes:
                    if not object_name in classes[known_class]:
                        continue

                    if classes[known_class][object_name] == definition:
                        is_base_method_only = True
                        break

                if not is_base_method_only:
                    classes[concrete_class][object_name] = definition

        return list(classes[the_class].keys())

    def __call__(self, testClass):
        """ called when instantiated; then we have to verify for the required test methods """
        self.verify(testClass)
        return testClass

    @staticmethod
    def getTestMethod(name, prefix=""):
        """ adjusting final test method name """
        # no underscores wanted (change "__init__" => "init")
        finalName = name.strip("_")
        # if we find "_" as separator between words ...
        if finalName.find("_") > 0:
            finalName = "".join(subName.title() for subName in finalName.split("_"))

        # ensure more readable name (like "equal" instead of "eq")
        if name == "__eq__":
            finalName = "equal"
        elif name == "__lt__":
            finalName = "less"
        elif name == "__gt__":
            finalName = "greater"

        return "test" + prefix + finalName[0].upper() + finalName[1:]

    def verify(self, test_class):
        """ verification that for each testable method a test method does exist """
        methodsInTestClass\
            = self.getEntries(test_class, inspect.isfunction)\
            + self.getEntries(test_class, inspect.ismethod)

        missing = []
        for testableMethod in self.methodsInTestableClass:
            prefix = ""
            if self.includeClassName:
                prefix = self.testableClass.__name__

            testMethod = self.getTestMethod(testableMethod, prefix)
            if testMethod in methodsInTestClass:
                continue

            missing.append((test_class.__name__ + "." + testMethod,
                            self.testableClass.__name__ + "." + testableMethod))

        if len(missing) > 0:
            # creates message with all missing methods throwing an exception for it
            message = ""
            for testMethod, testableMethod in missing:
                message += "\n...failed to provide test method '%s' for method '%s'" \
                           % (testMethod, testableMethod)
            raise Exception(message)

3 comments

Ned Batchelder 10 years, 4 months ago  # | flag

BTW, everywhere you use ValidateTestResponsibilityFor.xxx in your class, you can just use self.xxx, it will find the staticmethods, etc, and you won't have to repeat the long class name each time.

Tim Chase 10 years, 4 months ago  # | flag

This is pretty verbose and might do unexpected things in the presence of leading/trailing underscores that aren't dunder-marks, so instead of all this verbosity:

if finalName.startswith("__") and finalName.endswith("__"):
    finalName = finalName[2:-2]
if finalName.find("_") > 0:
    # when changing some like "scalar_product" to "ScalarProduct"
    tokens = finalName.split("_")
finalName = ""
for subName in tokens:
    finalName += subName[0].upper() + subName[1:]

I'd be tempted to just write

finalName = "".join(
    subname.title()
    for subname
    in finalName.strip("_").split("_")
    )

This would also make things like "this_is_an_HTML_test" into ThisIsAnHtmlTest rather than ThisIsAnHTMLTest, which I find a bit harder to read.

Thomas Lehmann (author) 10 years, 4 months ago  # | flag

Well, nice way to do it but a method name like "intersectionPoint" would change to "Intersectionpoint" because the "title" does lowercase of the rest.

Of course for the "this_is_an_HTML_test" example it does work well.

Anyway ... Ned's and yours comment taken into account I did some modification as a compromise... (see new revision)