Welcome, guest | Sign In | My Account | Store | Cart

This is a decorator that makes some of a function's or method's arguments into keyword-only arguments. As much as it possible, it emulates the Python 3 handling of keyword-only arguments, including messages for TypeErrors. It's compatible with Python 3 so it can be used in code bases that support both versions.

Python, 182 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
#!/usr/bin/python

from __future__ import print_function

import functools
import inspect


def decorator_factory(*kw_only_parameters):
    """Transforms a function with keyword arguments into one with
    keyword-only arguments.

    Call this decorator as @decorator_factory() for the default mode,
    which makes all keyword arguments keyword-only, or with the names
    of arguments to make keyword-only.  They must correspond with the
    names of arguments in the decorated function.  It works by
    collecting all the arguments into *args and **kws, then moving the
    arguments marked as keyword-only from **kws into *args.

    Args:
      *kw_only_parameters: Keyword-only arguments as strings.

    Returns:
      A decorator that modifies a function so it has keyword-only
      arguments.

    """
    def decorator(wrapped):
        """The decorator itself, assigns arguments as keyword-only and
        calculates sets for error checking.

        Args:
          wrapped: The function to decorate.

        Returns:
          A function wrapped so that it has keyword-only arguments.

        """

        # Each Python 3 argument has two independent properties: it is
        # positional-and-keyword *or* keyword-only, and it has a
        # default value or it doesn't.
        names, varargs, _, defaults = inspect.getargspec(wrapped)

        # If there are no default values getargpsec() returns None
        # rather than an empty iterable for some reason.
        if defaults is None:
            defaults = ()
        names_with_defaults = frozenset(names[len(names) - len(defaults):])
        names_to_defaults = dict(zip(reversed(names), reversed(defaults)))
        if kw_only_parameters:
            kw_only_names = frozenset(kw_only_parameters)
        else:
            kw_only_names = names_with_defaults.copy()

        @functools.wraps(wrapped)
        def wrapper(*args, **kws):
            """Wrapper function, checks arguments with set operations, moves args
            from **kws into *args, and then calls wrapped().

            Args:
              *args, **kws: The arguments passed to the original function.

            Returns:
              The original function's result when it's called with the
              modified arguments.

            Raises:
              TypeError: When there is a mismatch between the supplied
                and expected arguments.

            """

            new_args = []
            args_index = 0
            for name in names:
                if name in kws:
                    # Check first if there's a bound keyword for this name
                    new_args.append(kws.pop(name))
                elif name in kw_only_names:
                    # If this name is keyword-only, check for a
                    # default or raise.
                    if name in names_to_defaults:
                        new_args.append(names_to_defaults[name])
                    else:
                        _wrong_args(wrapped, names, 
                                    kw_only_names -
                                    (names_with_defaults | frozenset(kws)),
                                    'keyword-only')
                elif args_index < len(args):
                    # Check for a positional arg.
                    new_args.append(args[args_index])
                    args_index += 1
                elif name in names_to_defaults:
                    # Check for a default value.
                    new_args.append(names_to_defaults[name])
                else:
                    # No positional arg or default for this name so raise.
                    _wrong_args(wrapped, names,
                                frozenset(names) -
                                (names_with_defaults | frozenset(kws)),
                                'positional', len(args))

            if args_index != len(args) and not varargs:
                # Too many positional arguments and no varargs, so
                # raise after subtracting off the number of kw-only
                # arguments from those expected.
                raise TypeError(
                    '%s() takes %d positional arguments but %d were given' %
                    (wrapped.__name__,
                     len(names) - len(names_with_defaults | frozenset(kws)),
                     len(args)))
            else:
                # Pass the rest of the positional args, if any.
                new_args.extend(args[args_index:])

            return wrapped(*new_args, **kws)
        return wrapper

    return decorator


def _wrong_args(wrapped, names, missing_args, arg_type, number_of_args=0):
    """ Raise Python 3-style TypeErrors for missing arguments."""
    ordered_args = [a for a in names if a in missing_args]
    ordered_args = ordered_args[number_of_args:]
    error_message = ['%s() missing %d required %s argument' % 
                     (wrapped.__name__, len(ordered_args), arg_type)]
    if len(ordered_args) == 1:
        error_message.append(": '%s'" % ordered_args[0])
    else:
        error_message.extend(['s: ', 
                              ' '.join("'%s'" % a for a in ordered_args[:-1]),
                              " and '%s'" % ordered_args[-1]])
    raise TypeError(''.join(error_message))


if __name__ == '__main__':
    def test(f, *args, **kws):
        print(args, kws, '-> ', end='')
        try:
            f(*args, **kws)
        except TypeError as e:
            print(e.args[0])

    @decorator_factory()
    def f(a, b, c, d, e='e'):
        print(a, b, c, d, e)

    test(f)
    test(f, 0, 1, 2, 3)
    test(f, 0, 1, 2, 3, 4)

    @decorator_factory('c')
    def f(a, b, c='c', d='d', *args, **kws):
        print(a, b, c, d, args, kws)

    test(f)
    test(f, 0, 1)
    test(f, -1, b='b')
    test(f, b='b')
    test(f, 0)
    test(f, 0, 1, 2, 3, 4, 5, c='foo', d='bar', e='baz')

    @decorator_factory('b', 'c')
    def f(a, b, c='c', d='d', *args, **kws):
        print(a, b, c, d, args, kws)

    test(f, 0)
    test(f, 0, b='b')
    test(f, 0, 1, 2)
    test(f, 0, 1, 2, b='b')
    test(f, 0, 1, 2, 3, 4, 5, c='foo', d='bar', e='baz')
    test(f, 0, 1, 2, 3, 4, 5, b='foo', c='bar', e='baz')

    class C(object):
        @decorator_factory('b', 'c')
        def __init__(self, a, b, c='c', d='d', *args, **kws):
            print(a, b, c, d, args, kws)

    test(C, 0, 1, 2, b='b')
    test(C, 0, 1, 2, 3, 4, 5, b='foo', c='bar', e='baz')

Some simplifications.

4 comments

Michael Cuthbert 8 years, 9 months ago  # | flag

Great Recipe! One potential bug I found. line 42:

args_with_defaults = set(positional_args[len(defaults):])

I believe should be:

args_with_defaults = set(positional_args[len(positional_args) - len(defaults):])

Your tests worked because you always had the same number of positional args w/o defaults as positional args w/ defaults. Change this around and you'll see the difference.

I was ready to take a project of mine to Python 3 only because I absolutely wanted to require certain args to be keyword only. Now I can continue Py2.7 support for a bit longer.

Michael Cuthbert 8 years, 9 months ago  # | flag

I needed to make a few more changes to have it work in every situation in Python 2 and 3; probably too many changes to incorporate easily, but it might be useful to someone:

    # we want to preserve default=None, so we need to give a very implausible value for a default
    noDefaultString = '***NO_DEFAULT_PROVIDED***'
    positional_args, unused_varargs, unused_keywords, defaults = inspect.getargspec(func) 
    args_with_defaults = set(positional_args[len(positional_args) - len(defaults):])

    kw_only_args = set(included_keywords) if len(included_keywords) > 0 else args_with_defaults.copy()
    args_and_defaults = list(zip_longest(reversed(positional_args), reversed(defaults), fillvalue=noDefaultString))
    args_and_defaults.reverse()
    positional_args = set(positional_args)

    @functools.wraps(func)
    def wrapper(*callingArgs, **keywordDict):
        keywordSet = set(keywordDict)
        # Are all the keyword-only args covered either by a passed
        # argument or a default?
        kw_only_args_specified_by_keyword_or_default = keywordSet | args_with_defaults
        if not kw_only_args <= kw_only_args_specified_by_keyword_or_default:
            missing_args = kw_only_args - kw_only_args_specified_by_keyword_or_default
            wrong_args(func, args_and_defaults, missing_args, 'keyword-only')
        # Are there enough positional args to cover all the
        # arguments not covered by a passed argument or a default?
        if len(callingArgs) < len(positional_args - kw_only_args_specified_by_keyword_or_default):
            missing_args = positional_args - kw_only_args_specified_by_keyword_or_default
            wrong_args(func, args_and_defaults, missing_args, 'positional', len(callingArgs))

        finalArgs = []
        maxIndex = 0
        for index, (name, default) in enumerate(args_and_defaults):
            fArg = noDefaultString
            if name in keywordDict:
                fArg = keywordDict[name]
                keywordDict.pop(name)
            else:
                if maxIndex < len(callingArgs):
                    fArg = callingArgs[maxIndex]
                    maxIndex += 1
                elif name not in keywordDict and default is not noDefaultString:
                    fArg = default
            if fArg is not noDefaultString:
                finalArgs.append(fArg)
        if len(callingArgs) > maxIndex: #  *args                
            finalArgs.extend(callingArgs[maxIndex:])
        return func(*finalArgs, **keywordDict)
    return wrapper
Cara (author) 8 years, 7 months ago  # | flag

I'm getting back to this after being busy with some other projects. I'd like to update this to incorporate your bugfixes, but I'm not sure I understand what some of your code is doing. I see your point about the length of positional args and the need for a placeholder for args without defaults, but I'm not sure what some of your other changes are for.

Cara (author) 8 years, 7 months ago  # | flag

The latest revision should fix Michael Cuthbert's bugs as well as some other bugs I found.