Improved handling of mutable and deferred default arguments. After the recipe you'll find an explanation of what that means.
Works for 2.7 with minor tweaks (getfullargspec --> getargspec).
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | """smart_default_arguments module
DEFERRED is the singleton to use in place of None as the sentinel
for deferred default arguments.
"""
import inspect
# mutable default arguments
def recalculate_defaults(f, newdefaults):
argspec = inspect.getfullargspec(f)
# take care of defaults
args = argspec.args
defaults = list(argspec.defaults)
num_bare_args = len(args) - len(defaults)
for i in range(len(args)-num_bare_args):
defaults[i] = newdefaults.pop(args[i+num_bare_args], defaults[i])
bare_args = args[:num_bare_args]
while bare_args:
arg = bare_args.pop()
if arg in newdefaults:
defaults = [newdefaults.pop(arg)] + defaults
else:
break
# take care of kwonly defaults
kwonly = argspec.kwonlyargs[:]
kwdefaults = argspec.kwonlydefaults
num_bare_kwonly = len(kwonly) - len(kwdefaults or ())
for arg in kwonly[num_bare_kwonly:]:
kwdefaults[arg] = newdefaults.pop(arg, kwdefaults[arg])
bare_kwonly = kwonly[:num_bare_kwonly]
while bare_kwonly:
arg = bare_kwonly.pop()
if arg in newdefaults:
kwdefaults[arg] = newdefaults.pop(arg)
else:
break
# finish up
if newdefaults:
raise TypeError("Unexpected new defaults: %s" % newdefaults)
return tuple(defaults), kwdefaults
def has_default_arguments(**kwargs):
"""A decorator factory that applies default arguments.
It handles mutable default arguments, which sets it apart from the
normal handling of default arguments.
Trust that kwargs matches parameters of the decorated function.
"""
def decorator(f):
f.__defaults__, f.__kwdefaults__ = recalculate_defaults(f, kwargs)
return f
return decorator
# deferred default arguments
import functools
class DEFER_DEFAULT_TYPE:
"""Indicates that the default argument should be deferred to the callee."""
DEFERRED = DEFER_DEFAULT_TYPE()
ERROR_MSG = "A DEFERRED object cannot be used for a non-default argument"
def recalculate_arguments(args, kwargs,
spec_args, num_spec_args, num_bare_args, defaults,
kwonlyargs, kwonlydefaults):
args = list(args)
num_args = len(args)
# handle argspec.args
for i in range(num_spec_args):
arg = spec_args[i]
if i >= num_bare_args:
if i < num_args and args[i] is DEFERRED:
args[i] = defaults[i-num_bare_args]
elif kwargs.get(arg) is DEFERRED:
kwargs[arg] = defaults[i-num_bare_args]
continue
if i < num_args and args[i] is DEFERRED:
raise TypeError(ERROR_MSG)
elif kwargs.get(arg) is DEFERRED:
raise TypeError(ERROR_MSG)
# handle argspec.kwonlyargs
defaults = kwonlydefaults
for arg in kwonlyargs:
if kwargs[arg] != DEFERRED:
continue
if arg not in kwonlydefaults:
raise TypeError(ERROR_MSG)
kwargs[arg] = kwonlydefaults[arg]
return args, kwargs
def accepts_deferred_defaults(f):
"""A decorator that handles DEFERRED arguments.
Because this wraps the decorated function with another function,
performance will take a hit. However, this is unavoidable since the
arguments are not known until runtime. Some effort has been made to
optimize the new function, though it could certainly be improved.
"""
argspec = inspect.getfullargspec(f)
spec_args = argspec.args
num_spec_args = len(spec_args)
defaults = argspec.defaults
num_bare_args = num_spec_args - len(defaults)
kwonlyargs = argspec.kwonlyargs
kwonlydefaults = argspec.kwonlydefaults
@functools.wraps(f)
def newfunc(*args, **kwargs):
args, kwargs = recalculate_arguments(args, kwargs,
spec_args, num_spec_args, num_bare_args, defaults,
kwonlyargs, kwonlydefaults)
return f(*args, **kwargs)
return newfunc
|
Python handles default arguments just fine. However, there are two situations that require some extra effort to handle a default argument correctly.
Deferred Default Arguments
If you have a function that calls another, passing through an argument, you likely want the default argument of your function to match that of the called function.
Deferred default arguments is what I call the technique of using a sentinel for the default argument of your function and having the called function translate the sentinel into its own default argument. See my "introduction to deferred default arguments" (recipe 577789) for more information.
This recipe provides a much smoother technique for handling deferred default arguments. The "introduction" recipe requires that you call a special handler function inside each target function, and unnecessarily exposes some of the boilerplate of the solution.
The alternative given here is a decorator. The boilerplate is hidden away and you don't have to make a single change to any function that will be receiving the sentinel argument. Just tack on the decorator and the target function is good to go. See the first example for a demonstration.
Mutable Default Arguments
When a function is defined, any default argument values are evaluated and the resulting objects are bound to the function's __defaults__ and __kwdefaults__. These objects are implicitly passed at each call of the function where the argument is not explicitly passed.
Since the default value is evaluated at definition time and stored away, if a default argument is mutable it will save state between function calls. So if you are expecting that the default value is evaluated at each call, you may be surprised when your changes to that default value are still reflected during your next call.
Currently if you want to avoid any confusion with mutable defaults, you set the default argument to some (immutable) sentinel that indicates the real default argument should be used:
def f(x=None):
if x is None:
x = []
There are downsides though:
- The default argument no longer reflects the expectation for the argument type.
- You can no longer introspect the default argument.
- If you want to have the default actually be None, you will have to use some other value for the sentinel. At that point you'll probably then have two different sentinels in use: None and and its surrogate.
- The real default must be re-evaluated during each call it is used.
- There are two more [seemingly superfluous] lines cluttering up your function body.
This recipe provides a way to mitigate the surprise and downsides without sacrificing the security of your arguments. You only have to add the decorator from the recipe onto any function for which you want to override (with immutable values) any of the default argument values. Go to the second example to see how it works.
This implementation actually replaces the specified default argument values. Thus you still lose the original value. One alternative would be for the decorator to wrap the function in another that would plug in the new default argument values at each call, as appropriate. The trade-off is in performance, which, for functions, is a big deal.
Example 1
redefining default arguments
deferred default arguments
@accepts_deferred_defaults
def f(x=1):
return x
def g(x=DEFERRED):
return f(x)
assert g(2) == 2 == f(2)
assert g() == 1 == f()
Example 2
@has_default_arguments(a=2, x=())
def f(a, x=[]):
result = 0
for val in x:
result += val*a
return result
assert f() == 0
assert f(x=range(3)) == 6
assert f(1) == 0
assert f(1,range(3)) == 3
