Important caveat for the pass-through approach: if any of your build_dataset() functions accept **kwargs, you have to be very careful about how they’re handled to preserve the property that “calling a function with unused arguments is an error”. It was a lotof work to clean this up in Matplotlib...
The general lesson is that “magic” interfaces which try to ‘do what I mean’ are nice to work with at the top-level, but it’s a lot easier to reason about composing primitives if they’re all super-strict.
Another example: $hypothesis write numpy.matmul produces code against a very strict (and composable) runtime API, but you probably don’t want to look at how.
The general lesson is that “magic” interfaces which try to ‘do what I mean’ are nice to work with at the top-level, but it’s a lot easier to reason about composing primitives if they’re all super-strict.
100% agree. In general I usually aim to have a thin boundary layer that does validation and converts everything to nice types/data structures, and then a much stricter core of inner functionality. Part of the reason I chose to write about this example is because it’s very different from what I normally do.
Important caveat for the pass-through approach: if any of your build_dataset() functions accept **kwargs, you have to be very careful about how they’re handled to preserve the property that “calling a function with unused arguments is an error”. It was a lotof work to clean this up in Matplotlib...
To make the pass-through approach work, the build_dataset functions do accept excess parameters and throw them away. That’s definitely a cost. The easiest way to handle it is to have the build_dataset functions themselves just pass the actually needed arguments to a stricter, core function, e.g.:
def build_dataset(a, b, **kwargs):
build_dataset_strict(a, b)
build_dataset(**parameters) # Succeeds as long as keys named "a" and "b" are in parameters
Important caveat for the pass-through approach: if any of your
build_dataset()functions accept**kwargs, you have to be very careful about how they’re handled to preserve the property that “calling a function with unused arguments is an error”. It was a lot of work to clean this up in Matplotlib...The general lesson is that “magic” interfaces which try to ‘do what I mean’ are nice to work with at the top-level, but it’s a lot easier to reason about composing primitives if they’re all super-strict.
Another example:
$hypothesis write numpy.matmulproduces code against a very strict (and composable) runtime API, but you probably don’t want to look at how.100% agree. In general I usually aim to have a thin boundary layer that does validation and converts everything to nice types/data structures, and then a much stricter core of inner functionality. Part of the reason I chose to write about this example is because it’s very different from what I normally do.
To make the pass-through approach work, the
build_datasetfunctions do accept excess parameters and throw them away. That’s definitely a cost. The easiest way to handle it is to have thebuild_datasetfunctions themselves just pass the actually needed arguments to a stricter, core function, e.g.: