Functional Python "threading" pattern

Consider this code in Python:

import re

from functools import partial, reduce


def enumify(s: str) -> str:
    return reduce(
        lambda x, f: f(x),
        [partial(re.sub, r"[\(\)\.\,\!]", ""), partial(re.sub, r"[\s\-\/]", "_"), str.upper],
        s,
    )

The enumify function applies some transformations to a string, returning a string suitable for deserializing into an enum value.

enumify("Hello, world!")

HELLO_WORLD

enumify("some-kebab-var")

SOME_KEBAB_VAR

These transformations are just pure functions that modify a string. Python's functools.reduce is used to apply these pure functions sequentially, with each function producing the input for the next.

The arguments to functools.reduce are a reducing function, which itself accepts an accumulator and a value, an iterable to supply the value in the reducing function, and optionally an initial value for the accumulator. To create the enumify function, we supply our aguments:

• The first argument to functools.reduce a generic lambda whose accumulator argument is possibly the result of one of the pure functions, and the value argument is a function to be applied.

• The second argument — a sequence of pure functions to apply to the accumulating result. Note the use of functools.partial with re.sub to produce a fucntion of singular arity.

• Lastly, we pass in the initial accumulator value; here it is the input to our “pipeline”. However, an initial value which is of type expected by the sequence of input functions is mandatory for this pattern.

Generalizing this pattern, we get something like the following.

def thread(input, *fs):
    return reduce(lambda x, f: f(x), fs, input)


def enumify(s: str) -> str:
    return thread(
        s, partial(re.sub, r"[\(\)\.\,\!]", ""), partial(re.sub, r"[\s\-\/]", "_"), str.upper
    )

Refactoring the function even makes it easier to reason about. This pattern is inspired by Clojure's threading macros.

Python is regarded as a highly readable programming language with a syntax that resembles psuedo-code or even natural language, but this is to demonstrate that taking advantage of Python's functional features can make it into a very expressive language.

add_b = lambda x: {**x, "b": "hello"}
capitalize_vals = lambda x: {k: v.upper() for k, v in x.items()}

thread({"a": "value1"},
       add_b,
       capitalize_vals)

{'a': 'VALUE1', 'b': 'HELLO'}

thread({"x-some-header": "header value", "x-other-header": "other value"},
       add_b,
       capitalize_vals)

{'x-some-header': 'HEADER VALUE', 'x-other-header': 'OTHER VALUE', 'b': 'HELLO'}

So far, these examples attempt to mimic Clojure's -> macro. Let's go a step further and try to replicate the cond-> macro.

def thread_cond(input, *pftups):
    def thread_cond_reduce_helper(acc, pftup):
        pred, f = pftup
        return f(acc) if pred() else acc
    return reduce(thread_cond_reduce_helper, pftups, input)

As you can see, this is very similar to our thread function, but instead of a variable number of pure functions, thread_cond accepts a variable number of predicate and function tuples. The predicate is a simple supplier function.

def describe_num(n: int) -> str:
    return thread_cond("",
                       (lambda: n < 0, lambda x: x + "NEGATIVE "),
                       (lambda: n >= 0, lambda x: x + "POSITIVE "),
                       (lambda: n % 2 != 0, lambda x: x + "ODD "),
                       (lambda: n % 2 == 0, lambda x: x + "EVEN "),
                       (lambda: True, str.strip))

describe_num(11)

POSITIVE ODD

describe_num(26)

POSITIVE EVEN

describe_num(-3)

NEGATIVE ODD

describe_num(-14)

NEGATIVE EVEN

The predicate being a simple supplier function allows us to define concise closures; in this last example we created a predicate that closes over the variable n. Compare this to the if ladder version.

def describe_num_other(n: int) -> str:
    res = ""
    if n < 0:
        x += "NEGATIVE "
    if n >= 0:
        x += "POSITIVE"
    if n % 2 != 0:
        x += "ODD "
    if n % 2 == 0:
        x += "EVEN "
    return res.strip()

Which version you think is better is really a matter of preference and familiarity — and maybe taste.