Crimes with Python's Pattern Matching

One of my favorite little bits of python is __subclasshook__. Abstract Base Classes with __subclasshook__ can define what counts as a subclass of the ABC, even if the target doesn’t know about the ABC. For example:

class PalindromicName(ABC):
   
  @classmethod
  def __subclasshook__(cls, C):
    name = C.__name__.lower()
    return name[::-1] == name

class Abba:
  ...

class Baba:
  ...

>>> isinstance(Abba(), PalindromicName)
True
>>> isinstance(Baba(), PalindromicName)
False

You can do some weird stuff with this. Back in 2019 I used it to create non-monotonic types, where something counts as a NotIterable if it doesn’t have the __iter__ method. There wasn’t anything too diabolical you could do with this: nothing in Python really interacted with ABCs, limiting the damage you could do with production code.

Then Python 3.10 added pattern matching.

A quick overview of pattern matching

From the pattern matching tutorial:

match command.split():
    case ["quit"]:
        print("Goodbye!")
        quit_game()
    case ["look"]:
        current_room.describe()
    case ["get", obj]:
        character.get(obj, current_room)

You can match on arrays, dictionaries, and custom objects. To support matching objects, Python uses isinstance(obj, class), which checks

If obj is of type class
If obj is a transitive subtype of class
If class is an ABC and defines a __subclasshook__ that matches the type of obj.

That made me wonder if ABCs could “hijack” a pattern match. Something like this:

from abc import ABC

class NotIterable(ABC):

    @classmethod
    def __subclasshook__(cls, C):
        return not hasattr(C, "__iter__")

def f(x):
    match x:
        case NotIterable():
            print(f"{x} is not iterable")
        case _:
            print(f"{x} is iterable")

if __name__ == "__main__":
    f(10)
    f("string")
    f([1, 2, 3])

But surely Python clamps down on this chicanery, right?

$ py10 abc.py
10 is not iterable
string is iterable
[1, 2, 3] is iterable

Oh.

Oh my.

Making it worse

Pattern matching can also destructure object fields:

match event.get():
    case Click(position=(x, y)):
        handle_click_at(x, y)

We can only get the field after we’ve decided the object. We can’t match “any object that has the foo field”… unless we use ABCs.¹

from abc import ABC
from dataclasses import dataclass
from math import sqrt

class DistanceMetric(ABC):

    @classmethod
    def __subclasshook__(cls, C):
        return hasattr(C, "distance")

def f(x):
    match x:
        case DistanceMetric(distance=d):
            print(d)
        case _:
            print(f"{x} is not a point")

@dataclass
class Point2D:
    x: float
    y: float

    @property
    def distance(self):
        return sqrt(self.x**2 + self.y**2)

@dataclass
class Point3D:
    x: float
    y: float
    z: float

    @property
    def distance(self):
        return sqrt(self.x**2 + self.y**2 + self.z**2)

if __name__ == "__main__":
    f(Point2D(10, 10))
    f(Point3D(5, 6, 7))
    f([1, 2, 3])

14.142135623730951
10.488088481701515
[1, 2, 3] is not a point

It gets better! While the ABC decides the match, the object decides the destructuring, meaning we can do stuff like this:

def f(x):
    match x:
        case DistanceMetric(z=3):
            print(f"A point with a z-coordinate of 3")
        case DistanceMetric(z=z):
            print(f"A point with a z-coordinate that's not 3")
        case DistanceMetric():
            print(f"A point without a z-coordinate")
        case _:
            print(f"{x} is not a point")

Combinators

The pattern matching is flexible but also fairly limited. It can only match on an object’s type, meaning we have to make a separate ABC for each thing we want to test. Fortunately, there’s a way around this. Python is dynamically typed. 99% of the time this just means “you don’t need static types if you’re okay with things crashing at runtime”. But it also means that type information exists at runtime, and that types can be created at runtime.

Can we use this for pattern matching? Let’s try it:

def Not(cls):
    class _Not(ABC):
        @classmethod
        def __subclasshook__(_, C):
            return not issubclass(C, cls)
    return _Not

def f(x):
    match x:
        case Not(DistanceMetric)(): 
            print(f"{x} is not a point")
        case _:
            print(f"{x} is a point")

Not is a function that takes a class, defines a new ABC, sets the hook for that ABC to “anything that’s not the class”, and then returns that ABC.

We try this and…

    case Not(DistanceMetric)():
                            ^
SyntaxError: expected ':'

It’s an error! We’ve finally hit the limits of pattern matching on ABCs. Then again, it’s “just” a syntax error. Maybe it would work if we tweak the syntax a little?

+   n = Not(DistanceMetric)
    match x:
-       case Not(DistanceMetric)(): 
+       case n():

PlanePoint(x=10, y=10) is a point
SpacePoint(x=5, y=6, z=7) is a point
[1, 2, 3] is not a point

Success! And just to test that this is composable, let’s write an And.

from abc import ABC
from dataclasses import dataclass
from collections.abc import Iterable

def Not(cls):
    class _Not(ABC):
        @classmethod
        def __subclasshook__(_, C):
            return not issubclass(C, cls)
    return _Not

def And(cls1, cls2):
    class _And(ABC):
        @classmethod
        def __subclasshook__(_, C):
            return issubclass(C, cls1) and issubclass(C, cls2)
    return _And


def f(x):
    n = And(Iterable, Not(str))
    match x:
        case n():
            print(f"{x} is a non-string iterable")
        case str():
            print(f"{x} is a string")
        case _:
            print(f"{x} is a string or not-iterable")


if __name__ == "__main__":
    f("abc")
    f([1, 2, 3])

This works as “”“expected”“”.

Caching Rules Everything Around Me

This got me thinking: what if __subclasshook__ wasn’t a pure function? Could I make an ABC that matched the first value of each type passed in, but not subsequent ones?

from abc import ABC

class OneWay(ABC):
    seen_classes = set()

    @classmethod
    def __subclasshook__(cls, C):
        print(f"trying {C}")
        if C in cls.seen_classes:
            return False
        cls.seen_classes |= {C}
        return True




def f(x):
    match x:
        case OneWay():
            print(f"{x} is a new class")
        case _:
            print(f"we've seen {x}'s class before")


if __name__ == "__main__":
    f("abc")
    f([1, 2, 3])
    f("efg")

Sadly, this was all for naught.

trying <class 'str'>
abc is a new class
trying <class 'list'>
[1, 2, 3] is a new class
efg is a new class

It looks like __subclasshook__ caches the results for a given type check. CPython assumes that people don’t want to shove side effects into esoteric corners of the language. Show’s how much they know.

We can still have fun with side effects, though. This ABC lets through every-other type.

class FlipFlop(ABC):
    flag = False

    @classmethod
    def __subclasshook__(cls, _):
        cls.flag = not cls.flag
        return cls.flag

And this ABC asks the user what it should do for each type.

class Ask(ABC):
    first_class = None

    @classmethod
    def __subclasshook__(cls, C):
        choice = input(f"hey should I let {C} though [y/n]  ")
        if choice == 'y':
            print("okay we'll pass em through")
            return True
        return False

Try them in a pattern match. They both work!

Should I use this?

God no.

The pattern matching feature is, on the whole, pretty reasonably designed, and people will expect it to behave in reasonable ways. Whereas __subclasshook__ is extremely dark magic. This kind of chicanery might have a place in the dark beating heart of a complex library, certainly not for any code your coworkers will have to deal with.

So yeah, you didn’t learn anything useful. I just like horrible things ¯\_(ツ)_/¯

Thanks to Predrag Gruevski for feedback. Title is from Crimes with Go Generics.

I shared an early version of this post on my weekly newsletter where I announce new blog posts and write additional essays. If you enjoyed this, why not subscribe?

The @property decorator makes distance readable and writeable as an attribute. It means we can write point.distance instead of point.distance(), and makes it easier for the subclasshook to pick up on it. ^[return]