Is there a place for non-@dataclass classes in Python any more?

I have previously — and somewhat famously — written favorably about @dataclass’s venerable progenitor, attrs, and how you should use it for pretty much everything.

At the time, attrs was an additional dependency, a piece of technology that you could bolt on to your Python stack to make your particular code better. While I advocated for it strongly, there are all the usual implicit reasons against using a new thing. It was an additional dependency, it might not interoperate with other convenience mechanisms for type declarations that you were already using (i.e. NamedTuple), it might look weird to other Python programmers familiar with existing tools, and so on. I don’t think that any of these were good counterpoints, but there was nevertheless a robust discussion to be had in addressing them all.

But for many years now, dataclasses have been — and currently are — built in to the language. They are increasingly integrated to the toolchain at a deep level that is difficult for application code — or even other specialized tools — to replicate. Everybody knows what they are. Few or none of those reasons apply any longer.

For example, classes defined with @dataclass are now optimized as a C structure might be when you compile them with mypyc, a trick that is extremely useful in some circumstances, which even attrs itself now has trouble keeping up with.

This all raises the question for me: beyond backwards compatibility, is there any point to having non-@dataclass classes any more? Is there any remaining justification for writing them in new code?

Consider my original example, translated from attrs to dataclasses. First, the non-dataclass version:

1
2
3
4
5

class Point3D:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

And now the dataclass one:

1
2
3
4
5
6
7

from dataclasses import dataclass

@dataclass
class Point3D:
    x: int
    y: int
    z: int

Many of my original points still stand. It’s still less repetitive. In fewer characters, we’ve expressed considerably more information, and we get more functionality (repr, sorting, hashing, etc). There doesn’t seem to be much of a downside besides the strictness of the types, and if typing.Any were a builtin, x: any would be fine for those who don’t want to unduly constrain their code.

The one real downside of the latter over the former right now is the need for an import. Which, at this point, just seems… confusing? Wouldn’t it be nicer to be able to just write this:

1
2
3
4

class Point3D:
    x: int
    y: int
    z: int

and not need to faff around with decorator semantics and fudging the difference between Mypy (or Pyright or Pyre) type-check-time and Mypyc or Cython compile time? Or even better, to not need to explain the complexity of all these weird little distinctions to new learners of Python, and to have to cover import before class?

These tools all already treat the @dataclass decorator as a totally special language construct, not really like a decorator at all, so to really explore it you have to explain a special case and then a special case of a special case. The extension hook for this special case of the special case notwithstanding.

If we didn’t want any new syntax, we would need a from __future__ import dataclassification or some such for a while, but this doesn’t seem like an impossible bar to clear.

There are still some folks who don’t like type annotations at all, and there’s still the possibility of awkward implicit changes in meaning when transplanting code from a place with dataclassification enabled to one without, so perhaps an entirely new unambiguous syntax could be provided. One that more closely mirrors the meaning of parentheses in def, moving inheritance (a feature which, whether you like it or not, is clearly far less central to class definitions than ‘what fields do I have’) off to its own part of the syntax:

1
2
3

data Point3D(x: int, y: int, z: int) from Vector:
    def method(self):
        ...

which, for the “I don’t like types” contingent, could reduce to this in the minimal case:

1
2

data Point3D(x, y, z):
    pass

Just thinking pedagogically, I find it super compelling to imagine moving from teaching def foo(x, y, z):... to data Foo(x, y, z):... as opposed to @dataclass class Foo: x: int....

I don’t have any desire for semantic changes to accompany this, just to make it possible for newcomers to ignore the circuitous historical route of the @dataclass syntax and get straight into defining their own types with legible reprs from the very beginning of their Python journey.

(And make it possible for me to skip a couple of lines of boilerplate in short examples, as a bonus.)

I’m curious to know what y’all think, though. Shoot me an email or a toot and let me know.

In particular:

1. Do you think there’s some reason I’m missing why Python’s current method for defining classes via a bunch of dunder methods is still better than dataclasses, or should stick around into the future for reasons beyond “compatibility”?
2. Do you think “compatibility” is sufficient reason to keep the syntax the way it is forever, and I’m underestimating the cost of adding a keyword like this?
3. If you do think that a change should be made, would you prefer:
   1. changing the meaning of class itself via a __future__ import,
   2. a new data keyword like the one I’ve proposed,
   3. a new keyword that functions exactly like the one I have proposed but really want to bikeshed the word data a bunch,
   4. something more incremental like just putting dataclass and field in builtins,
   5. or an option I haven’t even contemplated here?

If I find I’m not alone in this perhaps I will wander over to the Python discussion boards to have a more substantive conversation...

Thank you to my patrons who are helping me while I try to turn… whatever this is… along with open source maintenance and application development, into a real job. Do you want to see me pursue ideas like this one further? If so, you can support my work as a sponsor!

One of the persistently lesser-known symptoms of ADHD is hyperfocus. It is sometimes quasi-accurately described as a “superpower”^{1 2}, which it can be. In the right conditions, hyperfocus is the ability to effortlessly maintain a singular locus of attention for far longer than a neurotypical person would be able to.

However, as a general rule, it would be more accurate to characterize hyperfocus not as an “ability to focus on X” but rather as “an inability to focus on anything other than X”. Sometimes hyperfocus comes on and it just digs its claws into you and won’t let go until you can achieve some kind of closure.

Recently, the X I could absolutely not stop focusing on — for days at a time — was this extremely annoying picture:

Which lead to me writing the silliest computer program I have written in quite some time.

You see, for some reason, macOS seems to prefer YUV422 chroma subsampling³ on external displays, even when the bitrate of the connection and selected refresh rate support RGB.⁴ Lots of people have been trying to address this for a literal decade^{5 6 7 8 9 10 11}, and the problem has gotten worse with Apple Silicon, where the operating system no longer even supports the EDID-override functionality available on every other PC operating system that supports plugging in a monitor.

In brief, this means that every time I unplug my MacBook from its dock and plug it back in more than 5 minutes later, its color accuracy is destroyed and red or blue text on certain backgrounds looks like that mangled mess in the picture above. Worse, while the color distinction is definitely noticeable, it’s so subtle that it’s like my display is constantly gaslighting me. I can almost hear it taunting me:

Magenta? Yeah, magenta always looked like this. Maybe it’s the ambient lighting in this room. You don’t even have a monitor hood. Remember how you had to use one of those for print design validation? Why would you expect it to always look the same without one?

Still, I’m one of the luckier people with this problem, because I can seem to force RGB / 444 color format on my display just by leaving the display at 120Hz rather than 144, then toggling HDR on and then off again. At least I don’t need to plug in the display via multiple HDMI and displayport cables and go into the OSD every time. However, there is no API to adjust, or even discover the chroma format of your connected display’s link, and even the accessibility features that supposedly let you drive GUIs are broken in the system settings “Displays” panel¹², so you have to do it by sending synthetic keystrokes and hoping you can tab-focus your way to the right place.

Anyway, this is a program which will be useless to anyone else as-is, but if someone else is struggling with the absolute inability to stop fiddling with the OS to try and get colors to look correct on a particular external display, by default, all the time, maybe you could do something to hack on this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import os
from Quartz import CGDisplayRegisterReconfigurationCallback, kCGDisplaySetMainFlag, kCGDisplayBeginConfigurationFlag
from ColorSync import CGDisplayCreateUUIDFromDisplayID
from CoreFoundation import CFUUIDCreateString
from AppKit import NSApplicationMain, NSApplicationActivationPolicyAccessory, NSApplication

NSApplication.sharedApplication().setActivationPolicy_(NSApplicationActivationPolicyAccessory)

CGDirectDisplayID = int
CGDisplayChangeSummaryFlags = int

MY_EXTERNAL_ULTRAWIDE = '48CEABD9-3824-4674-9269-60D1696F0916'
MY_INTERNAL_DISPLAY = '37D8832A-2D66-02CA-B9F7-8F30A301B230'

def cb(display: CGDirectDisplayID, flags: CGDisplayChangeSummaryFlags, userInfo: object) -> None:
    if flags & kCGDisplayBeginConfigurationFlag:
        return
    if flags & kCGDisplaySetMainFlag:
        displayUuid = CGDisplayCreateUUIDFromDisplayID(display)
        uuidString = CFUUIDCreateString(None, displayUuid)
        print(uuidString, "became the main display")
        if uuidString == MY_EXTERNAL_ULTRAWIDE:
            print("toggling HDR to attempt to clean up subsampling")
            os.system("/Users/glyph/.local/bin/desubsample")
            print("HDR toggled.")

print("registered", CGDisplayRegisterReconfigurationCallback(cb, None))

NSApplicationMain([])

and the linked desubsample is this atrocity, which I substantially cribbed from this helpful example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

#!/usr/bin/osascript

use AppleScript version "2.4" -- Yosemite (10.10) or later
use framework "Foundation"
use framework "AppKit"
use scripting additions

tell application "System Settings"
    quit
    delay 1
    activate
    current application's NSWorkspace's sharedWorkspace()'s openURL:(current application's NSURL's URLWithString:"x-apple.systempreferences:com.apple.Displays-Settings.extension")
    delay 0.5

    tell application "System Events"
    tell process "System Settings"
        key code 48
        key code 48
        key code 48
            delay 0.5
        key code 49
        delay 0.5
        -- activate hdr on left monitor

        set hdr to checkbox 1 of group 3 of scroll area 2 of ¬
                group 1 of group 2 of splitter group 1 of group 1 of ¬
                window "Displays"
        tell hdr
                click it
                delay 1.0
                if value is 1
                    click it
                end if
        end tell

    end tell
    end tell
    quit
end tell

This ridiculous little pair of programs does it automatically, so whenever I reconnect my MacBook to my desktop dock at home, it faffs around with clicking the HDR button for me every time. I am leaving it running in a background tmux session so — hopefully — I can finally stop thinking about this.

One potato, two potato, three potato, four
Five potato, six potato, seven potato, more.

Traditional Children’s Counting Rhyme

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

Knuth, Donald
“Structured Programming with go to statements”
Computing Surveys, Vol. 6, No. 4, December 1974
(p. 268)
(Emphasis mine)

Knuth’s admonition about premature optimization is such a cliché among software developers at this point that even the correction to include the full context of the quote is itself a a cliché.

Still, it’s a cliché for a reason: the speed at which software can be written is in tension — if not necessarily in conflict — with the speed at which it executes. As Nelson Elhage has explained, software can be qualitatively worse when it is slow, but spending time optimizing an algorithm before getting any feedback from users or profiling the system as a whole can lead one down many blind alleys of wasted effort.

In that same essay, Nelson further elaborates that performant foundations simplify architecture¹. He then follows up with several bits of architectural advice that is highly specific to parsing—compilers and type-checkers specifically—which, while good, is hard to generalize beyond “optimizing performance early can also be good”.

So, here I will endeavor to generalize that advice. How does one provide a performant architectural foundation without necessarily wasting a lot of time on early micro-optimization?

Enter The Potato

Many years before Nelson wrote his excellent aforementioned essay, my father coined a related term: “Potato Programming”.

In modern vernacular, a potato is very slow hardware, and “potato programming” is the software equivalent of the same.

The term comes from the rhyme that opened this essay, and is meant to evoke a slow, childlike counting of individual elements as an algorithm operates upon them. it is an unfortunately quite common software-architectural idiom whereby interfaces are provided in terms of scalar values. In other words, APIs that require you to use for loops or other forms of explicit, individual, non-parallelized iteration. But this is all very abstract; an example might help.

For a generic business-logic example, let’s consider the problem of monthly recurring billing. Every month, we pull in the list of all of all subscriptions to our service, and we bill them.

Since our hypothetical company has an account-management team that owns the UI which updates subscriptions and a billing backend team that writes code to interface with 3rd-party payment providers, we’ll create 2 backends, here represented by some Protocols.

Finally, we’ll have an orchestration layer that puts them together to actually run the billing. I will use async to indicate which things require a network round trip:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

class SubscriptionService(Protocol):
    async def all_subscriptions(self) -> AsyncIterable[Subscription]:
        ...

class Subscription(Protocol):
    account_id: str
    to_charge_per_month: money

class BillingService(Protocol):
    async def bill_amount(self, account_id: str, amount: money) -> None:
        ...

To many readers, this may look like an entirely reasonable interface specification; indeed, it looks like a lot of real, public-facing “REST” APIs. An equally apparently-reasonable implementation of our orchestration between them might look like this:

1
2
3

async def billing(s: SubscriptionService, b: BillingService) -> None:
    async for sub in s.all_subscriptions():
        await b.bill_amount(sub.account_id, sub.to_charge_per_month)

This is, however, just about the slowest implementation of this functionality that it’s possible to implement. So, this is the bad version. Let’s talk about the good version: no-tato programming, if you will. But first, some backstory.

Some Backstory

My father began his career as an APL programmer, and one of the key insights he took away from APL’s architecture is that, as he puts it:

Computers like to do things over and over again. They like to do things on arrays. They don’t want to do things on scalars. So, in fact, it’s not possible to write a program that only does things on a scalar. [...] You can’t have an ‘integer’ in APL, you can only have an ‘array of integers’. There’s no ‘loop’s, there’s no ‘map’s.

APL, like Python², is typically executed via an interpreter. Which means, like Python, execution of basic operations like calling functions can be quite slow. However, unlike Python, its pervasive reliance upon arrays meant that almost all of its operations could be safely parallelized, and would only get more and more efficient as more and more parallel hardware was developed.

I said ‘unlike Python’ there, but in fact, my father first related this concept to me regarding a part of the Python ecosystem which follows APL’s design idiom: NumPy. NumPy takes a similar approach: it cannot itself do anything to speed up Python’s fundamental interpreted execution speed³, but it can move the intensive numerical operations that it implements into operations on arrays, rather than operations on individual objects, whether numbers or not.

The performance difference involved in these two styles is not small. Consider this case study which shows a 5828% improvement⁴ when taking an algorithm from idiomatic pure Python to NumPy.

This idiom is also more or less how GPU programming works. GPUs cannot operate on individual values. You submit a program⁵ to the GPU, as well as a large array of data⁶, and the GPU executes the program on that data in parallel across hundreds of tiny cores. Submitting individual values for the GPU to work on would actually be much slower than just doing the work on the CPU directly, due to the bus latency involved to transfer the data back and forth.

Back from the Backstory

This is all interesting for a class of numerical software — and indeeed it works very well there — but it may seem a bit abstract to web backend developers just trying to glue together some internal microservice APIs, or indeed most app developers who aren’t working in those specialized fields. It’s not like Stripe is going to let you run their payment service on your GPU.

However, the lesson generalizes quite well: anywhere you see an API defined in terms of one-potato, two-potato iteration, ask yourself: “how can this be turned into an array”? Let’s go back to our example.

The simplest change that we can make, as a consumer of these potato-shaped APIs, is to submit them in parallel. So if we have to do the optimization in the orchestration layer, we might get something more like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

from asyncio import Semaphore, AbstractEventLoop

async def one_bill(
    loop: AbstractEventLoop,
    sem: Semaphore,
    sub: Subscription,
    b: BillingService,
) -> None:
    await sem.acquire()
    async def work() -> None:
        try:
            await b.bill_amount(sub.account_id, sub.to_charge_per_month)
        finally:
            sem.release()
    loop.create_task(work)

async def billing(
    loop: AbstractEventLoop,
    s: SubscriptionService,
    b: BillingService,
    batch_size: int,
) -> None:
    sem = Semaphore(batch_size)
    async for sub in s.all_subscriptions():
        await one_bill(loop, sem, sub, b)

This is an improvement, but it’s a bit of a brute-force solution; a multipotato, if you will. We’ve moved the work to the billing service faster, but it still has to do just as much work. Maybe even more work, because now it’s potentially got a lot more lock-contention on its end. And we’re still waiting for the Subscription objects to dribble out of the SubscriptionService potentially one request/response at a time.

In other words, we have used network concurrency as a hack to simulate a performant design. But the back end that we have been given here is not actually optimizable; we do not have a performant foundation. As you can see, we have even had to change our local architecture a little bit here, to include a loop parameter and a batch_size which we had not previously contemplated.

A better-designed interface in the first place would look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

class SubscriptionService(Protocol):
    async def all_subscriptions(
        self, batch_size: int,
    ) -> AsyncIterable[Sequence[Subscription]]:
        ...

class Subscription(Protocol):
    account_id: str
    to_charge_per_month: money

@dataclass
class BillingRequest:
    account_id: str
    amount: money

class BillingService(Protocol):
    async def submit_bills(
        self,
        bills: Sequence[BillingRequest],
    ) -> None:
        ...

Superficially, the implementation here looks slightly more awkward than our naive first attempt:

1
2
3
4
5
6
7
8

async def billing(s: SubscriptionService, b: BillingService, batch_size: int) -> None:
    async for sub_batch in s.all_subscriptions(batch_size):
        await b.submit_bills(
            [
                BillingRequest(sub.account_id, sub.to_charge_per_month)
                for sub in sub_batch
            ]
        )

However, while the implementation with batching in the backend is approximately as performant as our parallel orchestration implementation, backend batching has a number of advantages over parallel orchestration.

First, backend batching has less internal complexity; no need to have a Semaphore in the orchestration layer, or to create tasks on an event loop. There’s less surface area here for bugs.

Second, and more importantly: backend batching permits for future optimizations within the backend services, which are much closer to the relevant data and can achieve more substantial gains than we can as a client without knowledge of their implementation.

There are many ways this might manifest, but consider that each of these services has their own database, and have got to submit queries and execute transactions on those databases.

In the subscription service, it’s faster to run a single SELECT statement that returns a bunch of results than to select a single result at a time. On the billing service’s end, it’s much faster to issue a single INSERT or UPDATE and then COMMIT for N records at once than to concurrently issue a ton of potentially related modifications in separate transactions.

Potato No Mo

The initial implementation within each of these backends can be as naive and slow as necessary to achieve an MVP. You can do a SELECT … LIMIT 1 internally, if that’s easier, and performance is not important at first. There can be a mountain of potatoes hidden behind the veil of that batched list. In this way, you can avoid the potential trap of premature optimization. Maybe this is a terrible factoring of services for your application in the first place; best to have that prototype in place and functioning quickly so that you can throw it out faster!

However, by initially designing an interface based on lists of things rather than individual things, it’s much easier to hide irrelevant implementation details from the client, and to achieve meaningful improvements when optimizing.

Acknowledgements

This is the first post supported by my patrons, with a topic suggested by a member of my Patreon!

Python’s standard datetime module is very powerful. However, it has a couple of annoying flaws.

Firstly, datetimes are considered a kind of date¹, which causes problems. Although datetime is a literal subclass of date so Mypy and isinstance believe a datetime “is” a date, you cannot substitute a datetime for a date in a program without provoking errors at runtime.

To put it more precisely, here are two programs which define a function with type annotations, that mypy finds no issues with. The first of which even takes care to type-check its arguments at run-time. But both raise TypeErrors at runtime:

Comparing datetime to date:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

from datetime import date, datetime

def is_after(before: date, after: date) -> bool | None:
    if not isinstance(before, date):
        raise TypeError(f"{before} isn't a date")
    if not isinstance(after, date):
        raise TypeError(f"{after} isn't a date")
    if before == after:
        return None
    if before > after:
        return False
    return True

is_after(date.today(), datetime.now())

1
2
3
4
5
6

Traceback (most recent call last):
  File ".../date_datetime_compare.py", line 14, in <module>
    is_after(date.today(), datetime.now())
  File ".../date_datetime_compare.py", line 10, in is_after
    if before > after:
TypeError: can't compare datetime.datetime to datetime.date

Comparing “naive” and “aware” datetime:

1
2
3
4
5
6

from datetime import datetime, timezone, timedelta

def compare(a: datetime, b: datetime) -> timedelta:
    return a - b

compare(datetime.now(), datetime.now(timezone.utc))

1
2
3
4
5
6

Traceback (most recent call last):
  File ".../naive_aware_compare.py", line 6, in <module>
    compare(datetime.now(), datetime.now(timezone.utc))
  File ".../naive_aware_compare.py", line 4, in compare
    return a - b
TypeError: can't subtract offset-naive and offset-aware datetimes

In some sense, the whole point of using Mypy - or, indeed, of runtime isinstance checks - is to avoid TypeError getting raised. You specify all the types, the type-checker yells at you, you fix it, and then you can know your code is not going to blow up in unexpected ways.

Of course, it’s still possible to avoid these TypeErrors with runtime checks, but it’s tedious and annoying to need to put a check for .tzinfo is not None or not isinstance(..., datetime) before every use of - or >.

The problem here is that datetime is trying to represent too many things with too few types. datetime should not be inheriting from date, because it isn’t a date, which is why > raises an exception when you compare the two.

Naive datetimes represent an abstract representation of a hypothetical civil time which are not necessarily tethered to specific moments in physical time. You can’t know exactly what time “today at 2:30 AM” is, unless you know where on earth you are and what the rules are for daylight savings time in that place. However, you can still talk about “2:30 AM” without reference to a time zone, and you can even say that “3:30 AM” is “60 minutes after” that time, even if, given potential changes to wall clock time, that may not be strictly true in one specific place during a DST transition. Indeed, one of those times may refer to multiple points in civil time at a particular location, when attached to different sides of a DST boundary.

By contrast, Aware datetimes represent actual moments in time, as they combine civil time with a timezone that has a defined UTC offset to interpret them in.

These are very similar types of objects, but they are not in fact the same, given that all of their operators have slightly different (albeit closely related) semantics.

Using datetype

I created a small library, datetype, which is (almost) entirely type-time behavior. At runtime, despite appearances, there are no instances of new types, not even wrappers. Concretely, everything is a date, time, or datetime from the standard library. However, when type-checking with Mypy, you will now get errors reported from the above scenarios if you use the types from datetype.

Consider this example, quite similar to our first problematic example:

Comparing AwareDateTime or NaiveDateTime to date:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from datetype import Date, NaiveDateTime

def is_after(before: Date, after: Date) -> bool | None:
    if before == after:
        return None
    if before > after:
        return False
    return True

is_after(Date.today(), NaiveDateTime.now())

Now, instead of type-checking cleanly, it produces this error, letting you know that this call to is_after will give you a TypeError.

1
2

date_datetime_datetype.py:10: error: Argument 2 to "is_after" has incompatible type "NaiveDateTime"; expected "Date"
Found 1 error in 1 file (checked 1 source file)

Similarly, attempting to compare naive and aware objects results in errors now. We can even use the included AnyDateTime type variable to include a bound similar to AnyStr from the standard library to make functions that can take either aware or naive datetimes, as long as you don’t mix them up:

Comparing AwareDateTime to NaiveDateTime:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

from datetime import datetime, timezone, timedelta
from datetype import AwareDateTime, NaiveDateTime, AnyDateTime


def compare_same(a: AnyDateTime, b: AnyDateTime) -> timedelta:
    return a - b


def compare_either(
    a: AwareDateTime | NaiveDateTime,
    b: AwareDateTime | NaiveDateTime,
) -> timedelta:
    return a - b


compare_same(NaiveDateTime.now(), AwareDateTime.now(timezone.utc))

compare_same(AwareDateTime.now(timezone.utc), AwareDateTime.now(timezone.utc))
compare_same(NaiveDateTime.now(), NaiveDateTime.now())

1
2
3
4
5
6

naive_aware_datetype.py:13: error: No overload variant of "__sub__" of "_GenericDateTime" matches argument type "NaiveDateTime"
...
naive_aware_datetype.py:13: error: No overload variant of "__sub__" of "_GenericDateTime" matches argument type "AwareDateTime"
...
naive_aware_datetype.py:16: error: Value of type variable "AnyDateTime" of "compare_same" cannot be "_GenericDateTime[Optional[tzinfo]]"
Found 3 errors in 1 file (checked 1 source file)

Telling the Difference

Although the types in datetype are Protocols, there’s a bit of included magic so that you can use them as type guards with isinstance like regular types. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

from datetype import NaiveDateTime, AwareDateTime
from datetime import datetime, timezone

nnow = NaiveDateTime.now()
anow = AwareDateTime.now(timezone.utc)


def check(d: AwareDateTime | NaiveDateTime) -> None:
    if isinstance(d, NaiveDateTime):
        print("Naive!", d - nnow)
    elif isinstance(d, AwareDateTime):
        print("Aware!", d - anow)


check(NaiveDateTime.now())
check(AwareDateTime.now(timezone.utc))

Try it out, carefully

This library is very much alpha-quality; in the process of writing this blog post, I made a half a dozen backwards-incompatible changes, and there are still probably a few more left as I get feedback. But if this is a problem you’ve had within your own codebases - ensuring that dates and datetimes don’t get mixed up, or requiring that all datetimes crossing some API boundary are definitely aware and not naive, give it a try with pip install datetype and let me know if it catches any bugs!

In this post I’d like to convince you that you should be running Mypyc over your code¹ — especially if your code is a library you upload to PyPI — for both your own benefit and that of the Python ecosystem at large.

But first, let me give you some background.

Python is Slow, And That’s Fine, Because It’s Fast Enough

A common narrative about Python’s value proposition, from the very earliest days of the language², often recited in response to a teammate saying “shouldn’t we just write this in $HIGHER_PERFORMANCE_LANGUAGE instead?” goes something like this:

Sure, Python is slow.

But that’s okay, because it saves you so much time over implementing your code in $HIGHER_PERFORMANCE_LANGUAGE that you’ll have so much more time for optimizing those critical hot-spots where performance is really critical.

And if the language’s primitives are too slow to micro-optimize those hot-spots enough, that’s okay too, because you can always re-write just those small portions of the program as a C extension module.

Python’s got you covered!

There is some truth to this narrative, and I’ve quoted from it myself on many occasions. When I did so, I was not quoting it as some facile, abstract hypothetical, either. I had a few projects, particularly very early in my Python career, where I replaced performance-critical C++ code with a one tenth the number of lines of Python, and improved performance by orders of magnitude in the process³.

When you have algorithmically interesting, performance-sensitive code that can benefit from a high-level expressive language, and the resources to invest in making it fast, this process can be counterintuitively more efficient than other, “faster” tools. If you’re working on massively multiplayer online games⁴ or something equally technically challenging, Python can be a surprisingly good idea.

But… Is It Fine, Though?

This little nugget of folk wisdom does sound a bit defensive, doesn’t it? If Python were just fast, you could just use it, you wouldn’t need this litany of rationalizations. Surely if we believed that performance is important in our own Python code, we wouldn’t try to wave away the performance of Python itself.

Most projects are not massively multiplayer online games. On many straightforward business automation projects, this sort of staged approach to performance is impractical.

Not all performance problems are hot spots. Some programs have to be fast all the way through. This is true of some complex problems, like compilers and type checkers, but is also often the case in many kinds of batch processing; there are just a lot of numbers, and you have to add them all up.

More saliently for the vast majority of average software projects, optimization just isn’t in the budget. You do your best on your first try and hope that none of those hot spots get too hot, because as long as the system works within a painfully generous time budget, the business doesn’t care if it’s slow.

The progression from “idiomatic Python” to “optimized Python” to “C” is a one-way process that gradually loses the advantages that brought us to Python in the first place.

The difficult-to-reverse nature of each step means that once you have prototyped out a reasonably optimized data structure or algorithm, you need to quasi-permanently commit to it in order to squeeze out more straight-line performance of the implementation.

Plus, the process of optimizing Python often destroys its readability, for a few reasons:

1. Optimized Python relies on knowledge of unusual tricks. Things like “use the array module instead of lists”, and “use % instead of .format”.
2. Optimized Python requires you to avoid the things that make Python code nicely organized:
   1. method lookups are slow so you should use functions.
   2. object attribute accesses are slow so you should use tuples with hard-coded numeric offsets.
   3. function calls are slow so you should copy/paste and inline your logic
3. Optimized Python requires very specific knowledge of where it’s going to be running, so you lose the flexibility of how to run it: making your code fast on CPython might make it much slower on PyPy, for example. Native extension modules can make your code faster, but might also make it fail to run inside a browser, or add a ton of work to get it set up on a new operating system.

Maintaining good performance is part of your software’s development lifecycle, not just a thing you do once and stop. So by moving into this increasingly arcane dialect of “fast” python, and then into another programming language entirely with a C rewrite, you end up having to maintain C code anyway. Not to mention the fact that rewriting large amounts of code in C is both ludicrously difficult (particularly if your team primarily knows Python) and also catastrophically dangerous. In recent years, safer tools such as PyO3 have become available, but they still involve switching programming languages and rewriting all your code as soon as you care about speed⁵.

So, for Python to be a truly general-purpose language, we need some way to just write Python, and have it be fast.

It would benefit every user of Python for there to be an easy, widely-used way to make idiomatic, simple Python that just does stuff like adding numbers, calling methods, and formatting strings in a straight line go really fast — exactly the sorts of things that are the slowest in Python, but are also the most common, particularly before you’ve had an opportunity to cleverly optimize.

We’ve Been Able To At Least Make Do

There are also a number of tools that have long been in use for addressing this problem: PyPy, Pyrex, Cython, Numba, and Numpy to name a few. Their maintainers all deserve tremendous amounts of credit, and I want to be very clear that this post is not intended to be critical of anyone’s work here. These tools have drawbacks, but many of those drawbacks make them much better suited to specialized uses beyond the more general 80% case I’m talking about in this post, for which Mypyc would not be suitable.

Each one of these tools impose limitations on either the way that you write code or where you can deploy it.

Cython and Numba aren’t really “Python” any more, because they require special-purpose performance-oriented annotations. Cython has long supported pure-Python type annotations, but you won’t get any benefit from telling it that your variable is an int, only a cython.int. It can’t optimize a @dataclass, only a @cython.cclass. And so on.

PyPy gets the closest — it’s definitely regular Python — but its strategy has important limitations. Primarily, despite the phenomenal and heroic effort that went into cpyext, it seems like there’s always just one PyPy-incompatible library in every large, existing project’s dependency list which makes it impossible to just drop in PyPy without doing a bunch of arcane debugging first.

PyPy might make your program magically much faster, but if it doesn’t work, you have to read the tea leaves on the JIT’s behavior in a profiler which practically requires an online component that doesn’t even work any more. So mostly you just simplify your code to use more straightforward data structures and remove CPython-specific tricks that might trip up the JIT, and hope for the best.

PyPy also introduces platform limitations. It’s always — understandably, since they have to catch up after the fact — lagging a bit behind the most recently released version of CPython, so there’s always some nifty language feature that you have to refrain from using for at least one more release cycle.

It also has architectural limitations. For example, it performs quite poorly on an M1 Mac since it still runs under x86_64 emulation on that platform. And due to iOS forbidding 3rd-party JITs, it won’t ever be able to provide better performance in one of the more constrained environments that needs it more that other places. So you might need to rely on CPython on those platforms anyway… and you just removed all your CPython-specific hacks to try to please the JIT on the other platforms you support.

So while I would encourage everyone to at least try their code on PyPy — if you’re running a web-based backend, it might save you half your hardware budget⁶ — it’s not going to solve “python is slow” in the general case.

It’ll Eventually Be All Right

This all sounds pretty negative, so I would be remiss if I did not also point out that the core team is well aware that Python’s default performance needs to be better, and Guido van Rossum literally came out of retirement for one last job to fix it, and we’ve already seen a bunch of benefits from that effort.

But there are some fundamental limitations on the long-term strategy for these optimizations; one of the big upcoming improvements is a JIT, which suffers from some (but not all) of the same limitations as PyPy, and the late-bound, freewheeling nature of Python inherently comes with some performance tradeoffs.

So it would still behoove us to have a strategy for production-ized code that gives good, portable, ahead-of-time performance.

But What About Right Now?

Mypyc takes the annotations meant for Mypy and generates C with them, potentially turning your code into a much more efficient extension module. As part of Mypy itself, it does this with your existing Python type-hints, the kind you’d already use Mypy with to check for correctness, so it doesn’t entail much in the way of additional work.

I’d been curious about this since it was initially released, but I still haven’t had a hard real-world performance problem to really put it through its paces.

So when I learned about the High Throughput Fizzbuzz Challenge via its impressive assembler implementation that achieves 56GiB/s, and I saw even heavily-optimized Python implementations sitting well below the performance of a totally naïve C reference implementation, I thought this would be an interesting miniature experiment to use to at least approximate practical usage.

In Which I Design A Completely Unfair Fight Which I Will Then Handily Win

The dizzying heights of cycle-counting hand-tuned assembler implementations of this benchmark are squarely out of our reach, but I wanted to see if I could beat the performance of this very naïve C implementation with Python that was optimized, but at least, somewhat idiomatic and readable.

I am about to compare a totally naïve C implementation with a fairly optimized hand-tuned Python one, which might seem like an unfair fight. But what I’m trying to approximate here is a micro-instance of the real-world development-team choice that looks like this:

Since Python is more productive, but slower, the effort to deliver each of the following is similar:

1. a basic, straightforward implementation of our solution in C
2. a moderately optimized Python implementation of our solution

and we need to choose between them.

This is why I’ll just be showing naïve C and not unrolling any loops; I’ll use -O3 because any team moderately concerned with performance would at least turn on the most basic options, but nothing further.

Furthermore, our hypothetical team also has this constraint, which really every reasonable team should:

We can trade off some readability for efficiency, but it’s important that our team be able to maintain this code going forward.

This is why I’m doing a bit of optimizing in Python but not going all out by calling mmap or pulling in numpy or attempting to use something super esoteric like a SIMD library to emulate what the assembler implementations do. The goal is that this is normal Python code with a reasonable level of systems-level understanding (i.e. accounting for the fact that pipes have buffers in the kernel and approximately matching their size maximizes throughput).

If you want to see FizzBuzz pushed to its limit, you can go check out the challenge itself. Although I think I do coincidentally beat the performance of the Python versions they currently have on there, that’s not what I’m setting out to do.

So with that elaborate framing of this slightly odd experiment out of the way, here’s our naïve C version:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

#include <stdio.h>

int main() {
    for (int i = 1; i < 1000000000; i++) {
        if ((i % 3 == 0) && (i % 5 == 0)) {
            printf("FizzBuzz\n");
        } else if (i % 3 == 0) {
            printf("Fizz\n");
        } else if (i % 5 == 0) {
            printf("Buzz\n");
        } else {
            printf("%d\n", i);
        }
    }
}

First, let’s do a quick head-to-head comparison with a naïve Python implementation of the algorithm:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

def fizzbuzz() -> None:
    for counter in range(1, 1000000000):
        fizz = counter % 3 == 0
        buzz = counter % 5 == 0
        if fizz:
            print("Fizz", end="")
        if buzz:
            print("Buzz", end="")
        if not (fizz or buzz):
            print(counter, end="")
        print()

if __name__ == "__main__":
    fizzbuzz()

Running both of these on my M1 Max MacBook, the naïve C implementation yields 127 MiB/s of Fizzbuzz output. But, as I said, although we’re not going to have time for testing a more complex optimized C version, we would want to at least build it with the performance benefits we get for free with the -O3 compiler option. It turns out that yields us a 27 MiB/s speedup. So 154 MiB/s is the number we have to beat.⁷

The naïve Python version achieves a dismal 24.3 MiB/s, due to a few issues. First of all, although it’s idiomatic, print() is doing a lot of unnecessary work here. Among other things, we are encoding Unicode, which the C version isn’t. Still, our equivalent of adding the -O3 option for C is running mypyc without changing anything, and that yields us a 6.8MiB/s speedup immediately. We still aren’t achieving comparable performance, but a roughly 25% performance improvement for no work at all is a promising start!⁸

In keeping with the “some optimizations, but not so much that it’s illegible” constraint described above, the specific optimizations I’ve chosen to pursue here are:

1. switch to using bytes objects and sys.stdout.buffer to avoid encoding overhead
2. take advantage of the repeating nature of the pattern in FizzBuzz output and pre-generate a template rather than computing each line independently
3. fill out the buffer with the relevant integers from a sequence as we go
4. tune the repetition of that template to a size that roughly fills a pipe buffer on my platform of choice

Hopefully, with that explanation, this isn’t too bad:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

from sys import stdout
from typing import Tuple, Iterable


def precompute_template() -> Iterable[bytes]:
    for counter in range(1, 16):
        fizz = counter % 3 == 0
        buzz = counter % 5 == 0
        if fizz:
            yield b"Fizz"
        if buzz:
            yield b"Buzz"
        if not (fizz or buzz):
            yield b"%d"
        yield b"\n"


chunk_copies = 4
precomputed_template_chunks = list(precompute_template())
format_string = b"".join(precomputed_template_chunks)
number_indexes = [
    number_index
    for number_index, line_content in enumerate(format_string.split(b"\n"))
    if line_content == b"%d"
]
format_string *= chunk_copies


def fizzbuzz() -> None:
    num: int = 1
    output = stdout.buffer.write
    for num in range(1, 1000000001, 15 * chunk_copies):
        t: Tuple[int, ...] = tuple(
            (
                x + number_index
                for x in range(num, num + (15 * chunk_copies), 15)
                for number_index in number_indexes
            )
        )
        output(format_string % t)


if __name__ == "__main__":
    fizzbuzz()

Running this optimized version actually gets us within the ballpark of the naïve C version, even beating it by a hair; my measurement was 159 MiB/s, a small improvement even over -O3. So, per the “litany against C” from the beginning of this post, algorithmic optimization of Python really does help a lot; it’s not just a rationalization. This is a much bigger boost than our original no-effort Mypyc run, giving us more like an 85% speedup; definitely bigger than 25%.

But clearly we’re still being slowed down by Python’s function call overhead, object allocations for small integers, and so on, so Mypyc should help us out here: and indeed it does. On my machine, it nets a whopping 233 MiB/s. Now that we are accounting for performance and optimizing a bit, Mypyc’s relative advantage has doubled to a 50% improvement in performance on both the optimized-but-interpreted Python and naïve C versions.

It’s worth noting that the technique I used to produce the extension modules to test was literally pip install mypy; mypyc .../module.py, then python -c “import module”. I did already have a C compiler installed, but other than that, there was no setup.

I just wrote Python, and it just worked.

The Call To Adventure

Here’s what I want you to take away from all this:

1. Python can be fast.
2. More importantly, your Python can be fast.
3. For a fairly small investment of effort, your Python code can be made meaningfully faster.

Unfortunately, due to the limitations and caveats of existing powerful performance tools like Cython and PyPy, over the last few years in the Python community a passive consensus has emerged. For most projects, in most cases, it’s just not worth it to bother to focus on performance. Everyone just uses the standard interpreter, and only fixes the worst performance regressions.

We should, of course, be glad that the standard interpreter is reliably getting faster all the time now, but we shouldn’t be basing our individual libraries’ and applications’ performance strategies on that alone.

The projects that care the most about performance have made the effort to use some of these tools, and they have often invested huge amounts of effort to good effect, but often they care about performance too much. They make the problem look even harder for everyone else, by essentially stipulating that step 1 is to do something extreme like give up and use Fortran for all the interesting stuff.

My goal with this post is to challenge that status quo, spark interest in revisiting the package ecosystem’s baseline performance expectations, and to get more projects — particularly libraries on PyPI — to pick up Mypyc and start giving Python a deserved reputation for being surprisingly fast.

The Last Piece of the Puzzle

One immediate objection you might be thinking of is the fact that, under the hood, Mypyc is emitting some C code and building it, and so this might create a problem for deployment: if you’ve got a Linux machine but 30% of your users are on Windows, moving from pure-Python to this hybrid workflow might create installation difficulties for them, or at least they won’t see the benefits.

Luckily a separate tool should make that a non-issue: cibuildwheel. “CI Build Wheel”, as its name suggests, lets you build your wheels in your continuous integration system, and upload those builds automatically upon tagging a release.

Often, the bulk of the work in using it is dealing with the additional complexities involved in setting up your build environment in CI to make sure you’re appropriately bundling in any native libraries you depend upon, and linking to them in the correct way. Mypyc’s limitation relative to Cython is a huge advantage here: it doesn’t let you link to other native libraries, so you can always skip the worst step here.

So, for maintainers, you don’t need to maintain a pile of janky VMs on your personal development machine in order to serve your users. For users, nobody needs to deal with the nightmare of setting up the right C compiler on their windows machine, because the wheels are prebuilt. Even users without a compiler who want to contribute new code or debug it can run it with the interpreter locally, and let the cloud handle the complicated compilation steps later. Once again, the fact that you can’t require additional, external C libraries here is a big advantage; it prevents you from making the user’s experience inadvertently worse.

cibuildwheel supports all major operating systems and architectures, and supported versions of Python, and even lets you build wheels for PyPy while you’re at it.⁹

Putting It All Together

Using Mypyc and cibuildwheel, we, as PyPI package maintainers, can potentially produce an ecosystem of much faster out-of-the-box experiences via prebuilt extension modules, written entirely in Python, which would make the average big Python application with plenty of dependencies feel snappier than expected. This doesn’t have to come with the pain that we have unfortunately come to expect from C extensions, either as maintainers or users.

Another nice thing is that this is not an all-or-nothing proposition. If you try PyPy and it blows up in some obscure way on your code, you have to give up on it unless you want to fully investigate what’s happening. But if you trip over a bug in Mypyc, you can report the bug, drop the module where you’re having the problem from the list of things you’re trying to compile, and move on. You don’t even have to start out by trying to jam your whole project through it; just pick a few key modules to get started, and gradually expand that list over time, as it makes sense for your project.

In a future post, I’ll try to put all of this together myself, and hopefully it’s not going to be embarrassingly difficult and make me eat my words.

Despite not having done that yet, I wanted to put this suggestion out now, to get other folks thinking about getting started with it. For older projects¹⁰, retrofitting all the existing infrastructure to put Mypyc in place might be a bit of a challenge. But for new projects starting today, putting this in place when there’s very little code might be as simple as adding a couple of lines to pyproject.toml and copy-pasting some YAML into a Github workflow.

If you’re thinking about making some new open source Python, give Mypyc a try, and see if you can delight some users with lightning speed right out of the box. If you do, let me know how it turns out.

Acknowledgments

Thanks to Donald Stufft, Moshe Zadka, Nelson Elhage, Itamar Turner-Trauring, and David Reid for extensive feedback on this post. As always, any errors or inaccuracies remain my own.