When you build Python, you can pass various options to ./configure that
change aspects of how it is built. There is documentation for all of these
options, and they are
things like --prefix to tell the build where to install itself,
--without-pymalloc if you have some esoteric need for everything to go
through a custom memory allocator, or --with-pydebug.
One of these options only matters on macOS, and its effects are generally poorly understood. The official documentation just says “Create a Python.framework rather than a traditional Unix install.” But… do you need a Python.framework? If you’re used to running Python on Linux, then a “traditional Unix install” might sound pretty good; more consistent with what you are used to.
If you use a non-Framework build, most stuff seems to work, so why should anyone care? I have mentioned it as a detail in my previous post about Python on macOS, but even I didn’t really explain why you’d want it, just that it was generally desirable.
The
traditional
answer to this question is that you need a Framework build “if you want to use
a GUI”, but this is demonstrably not true. At first it might not seem so,
since the go-to Python GUI test is “run IDLE”; many non-Framework builds also
omit Tkinter because they don’t ship a Tk dependency, so IDLE won’t start. But
other GUI libraries work fine. For example, uv tool install runsnakerun /
runsnake will happily pop open a GUI window, Framework build or not. So it bears some explaining
Wait, what is a “Framework” anyway?
Let’s back up and review an important detail of the mac platform.
On macOS, GUI applications are not just an executable file, they are organized
into a bundle, which is a directory with a particular layout, that includes
metadata, that launches an executable. A thing that, on Linux, might live in a
combination of /bin/foo for its executable and /share/foo/ for its
associated data files, is instead on macOS bundled together into Foo.app, and
those components live in specified locations within that directory.
A
framework
is also a bundle, but one that contains a library. Since they are
directories, Applications can contain their own Frameworks and Frameworks can
contain helper Applications. If /Applications is roughly equivalent to the
Unix /bin, then /Library/Frameworks is roughly equivalent to the Unix
/lib.
App bundles are contained in a directory with a .app suffix, and frameworks
are a directory with a .framework suffix.
So what do you need a Framework for in Python?
The truth about Framework builds is that there is not really one specific thing that you can point to that works or doesn’t work, where you “need” or “don’t need” a Framework build. I was not able to quickly construct an example that trivially fails in a non-framework context for this post, but I didn’t try that many different things, and there are a lot of different things that might fail.
The biggest issue is not actually the Python.framework itself. The
metadata on the framework is not used for much outside of a build or linker
context. However, Python’s Framework builds also ship with a stub application
bundle, which places your Python process into a normal application(-ish)
execution context all the time, which allows for various platform APIs like
[NSBundle
mainBundle]
to behave in the normal, predictable ways that all of the numerous, various
frameworks included on Apple platforms
expect.
Various Apple platform features might want to ask a process questions like “what is your unique bundle identifier?” or “what entitlements are you authorized to access” and even beginning to answer those questions requires information stored in the application’s bundle.
Python does not ship with a wrapper around the core macOS “cocoa” API itself,
but we can use pyobjc to interrogate this. After installing
pyobjc-framework-cocoa, I can do this
1 2 | |
On a non-Framework build, it might look like this:
1 | |
But on a Framework build (even in a venv in a similar location), it might look like this:
1 | |
This is why, at various points in the past, GUI access required a framework build, since connections to the window server would just be rejected for Unix-style executables. But that was an annoying restriction, so it was removed at some point, or at least, the behavior was changed. As far as I can tell, this change was not documented. But other things like user notifications or geolocation might need to identity an application for preferences or permissions purposes, respectively. Even something as basic as “what is your app icon” for what to show in alert dialogs is information contained in the bundle. So if you use a library that wants to make use of any of these features, it might work, or it might behave oddly, or it might silently fail in an undocumented way.
This might seem like undocumented, unnecessary cruft, but it is that way because it’s just basic stuff the platform expects to be there for a lot of different features of the platform.
/etc/ builds
Still, this might seem like a strangely vague description of this feature, so
it might be helpful to examine it by a metaphor to something you are more
familiar with. If you’re familiar with more Unix style application development,
consider a junior developer — let’s call him Jim — asking you if they should
use an “/etc build” or not as a basis for their Docker containers.
What is an “/etc build”? Well, base images like ubuntu come with a bunch
of files in /etc, and Jim just doesn’t see the point of any of them, so he
likes to delete everything in /etc just to make things simpler. It seems to
work so far. More experienced Unix engineers that he has asked react
negatively and make a face when he tells them this, and seem to think that
things will break. But their app seems to work fine, and none of these
engineers can demonstrate some simple function breaking, so what’s the problem?
Off the top of your head, can you list all the features that all the files that
/etc is needed for? Why not? Jim thinks it’s weird that all this stuff is
undocumented, and it must just be unnecessary cruft.
If Jim were to come back to you later with a problem like “it seems like
hostname resolution doesn’t work sometimes” or “ls says all my files are
owned by 1001 rather than the user name I specified in my Dockerfile” you’d
probably say “please, put /etc back, I don’t know exactly what file you need
but lots of things just expect it to be there”.
This is what a framework vs. a non-Framework build is like. A Framework build
just includes all the pieces of the build that the macOS platform expects to be
there. What pieces do what features need? It depends. It changes over time.
And the stub that Python’s Framework builds include may not be sufficient for
some more esoteric stuff anyway. For example, if you want to use a feature
that needs a bundle that has been signed with custom
entitlements
to access something specific, like the virtualization
API,
you might need to build your own app
bundle. To extend our analogy with Jim, the fact that /etc exists and has
the default files in it won’t always be sufficient; sometimes you have to add
more files to /etc, with quite specific contents, for some features to work
properly. But “don’t get rid of /etc (or your application bundle)” is pretty
good advice.
Do you ever want a non-Framework build?
macOS does have a Unix subsystem, and many Unix-y things work, for Unix-y tasks. If you are developing a web application that mostly runs on Linux anyway and never care about using any features that touch the macOS-specific parts of your mac, then you probably don’t have to care all that much about Framework builds. You’re not going to be surprised one day by non-framework builds suddenly being unable to use some basic Unix facility like sockets or files. As long as you are aware of these limitations, it’s fine to install non-Framework builds. I have a dozen or so Pythons on my computer at any given time, and many of them are not Framework builds.
Framework builds do have some small drawbacks. They tend to be larger, they
can be a bit more annoying to relocate, they typically want to live in a
location like /Library or ~/Library. You can move Python.framework into
an application bundle according to certain rules, as any bundling tool for
macOS will have to do, but it might not work in random filesystem locations.
This may make managing really large number of Python versions more annoying.
Most of all, the main reason to use a non-Framework build is if you are building a tool that manages a fleet of Python installations to perform some automation that needs to know about Python installs, and you want to write one simple tool that does stuff on Linux and on macOS. If you know you don’t need any platform-specific features, don’t want to spend the (not insignificant!) effort to cover those edge cases, and you get a lot of value from that level of consistency (for example, a teaching environment or interdisciplinary development team with a lot of platform diversity) then a non-framework build might be a better option.
Why do I care?
Personally, I think it’s important for Framework builds to be the default for most users, because I think that as much stuff should work out of the box as possible. Any user who sees a neat library that lets them get control of some chunk of data stored on their mac - map data, health data, game center high scores, whatever it is - should be empowered to call into those APIs and deal with that data for themselves.
Apple already makes it hard enough with their thicket of code-signing and notarization requirements for distributing software, aggressive privacy restrictions which prevents API access to some of this data in the first place, all these weird Unix-but-not-Unix filesystem layout idioms, sandboxing that restricts access to various features, and the use of esoteric abstractions like mach ports for communications behind the scenes. We don't need to make it even harder by making the way that you install your Python be a surprise gotcha variable that determines whether or not you can use an API like “show me a user notification when my data analysis is done” or “don’t do a power-hungry data analysis when I’m on battery power”, especially if it kinda-sorta works most of the time, but only fails on certain patch-releases of certain versions of the operating system, becuase an implementation detail of a proprietary framework changed in the meanwhile to require an application bundle where it didn’t before, or vice versa.
More generally, I think that we should care about empowering users with local computation and platform access on all platforms, Linux and Windows included. This just happens to be one particular quirk of how native platform integration works on macOS specifically.
Acknowledgments
Thank you to my patrons who are supporting my writing on this blog. For this one, thanks especially to long-time patron Hynek who requested it specifically. If you like what you’ve read here and you’d like to read more of it, or you’d like to support my various open-source endeavors, you can support my work as a sponsor! I am also available for consulting work if you think your organization could benefit from expertise on topics like “how can we set up our Mac developers’ laptops with Python”.