In honor of Eevee’s delightful Games Made Quick???, I’d like to help you package your games even quicker than you made them.
Who is this for?
About ten years ago I made a prototype of a little PyGame thing which I wanted to share with a few friends. Building said prototype was quick and fun, and very different from the usual sort of work I do. But then, the project got just big enough that I started to wonder if it would be possible to share the result, and thus began the long winter of my discontent with packaging tools.
I might be the only one, but... I don’t think so. The history of PyWeek, for example, looks to be a history of games distributed as Github repositories, or, at best, apps which don’t launch. It seems like people who participate in game jams with Unity push a button and publish their games to Steam; people who participate in game jams with Python wander away once the build toolchain defeats them.
So: perhaps you’re also a Python programmer, and you’ve built something with PyGame, and you want to put it on your website so your friends can download it. Perhaps many or most of your friends and family are Mac users. Perhaps you tried to make a thing with py2app once, and got nothing but inscrutable tracebacks or corrupt app bundles for your trouble.
If so, read on and enjoy.
If things didn’t work for me when I first tried to do this, what’s different now?
- the packaging ecosystem in general is far less buggy, and py2app’s dependencies, like setuptools, have become far more reliable as well. Many thanks to Donald Stufft and the whole PyPA for that.
- Binary wheels exist, and the community has been getting better and better at building self-contained wheels which include any necessary C libraries, relieving the burden on application authors to figure out gnarly C toolchain issues.
- The PyGame project now ships just such wheels for a variety of Python versions on Mac, Windows, and Linux, which removes a whole huge pile of complexity both in generally understanding the C toolchain and specifically understanding the SDL build process.
- py2app has been actively maintained and many bugs have been fixed - many thanks to Ronald Oussoren et. al. for that.
- I finally broke down and gave Apple a hundred dollars so I can produce an app that normal humans might actually be able to run.
There are still weird little corner cases you have to work around — hence this post – but mostly this is the story of how years of effort by the Python packaging community have resulted in tools that are pretty close to working out of the box now.
Step 0: Development Setup
Get a good Python.
My recommendation is to use an official build from python.org; these are already compiled in such a way that they will run on a wide range of macs, both new and old. Use a recent Python 3 version, if you can; there are a variety of low-level improvements which make it better for redistribution.
- My previous recommendation was to use Homebrew; this is wrong. Don’t use homebrew; it might build from source, and if it does, it might do it in a way which doesn’t work on a lot of macs out there. If you’re going to compile your own python from source, you need to familiarize yourself with a bunch of tips and tricks for making sure you don’t enable CPU-specific optimizations, too-recent SDK requirements, and so on.
- This goes for pyenv too; it can accidentally configure Python in ways that are not good for redistributables.
- Definitely don’t use the System python. Probably nothing will work.
You probably also want to use a
virtualenv for development. This post is
about how to build a for-real thing that other people can download, but part of
the magic of Python is the interactive, real-time dynamic nature of everything.
Running the full build pipeline every time you change a file or an asset is
slow and annoying. However, there’s a weird thing where certain parts of the
macOS GUI won’t work right (in PyGame’s case, mostly keyboard focus) unless
your code appears to be in an application bundle.
I made this dumb little
which lets you fake out enough of this that the OS won’t hassle you: you need
pip install venvdotapp; venvdotapp inside the virtualenv where you’re
making your pygame app.
pip install all your requirements into your
Step 1: Make an icon
All good apps need an icon, right?
When I was young, one would open up
Resorcerer MPW CodeWarrior
Project Builder Icon Composer Xcode and
create a new ICON resource cicn resource
.icns file. Nowadays there’s some weird opaque
xcassets files and
Contents.json and “Copy Bundle Resources” in
the default Swift and Objective C project templates and honestly I can’t be
bothered to keep track of what’s going on with this nonsense any more.
Luckily the OS ships with the macOS-specific “scriptable image processing system”, which can helpfully convert an icon for you. Make yourself a 512x512 PNG file in your favorite image editor (with an alpha channel!) that you want to use as your icon, then run it something like this:
somewhere in your build process, to produce an icon in the appropriate format.
There’s also one additional wrinkle with PyGame: once you’ve launched the
game, PyGame helpfully assigns the cute, but ugly, default PyGame icon to
your running process. To avoid this, you’ll need these two lines somewhere in
your initialization code, somewhere before
pygame.display.init (or, for that
Obviously this is pretty Mac-specific so you probably want this under some kind of platform-detection conditional, perhaps this one.
Step 2: Include All The Dang Files, I Don’t Care About Performance
Unfortunately py2app still tries really hard to jam all your code into a
file, which breaks the world in various hilarious ways. Your app will probably
have some resources you want to load, as will PyGame itself.
packages=["your_package"] in your setup.py should address this,
and it comes with a “pygame” recipe, but neither of these things worked for me.
Instead, I convinced py2app to splat out all the files by using the
not-quite-public “recipe” plugin API:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
This is definitely somewhat less efficient than py2app’s default of stuffing the code into a single zip file, but, as a counterpoint to that: it actually works.
Step 3: Build it
Hopefully, at this point you can do
python setup.py py2app and get a shiny
new app bundle in
dist/$NAME.app. We haven’t had to go through the hell of
yet, so it should launch at this point. If it doesn’t, sorry :-(.
You can often debug more obvious fail-to-launch issues by running the
executable in the command line, by running
./dist/$NAME.app/Contents/MacOS/$NAME. Although this will run in a slightly
different environment than double clicking (it will have all your shell’s env
vars, for example, so if your app needs an env var to work it might
mysteriously work there) it will also print out any tracebacks to your
terminal, where they’ll be slightly easier to find than in Console.app.
Once your app at least runs locally, it’s time to...
Step 4: Code sign it
All the tutorials that I’ve found on how to do this involve doing Xcode project goop where it’s not clear what’s happening underneath. But despite the fact that the introductory docs aren’t quite there, the underlying model for codesigning stuff is totally common across GUI and command-line cases. However, actually getting your cert requires Xcode, an apple ID, and a credit card.
After paying your hundred dollars, go into Xcode, go to Accounts, hit “+”, “Apple ID”, then log in. Then, in your shiny new account, go to “Manage Certificates”, hit the little “+”, and (assuming, like me, you want to put something up on your own website, and not submit to the Mac App Store), and choose Developer ID Application. You probably think you want “mac app distribution” because you are wanting to distribute a mac app! But you don’t.
Next, before you do anything else, make sure you have backups of your certificate and private key. You really don’t want to lose the private key associated with that cert.
Now quit Xcode; you’re done with the GUI.
You will need to know the identifier of your signing key though, which should be output from the command:
You probably want to put that in your build script, since you want to sign with
the same identity every time. Further commands here will assume you’ve copied one of the lines of results from that command and done
export IDENTITY="..." with it.
Step 4a: Become Aware Of New Annoying Requirements
Update for macOS Catalina: In Catalina, Apple has added a new code-signing requirement; even for apps distributed outside of the app store, they still have to be submitted to and approved by Apple.
In order to be notarized, you will need to codesign not only your app itself, but to also:
- add the hardened-runtime exception entitlements that allow Python to work, and
- directly sign every shared library that is part of your app bundle.
So the actual code-signing step is now a little more complicated.
Step 4b: Write An Entitlements Plist That Allows Python To Work
One of the features that notarization is intended to strongly encourage1 is the “hardened runtime”, a feature of macOS which opts in to stricter run-time behavior designed to stop malware. One thing that the hardened runtime does is to disable writable, executable memory, which is used by JITs, FFIs ... and malware.
Unfortunately, both Python’s built-in
ctypes module and various popular bits
of 3rd-party stuff that uses
pyOpenSSL, require writable,
executable memory to work. Furthermore,
py2app actually imports
during its bootstrapping phase, so you can’t even get your own code to start
running to perform any workarounds unless this is enabled. So this is just
if you want to use Python, not if your project requires
To make this long, sad story significantly shorter and happier, you can create an entitlements property list that enables the magical property which allows this to work. It looks like this:
1 2 3 4 5 6 7 8
Subsequent steps assume that you’ve put this into a file called
entitleme.plist in your project root.
Step 4c: SIGN ALL THE THINGS
Notarization also requires that all the executable files in your bundle, not
just the main executable, are properly code-signed before submitting. So
you’ll need to first run the
codesign command across all your shared
libraries, something like this:
1 2 3 4 5 6 7 8 9
Then finally, sign the bundle itself.
1 2 3 4 5
Now, your app is code-signed.
Step 5: Archive it
The right way to do this is probably to use dmgbuild or something like it, but what I promised here was quick and dirty, not beautiful and best practices.
You have to make a Zip archive that preserves symbolic links. There are a couple of options for this:
open dist/, then in the Finder window that comes up, right click on the app and “compress” it
cd dist; zip -yr $NAME.app.zip $NAME.app
Most importantly, if you use the
zip command line tool, you must use the
-y option. Without it, your downloadable app bundle will be somewhat
mysteriously broken even though the one before you
zipped it will be fine.
Step 6: Actually The Rest Of Step 4: Request Notarization
Notarization is a 2-step process, which is somewhat resistant to fully automating. You submit to Apple, then they email you the results of doing the notarization, then if that email indicates that your notarization succeded, you can “staple” the successful result to your bundle.
The thing you notarize is an archive, which is why you need to do step 5 first. Then, you need to do this:
1 2 3 4 5
Be sure that
YOUR_BUNDLE_ID matches the
CFBundleIdentifier you told py2app
about before, so that the tool can find your app bundle inside the archive.
You’ll also need to type in the iCloud password for your Developer ID account here.2
Step 6a: Wait A Minute
Anxiously check your email for an hour or so. Hope you don’t get any errors.
Step 6b: Finish Notarizing It, Finally!
Once Apple has a record of the app’s notarization, their tooling will recognize
it, so you don’t need any information from the confirmation email or the
previous command; just make sure that you are running this on the exact same
.app directory you just built and archived and not a version that differs in
Finally, you will want to archive it again:
Step 7: Download it
Ideally, at this point, everything should be working. But to make sure that code-signing and archiving and notarizing and re-archiving went correctly, you should have either a pristine virtual machine with no dev tools and no Python installed, or a non-programmer friend’s machine that can serve the same purpose. They probably need a relatively recent macOS - my own experience has shown that apps made using the above technique will definitely work on High Sierra (and later) and will definitely break on Yosemite (and earlier); they probably start working at some OS version between those.
There’s no tooling that I know of that can clearly tell you whether your mac
app depends on some detail of your local machine.
Even for your
dependencies, there’s no auditwheel for macOS.
Updated 2019-06-27: It turns out there is an
auditwheel like thing for macOS:
fact, it predated and inspired
Thanks to Nathaniel Smith for the update (which he provided in, uh, January of 2018 and I’ve only just now gotten around to updating...).
Nevertheless, it’s always a good idea to check your final app build on a fresh computer before you announce it.
If you were expecting to get to the end and download my cool game, sorry to disappoint! It really is a half-broken prototype that is in no way ready for public consumption, and given my current load of personal and professional responsibilities, you definitely shouldn’t expect anything from me in this area any time soon, or, you know, ever.
But, from years of experience, I know that it’s nearly impossible to summon any motivation to work on small projects like this without the knowledge that the end result will be usable in some way, so I hope that this helps someone else set up their Python game-dev pipeline.
I’d really like to turn this into a 3-part series, with a part for Linux (perhaps using flatpak? is that a good thing?) and a part for Windows. However, given my aforementioned time constraints, I don’t think I’m going to have the time or energy to do that research, so if you’ve got the appropriate knowledge, I’d love to host a guest post on this blog, or even just a link to yours.
If this post helped you, if you have questions or corrections, or if you’d like to write the Linux or Windows version of this post, let me know.
The hardened runtime was originally required when notarization was introduced. Apparently this broke too much software and now the requirement is relaxed until January 2020. But it’s probably best to treat it as if it is required, since the requirement is almost certainly coming back, and may in fact be back by the time you’re reading this. ↩
You can pass it via the
--passwordoption but there are all kinds of security issues with that so I wouldn’t recommend it. ↩