I’m pleased to announce the initial release of PyOxy. Binaries are available on GitHub.

(Yes, I used my pure Rust Apple code signing implementation to remotely sign the macOS binaries from GitHub Actions using a YubiKey plugged into my Windows desktop: that experience still feels magical to me.)

PyOxy is all of the following:

• An executable program used for running Python interpreters.
• A single file and highly portable (C)Python distribution.
• An alternative python driver providing more control over the interpreter than what python itself provides.
• A way to make some of PyOxidizer’s technology more broadly available without using PyOxidizer. Read the following sections for more details.

pyoxy Acts Like python

The pyoxy executable has a run-python sub-command that will essentially do what python would do:

$ pyoxy run-python
Python 3.9.12 (main, May  3 2022, 03:29:54)
[Clang 14.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

A Python REPL. That’s familiar!

You can even pass python arguments to it:

$ pyoxy run-python -- -c 'print("hello, world")'
hello, world

When a pyoxy executable is renamed to any filename beginning with python, it implicitly behaves like pyoxy run-python --.

$ mv pyoxy python3.9
$ ls -al python3.9
-rwxrwxr-x  1 gps gps 120868856 May 10  2022 python3.9

$ ./python3.9
Python 3.9.12 (main, May  3 2022, 03:29:54)
[Clang 14.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Single File Python Distributions

The official pyoxy executables are built with PyOxidizer and leverage the Python distributions provided by my python-build-standalone project. On Linux and macOS, a fully featured Python interpreter and its library dependencies are statically linked into pyoxy. The pyoxy executable also embeds a copy of the Python standard library and imports it from memory using the oxidized_importer Python extension module.

What this all means is that the official pyoxy executables can function as single file CPython distributions! Just download a pyoxy executable, rename it to python, python3, python3.9, etc and it should behave just like a normal python would!

Your Python installation has never been so simple. And fast: pyoxy should be a few milliseconds faster to initialize a Python interpreter mostly because of oxidized_importer and it avoiding filesystem overhead to look for and load .py[c] files.

Low-Level Control Over the Python Interpreter with YAML

The pyoxy run-yaml command is takes the path to a YAML file defining the embedded Python interpreter configuration and then launches that Python interpreter in-process:

$ cat > hello_world.yaml <<EOF
---
allocator_debug: true
interpreter_config:
run_command: 'print("hello, world")'
...
EOF

$ pyoxy run-yaml hello_world.yaml
hello, world

Under the hood, PyOxy uses the pyembed Rust crate to manage embedded Python interpreters. The YAML document that PyOxy uses is simply deserialized into a pyembed::OxidizedPythonInterpreterConfig Rust struct, which pyembed uses to spawn a Python interpreter. This Rust struct offers near complete control over how the embedded Python interpreter behaves: it even allows you to tweak settings that are impossible to change from environment variables or python command arguments! (Beware: this power means you can easily cause the interpreter to crash if you feed it a bad configuration!)

YAML Based Python Applications

pyoxy run-yaml ignores all file content before the YAML --- start document delimiter. This means that on UNIX-like platforms you can create executable YAML files defining your Python application. e.g.

$ mkdir -p myapp
$ cat > myapp/__main__.py << EOF
print("hello from myapp")
EOF

$ cat > say_hello <<"EOF"
#!/bin/sh
"exec" "`dirname $0`/pyoxy" run-yaml "$0" -- "$@"
---
interpreter_config:
run_module: 'myapp'
module_search_paths: ["$ORIGIN"]
...
EOF

$ chmod +x say_hello

$ ./say_hello
hello from myapp

This means that to distribute a Python application, you can drop a copy of pyoxy in a directory then define an executable YAML file masquerading as a shell script and you can run Python code with as little as two files!

The Future of PyOxy

PyOxy is very young. I hacked it together on a weekend in September 2021. I wanted to shore up some functionality before releasing it then. But I got perpetually sidetracked and never did the work. I figured it would be better to make a smaller splash with a lesser-baked product now than wait even longer. Anyway...

As part of building PyOxidizer I’ve built some peripheral technology:

• Standalone and highly distributable Python builds via the python-build-standalone project.
• The pyembed Rust crate for managing an embedded Python interpreter.
• The oxidized_importer Python package/extension for importing modules from memory, among other things.
• The Python packed resources data format for representing a collection of Python modules and resource files for efficient loading (by oxidized_importer). I conceived PyOxy as a vehicle to enable people to leverage PyOxidizer’s technology without imposing PyOxidizer onto them. I feel that PyOxidizer’s broader technology is generally useful and too valuable to be gated behind using PyOxidizer.

PyOxy is only officially released for Linux and macOS for the moment. It definitely builds on Windows. However, I want to improve the single file executable experience before officially releasing PyOxy on Windows. This requires an extensive overhaul to oxidized_importer and the way it serializes Python resources to be loaded from memory.

I’d like to add a sub-command to produce a Python packed resources payload. With this, you could bundle/distribute a Python application as pyoxy plus a file containing your application’s packed resources alongside YAML configuring the Python interpreter. Think of this as a more modern and faster version of the venerable zipapp approach. This would enable PyOxy to satisfy packaging scenarios provided by tools like Shiv, PEX, and XAR. However, unlike Shiv and PEX, pyoxy also provides an embedded Python interpreter, so applications are much more portable since there isn’t reliance on the host machine having a Python interpreter installed.

I’m really keen to see how others want to use pyoxy.

The YAML based control over the Python interpreter could be super useful for testing, benchmarking, and general Python interpreter configuration experimentation. It essentially opens the door to things previously only possible if you wrote code interfacing with Python’s C APIs.

I can also envision tools that hide the existence of Python wanting to leverage the single file Python distribution property of pyoxy. For example, tools like Ansible could copy pyoxy to a remote machine to provide a well-defined Python execution environment without having to rely on what packages are installed. Or pyoxy could be copied into a container or other sandboxed/minimal environment to provide a Python interpreter.

And that’s PyOxy. I hope you find it useful. Please file any bug reports or feature requests in PyOxidizer’s issue tracker.