How to use Rust with Python, and Python with Rust

Python and Rust occupy seemingly opposite ends of the language spectrum. Python, interpreted at runtime, offers developers a flexible and comfortable programming environment, but at the cost of raw speed. Rust provides the speed, plus guarantees of memory safety, but requires that you learn a new paradigm for handling memory operations.

In theory, these languages shouldn’t compete; they should cooperate. And in practice, they can. Rust can benefit from Python’s ease of use, and Python can benefit from Rust’s speed and safety.

If you want to use Rust with Python, or Python with Rust, you’ll need to have at least passing familiarity with both languages to get the best results. You’ll also need to decide which of the two is your primary language, as the options for each approach are significantly different.

Calling Rust from Python with PyO3

If Python is your primary language, integrating with Rust works in conceptually the same way as integrating Python with C. The default implementation of Python, written in C, uses extensions either written in C or using a C-compatible ABI. Extensions written in Rust that use the same ABI will also work, although that isn’t automatic—you have to use crates designed to provide bindings for Rust functions to the Python C API.

Creating Rust bindings in Python

The most widely recognized project for creating Rust bindings in Python is PyO3. It can be used to write Python modules in Rust, or to embed the Python runtime in a Rust binary.

PyO3 leverages another project, Maturin, which is a tool for authoring Rust crates with Python packaging and bindings. When installed in a Python virtual environment, Maturin can be used from the command line to initialize a new Rust project with Python bindings enabled. The developer uses directives in the Rust code to indicate what Rust functions to expose to Python, and how to expose the whole of the Rust project to Python as an importable module.

Mapping Rust and Python types

One of PyO3’s useful aspects is its mappings between Rust and Python types. Functions written in Rust can accept either native Python types or Rust types converted from Python types. For instance, a bytearray or bytes object in Python maps elegantly to a Vec<u8> in Rust, and a str in Python can be rendered as a Rust String.

More complex types, like a Python dictionary or an integer that is too big for a machine-native integer, also have Rust conversions, but some require optional components. For instance, if you want to use Python integers of arbitrary size, you’d install the num-bigint optional feature in PyO3, and have those integers expressed in Rust as num_bigint::BigInt or num_bigint::BigUint types.

Converting from Python to Rust incurs a per-call cost, but it frees you from having to use Python types entirely in the Rust code. In the Cython world, this is akin to the conversions to C types: there’s a cost for each conversion, but they bring major speedups if your goal is numerical processing entirely in C.

Calling Python from Rust 

If you’re primarily a Rust developer but want to use Python inside a Rust application, PyO3 also has mechanisms for calling Python from within Rust code.

Python code can be defined in-line in a Rust program, compiled to Python bytecode, and executed using Rust calls. Rust structures like HashMap or BTreeMap can be used to pass positional or keyword argument lists. You can even just evaluate single expressions if that’s all you need.

Rust programs can invoke the CPython interpreter and work with it, allowing you to create and manipulate Python objects in Rust and make library calls. Existing Python code files can also be loaded in and used, but it’s a potential security hazard, so don’t load untrusted code or use this option in a situation where you could potentially do so.

If you’re more familiar with Rust than Python, it’s a good idea to have at least passing familiarity with the Python C API and the various Python object types before diving in.

A previous project, the cpython crate, also allowed Python inside a Rust application, but it’s no longer actively maintained and its developers recommend using PyO3 instead.

Performance tip

An important caveat with PyO3 is to always minimize the number of times data is passed back and forth between the two languages. Each call from Python to Rust or vice versa incurs some overhead. If the overhead outweighs the work you’re doing in Rust, you won’t see any significant performance improvement.

As an example, if you’re looping over an object collection, send the object to Rust and perform the looping there. This is more efficient than looping on the Python side and calling the Rust code with each iteration of the loop.

This guideline also applies generally to integrations between Python and other code that uses the Python C ABI, such as Cython modules.