Interacting with Python
ScalaPy offers a variety of ways to interact with the Python interpreter, enabling you to calculate any Python expression from Scala code.
The primary entrypoint into the Python interpreter from ScalaPy is
py.global, which acts similarly to Scala.js's
js.Dynamic.global to provide a dynamically-typed interface for the interpreter's global scope. With
py.global, you can call any global method and access any global value.
For example, we can create a Python range with the
range() method, and calculate the sum of its elements with
If you're working with a Python library, you'll likely need to import some modules. You can do this in ScalaPy with the
py.module method. This method returns an object representing the imported module, which can be used just like
py.global but with the contents referencing the module instead of the global scope.
For example we can import NumPy, a popular package for scientific computing with Python.
In the previous example, you'll notice that we passed in a Scala
np.array, which usually takes a Python list. When using Python APIs, ScalaPy will automatically convert scalar Scala values into their Python equivalents (through the
Writer type). This handles the integer values, but not sequences, which have multiple options for conversion in ScalaPy.
If you'd like to create a copy of the sequence, which can be accessed by Python code with high performance but miss any mutations you later make in Scala code, you can use
If you'd like to create a proxy of the sequence instead, which uses less memory and can observe mutations but comes with a larger overhead for repeated access from Python, you can use
To convert Python values back into their Scala equivalents, ScalaPy comes with the
.as API to automatically perform conversions for supported types (those that have a
Reader implementation). Unlike writing, where there were multiple options for converting sequence types, there is a single
.as API for converting back. If you load a collection into an immutable Scala sequence type, it will be loaded as a copy. If you load it as a
mutable.Seq, however, it will be loaded as a proxy and can observe underlying changes
#Custom Python Snippets
Sometimes, you might run into a situation where you need to express a Python construct that can't be done through an existing ScalaPy API. For this situation and to make converting Python code easier, ScalaPy provides an escape hatch via the
py"" string interpolator. This lets you run arbitrary strings as Python code with the additional power of being able to interpolate in Scala values.
For example, we might want to use Python
map which takes a
lambda. Instead, we can use the
py"" interpolator to write the expression as a piece of Python code.
If you need to run arbitrary strings of Python that are dynamically generated, you can use
#Special Python Syntax
ScalaPy includes APIs to make it possible to use Python features that require special syntax from Scala.
Python includes a "try-with-resources" feature in the form of the
with keyword. In ScalaPy, you can use this feature by calling
py.with with the value you want to open and a curried function using that value. For example, we can open a file with the following code:
To index into a sequence-like Python value,
py.Dynamic offers the
bracketDelete APIs to load, set, or delete a value through an indexing operation. For example, we could update values of a Python list:
We can also delete elements of a Python dictionary:
On supported objects, you can also delete an attribute with the
Some Python APIs require you to explicitly delete a reference to a value with the
del keyword. In ScalaPy, you can perform the equivalent operation by calling
del on a Python value.
There are two key points to note when using this API. First, although the Python value is still available in Scala, any attempts to access it will result in an exception since the value has been released. Second, if there are multiple references to a single Python value from your Scala code,
del will only delete a single reference and the underlying value will not be freed since other Scala code still holds a reference to it.
#Zoned Memory Management
By default, ScalaPy uses the JVM's garbage collector to determine when to free memory on the Python side. However, if your code allocates many Python values and immediately releases them, your application may use more memory than needed since the JVM lazily performs garbage collection. In addition, Scala Native currently does not support the necessary APIs to automatically release memory on the Python side. In these situations, you can use the
py.local API to mark a chunk of your application where all new Python values will be freed at the end of the block.
For example, we can use
py.local to allocate a large number of Python values and then immediately release them:
In this code, we can manipulate the values allocated inside the
py.local block as long as we are still in its scope. If we try to access the values after the block ends, we will get an exception.