coding

How to Set Up Poetry for Python: A Step-by-Step Guide to Better Package Management

Sarah Glasmacher

01 May 2024 • 3 min read

I've always been a pip + venv (yes, not even virtualenv) user. Conda never played nice with me. However, there have been some times when I just could not install a requirements .txt from a repository due to it always crashing with some dependency mismatches. Recently, I have heard more and more people mention how they use Poetry for their virtual environments and package management, so I decided to give it a try.

Since I'm mentioning it in a few of my blog posts now, I thought I would quickly write down how I currently use it - specifically I'm developing on macOS, so some instructions might be targeted towards that OS.

Installing poetry

First off, you need Python 3.8+ to use this.

The official poetry documentation then recommends installing it via pipx, which I had admittedly never used before either.

Pipx is installed either through brew on macOS or scoop on Windows, though there are some alternative options, which you can check on the official install page. Here's how that looks on Mac:

brew install pipx
pipx ensurepath
sudo pipx ensurepath --global # optional to allow pipx actions in global scope

Then you can simply install poetry using

pipx install poetry

Creating a new project and virtual environment.

The following creates a new folder in the current directory with the given name and this new folder will be a poetry project:

poetry new llm-langfuse-demo

You can also initiate a poetry project in an already existing directory with

poetry init

In both cases, poetry creates a virtual environment for that project. By default, this environment is saved under {cache-dir}/virtualenvs, with cache-dir=~/Library/Caches/pypoetry as a default on macOS but you can change that.

The virtual environment & interpreter

For many use cases, like adding new packages or running scripts, you don't need to know where the virtual environment files are saved. Poetry automatically manages the environment as we will see below. However, there are a few exceptions:

Setting the poetry environment as Python interpreter in your VSCode project

VSCode checks your code for errors and notifies you with warnings if you don't have specific packages installed that you import in your scripts. To keep this information accurate, you need to tell it which environment and Python interpreter you are currently using.

To fix this, use the following terminal command to copy the path to the interpreter:

poetry env info --path | pbcopy

Then press Cmd + Shift + P to open the prompt for "Python > Set Interpreter", then click "Enter interpreter path..." then paste.

Problem solved 😊 I haven't yet found a way to do this automatically, but at least this is fast.

Source: VSCode: Adding Poetry Python Interpreter | Mark Needham

Installing new packages with poetry

While inside the folder of your project, simply running

poetry add <PyPI package>

adds the package as a dependency to your project (kind of like adding it to the requirements.txt) and also installs the package into the virtual environment of your current directory.

You can check which virtual environment is currently being used by running:

poetry env info

You can also modify the pyproject.toml file (sort of the replacement for requirements.txt) by hand and add your packages there.

Running Scripts

To run a Python script within the created project, use

poetry run python your_script.py

If executed in the project directory created through poetry new ... or poetry init, this will use the underlying virtual environment without you needing to activate it explicitly. One less thing to remember, though it also can feel spooky if you are used to manually activating it.

Activating the virtual environment with poetry manually

Although the above commands already cover most use cases I encounter in my daily coding and deal with the underlying environment automatically, you can also activate the virtual environment explicitly by creating a nested shell with poetry shell.

You deactivate the environment by simply exiting this shell with exit.

This might be useful if you have to run scripts again and again and are tired of typing poetry run or if your IDE automatically detects activated environments.