Using poetry, and never going back to virtualenvs
In this edition of 3 act engineering, let’s go back in time back in ’07. Its the year when virtualenv was born.
The Setup
Python has had import
statements which allow it to do something called dynamic linking, which is an alternative to static linking. Below is a brief of what both of them mean:
Static vs Dynamic Linking: Node, Go, Rust and Python

Let’s say you want to import some code which has been written somewhere else by someone else, but is equally useful to you for your current problem at hand. One way to do this would be to copy paste all of it from its current location and paste it alongside yours, calling it static linking. Lot of modern programming languages do it till date, like, Go and Rust.
An alternate to this is dynamic linking, where the libraries are somewhere else on the same machine, but aren’t loaded when the program runs. This gives you an advantage of using the same set of code in multiple programs without ever creating its copies, unless explicitly needed.
Then sometime in ’07, virtualenv
was born. The idea was simple, isolate different versions of same package under different umbrellas, and use whichever is needed at that point. Later, if you want to switch to a different version, just a handful of commands are needed and there you go.
Additionally, you can run multiple versions of the same program on the same system all at the same time, without ever facing dependency conflicts since they all practically live in different floors in the same house.
This is how virtualenv (and some other variants of it including: pyenv , venv) made everyone feel that:

Let’s first discuss why do you need to move from virtualenv.
>Virtualenv is pretty bad when it comes to automatically resolving dependency conflicts.
In simple words, if you have 2 packages A1 and A2 which require different versions of a package B, then you’ll be invited to a battle which will have plenty of red color logs telling you to either upgrade one, downgrade the other or quit your job.
>Separate dependency management for each of your environments
A lot of manual work. Why? have a separate requirements.txt
for each of your repos. The minute you upgrade from one python version to another, the god promises you that you have to do Round 2 to figure out which library suits all the neighbors on that floor.
>No version locks. You rarely need it anyway, but when you do, you do.
Virtualenv has no such mechanism for locking dependencies to specific versions of Python. High chance of inconsistencies across environments.
>Publishing your work? Forget about it
If you want to publish your packages, you’ll need to handle that process separately, which can be more complicated without dedicated support. Virtualenv doesn’t have such a logic built-in out of the box
Alright, alright, alright

The Build
Below would be a speed run on how to setup your first poetry project:
Poetry, the first steps
Using the global python interpreter, install Poetry
python -m pip install poetry
A version check if successful will give you whatever current version you’ve installed for

Let’s create a new project now using Poetry
poetry new my_first_poetry
cd my_first_poetry
tree
This should give you something like:

A few other variants, which give you bit of customization
# use this_is_my_project_folder as project folder
poetry new my_first_poetry --name this_is_my_project_folder
# use src as project folder
poetry new my_first_poetry --src
A new important file created, pyproject.toml
will have the following details:
name of the project
the current version
python dependency declaration
Creating virtual environment in Poetry
With the following command, you can see what virtual environments are currently being maintained in this project folder
poetry env list

Since we haven’t created any, as expected, it didn’t show any. But we can fix that
poetry env use python
And voila!, this time it creates a virtual environment specific to our project and also shows the current path where its installed

The installation path is among one of the properties you should worry about. Thankfully, we can always check it out using:
poetry config --list

Installing dependencies in Poetry
The python version I started out with is 3.8, so trying to install some dependencies got me this error
poetry add pandas numpy requests

As requested by Poetry, a possible solution can be to get a new version of Python (preferably Python 3.10 and above).
Below are some easy steps to switch to a different version
# your python executive path goes here
poetry env use "C:\Python310\python.exe"
Now the interesting part
poetry env list
This gives you 2 virtual environments, and the default one is using Python 3.10

And that solves all my issues from installation, this time the same poetry add
command

The last line is interesting,
“Writing lock file”
A tree
command will show a new file in the folder: poetry.lock
Its a synonym to ‘going down a rabbit hole’.
It contains the following information about each of the packages we’ve installed:
Description of the package
Required python version for it
Is it optional or not?
For different OS, what exact build/wheel file to use
TLDR: To ensure stability and have the exact same setup in every possible environment, this is what gets the job done.
Cloning an existing poetry project
Imagine you clone a git repo which uses poetry as package manager. Below would be the steps to setup the project:
rm poetry.lock
poetry env remove --all # optional: remove existing python environment for this one if any
poetry install
That should be more than enough to get started.
But that’s not all, Poetry has a plethora of tools/commands for various kinds of scenarios. Some of these scenarios (google if you care about it) are:
— Group dependencies in Poetry
— Optional dependencies in Poetry
— Extra dependencies in Poetry
The Payoff
Alright, final round: Let’s migrate an existing project of yours

Before jumping right in, below is my current project, running fast-api

cd migrate_this
poetry init --no-interaction
pip freeze > requirements.txt
deactivate # get out of any virtual environment if inside one
If you’re on Windows, you can run
@(cat requirements.txt) | %{&poetry add $_}
Otherwise, for everywhere else:
poetry add ${cat requirements.txt}
Finally, let’s do bit of cleanup (optional)
rm -f venv
rm -f __pycache__
rm requirements.txt
The project now looks like this:

And can be run as:
poetry run uvicorn main:app --reload
THIS IS THE END
I hope this post helped you know more about my favorite dependency manager as of 2024: Poetry.
A couple of good alternatives one can take a look, and maybe I might too:
- Hatch
- PDM
Outside work, some recent recommendations I’ve spent time enjoying:
Alien: Romulus, The Substance, Berlin & Longlegs
Patriot, Lastman, Blackbird & Monster
Fred again…, Marcin, Clancy & Beta
Till next time,
fin