Virtual Environments and Dependency Management in Python: From venv to Poetry

Anyone who has spent more than a few days with Python has likely run into library version issues. A classic scenario: one project requires an older version of the Django framework, while a new one needs the latest features. Installing everything globally in the operating system is the easiest path to what is known as dependency hell.

In this article, we will look at how to professionally approach environment isolation. We will go from the traditional, built-in `venv` tool to the modern standard that `Poetry` is becoming, analyzing their pros, cons, and use cases in daily developer work and CI/CD systems.

Why is global installation a mistake?

Most operating systems (especially Linux and macOS) rely on Python to run internal system scripts. Overwriting a system library version using the `sudo pip install` command can break critical system tools. What's more, a lack of isolation makes it impossible to work on multiple projects simultaneously if they require different versions of the same libraries.

Virtual environments are the solution to this problem. They create an isolated space (directory) for each project, with its own Python interpreter and an independent tree of installed packages. This ensures that changes in one project do not affect others.

The classic duo: venv and pip

The Python standard library offers the `venv` module, which has been the default tool for creating virtual environments since version 3.3. Combined with the `pip` package manager, it forms the foundation of most older and simpler projects.

# Create a virtual environment named .venv
python3 -m venv .venv

# Activate the environment (Linux/macOS)
source .venv/bin/activate

# Activate the environment (Windows - PowerShell)
.venv\Scripts\Activate.ps1

Once the environment is activated, every `pip install` call installs packages inside the `.venv` directory. The standard for saving dependencies in this approach is the `requirements.txt` file. It is generated using the `pip freeze > requirements.txt` command and recreated on another machine via `pip install -r requirements.txt`.

Where does the classic venv + pip duo start to fail?

While `venv` and `pip` are great to start with, they quickly reveal their limitations in professional team projects:

• Lack of determinism: The `requirements.txt` file often contains loosely specified versions (e.g., `requests>=2.25.0`), meaning that installation on another machine might fetch a newer, potentially breaking version of a sub-dependency.
• No environment separation: There is no simple way to separate libraries needed only for testing and development (e.g., `pytest`, `black`) from those required in production. This leads to creating artificial files like `requirements-dev.txt`.
• Weak conflict resolution: For years, `pip` installed packages "blindly", sometimes overwriting another library's dependency without warning.

A new era: Poetry as a modern standard

In response to these pain points, the Poetry tool was created. It combines virtual environment management, a package manager, an application build system, and publishing capabilities to repositories (e.g., PyPI). Poetry is based on the PEP 518 standard and the `pyproject.toml` file, which replaces not only `requirements.txt` but also configuration files for other tools.

How does Poetry work under the hood?

The key to Poetry's success is splitting dependencies into two files:

1. `pyproject.toml` – contains direct dependency declarations defined by the developer (e.g., "I need FastAPI version ^0.95.0").
2. `poetry.lock` – an automatically generated file containing the exact, locked versions of all installed packages along with their checksums (hashes). This ensures that every developer on the team and the CI/CD server installs the exact same set of bits.

# Initialize a new project with Poetry
poetry new moj-projekt
cd moj-projekt

# Add a production dependency
poetry add fastapi

# Add a development dependency (for testing only)
poetry add pytest --group dev

# Install the environment based on the lock file (e.g., on CI/CD)
poetry install --no-root

Comparison: venv + pip vs Poetry

The table below summarizes the key differences between the traditional approach and the modern standard, Poetry.

Best practices in teamwork

Implementing the right tools is only half the battle. Equally important is the culture of working with dependencies in a team and in CI/CD environments:

1. Always commit the lock file: The `poetry.lock` file MUST be committed to the Git repository. Without this, teamwork loses the guarantee of environment consistency.
2. Leverage caching in CI/CD: Installing packages from scratch on every pipeline run drastically increases build times. Cache package directories (e.g., `~/.cache/pypoetry`) based on the lock file hash.
3. Regular updates: Use tools like Dependabot, which automatically open Pull Requests with dependency updates and test whether they break the application.

Summary

The choice between `venv` and `Poetry` depends on the scale of the project. For small, one-off scripts, `venv` is perfectly sufficient and does not require installing additional tools. However, for commercial projects, teamwork, and building CI/CD pipelines, Poetry is an absolute standard that saves time and prevents production errors.