No regrets with Poetry!

Samuel Roeca, Head of Engineering

Published September 15

In late 2019, I presented our team's favorite Python "Dependency Management" and "Packaging" tool (Poetry) to an audience at PyGotham, an annual NYC-based Python conference. Before discovering Poetry, I had struggled with Python's convoluted dependency management and packaging ecosystem for years, balancing distutils, setuptools, pip, twine, and other occult mechanisms by which Python projects were assembled, built, packaged, distributed. In late 2018, on the packaging front, our team had begun discussing an ambitious internal goal to move some shared logic (like failure alerts, etc) to libraries that could be hosted on an internally-managed private Python Package Index and version controlled. On the dependency management front, after a kerfluffle involving breaking changes with itsdangerous, our team realized with urgency that we needed to prevent the dependencies of our dependencies from creating hard-to-debug problems. With our "Dependency Management Taming" and "Library Packaging" goals in mind, we actively considered two early-stage tools in the Python ecosystem: Pipenv and Poetry.

On the surface, Pipenv and Poetry seemed pretty similar in 2018. Both claimed to be related to packaging and dependency management, and the major differences at that time were that Pipenv had obtained an official recommendation from the Python packaging authority and was already hosted by the Python Packaging Authority organization on GitHub. Beyond those advantages, in the first lines of its README, Pipenv compared itself directly to cargo, yarn, npm, and other packaging and dependency management tools that our polyglot-programming team was already familiar with.

Given all of Pipenv's advantages at the time, we gave it the old college try. After some initial tests, we observed that Pipenv had a several serious disadvantages (many outlined in an interesting-and-contentious discussion here):

• It was not, in fact, a packaging tool like cargo, yarn, and npm. Dependency management, yes, packing, no.
• It was very slow.
• It was buggy.
• Its cli felt inconsistent with other packaging tools we were used to (like cargo).

I hope that by December 2021, many of the above issues have been resolved, but in 2018 our engineering team concluded that Pipenv's most important disadvantage was the fact that it wasn't what it claimed to be. "Packaging" and "Dependency Management" are different concepts and Pipenv appears to have conflated those terms in the first line of its README. Given Pipenv's unclear purpose and its general slowness, we decided to give Poetry -- then a relatively new tool whose first commit was on February 20, 2018 -- a try. As of December 2021, we use Poetry to manage our Python project dependencies, build local libraries / packages, and deploy these libraries to our private package index. After several years of testing, I'm happy to report that we have no regrets with Poetry!