There have been many updates to Atlassian Bitbucket since I wrote the article CICD to PyPI with Bitbucket Pipelines. Because of these updates, I’ve decided to re-publish my Pipelines configuration along with how I’m now using Pipelines to test and automatically publish my Python Modules and Packages to PyPi.
Here is my Bitbucket Pipeline configuration for an example project:
image: python:latest
pipelines:
default:
- step: &py37
name: Python 3.7 Tests
image: python:3.7
script:
- python -m pip install -U -r requirements.txt
- coverage run -m unittest discover tests/
caches:
- pip
- step: &py38
name: Python 3.8 Tests
image: python:3.8
script:
- python -m pip install -U -r requirements.txt
- coverage run -m unittest discover tests/
caches:
- pip
- step: &py39
name: Python 3.9 Tests
image: python:3.9
script:
- python -m pip install -U -r requirements.txt
- coverage run -m unittest discover tests/
caches:
- pip
- step: &py310
name: Python 3.10 Tests
image: python:3.10
script:
- python -m pip install -U -r requirements.txt
- coverage run -m unittest discover tests/
caches:
- pip
- step: &pypy3
name: Pypy 3 Tests
image: pypy:3
script:
- python -m pip install -U -r requirements.txt
- coverage run -m unittest discover tests/
caches:
- pip
branches:
release/*:
- step: *py37
- step: *py38
- step: *py39
- step: *py310
- step: *pypy3
- step:
name: Build Documentation
script:
- python -m pip install Sphinx
- sphinx-build -b html docs/source/ docs/build/html/
caches:
- pip
- step:
name: Deploy To Test PyPi
trigger: manual
script:
- pip install build twine
- python -m build
- twine upload --repository testpypi --non-interactive --username __token__ --password ${TEST_PYPI_PASSWORD} dist/*;
master:
- step: *py37
- step: *py38
- step: *py39
- step: *py310
- step: *pypy3
- step:
name: Deploy To PyPi
script:
- pip install build twine
- python -m build
- twine upload --non-interactive --username __token__ --password ${PYPI_PASSWORD} dist/*;
This configuration runs tests for Python3.7 and later on all branches. The only difference on master and any release/*
branch is the additional step to actually do the deploy to PyPI. Of course, the release/* branches are also doing a test
publish to the testpypi instance.
You might have noticed that I’ve moved away from running tox for my unit tests in favor
of simply running python -m unittest discover tests/. This is because tox is overkill in this situation where Pipelines
is running each set of unit tests in a specific Docker Container running a specific version of Python.
I have also moved away from my custom deploy.bash script in favor of the twine module.
The reason for this is twofold. First, using a supported module that already allows for token-based authentication to PyPi
is preferable over using a simple bash script that could easily be hacked to publish secrets. Second, it simplifies
the Pipeline and allows easy publishing to the testpypi repo on pre-release branches.
Keep in mind, you must configure token-based authentication as well as create a secure environment variable to store it in so that not just anyone can read the token used to authenticate with PyPi. Once you’ve done this, the above “recipe” will work and your secure token will stay secure within Bitbucket Pipelines logs.
And that is it. That is the update! Hope you enjoy using Bitbucket Pipelines to make your testing and publishing of Python Modules and Packages easier and automated!