Python packaging has evolved significantly, with modern tools and standards making it easier than ever to distribute your code.

This guide walks you through building professional Python packages and publishing them to PyPI. If you’re new to Python or need a quick reference, check out our Python Cheatsheet to get up to speed with Python fundamentals.

Why Package Your Python Code?

Packaging your Python project offers multiple benefits:

• Reusability: Share code across multiple projects without copy-pasting
• Distribution: Enable others to install your code with a simple pip install
• Dependency Management: Clearly specify and manage dependencies
• Versioning: Track releases and maintain backwards compatibility
• Professional Standards: Follow Python community best practices
• Documentation: Structure encourages proper documentation and testing

Modern Python Package Structure

A well-organized package follows this structure:

my-package/
├── src/
│   └── mypackage/
│       ├── __init__.py
│       ├── core.py
│       └── utils.py
├── tests/
│   ├── __init__.py
│   └── test_core.py
├── docs/
│   └── index.md
├── .github/
│   └── workflows/
│       └── publish.yml
├── pyproject.toml
├── README.md
├── LICENSE
└── .gitignore

The src-layout is now preferred over the flat layout because it:

• Prevents accidental imports of uninstalled code during development
• Makes testing more reliable by forcing installation
• Clearly separates source code from other project files

When organizing your package’s internal structure, consider applying clean architecture principles to make your code more maintainable and testable. Our guide on Python Design Patterns for Clean Architecture covers SOLID principles, dependency injection, and layered architecture patterns that work excellently with Python packages.

The pyproject.toml File: Modern Configuration

pyproject.toml (PEP 518, 621) is the modern standard for Python project configuration, replacing the legacy setup.py:

[build-system]
requires = ["setuptools>=68.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "mypackage"
version = "0.1.0"
description = "A fantastic Python package"
readme = "README.md"
requires-python = ">=3.8"
license = {text = "MIT"}
authors = [
{name = "Your Name", email = "you@example.com"}
]
keywords = ["example", "tutorial", "package"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
dependencies = [
"requests>=2.28.0",
"click>=8.1.0",
]

[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"black>=23.0.0",
"flake8>=6.0.0",
"mypy>=1.5.0",
]
docs = [
"sphinx>=7.0.0",
"sphinx-rtd-theme>=1.3.0",
]

[project.urls]
Homepage = "https://github.com/yourusername/mypackage"
Documentation = "https://mypackage.readthedocs.io"
Repository = "https://github.com/yourusername/mypackage"
Issues = "https://github.com/yourusername/mypackage/issues"

[project.scripts]
mypackage-cli = "mypackage.cli:main"

[tool.setuptools.packages.find]
where = ["src"]

Key Configuration Sections

1. build-system: Specifies build backend (setuptools, hatchling, poetry-core, flit-core)
2. project: Core metadata and dependencies
3. project.optional-dependencies: Extra feature dependencies (dev, docs, testing)
4. project.scripts: Command-line entry points
5. tool.*: Tool-specific configuration (black, pytest, mypy, etc.)

Choosing a Build Backend

Setuptools (Standard Choice)

The most widely used and compatible option:

[build-system]
requires = ["setuptools>=68.0", "wheel"]
build-backend = "setuptools.build_meta"

Pros: Universal compatibility, extensive features, large ecosystem Cons: More verbose configuration, slower than newer alternatives

Hatchling (Modern & Fast)

Lightweight and performant modern backend:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Pros: Fast builds, simple config, good defaults Cons: Fewer plugins than setuptools

Poetry (All-in-One)

Complete dependency and environment management:

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "mypackage"
version = "0.1.0"
description = "A fantastic Python package"
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.28.0"

Pros: Dependency resolution, lock files, integrated workflow Cons: Different workflow, larger tool surface area

Flit (Minimalist)

Simple tool for simple packages:

[build-system]
requires = ["flit_core >=3.2,<4"]
build-backend = "flit_core.buildapi"

Pros: Extremely simple, minimal configuration Cons: Limited features for complex packages

Building Your Package

Install Build Tools

# Install the modern build tool
pip install build

# Or for Poetry
pip install poetry

# Or for Hatchling
pip install hatch

Build Distribution Files

Using the build package (works with any backend):

# Clean previous builds
rm -rf dist/ build/ *.egg-info

# Build source distribution and wheel
python -m build

# This creates:
# dist/mypackage-0.1.0.tar.gz    (source distribution)
# dist/mypackage-0.1.0-py3-none-any.whl  (wheel)

Using Poetry:

poetry build

Using Hatch:

hatch build

Understanding Distribution Formats

Source Distribution (sdist) - .tar.gz

• Contains source code and build instructions
• Users’ pip builds it during installation
• Includes tests, docs, and other development files

Wheel - .whl

• Pre-built binary distribution
• Fast installation (no build step)
• Platform-specific or pure Python
• Recommended format for distribution

Publishing to PyPI

Method 1: Manual Upload with Twine (Traditional)

# Install twine
pip install twine

# Check package before uploading
twine check dist/*

# Upload to TestPyPI first (recommended)
twine upload --repository testpypi dist/*

# Test installation from TestPyPI
pip install --index-url https://test.pypi.org/simple/ mypackage

# Upload to production PyPI
twine upload dist/*

Configure credentials in ~/.pypirc:

[distutils]
index-servers =
pypi
testpypi

[pypi]
username = __token__
password = pypi-AgEIcHlwaS5vcmc...

[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = pypi-AgENdGVzdC5weXBpLm9yZw...

Method 2: Trusted Publishing (Recommended for CI/CD)

Trusted Publishing uses OpenID Connect (OIDC) to authenticate from CI/CD platforms without storing tokens.

Setup in PyPI:

1. Go to your PyPI project settings
2. Navigate to “Publishing” section
3. Add a new “pending publisher”
4. Configure:

• Owner: Your GitHub username/organization
• Repository: Repository name
• Workflow: publish.yml
• Environment: release (optional but recommended)

GitHub Actions Workflow (.github/workflows/publish.yml):

name: Publish to PyPI

on:
release:
types: [published]

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install build

- name: Build package
run: python -m build

- name: Store distribution packages
uses: actions/upload-artifact@v3
with:
name: python-package-distributions
path: dist/

publish:
needs: build
runs-on: ubuntu-latest
environment: release
permissions:
id-token: write

steps:
- name: Download distributions
uses: actions/download-artifact@v3
with:
name: python-package-distributions
path: dist/

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

Advantages:

• No API tokens to manage or secure
• Automatic authentication via OIDC
• Enhanced security through environment protection rules
• Audit trail of all publications

Best Practices

1. Version Management

Use semantic versioning (SemVer): MAJOR.MINOR.PATCH

# Install version bumping tool
pip install bump2version

# Bump version
bump2version patch  # 0.1.0 -> 0.1.1
bump2version minor  # 0.1.1 -> 0.2.0
bump2version major  # 0.2.0 -> 1.0.0

Configure in .bumpversion.cfg:

[bumpversion]
current_version = 0.1.0
commit = True
tag = True

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:src/mypackage/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"

2. Include Essential Files

README.md: Clear project description, installation, usage examples LICENSE: Choose appropriate license (MIT, Apache 2.0, GPL, etc.) CHANGELOG.md: Document changes between versions .gitignore: Exclude build artifacts, caches, virtual environments

3. Comprehensive Testing

# Install testing dependencies
pip install pytest pytest-cov

# Run tests with coverage
pytest --cov=mypackage tests/

# Generate coverage report
pytest --cov=mypackage --cov-report=html tests/

Configure in pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
addopts = "--strict-markers --cov=mypackage"

4. Code Quality Tools

# Formatting
black src/ tests/

# Linting
flake8 src/ tests/

# Type checking
mypy src/

# Import sorting
isort src/ tests/

Integrate into pre-commit hooks (.pre-commit-config.yaml):

repos:
- repo: https://github.com/psf/black
rev: 23.9.1
hooks:
- id: black

- repo: https://github.com/pycqa/flake8
rev: 6.1.0
hooks:
- id: flake8

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.5.1
hooks:
- id: mypy

5. Documentation

Use Sphinx for documentation:

# Install Sphinx
pip install sphinx sphinx-rtd-theme

# Initialize docs
cd docs
sphinx-quickstart

# Build documentation
make html

Or use MkDocs for simpler Markdown-based docs:

pip install mkdocs mkdocs-material
mkdocs new .
mkdocs serve

6. Continuous Integration

Testing your Python packages across different platforms and environments is crucial for reliability. For insights on Python performance in different deployment scenarios, see our comparison of AWS Lambda performance: JavaScript vs Python vs Golang, which explores how Python performs in serverless environments.

Complete CI/CD workflow (.github/workflows/ci.yml):

name: CI

on:
push:
branches: [main, develop]
pull_request:
branches: [main]

jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"

- name: Run tests
run: pytest --cov=mypackage --cov-report=xml

- name: Upload coverage
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml

lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install dependencies
run: |
pip install black flake8 mypy

- name: Black formatting check
run: black --check src/ tests/

- name: Flake8
run: flake8 src/ tests/

- name: MyPy
run: mypy src/

Common Pitfalls and Solutions

Issue 1: Import Errors After Installation

Problem: Package installed but imports fail

Solution: Ensure proper package structure with __init__.py files and correct pyproject.toml configuration:

[tool.setuptools.packages.find]
where = ["src"]
include = ["mypackage*"]

Issue 2: Missing Dependencies

Problem: Package installs but fails at runtime due to missing dependencies

Solution: Declare all runtime dependencies in pyproject.toml:

[project]
dependencies = [
"requests>=2.28.0",
"click>=8.1.0",
]

Issue 3: Version Conflicts

Problem: Package works in development but fails in production

Solution: Use virtual environments and specify minimum versions:

# Create isolated environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install in editable mode for development
pip install -e ".[dev]"

Issue 4: Large Package Size

Problem: Package takes too long to download/install

Solution: Exclude unnecessary files using MANIFEST.in:

include README.md
include LICENSE
include pyproject.toml
recursive-include src *.py
recursive-exclude * __pycache__
recursive-exclude * *.py[co]
recursive-exclude tests *
recursive-exclude docs *

Issue 5: Platform-Specific Issues

Problem: Package works on one OS but fails on another

Solution: Test on multiple platforms using CI/CD matrix builds (see CI example above)

Publishing Checklist

Before publishing your package to PyPI, verify:

• [ ] All tests pass across Python versions
• [ ] Code is formatted and linted
• [ ] README.md is complete with examples
• [ ] LICENSE file is included
• [ ] Version number is updated
• [ ] CHANGELOG.md is updated
• [ ] Dependencies are correctly specified
• [ ] Package builds without errors (python -m build)
• [ ] Package tested on TestPyPI
• [ ] Documentation is up to date
• [ ] Git repository is tagged with version
• [ ] GitHub release is created (for trusted publishing)

Advanced Topics

Entry Points and CLI Tools

Create command-line interfaces:

[project.scripts]
mypackage = "mypackage.cli:main"

Implementation in src/mypackage/cli.py:

import click

@click.command()
@click.option('--name', default='World', help='Name to greet')
def main(name):
"""Simple CLI tool example"""
click.echo(f'Hello, {name}!')

if __name__ == '__main__':
main()

For a real-world example of creating Python packages that interact with external APIs, check out our guide on Integrating Ollama with Python, which demonstrates building Python clients with both REST API and official library integrations.

Plugin Systems

Enable plugin discovery:

[project.entry-points."mypackage.plugins"]
plugin_a = "mypackage_plugin_a:PluginA"

C Extensions

For performance-critical code:

[tool.setuptools.ext-modules]
name = "mypackage._speedups"
sources = ["src/mypackage/_speedups.c"]

Namespace Packages

For distributed packages under a common namespace:

[tool.setuptools.packages.find]
where = ["src"]
include = ["company.*"]

[tool.setuptools.package-data]
"company.mypackage" = ["py.typed"]

Useful Tools and Resources

Package Development Tools

• cookiecutter: Project templates for Python packages
• tox: Test automation across Python versions
• nox: Flexible test automation (tox alternative)
• pre-commit: Git hook framework for code quality
• commitizen: Standardize commit messages and versioning

Documentation Platforms

• Read the Docs: Free documentation hosting
• GitHub Pages: Host MkDocs or Sphinx documentation
• pdoc: Simple API documentation generator

Package Registries

• PyPI: The official Python Package Index (pypi.org)
• TestPyPI: Testing environment (test.pypi.org)
• Anaconda.org: Conda package distribution
• GitHub Packages: Private package hosting

Monitoring and Analytics

• PyPI Stats: Download statistics for packages
• Libraries.io: Dependency monitoring and alerts
• Snyk: Security vulnerability scanning
• Dependabot: Automated dependency updates

Conclusion

Modern Python packaging has evolved to be developer-friendly with standards like pyproject.toml, powerful build backends, and secure publishing via Trusted Publishing. Following this guide, you can create professional, maintainable Python packages that serve your users effectively.

Key takeaways:

1. Use pyproject.toml for all new projects
2. Choose the right build backend for your needs
3. Implement automated testing and CI/CD
4. Use Trusted Publishing for secure, token-free deployment
5. Follow semantic versioning and maintain changelogs
6. Provide comprehensive documentation and examples

The Python packaging ecosystem continues to improve with better tooling, standards, and security features. Stay updated with Python Enhancement Proposals (PEPs) and community best practices to keep your packages modern and maintainable.

Useful Links

• Python Packaging User Guide
• PEP 518 - Specifying Build System Requirements
• PEP 621 - Storing project metadata in pyproject.toml
• PyPI Trusted Publishing Documentation
• Setuptools Documentation
• Poetry Documentation
• Hatch Documentation
• TestPyPI
• Python Packaging Authority GitHub
• Semantic Versioning
• Choose a License

Other Useful Articles on this Site

• Python Cheatsheet
• Python Design Patterns for Clean Architecture
• Integrating Ollama with Python: REST API and Python Client Examples
• AWS lambda performance: JavaScript vs Python vs Golang