Why Project Structure Matters

A well-organized Python project is easier to understand, test, maintain, and scale. Whether you're building a small script or a production application, establishing a clear structure from the start saves significant headaches down the road — especially when working in a team or returning to a project months later.

The Recommended Layout

Here's a widely accepted structure for a Python package or application:

my_project/
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── models.py
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── test_utils.py
├── docs/
├── .env.example
├── .gitignore
├── pyproject.toml
├── README.md
└── requirements.txt

Key Directories and Files Explained

src/ Layout

The src/ layout keeps your package code isolated from project-level files. This prevents accidentally importing the local package instead of an installed version during testing — a subtle but real bug. It's the recommended approach for packages you intend to distribute.

tests/

Keep all tests in a dedicated tests/ directory, mirroring the structure of your source package. Every test file should be named test_*.py so testing tools like pytest discover them automatically.

pyproject.toml

The modern standard for Python project configuration. It replaces setup.py and setup.cfg for defining package metadata, build systems, and tool configurations (like Black, mypy, pytest):

[project]
name = "my_package"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["requests", "pydantic"]

[tool.pytest.ini_options]
testpaths = ["tests"]

Virtual Environments: Always Use One

Never install project dependencies into your global Python environment. Create a dedicated virtual environment for each project:

python -m venv .venv
source .venv/bin/activate   # Linux/macOS
.venv\Scripts\activate      # Windows

pip install -r requirements.txt

Managing Dependencies

  • requirements.txt — List direct dependencies for applications. Pin versions (requests==2.31.0) for reproducibility.
  • pyproject.toml — Preferred for libraries; use version ranges (requests>=2.28).
  • pip-tools or Poetry — Consider these tools for dependency locking and management in larger projects.

Configuration and Secrets

Never hardcode configuration values or secrets. Use environment variables, loaded via a .env file during development:

# .env (never commit this)
DATABASE_URL=postgresql://user:password@localhost/mydb
SECRET_KEY=supersecretkey123
# In code, use python-dotenv or pydantic-settings
from dotenv import load_dotenv
import os

load_dotenv()
db_url = os.getenv("DATABASE_URL")

Always add .env to .gitignore and commit only a .env.example template.

The __init__.py File

Use __init__.py to define your package's public API. Keep it minimal — only expose what users of your package should access:

# my_package/__init__.py
from .core import MainClass
from .utils import helper_function

__all__ = ["MainClass", "helper_function"]

Checklist for a Healthy Python Project

  1. Use a virtual environment
  2. Use pyproject.toml for configuration
  3. Separate source code from tests
  4. Never commit secrets or .env files
  5. Write tests from day one
  6. Use a linter (Ruff) and formatter (Black)
  7. Document with a clear README.md

Good project structure isn't about following rules for their own sake — it's about making your codebase predictable and navigable for everyone who touches it, including future you.