How to Build Small, Fast Docker Images for Python AI Services

Python AI services have a reputation for huge container images. A few gigabytes means slow builds, slow deployments and slow scaling when traffic arrives. Most of that size is avoidable.

Step 1: Start from a slim base

Use python:3.12-slim rather than the full image. If your service only calls model APIs, you don't need CUDA or a GPU base image at all. Those add gigabytes and belong only in images that run models locally.

Step 2: Use a multi-stage build

Install dependencies in a build stage and copy only the result into the final image:

FROM python:3.12-slim AS build
COPY --from=ghcr.io/astral-sh/uv:0.8 /uv /bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-install-project

FROM python:3.12-slim
WORKDIR /app
COPY --from=build /app/.venv /app/.venv
COPY src ./src
ENV PATH="/app/.venv/bin:$PATH"
USER nobody
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080"]

Compilers and build tools stay in the first stage and never reach production.

Step 3: Order layers for caching

Copy the dependency files and install them before copying your source code. A code change then rebuilds only the last layers, and the dependency layer comes from cache.

Step 4: Keep models and data out

Don't copy model weights, datasets or notebooks into the image. Add a .dockerignore that excludes .git, data/, notebooks/ and local virtual environments.

Step 5: Run as a non-root user

The USER line stops the process from running as root. Many Kubernetes security policies reject root containers anyway.

Step 6: Check the result

Run docker image ls to see the size and docker history to find the layers that grew. A tool like dive shows exactly which files take up the space.

Things to watch

• Some packages pull in large optional extras. Install only what you import.
• Pin the uv image and the base image to specific versions, so builds don't change unexpectedly.
• Alpine-based images can make Python packages slower to install and harder to debug. Slim Debian images are usually the safer choice.

With these steps, a typical service that calls model APIs fits in a few hundred megabytes.