Docker compose files overview

Use compose files to describe docker services

view on github

The Compose specification

✔️ The Compose file is a .yaml file defining the following components of a Compose application :

  • Services
  • Networks
  • Volumes

✔️ The Compose file specification specifies properties designed to :

  1. Target a local OCI container runtime (example : Docker)
  2. Expose Linux kernel specific configuration options
  3. Expose cloud platform features (resource placement on a cluster, replicated application distribution, scalability)

✔️ A Compose implementation may parse a Compose file using one of the following modes :

  • loose: accept the Compose file, ignore unsupported attributes and unknown attributes
  • default: accept the Compose file, warn user about unsupported attributes and ignore them
  • strict: reject the Compose file, warn user about unsupported attributes

Important notes

  • The best practices documented hereby are meant to be use with version 3 of the Compose file specification
  • Also, this documentation is meant to be relevant only to the deployment of services in swarm mode by using docker stack deploy

The Compose application model

✔️ A Compose application is designed as a set of containers which have to both :

  • Run simultaneously
  • Share resources and communication channels.

✔️ The components of a Compose applications are as follows :

  1. Services

    • Services are the application components that perform computing tasks.
    • A Service is the abstract concept of simultaneously running multiple containers based on the same image / configuration :
      • Services are sets of containers orchestrated by the platform according to replication requirements and placement constraints.
      • Service can be scaled/replaced independently from other components.
      • Being backed by containers, Services are defined by a Docker image and set of runtime arguments.
      • All containers within a service are identically created with these arguments.

  2. Networks

    • Networks provide application-wide channels for inter-service communication.
    • A Network is a platform capability abstraction to establish an IP route between containers within services connected together.

  3. Volumes

    • Volumes provide data persistence capabilities to services.
    • Volumes implement data persistence as high-level filesystem mounts with global options

  4. Configs

    • Configs provide the ability to configure services in accordance to a specific runtime or platform.
    • Configs are comparable to Volumes in that they are files mounted into containers, but they are not used to persist data

  5. Secrets

    • Secrets are a distinct type of Configs for sensitive data that SHOULD NOT be exposed without security considerations.
    • Secrets are comparable to Configs in that they are files mounted into containers, but they are never exposed outside of the container itself

Example of a Compose application

✔️ The following example leverages all of the above concepts:

  • 2 services, backed by Docker images: webapp and database
  • 1 secret (HTTPS certificate), injected into the frontend
  • 1 configuration (HTTP), injected into the frontend
  • 1 persistent volume, attached to the backend
  • 2 networks
(External user) --> 443 [frontend network]
                    |
            +--------------------+
            |  frontend service  |...ro...<HTTP configuration>
            |      "webapp"      |...ro...<server certificate> #secured
            +--------------------+
                    |
                [backend network]
                    |
            +--------------------+
            |  backend service   |  r+w   ___________________
            |     "database"     |=======( persistent volume )
            +--------------------+        \_________________/

Compose file top-level elements :

✔️ version (deprecated)

  • The top-level version element is only informative.
  • As a best practice, it should always mention the latest Compose specification version.

✔️ name (optional)

  • User-specified name for the application, used internally by the runtime .

✔️ services (required)

  • Map of service name / definition pairs for the application's services :
    • A service definition contains the configuration to apply to each container started for that service.
    • It may include an optional deploy element (groups constraints and requirements for the deployment strategy).
  • List of relevant definition directives (AR = array, LS = long syntax) :
directive AR LS usage
command override Dockerfile COMMAND directive (use sparsely)
configs Y Y configs the containers will be granted access to (specified under the configs top-level element)
deploy\mode global : 1 container per docker daemon) / replicated : (default)
deploy\placement specifies placement constraints and preferences for the service containers (see footnote)
deploy\replicas ideal number of replicas the service should run at any moment
deploy\resources configure host resources thresholds and/or limits a container can use (CPUs, memory, devices ...)
deploy\restart_policy whether/how containers restart on exit (see footnote)
deploy\rollback_config configuration on how to revert a service to its previous working state when an update fails
deploy\update_config configuration on how to update a service container by container without stopping it (update image, add/remove volumes ...)
entrypoint override Dockerfile ENTRYPOINT directive (investigate)
env_file Y adds environment variables to containers based on dotenv files relative to the Compose file folder
environment Y adds environment variables to containers based on explicit variable declarations
healthcheck override Dockerfile HEALTHCHECK directive (investigate)
image image for the service containers (use this instead)
logging\driver logging driver for the service (specific to the host platform)
logging\options logging configuration for the service
networks Y networks to which the containers will attach (specified under the networks top-level element)
ports Y Y containers ports that the docker daemon will expose, specified as host:container/protocol
secrets Y Y secrets the containers will be granted access to (specified under the secrets top-level element)
stop_grace_period time to wait to send SIGKILL after sending SIGTERM if the container hasn't stopped (1m30s)
sysctls Y set kernel variables values inside containers
volumes Y Y volumes to which the containers will attach (specified under the volumes top-level element or at service level)
  • Fine tuning of the deploy\restart_policy subdirectives :
    • condition: none when containers are supposed to exit on error (default)
    • condition: on-failure when services orchestration needs to be fine tuned
      • if the service depends on other services, then assess the time by which said services will be up and running
      • The service containers will fail (exit with code 1) at startup, then will be successfully restarted if assessed time is correct : 
# since depends_on is not supported in swarm mode, service orchestration must rely on restart policy
# in the current scenario we expect the depended on services to be available after (15 + 15 = 30) seconds
restart_policy:
  # restart the dependent service when it fails
  condition: on-failure
  # duration after which the daemon attempts to restart the service containers
  delay: 15s
  # the first restart attempt should be successful
  max_attempts: 1
  # duration after which the docker daemon will consider the initial start as a fail
  window: 15s
  • Fine tuning of the deploy\placement subdirectives :
    • constraints limit the set of nodes where the service containers will be deployed
    • preferences allow load balancing by spreading containers deployment across several nodes

✔️ networks

  • Map of networks name / definition pairs for the application's networks :
    • The networking model exposed to a given service is a simple IP connection that targets other services and external resources.
    • The network definition allows fine-tuning of the actual implementation provided by the platform.

✔️ volumes

  • Map of volumes name / definition pairs for the application's volumes :
    • Allows configuration of named volumes that can be reused across multiple services.
    • The volumes definition allows fine tuning of volume allocation on underlying infrastructure.

✔️ configs

  • Map of configs name / definition pairs for the application's configs :
    • Configs allow adaptative container behavior without the need to rebuild the image.
    • By default, configs are mounted to / inside the container.
    • By default, the config MUST be owned by the user running the container command.
    • By default, the config MUST be have world-readable permissions (mode 0444).

✔️ secrets

  • Maps of secrets name / definition pairs for the application's secrets :
    • Specific type of configs focusing on sensitive data, with specific constraint for this usage.

Important notes on volumes usage in swarm mode

  • Named volumes declared in the top-level volume are created at service deployment on any host service containers are deployed to
  • To ensure data consistency, a service referencing a volume must have its containers deployed only on the host where the volume lives
  • Placement constraints can be used so that service containers are deployed and volume is created on a specific node
  • Volume data will be persisted in /var/lib/docker/volumes/<application-name>_<volume-name>/_data and remain on the node filesystem even when the services referencing it are not running
  • As an alternative, named volumes can be declared using a driver that supports writing files to an external storage system like NFS or Amazon S3