How to setup Django, Gunicorn, Nginx and PostgreSQL service using docker compose?

Published on October 09,2020 by Maulik

Docker helps to simplify and set up a uniform platform for development, staging, and production environments. DevOps efforts are reduced by using docker technology. This article will help you understand the process of setting up:

• Django running via a gunicorn server as a docker service.
• Nginx running as a docker service.
• PostgreSQL running as a docker service.

Github Repo of Django, Gunicorn, Nginx, and PostgreSQL using Docker Compose.

What is Docker?

Docker is a technology to set up isolated individual containers of operating system environments. Our application services can be deployed in such containers. In simpler terms, a virtual environment which gives the operating system and other libraries installed for running any application services.

A file with a set of executable commands while building a docker image, is a docker file. For building and running applications, a docker file is a must.

Why we should use Docker?

We should use docker because it gives us an isolated environment for running our applications. Docker builds the image of the application, we can relate it to the installation package which can be installed where docker technology is installed. It helps to ship software and it can run on any platform which supports docker technology. Developers can reduce and avoid the errors coming in the production environment like development, starting, and production environments are the same.

What is the cost of Docker?

Docker community edition is free and available to use for all. If you are planning an advanced professional services docker company provides tools to use. These tools make docker usage much simpler and easier. You may the docker pricing for details.

How to use the Docker?

We can download the docker and install it on our local machines. Download docker from their official download link.

Here are basic commands of docker:

• docker build – It builds docker images
• docker pull – Pulls image from a container registry
• docker exec – Execute command line command inside a docker container
• docker run – Start the docker container
• docker stop – Stop the docker container
• docker images – Shows a list of docker images

More docker commands can be found on docker commands documentation.

What is docker-compose?

Docker-compose helps to create multiple services configuration files and start all services at once. Docker-compose is an open-source project. Docker-compose is used to define and create multiple docker container services.

Following is the directory structure of the project, it can be seen on GitHub repo as well:

.
└── docker 
    │   └── django 
    │   │   ├── dockerfile 
    │   │   └── scripts
    │   │   │   ├── db_connectivity.sh 
    │   │   │   ├── gunicorn.sh
    │   │   ├── .django_local_env 
    │   └── nginx 
    │   │   ├── dockerfile 
    │   │   ├── nginx.conf 
    │   └── postgres 
    │   │   ├── dockerfile 
    │   │   ├── .postgres_local_env 
└── django_docker < Your main app >
├── docker-compose.yml
├── manage.py
├── requirements.txt

Steps to setup PostgresSQL as docker service

The following are the three major configurations for setting up PostgreSQL docker service:

1. Create a docker file for Postgres at docker/postgres/dockerfile, it will pull the Postgres Image from the docker hub.

   FROM postgres:10.12

    

2. Create docker/postgres/.postgres_local_env, it has the environment variables are used to configure the Postgress database server by the docker-compose process.

   POSTGRES_DB=django_db
   POSTGRES_PASSWORD=postgres
   POSTGRES_USER=postgres

    

3. Please review the docker-compose.yml file, it contains a service and volume section for the Postgres database server.

   version: '3'
   volumes:
    # static volume will be mounted to both nginx and django gunicorn services.
    static_volume:
    # postgres data volume will be mounted to postgres services
    postgres_data:
    
   services:
    postgres:
      command: postgres -c max_connections=100
      build:
        context: .
        # it points to the docker file which has instruction to build this service.
        dockerfile: ./docker/postgres/dockerfile
      volumes:
        - postgres_data:/var/lib/postgresql/data/
      ports:
        - "5432:5432"
      # all environment variables are defined in below file.
      env_file:
       - ./docker/postgres/.postgres_local_env

    

Steps to start Django via Gunicorn server as a docker service

Following are the configuration settings:

1. Create a docker file for Postgres at docker/django/dockerfile, it will pull the Python 3 Image from the docker hub and execute all commands in a sequence, these commands will be considered as build steps.

   FROM python:3
   # it will enable python to do stdout logs instead of being buffered
   ENV PYTHONUNBUFFERED 1
   ENV LANG en_US.utf8
    
   # creating
   RUN mkdir /app
    
   # copy all file in app folder
   COPY . /app
   # copy requriements.txt in app folder
   COPY requirements.txt /requirements.txt
   # copy db connectivity test script, because we have made /app as current working directory so script can be executed.
   COPY docker/django/scripts/db_connectivity.sh /db_connectivity.sh
   # copy start gunicorn server script, because we have made /app as current working directory so script can be executed.
   COPY docker/django/scripts/gunicorn.sh /gunicorn.sh
   # running pip command to install all dependencies
   RUN pip install -r requirements.txt
    
   RUN chmod +x /db_connectivity.sh
   RUN chmod +x /gunicorn.sh
    
   # making /app as a current working dir
   WORKDIR /app
    
   #It will check whether we are able to connect to postgres service or not.
   ENTRYPOINT ["/db_connectivity.sh"]

    

2. Update docker/django/.django_local_env, file as per your configurations, here is the sample:

   DEBUG=1
   SECRET_KEY=hfi&(e$#fyy1d^klhbg&u$ftx4(*[email protected]$yw*
   DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
   SQL_ENGINE=django.db.backends.postgresql_psycopg2
   SQL_DATABASE=django_db
   SQL_USER=postgres
   SQL_PASSWORD=postgres
   SQL_HOST=postgres
   SQL_PORT=5432
   DATABASE_URL=postgres://postgres:[email protected]:5432/django_db

    

3. docker/django/.db_connectivity.sh will be executed during docker build. It checks the connectivity of PostgreSQL from Django docker service

   #!/bin/bash
   set -e
   cmd="[email protected]"
    
   function postgres_ready(){
   python << END
   import sys
   from urllib import parse
   import psycopg2
   try:
      result = parse.urlparse("$DATABASE_URL")
      print(result)
      username = result.username
      password = result.password
      database = result.path[1:]
      hostname = result.hostname
      port = result.port
     
      conn = psycopg2.connect(
          database = database,
          user = username,
          password = password,
          host = hostname,
          port = port
      )
     
   except psycopg2.OperationalError as e:
      print(e)
      sys.exit(-1)
   sys.exit(0)
   END
   }
    
   until postgres_ready; do
    >&2 echo "Postgres is unavailable - sleeping"
    sleep 1
   done
    
   >&2 echo "Postgres is up - continuing..."
   exec $cmd

    

4. After docker/django/db_connectivity.sh execution docker/django/gunicorn.sh on docker build execution command. Here is gunicorn.sh :

   python /app/manage.py collectstatic --noinput
   python /app/manage.py migrate
   gunicorn django_docker.wsgi -b 0.0.0.0:8000 --timeout 900 --chdir=/app --log-level debug --log-file -

    

5. Now we need to add django docker service in docker-compose.yml:

   version: '3'
    
   volumes:
    # static volume will be mounted to both nginx and django gunicorn services.
    static_volume:
    # postgres data volume will be mounted to postgres services
    postgres_data:
    
   services:
    postgres:
      command: postgres -c max_connections=100
      build:
        context: .
        # it points to the docker file which has instruction to build this service.
        dockerfile: ./docker/postgres/dockerfile
      volumes:
        - postgres_data:/var/lib/postgresql/data/
      ports:
        - "5432:5432"
      # all environment variables are defined in below file.
      env_file:
       - ./docker/postgres/.postgres_local_env
    
    django:
      build:
        context: .
        # it points to the docker file which has instruction to build this service.
        dockerfile: ./docker/django/dockerfile
      # this command will execute after execution all build steps from './docker/django/dockerfile'
      command: /gunicorn.sh
      volumes:
        - static_volume:/app/static
      # links attribute will let postgres service become available first.
      links:
        - postgres
      expose:
        - "8000"
      restart: always
      env_file:
       - ./docker/django/.django_local_env

    

Steps to expose the Django service via Nginx docker service

1. Add/update docker/nginx/dockerfile file:

   FROM nginx:1.17
    
   # copying our custom configuration to our nginx service
   COPY ./docker/nginx/nginx.conf /etc/nginx/nginx.conf

    

2. Update/Configure docker/nginx/nginx.conf

   user  nginx;
   worker_processes  2;
    
   error_log  /var/log/nginx/error.log warn;
   pid        /var/run/nginx.pid;
    
   events {
    # as mentioned on line 2 there will be 2 worker process
    # so in total 2*1024 = 2048 connections can be handled at a time
    worker_connections  1024;
   }
    
   http {
    # it includes support for all generic mime types.
    include       /etc/nginx/mime.types;
    # It mentioned default mime type
    default_type  application/octet-stream;
    
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    '$status $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';
    
    access_log  /var/log/nginx/access.log  main;
    
    sendfile        on;
    
    upstream django_docker {
      # "django" is the web project as docker service.
      server django:8000;
    }
    
    server {
    
      # setting charset to utf-8
      charset     utf-8;
      # making nginx listen on port 8000
      listen      8000;
      # servername is assigned here
      server_name localhost;
    
      # routing all request which includes url meda to /app/media/ so this traffic can be served by nginx
      location /static/ {
          alias /app/static/;
      }
    
      # routing all request which includes url meda to /app/media/ so this traffic can be served by nginx
      location /media/ {
          alias /app/media/;
      }
    
      location / {
        # checks for static file, if not found proxy to app
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
       
        # django_docker is upstream object as mentioned on line 23,
        # basically it is pointing to django docker service as mentioned in docker-compose.yml
        proxy_pass http://django_docker;
    
      }
    }
   }

    

3. Add the Nginx server to docker-compose.yml:

   nginx:
      build:
        context: .
        dockerfile: ./docker/nginx/dockerfile
      ports:
        - "8000:8000"
      volumes:
        - static_volume:/app/static
      links:
        - django
      restart: always

    

4. Finally, full docker-compose.yaml should look like this:

   version: '3'
    
   volumes:
     # static volume will be mounted to both nginx and django gunicorn services.
     static_volume:
     # postgres data volume will be mounted to postgres services
     postgres_data:
    
   services:
     postgres:
       command: postgres -c max_connections=100
       build:
         context: .
         # it points to the docker file which has instruction to build this service.
         dockerfile: ./docker/postgres/dockerfile
       volumes:
         - postgres_data:/var/lib/postgresql/data/
       ports:
         - "5432:5432"
       # all environment variables are defined in below file.
       env_file:
         - ./docker/postgres/.postgres_local_env
    
     django:
       build:
         context: .
         # it points to the docker file which has instruction to build this service.
         dockerfile: ./docker/django/dockerfile
       # this command will execute after execution all build steps from './docker/django/dockerfile'
       command: /gunicorn.sh
       volumes:
         - static_volume:/app/static
       # links attribute will let postgres service become available first.
       links:
         - postgres
       expose:
         - "8000"
       restart: always
       env_file:
         - ./docker/django/.django_local_env
   
     nginx:
       build:
         context: .
         dockerfile: ./docker/nginx/dockerfile
       ports:
         - "8000:8000"
       volumes:
         - static_volume:/app/static
       links:
         - django
       restart: always

    

View Django application running via docker-compose

1. Let’s run the docker build: docker-compose up --build from project root directory.
2. You can see the following output on the following: http://localhost:8000