How to setup Django, Gunicorn, Nginx and PostgreSQL service using docker compose?

Published on October 09,2020 by Maulik

Docker helps to simplify and set up a uniform platform for development, staging, and production environments. DevOps efforts are reduced by using docker technology. This article will help you understand the process of setting up:

  • Django running via a gunicorn server as a docker service.
  • Nginx running as a docker service.
  • PostgreSQL running as a docker service.

Github Repo of Django, Gunicorn, Nginx, and PostgreSQL using Docker Compose.

What is Docker?

Docker is a technology to set up isolated individual containers of operating system environments. Our application services can be deployed in such containers. In simpler terms, a virtual environment which gives the operating system and other libraries installed for running any application services.

A file with a set of executable commands while building a docker image, is a docker file. For building and running applications, a docker file is a must.

Why we should use Docker?

We should use docker because it gives us an isolated environment for running our applications. Docker builds the image of the application, we can relate it to the installation package which can be installed where docker technology is installed. It helps to ship software and it can run on any platform which supports docker technology. Developers can reduce and avoid the errors coming in the production environment like development, starting, and production environments are the same.

What is the cost of Docker?

Docker community edition is free and available to use for all. If you are planning an advanced professional services docker company provides tools to use. These tools make docker usage much simpler and easier. You may the docker pricing for details.

How to use the Docker?

We can download the docker and install it on our local machines. Download docker from their official download link.

Here are basic commands of docker:

  • docker build – It builds docker images
  • docker pull – Pulls image from a container registry
  • docker exec – Execute command line command inside a docker container
  • docker run – Start the docker container
  • docker stop – Stop the docker container
  • docker images – Shows a list of docker images

More docker commands can be found on docker commands documentation.

What is docker-compose?

Docker-compose helps to create multiple services configuration files and start all services at once. Docker-compose is an open-source project. Docker-compose is used to define and create multiple docker container services.

Following is the directory structure of the project, it can be seen on GitHub repo as well:

.
└── docker 
    │   └── django 
    │   │   ├── dockerfile 
    │   │   └── scripts
    │   │   │   ├── db_connectivity.sh 
    │   │   │   ├── gunicorn.sh
    │   │   ├── .django_local_env 
    │   └── nginx 
    │   │   ├── dockerfile 
    │   │   ├── nginx.conf 
    │   └── postgres 
    │   │   ├── dockerfile 
    │   │   ├── .postgres_local_env 
└── django_docker < Your main app >
├── docker-compose.yml
├── manage.py
├── requirements.txt

Steps to setup PostgresSQL as docker service

The following are the three major configurations for setting up PostgreSQL docker service:

  1. Create a docker file for Postgres at docker/postgres/dockerfile, it will pull the Postgres Image from the docker hub.
    FROM postgres:10.12

     

  2. Create docker/postgres/.postgres_local_env, it has the environment variables are used to configure the Postgress database server by the docker-compose process.
    POSTGRES_DB=django_db
    POSTGRES_PASSWORD=postgres
    POSTGRES_USER=postgres

     

  3. Please review the docker-compose.yml file, it contains a service and volume section for the Postgres database server.
    version: '3'
    volumes:
     # static volume will be mounted to both nginx and django gunicorn services.
     static_volume:
     # postgres data volume will be mounted to postgres services
     postgres_data:
     
    services:
     postgres:
       command: postgres -c max_connections=100
       build:
         context: .
         # it points to the docker file which has instruction to build this service.
         dockerfile: ./docker/postgres/dockerfile
       volumes:
         - postgres_data:/var/lib/postgresql/data/
       ports:
         - "5432:5432"
       # all environment variables are defined in below file.
       env_file:
        - ./docker/postgres/.postgres_local_env

     

Steps to start Django via Gunicorn server as a docker service

Following are the configuration settings:

  1. Create a docker file for Postgres at docker/django/dockerfile, it will pull the Python 3 Image from the docker hub and execute all commands in a sequence, these commands will be considered as build steps.
    FROM python:3
    # it will enable python to do stdout logs instead of being buffered
    ENV PYTHONUNBUFFERED 1
    ENV LANG en_US.utf8
     
    # creating
    RUN mkdir /app
     
    # copy all file in app folder
    COPY . /app
    # copy requriements.txt in app folder
    COPY requirements.txt /requirements.txt
    # copy db connectivity test script, because we have made /app as current working directory so script can be executed.
    COPY docker/django/scripts/db_connectivity.sh /db_connectivity.sh
    # copy start gunicorn server script, because we have made /app as current working directory so script can be executed.
    COPY docker/django/scripts/gunicorn.sh /gunicorn.sh
    # running pip command to install all dependencies
    RUN pip install -r requirements.txt
     
    RUN chmod +x /db_connectivity.sh
    RUN chmod +x /gunicorn.sh
     
    # making /app as a current working dir
    WORKDIR /app
     
    #It will check whether we are able to connect to postgres service or not.
    ENTRYPOINT ["/db_connectivity.sh"]

     

  2. Update docker/django/.django_local_env, file as per your configurations, here is the sample:
    DEBUG=1
    SECRET_KEY=hfi&(e$#fyy1d^klhbg&u$ftx4(*[email protected]$yw*
    DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
    SQL_ENGINE=django.db.backends.postgresql_psycopg2
    SQL_DATABASE=django_db
    SQL_USER=postgres
    SQL_PASSWORD=postgres
    SQL_HOST=postgres
    SQL_PORT=5432
    DATABASE_URL=postgres://postgres:[email protected]:5432/django_db

     

  3. docker/django/.db_connectivity.sh will be executed during docker build. It checks the connectivity of PostgreSQL from Django docker service
    #!/bin/bash
    set -e
    cmd="[email protected]"
     
    function postgres_ready(){
    python << END
    import sys
    from urllib import parse
    import psycopg2
    try:
       result = parse.urlparse("$DATABASE_URL")
       print(result)
       username = result.username
       password = result.password
       database = result.path[1:]
       hostname = result.hostname
       port = result.port
      
       conn = psycopg2.connect(
           database = database,
           user = username,
           password = password,
           host = hostname,
           port = port
       )
      
    except psycopg2.OperationalError as e:
       print(e)
       sys.exit(-1)
    sys.exit(0)
    END
    }
     
    until postgres_ready; do
     >&2 echo "Postgres is unavailable - sleeping"
     sleep 1
    done
     
    >&2 echo "Postgres is up - continuing..."
    exec $cmd

     

  4. After docker/django/db_connectivity.sh execution docker/django/gunicorn.sh on docker build execution command. Here is gunicorn.sh :
    python /app/manage.py collectstatic --noinput
    python /app/manage.py migrate
    gunicorn django_docker.wsgi -b 0.0.0.0:8000 --timeout 900 --chdir=/app --log-level debug --log-file -

     

  5. Now we need to add django docker service in docker-compose.yml:
    version: '3'
     
    volumes:
     # static volume will be mounted to both nginx and django gunicorn services.
     static_volume:
     # postgres data volume will be mounted to postgres services
     postgres_data:
     
    services:
     postgres:
       command: postgres -c max_connections=100
       build:
         context: .
         # it points to the docker file which has instruction to build this service.
         dockerfile: ./docker/postgres/dockerfile
       volumes:
         - postgres_data:/var/lib/postgresql/data/
       ports:
         - "5432:5432"
       # all environment variables are defined in below file.
       env_file:
        - ./docker/postgres/.postgres_local_env
     
     django:
       build:
         context: .
         # it points to the docker file which has instruction to build this service.
         dockerfile: ./docker/django/dockerfile
       # this command will execute after execution all build steps from './docker/django/dockerfile'
       command: /gunicorn.sh
       volumes:
         - static_volume:/app/static
       # links attribute will let postgres service become available first.
       links:
         - postgres
       expose:
         - "8000"
       restart: always
       env_file:
        - ./docker/django/.django_local_env

     

Steps to expose the Django service via Nginx docker service

  1. Add/update docker/nginx/dockerfile file:
    FROM nginx:1.17
     
    # copying our custom configuration to our nginx service
    COPY ./docker/nginx/nginx.conf /etc/nginx/nginx.conf

     

  2. Update/Configure docker/nginx/nginx.conf
    user  nginx;
    worker_processes  2;
     
    error_log  /var/log/nginx/error.log warn;
    pid        /var/run/nginx.pid;
     
    events {
     # as mentioned on line 2 there will be 2 worker process
     # so in total 2*1024 = 2048 connections can be handled at a time
     worker_connections  1024;
    }
     
    http {
     # it includes support for all generic mime types.
     include       /etc/nginx/mime.types;
     # It mentioned default mime type
     default_type  application/octet-stream;
     
     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
     '$status $body_bytes_sent "$http_referer" '
     '"$http_user_agent" "$http_x_forwarded_for"';
     
     access_log  /var/log/nginx/access.log  main;
     
     sendfile        on;
     
     upstream django_docker {
       # "django" is the web project as docker service.
       server django:8000;
     }
     
     server {
     
       # setting charset to utf-8
       charset     utf-8;
       # making nginx listen on port 8000
       listen      8000;
       # servername is assigned here
       server_name localhost;
     
       # routing all request which includes url meda to /app/media/ so this traffic can be served by nginx
       location /static/ {
           alias /app/static/;
       }
     
       # routing all request which includes url meda to /app/media/ so this traffic can be served by nginx
       location /media/ {
           alias /app/media/;
       }
     
       location / {
         # checks for static file, if not found proxy to app
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header Host $http_host;
         proxy_redirect off;
        
         # django_docker is upstream object as mentioned on line 23,
         # basically it is pointing to django docker service as mentioned in docker-compose.yml
         proxy_pass http://django_docker;
     
       }
     }
    }

     

  3. Add the Nginx server to docker-compose.yml:
    nginx:
       build:
         context: .
         dockerfile: ./docker/nginx/dockerfile
       ports:
         - "8000:8000"
       volumes:
         - static_volume:/app/static
       links:
         - django
       restart: always

     

  4. Finally, full docker-compose.yaml should look like this:
    version: '3'
     
    volumes:
      # static volume will be mounted to both nginx and django gunicorn services.
      static_volume:
      # postgres data volume will be mounted to postgres services
      postgres_data:
     
    services:
      postgres:
        command: postgres -c max_connections=100
        build:
          context: .
          # it points to the docker file which has instruction to build this service.
          dockerfile: ./docker/postgres/dockerfile
        volumes:
          - postgres_data:/var/lib/postgresql/data/
        ports:
          - "5432:5432"
        # all environment variables are defined in below file.
        env_file:
          - ./docker/postgres/.postgres_local_env
     
      django:
        build:
          context: .
          # it points to the docker file which has instruction to build this service.
          dockerfile: ./docker/django/dockerfile
        # this command will execute after execution all build steps from './docker/django/dockerfile'
        command: /gunicorn.sh
        volumes:
          - static_volume:/app/static
        # links attribute will let postgres service become available first.
        links:
          - postgres
        expose:
          - "8000"
        restart: always
        env_file:
          - ./docker/django/.django_local_env
    
      nginx:
        build:
          context: .
          dockerfile: ./docker/nginx/dockerfile
        ports:
          - "8000:8000"
        volumes:
          - static_volume:/app/static
        links:
          - django
        restart: always

     

View Django application running via docker-compose

  1. Let’s run the docker build: docker-compose up --build from project root directory.
  2. You can see the following output on the following: http://localhost:8000django-docker-nginx-gunicorn-postgresql.png

0 Comments

Related Articles

How to create a common response format for 200, 400, 500 responses by creating custom exception handler in Django Rest Framework?

Published on May 28,2020 by Maulik

How to create a common response format for 200, 400, 500 responses by creating custom exception handler in Django Rest Framework?

In micro-services architecture, multiple client applications are consuming the backend API. The backend server does the core business logic and all heavy lifting. The client …

Read full article

How to create different custom logs formatter for Info, Warning and Error logs in Django web application?

Published on May 18,2020 by Maulik

How to create different custom logs formatter for Info, Warning and Error logs in Django web application?

If you are interested in something which helps you customize the logs format for Django web applications running in production, you have landed at the …

Read full article

copied to clipboard

Sign up for our newsletter

Please join our news letter which we share every month, you would love interesting python and django news letters.

We understand no one like spamming, your emails are safe with us.

Copyright © Django Circle All Rights Reserved.