Deploy Django


Published: 2 weeks, 6 days ago   101


About this Django Deployment Guide

This document will demonstrate the deployment of a Django project to the doprax cloud platform.  We will deploy the Django project with MySQL as the database and also we will use Nginx as a reverse proxy. We will use gunicorn as the WSGI HTTP server. The final deployed Django website is accessible here. You can find the repository of the Django project code here

Install Django

To use Django you must install it on your machine (and also the server). You can install it system-wide or in a virtual environment. To avoid conflicts with other packages it is strongly recommended to use virtual environments to install any python requirements. We are going to use the virtualenv package to manage the virtual environments. Also to install any Python packages you need to have pip on your machine. If you use virtualenv, pip will be automatically installed and available inside your virtual environment. But for the sake of completeness, we install both of them now.

sudo apt-get install python3-pip
sudo apt-get install virtualenv

You can also use venv which is a version of virtualenv that has been integrated into the Python standard library but virtualenv is a better choice. You can learn more about virtualenv here. Now create a virtual environment for your project.

virtualenv myenv -p python3

This will create a folder named myenv in your current directory. Activate your virtual environment using this command:

source ./myenv/bin/activate

Now it is time to install Django. 

pip install django == 2.2

We are installing Django version 2.2 which is still the latest LTS (Long Term Support) version of Django. Although at the time of this writing, the newest version of Django is 3.1.7. You can install a specific version by using pip install django == 3.1.7. It shouldn't be that much different. The future LTS version is going to be 3.2 and it is scheduled to be released in mid-2021. 

Create a Django project

Now that we have our virtual environment activated and Django installed, it is time to create a new project. Use this command to create a project 

django-admin startproject myproject

This command will create a new Django project named myproject in the current directory. Sofar the project structure should look like this:

Directory structure of the Django project

Now let's run this Django project to see if it is working or not. Make sure that your working directory is inside myproject where the manage.py file resides. 

cd myproject
python manage.py runserver

This command will run the development server provided by Django and it will be run in your localhost address on port 8000. You can now see the Django welcome page if you enter the following URL into your browser, or just click it (http://127.0.0.1:8000). You should see the following page:

Django welcome  page

You will see some complaints in your terminal about migrations. To get rid of it enter the command below to do the migrations. More on migrations later. 

python manage.py migrate

 

Create a new Django app

To make the Django project more realistic (complex) and test some features like migrations, let's create a Django app inside our Django project. The app will be named myapp and it is created like this:

python manage.py startapp myapp

Using the above command a new folder named myapp containing our Django app will be created. The first thing that we need to do when creating a new Django app is to add it to the installed apps list in settngs.py. So open myproject/settings.py file and add 'myapp', to the end of the INSTALLED_APPS list. 

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'myapp',
]

The new project structure will look like this:

added Django app

Notice the db.sqlite3 file in the project root. It has been created since we are using the default SQLite as the database for our Django project. SQLite is ok for development and test purposes but it is NOT recommended to use it in production. We will use a production-grade MySQL service as our database later. 

Create a simple page

Now let's continue by creating a simple page in our newly added Django app. Open myapp/views.py and create a new function to handle our first page like following

from django.shortcuts import render

def first_page(request):
   return render(request, 'first_page.html', {})

This is the simplest views function possible! It just renders the 'first_page.html' template file (which we will create next) and returns the response to the user. Now open myproject/urls.py and import the views function that we created and also add a new line to the urlpatterns list. In the end, it should look like this:

from django.contrib import admin
from django.urls import path
from myapp.views import first_page

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', first_page, name='index'),
]

Now create a folder named templates at the root of the project (where manage.py resides). Inside this folder create a file and name it first_page.html and then copy the following in it.

{% load static %}
<!doctype html>
<html lang="en">
    <body>
        <div style="background-color:blue; padding:50px">
            <h1>Welcome to django deployment Doc page</h1>
        </div>
        <div style="padding:20px;">
            <img src="{% static "doprax_logo.png" %}" alt="doprax">
            <p>Powered by <a href="https://www.doprax.com/"> Doprax.com </a></p>
        </div>
    </body>
</html>

Create a folder at the root of your project and name it static. This folder will host the static files used in your project like images, font, CSS and javascript files, and other static files. download the following image and copy it to the static folder

doprax logo


Three small changes are also needed in settings.py file to tell Django where to look for template files and static files. Open settings.py and find the TEMPLATES section. There  should be an entry in the dictionary like this 'DIRS': [], change it to the following:

'DIRS': [os.path.join(BASE_DIR, 'templates')],

It tells Django to look for templates in a folder named templates at the root of the project. Next, add the following to the settings.py file also. You can add it anywhere in the file but I recommend fining the STATIC_URL  item and paste the following below it so that static file settings would be near each other for ease of lookup. More on serving static files later. It should look like this:

STATIC_URL = '/static/'
STATIC_ROOT = '/app_data/static/'
STATICFILES_DIRS = [
    os.path.join(BASE_DIR, "static"),
]

Also, find ALLOWED_HOSTS = [] and add the following to it

ALLOWED_HOSTS = ['.doprax.com']

Notice that we have used a wildcard for allowed hosts, meaning that every domain name ending with doprax.com is acceptable. If you want to add a custom domain to your project, like example.com, you need to add it also to the ALLOWED_HOSTS list like this ALLOWED_HOSTS = ['.doprax.com', 'example.com'].  

No open your browser and go to this link (http://127.0.0.1:8000). It should look like this:

sample page

Config Django to use MySQL 

As I mentioned earlier, the default database server used in Django is SQLite, Which is only for development and test and is  NOT suitable for production websites. So we are going to configure the Django application to use MySQL as the database server. If you want to have MySQL for local development you need to install it on your machine. You can use SQLite for your local development and MySQL in production. This way you can avoid the overhead of installing and configuring MySQL server locally and for production, you can use doprax MySQL service which does not need any installation. Open settings.py file and find DATABASE section. By default it should be like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

It is completely ok to use it for your local development so we are going to keep it like this for your local development but we are going to also add database configuration for production. You must comment out the local database config when you deploy your project to doprax. For production use it should look like this:

# DATABASES = {
#     'default': {
#         'ENGINE': 'django.db.backends.sqlite3',
#         'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
#     }
# }

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mydb',
        'USER': 'root',
        'PASSWORD': os.environ['DB_PASS'],
        'HOST': os.environ['DB_HOST'],
        'PORT': '3306',
    }
}

Notice that we have not entered the password and host of the database, instead, we are using environment variables. So when the Django project runs in production, it will read the value of DB_PASS and DB_HOST dynamically. So we must not forget to add these environment values later when we create a project in doprax. 

Add Dockerfile

Docker is arguably the best way to package and deploy software these days and it is gaining huge popularity. Doprax deploys your Django application on Docker. You don't have to use Docker for your development environment although we strongly suggest it. Docker uses a Dockerfile as the instruction for building and running your application. So we need to create a file at the root of the project and name it Dockerfile, (notice the capital D letter).

FROM ubuntu:bionic
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y install \
    python3 python3-dev python3-dev python3-pip python3-venv python3-wheel \
    mysql-client libsqlclient-dev libssl-dev default-libmysqlclient-dev

ARG USER=root
USER $USER
RUN python3 -m venv venv
WORKDIR /app
COPY requirements.txt requirements.txt
RUN /venv/bin/pip install -r requirements.txt

COPY . .
EXPOSE 5000
RUN chmod +x /app/start.sh
ENTRYPOINT ["./start.sh"]

This file is the instruction for the deployment of a production-grade Django application. It uses the official ubuntu 18.04 Docker image as the base image. Then installs all necessary packages to run Django and interact with MySQL server. Then it will create a virtual environment and install all python dependencies needed from the requirements.txt file (which we will create next). Then it will expose port number 5000, and makes start.sh (which we will shortly create ) shell script executable and runs it. 

Add requirements.txt 

To manage the python requirements of the project we need to add a requirements.txt file to our project root and list every dependency in it. Since our project is very simple for now, we only need three packages. The first one is Django, the second one is mysqlclient which is need to make connections to a MySQL database server and the last one is gunicorn which is the WSGI webserver needed for production-grade Django application. More on gunicorn later. , If you also need any other packages, feel free to list them in the file. For now, enter the following in the requirements.txt file:

django==2.2
mysqlclient==2.0
gunicorn==19.4

To avoid surprise bugs, it is a good idea to explicitly specify the exact version of your dependencies. 

Add Start script

Docker needs one primary process to run. As It has been specified in the last line of our Dockerfile (ENTRYPOINT) our primary process is running start.sh shell script. We will now create this file and instruct it to do the necessary steps needed to run our Django web application. Create a start.sh file at the root of the project and copy the following line into it.

#!/bin/bash
source /venv/bin/activate
cd /app

echo "----- Collect static files ------ " 
python manage.py collectstatic --noinput

echo "-----------Apply migration--------- "
python manage.py makemigrations 
python manage.py migrate

echo "-----------Run gunicorn--------- "
gunicorn -b :5000 myapp.wsgi:application

This shell script does the following:

  • activates the virtual environment inside the container
  • changes directory to the /app which is the main working directory of the application inside the container
  • It does collect static (we have not instructed it yet to serve static files in production) 
  • Makes the migrations inf there is any change to the models
  • Executes the migrations (migrate)
  • It will run the WSGI HTTP server using Gunicorn (more on it next)

if your application  needs more initialization, for example, it needs to copy some files from another server, you can instruct it to do so in this start.sh shell script. 

Gunicorn as WSGI HTTP Server

Django by default comes with a built-in development HTTP server. You run it on your local machine by commanding python manage.py runserver. It will serve the request to the users. But it has several drawbacks mainly, being single-threaded, being slow, and being insecure! So nobody should use it in production AT ALL. Instead, you should use Gunicorn. As it says on the Gunicorn website, the Gunicorn server is fast, compatible with various web frameworks, and light-weight in terms of server resources. We install Gunicorn using pip (included it in requirements.txt). The basic gunicorn command that we have included in the start.sh shell script is fine for most cases but these are few important configs that you may want to use:

# basic gunicorn command, binding to port 5000
gunicorn -b :5000 myapp.wsgi:application

# specify the number of workers with a positive integer. Generally, 4*(num cores) defaults to  1
gunicorn -b :5000 --workers INT myapp.wsgi:application

# Outputs the access log to a file named logfile. Use - to output to stdout
gunicorn -b :5000 --access-logfile logfile myapp.wsgi:application

# specify log level default is info. Options for LEVEL: debug, info, warning, error, critical
gunicorn -b :5000 --log-level LEVEL myapp.wsgi:application

You can learn more about Gunicorn configurations at Gunicorn documentation.

Push code to GitHub

Our demo website is ready now and we will push it to GitHub and continue the deployment of the Django application on Doprax. First, you need to create a repository on your GitHub account. Then you need to commit your code and push it to GitHub. My repository name is django-example. 

git init
git add .
git commit -m"initial commit"
git branch -M main
git remote add origin git@github.com:hemenxyz/django-example.git
git push -u origin main

Now you need to connect your GitHub account to doprax to continue. Go to the account section (https://www.doprax.com/account/) and click on the "connect to github" button.

Connect to GitHub

You will be directed to GitHub to authorize doprax to access your repositories. If you are not logged in, you will be prompted to log in to your account, and then you will be shown the authorization page. It will look like the image below. Click on authorize dopraxcom to continue.

GitHub authorization page

You can always manage authorization settings from your GitHub account settings section (https://github.com/settings/applications). Now your doprax account is connected to your GitHub account.

Create a project in doprax

Now you need to create a project. Get more information about doprax projects here. In your project detail page under the source sub-section, click on the import from my github account button. Then choose the repository of our demo Django application and then click import. 

Add MySQL service

Under the services sub-section, click on add service button. On the next page, click on MySQL. You need to provide 4 configuration values. 

  • MYSQL_ROOT_PASSWORD is the root password of your MySQL server
  • MYSQL_DATABASE is the name of the database we want to create. In our example, we have used mydb, so you need to enter mydb.
  • MYSQL_USER is a user that is created and will be assigned a super-user role for MYSQL_DATABASE mentioned above
  • MYSQL_PASSWORD is the password for MYSQL_USER mentioned above.

If you do no enter any of the above values, a default value will be generated for it automatically. For passwords, it will be a random password. If you remember from when we configured Django to use MySQL as the database server we need to create two environment variables one for MySQL host and another for the password of the root user. The hostname of the MySQL service can be found in the services sub-section under MySQL name. In the image below, it has been marked by a red rectangle. You need to find your own MySQL hostname. Don't copy mine! 

MySQL database server for Django deployment

 

Go to the source sub-section and create two environment variables:

  • Key: DB_PASS 
    • Value: the password you chose for the root user database (MYSQL_ROOT_PASSWORD) 
  • Key: DB_HOST
    • Value: the hostname of the Mysql service you just copied (red rectangle in the above image)  

Now we need to add a volume to our project to save MySQL data. Go to the volumes sub-section and click on create volume. Give it a title. Next, choose MySQL to mount on and for the mount path, enter /var/lib/mysql/ and then click create.

Add volume to mysql

Add Nginx service 

Nginx is a high-performance HTTP server and reverse-proxy used by many of the world's biggest deployments of Django (like Instagram) and other programming languages and frameworks. We will use Nginx to act as reverse-proxy for our application and also serve the static files of our application. Websites need to serve images, CSS files and javascript files, and other kinds of files static files. Django development server by default serves the static files but it is highly inefficient for production use. Django has a built-in utility called collectstatic that collects all the static files in one place (STATIC_ROOT) so that they could be served by the Nginx or Apache or other webservers. So far we have configured STATIC_ROOT in settings.py file and we have used collectstatic utility in start.sh shell script to make the static files ready to be served. Next, we are going to serve the static files using Nginx.

Go to the services sub-section and click on add service. On the next page, click Nginx and then click add. The Nginx service is now added to our project. Now we need to configure it to proxy requests to our Django application and also serve static files from our static root. To make our static files available to Nginx, we need to create a volume and mount it to the main container (source), and also share it with Nginx. 

Go to the volumes sub-section and click on create volume. Give it a title and then select Main to mount the volume on. For the path enter /app_data/static/ and then click create. 

add volume for static files

Now we need to share this volume with the Nginx service so that it could serve static files to our users. To do that click on the share button on the static volume that we just created. It has been marked by a red rectangle in the image below

Share static volume with nginx

You will then be prompted to choose the service to mount the volume on and also enter the path to mount the volume on the target service. Choose Nginx and enter /app_data/static/ exactly like mount path of static volume on main. Choose read-only as mount mode so that Nginx could only read from it and not write on it. Now we have a volume that has been shared with two services. 

Share volume with nginx 

Now it is time to config the Nginx service to handle static files and also proxy the requests to our gunicorn process in the main container. Go to the services sub-section and find Nginx service and add a mounted config file config:

Mount a config file to nginx

For the name of the config file enter nginx.conf and for the mount path enter /etc/nginx/nginx.conf and click create

create a mounted config file to nginx

Then click the edit button beside the name of the config file and you will be directed to the online editor.

Start editing the config file

On the editor page, on the left side of the screen click nginx.cof to open it. Copy the following configuration to this file and click save. DON'T forget to change the server_name and proxy_pass (hostname of the main source) items. Find server_name if deploy sub-section which is the URL of your project (omit HTTPS) and find proxy_pass also in deploy section. 

worker_processes 8;
user root;
error_log  /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {}
http {
    include mime.types;
    server {
        listen 80; 
        server_name django-examplelhqq.eu-ckhzajsnkv.doprax.com; # copy Your server name here
        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_pass http://yc2e1q5zz8aep:5000; # copy main hostname here
        }
        location /static/ {
            alias /app_data/static/;
        }
    }
}

Now the incoming requests will be proxied to the main source and static files will be directly served by Nginx. Now it is time to deploy the project. 

Deploy the Django project

Now open the deploy sub-section and you will see the schematic of all the components that we have so far created. It should look like this:

Deploy section for our Django project

Click the run button and the deployment is done. It will first build the source code and then it will launch the project. The services will be run one by one. The first time that you run the project you may encounter a problem with connectivity with Mysql since it may take up to a minute to initialize it. This problem is fixable by simply restarting the project. Also, by some shell script magic, you can make sure that the deployment procedure waits until the MySQL is up and running. 

Create superuser admin 

To access the admin interface of your project you need to create a superuser. And to do that you need shell access to the project. When the deployment procedure is finished and the status of the main source and all services turned running, you can access the shell by clicking on the Open Shell button. Let's do that. Click on the Open Shell button as it is shown in the image below:

open shell button

A new shell window will appear on the screen and you will be logged in as the root user. This is a pseudo-shell, and not a real ssh client since doprax does not run ssh client on your container code for security reasons. The current working directory is /app since it has been defined as such in our Dockerfile. Remember that our virtual environment path is /venv/ so we need to activate our virtual environment to be able to execute any Django commands. 

doprax shell

Now you can do everything that you need to do with the Django project including, making the migrations, do the migrations, create the superuser, and other commands. Let's create a superuser first.

source /venv/bin/activate
python manage.py createsuperuser

It will ask you to choose a username and enter your email address. Then it will ask you to set a password and repeat the password. Now you can use this username and password to log in to your admin page. Go to your_project_url/admin and login using these credentials. Django admin will look like this. (note that we have not created any models yet and items in the admin are default Django models.)

Django Admin interface

 

Django migrations

Django migrations are import part of the Django deployment cycle. Migrations keep models and the database in sync. It means that when you first create a model, it will create a table in the database according to the specification of your model. Then if you change your model in future, you can apply these changes to the database tables using migrations. To do the migrations, you need to access the shell and activate the virtual environment. Open the shell (described in the previous section) and enter the following to create the migrations files. 

source /venv/bin/activate
python manage.py makemigrations

This command will create migrations files inside the app folders. Now you can apply the migrations using the following command.

python manage.py migrate