Setting up Django with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL

Django is an efficient, versatile and dynamically evolving web application development framework. When Django initially gained popularity, the recommended setup for running Django applications was based around Apache with mod_wsgi. The art of running Django advanced and these days the recommended configuration is more efficient and resilient, but also more complex and includes such tools as: Nginx, Gunicorn, virtualenv, supervisord and PostgreSQL.

In this text I will explain how to combine all of these components into a Django server running on Linux.

Prerequisites

I assume you have a server available on which you have root privileges. I am using a server running Debian 7, so everything here should also work on an Ubuntu server or other Debian-based distribution. If you’re using an RPM-based distro (such as CentOS), you will need to replace the apt-get commands by their yum counterparts and if you’re using FreeBSD you can install the components from ports.

If you don’t have a server to play with, I would recommend the inexpensive VPS servers offered by Digital Ocean. If you click through this link when signing up, you’ll pay a bit of my server bill :)

I’m also assuming you configured your DNS to point a domain at the server’s IP. In this text, I pretend your domain is example.com

Update your system

Let’s get started by making sure our system is up to date.

$ sudo apt-get update
$ sudo apt-get upgrade

PostgreSQL

To install PostgreSQL on a Debian-based system run this command:

$ sudo apt-get install postgresql postgresql-contrib

Create a database and user with privileges.

$ sudo su - postgres
postgres@django:~$ createdb hello
postgres@django:~$ createuser -P
Enter name of role to add: hello_django
Enter password for new role: 
Enter it again: 
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
postgres@django:~$ psql
psql (9.1.9)
Type "help" for help.

postgres=# GRANT ALL PRIVILEGES ON DATABASE hello TO hello_django;
GRANT
postgres=# \q
postgres@django:~$ logout
$

Install virtualenv and create an environment for you app

Virtualenv is a tool which allows you to create separate Python environments on your system. This allows you to run applications with different sets of requirements concurrently (e.g. one based on Django 1.5, another based on 1.6). virtualenv is easy to install on Debian:

$ sudo apt-get install python-virtualenv

Create and activate an environment for your application

I like to keep all my web apps in the /webapps/ directory. If you prefer /var/www/ or something else, use that instead.

$ cd /webapps/
$ virtualenv hello_django

New python executable in hello_django/bin/python
Installing distribute..............done.
Installing pip.....................done.

$ cd hello_django
$ source bin/activate
(hello_django) $ 

Your environment is now activated and you can proceed to install Django inside it.

(hello_django) $ pip install django

Downloading/unpacking django
(...)
Installing collected packages: django
(...)
Successfully installed django
Cleaning up...

Your environment with Django should be ready to use. Go ahead and create an empty Django project.

(hello_django) $ django-admin.py startproject hello

You can test it by running the development server:

(hello_django) $ cd hello
(hello_django) $ python manage.py runserver example.com:8000
Validating models...

0 errors found
June 09, 2013 - 06:12:00
Django version 1.5.1, using settings 'hello.settings'
Development server is running at http://example.com:8000/
Quit the server with CONTROL-C.

You should now be able to access your development server from http://example.com:8000

Configure PostgreSQL to work with Django

In order to use Django with PostgreSQL you will need to install the psycopg2 database adapter in your virtual environment. This step requires the compilation of a native extension (written in C). The compilation will fail if it cannot find header files and static libraries required for linking C programs with libpq (library for communication with Postgres) and building Python modules (python-dev package). We have to install these two packages first, then we can install psycopg2 using PIP.

Install dependencies:

$ sudo apt-get install libpq-dev python-dev

Install psycopg2 database adapter:

(hello_django) $ pip install psycopg2

You can now configure the databases section in your settings.py:

1
2
3
4
5
6
7
8
9
10

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'hello',
        'USER': 'hello_django',
        'PASSWORD': '1Ak5RTQt7mtw0OREsfPhJYzXIak41gnrm5NWYEosCeIduJck10awIzoys1wvbL8',
        'HOST': 'localhost',
        'PORT': '',                      # Set to empty string for default.
    }
}

And finally build the initial database for Django:

(hello_django) $ ./manage.py syncdb

Gunicorn

In production we won’t be using Django’s single-threaded development server, but a dedicated application server called gunicorn.

Install gunicorn in your application’s virtual environment:

(hello_django) $ pip install gunicorn
Downloading/unpacking gunicorn
  Downloading gunicorn-0.17.4.tar.gz (372Kb): 372Kb downloaded
  Running setup.py egg_info for package gunicorn

Installing collected packages: gunicorn
  Running setup.py install for gunicorn

    Installing gunicorn_paster script to /webapps/hello_django/bin
    Installing gunicorn script to /webapps/hello_django/bin
    Installing gunicorn_django script to /webapps/hello_django/bin
Successfully installed gunicorn
Cleaning up...

Now that you have gunicorn, you can test whether it can serve your Django application by running the following command:

(hello_django) $ gunicorn_django --bind example.com:8001

You should now be able to access the Gunicorn server from http://example.com:8001 . I intentionally changed port 8000 to 8001 to force your browser to establish a new connection.

Gunicorn is installed and ready to serve your app. Let’s set some configuration options to make it more useful. I like to set a number of parameters, so let’s put them all into a small BASH script, which I save as bin/gunicorn_start

#!/bin/bash
NAME="hello_app"                                  # Name of the application
DJANGODIR=/webapps/hello_django/hello             # Django project directory
SOCKFILE=/webapps/hello_django/run/gunicorn.sock  # we will communicte using this unix socket
USER=michal                                       # the user to run as
GROUP=michal                                      # the group to run as
NUM_WORKERS=3                                     # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=hello.settings             # which settings file should Django use
echo "Starting $NAME"
# Activate the virtual environment
cd $DJANGODIR
source ../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec ../bin/gunicorn_django \
  --name $NAME \
  --workers $NUM_WORKERS \
  --user=$USER --group=$GROUP \
  --log-level=debug \
  --bind=unix:$SOCKFILE

Set the executable bit on the gunicorn_start script:

$ chmod u+x bin/gunicorn_start

You can test your gunicorn_start script by running it:

$ ./bin/gunicorn_start 
Starting hello_app
2013-06-09 21:14:07 [2792] [INFO] Starting gunicorn 0.17.4
2013-06-09 21:14:07 [2792] [DEBUG] Arbiter booted
2013-06-09 21:14:07 [2792] [INFO] Listening at: unix:/webapps/hello_django/run/gunicorn.sock (2792)
2013-06-09 21:14:07 [2792] [INFO] Using worker: sync
2013-06-09 21:14:07 [2798] [INFO] Booting worker with pid: 2798
2013-06-09 21:14:07 [2799] [INFO] Booting worker with pid: 2799
2013-06-09 21:14:07 [2800] [INFO] Booting worker with pid: 2800

Note the parameters set in gunicorn_start. You’ll need to set the paths and filenames to match your setup.

As a rule-of-thumb set the --workers (NUM_WORKERS) according to the following formula: 2 * CPUs + 1. The idea being, that at any given time half of your workers will be busy doing I/O. For a single CPU machine it would give you 3.

The --name (NAME) argument specifies how your application will identify itself in programs such as top or ps. It defaults to gunicorn, which might make it harder to distinguish from other apps if you have multiple Gunicorn-powered applications running on the same server.

In order for the --name argument to have an effect you need to install a Python module called setproctitle. To build this native extension pip needs to have access to C header files for Python. You can add them to your system with the python-dev package and then install setproctitle.

$ sudo apt-get install python-dev
(hello_django) $ pip install setproctitle

Now when you list processes, you should see which gunicorn belongs to which application.

$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
(...)
michal   16124  0.0  1.9  56168  9860 ?        S    15:37   0:00 gunicorn: master [hello_app]
michal   16130  0.0  4.5  73520 23004 ?        S    15:37   0:00 gunicorn: worker [hello_app]
michal   16131  0.0  4.5  73496 23004 ?        S    15:37   0:00 gunicorn: worker [hello_app]
michal   16132  0.0  4.5  73504 23004 ?        S    15:37   0:00 gunicorn: worker [hello_app]

Starting and monitoring with Supervisor

Your gunicorn_start script should now be ready and working. We need to make sure that it starts automatically with the system and that it can automatically restart if for some reason it exits unexpectedly. These tasks can easily be handled by a service called supervisord. Installation is simple:

$ sudo apt-get install supervisor

When Supervisor is installed you can give it programs to start and watch by creating configuration files in the /etc/supervisor/conf.d directory. For our hello application we’ll create a file named /etc/supervisor/conf.d/hello.conf with this content:

[program:hello]
command = /webapps/hello_django/bin/gunicorn_start                    ; Command to start app
user = michal                                                         ; User to run as
stdout_logfile = /webapps/hello_django/logs/gunicorn_supervisor.log   ; Where to write log messages
redirect_stderr = true                                                ; Save stderr in the same log

You can set many other options, but this basic configuration should suffice.

After you save the configuration file for your program you can ask supervisor to reread configuration files and update (which will start your the newly registered app).

$ sudo supervisorctl reread
hello: available
$ sudo supervisorctl update
hello: added process group

You can also check the status of your app or start, stop or restart it using supervisor.

$ sudo supervisorctl status hello                       
hello                            RUNNING    pid 18020, uptime 0:00:50
$ sudo supervisorctl stop hello  
hello: stopped
$ sudo supervisorctl start hello                        
hello: started
$ sudo supervisorctl restart hello 
hello: stopped
hello: started

Your application should now be automatically started after a system reboot and automatically restarted if it ever crashed for some reason.

Nginx

Time to set up Nginx as a server for out application and its static files. Install and start Nginx:

$ sudo apt-get install nginx
$ sudo service nginx start

You can navigate to your server (http://example.com) with your browser and Nginx should greet you with the words “Welcome to nginx!”.

Create an Nginx virtual server configuration for Django

Each Nginx virtual server should be described by a file in the /etc/nginx/sites-available directory. You select which sites you want to enable by making symbolic links to those in the /etc/nginx/sites-enabled directory.

Create a new nginx server configuration file for your Django application running on example.com in /etc/nginx/sites-available/hello. The file should contain something along the following lines. A more detailed example is available from the folks who make Gunicorn.

upstream hello_app_server {
  # fail_timeout=0 means we always retry an upstream even if it failed
  # to return a good HTTP response (in case the Unicorn master nukes a
  # single worker for timing out).
  server unix:/webapps/hello_django/run/gunicorn.sock fail_timeout=0;
}
server {
    listen   80;
    server_name example.com;
    client_max_body_size 4G;
    access_log /webapps/hello_django/logs/nginx-access.log;
    error_log /webapps/hello_django/logs/nginx-error.log;
    location /static/ {
        alias   /webapps/hello_django/static/;
    }
    location /media/ {
        alias   /webapps/hello_django/media/;
    }
    location / {
        # an HTTP header important enough to have its own Wikipedia entry:
        #   http://en.wikipedia.org/wiki/X-Forwarded-For
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # enable this if and only if you use HTTPS, this helps Rack
        # set the proper protocol for doing redirects:
        # proxy_set_header X-Forwarded-Proto https;
        # pass the Host: header from the client right along so redirects
        # can be set properly within the Rack application
        proxy_set_header Host $http_host;
        # we don't want nginx trying to do something clever with
        # redirects, we set the Host: header above already.
        proxy_redirect off;
        # set "proxy_buffering off" *only* for Rainbows! when doing
        # Comet/long-poll stuff.  It's also safe to set if you're
        # using only serving fast clients with Unicorn + nginx.
        # Otherwise you _want_ nginx to buffer responses to slow
        # clients, really.
        # proxy_buffering off;
        # Try to serve static files from nginx, no point in making an
        # *application* server like Unicorn/Rainbows! serve static files.
        if (!-f $request_filename) {
            proxy_pass http://hello_app_server;
            break;
        }
    }
    # Error pages
    error_page 500 502 503 504 /500.html;
    location = /500.html {
        root /webapps/hello_django/static/;
    }
}

Restart Nginx:

$ sudo service nginx restart 

If you navigate to your site, you should now see your Django welcome-page powered by Nginx and Gunicorn. Go ahead and develop to your heart’s content.

If you run into any problems with the above setup, please drop me a line.

Final directory structure

If you followed this tutorial, you should have created a directory structure resembling this:

/webapps/hello_django/
├── bin                          <= Directory created by virtualenv
│   ├── activate                 <= Environment activation script
│   ├── django-admin.py
│   ├── gunicorn
│   ├── gunicorn_django
│   ├── gunicorn_start           <= Script to start application with Gunicorn
│   └── python
├── hello                        <= Django project directory, add this to PYTHONPATH
│   ├── manage.py
│   ├── project_application_1
│   ├── project_application_2
│   └── hello                    <= Project settings directory
│       ├── __init__.py
│       ├── settings.py          <= hello.settings - settings module Gunicorn will use
│       ├── urls.py
│       └── wsgi.py
├── include
│   └── python2.7 -> /usr/include/python2.7
├── lib
│   └── python2.7
├── lib64 -> /webapps/proxydemo/lib
├── logs                         <= Application logs directory
│   ├── gunicorn_supervisor.log
│   ├── nginx-access.log
│   └── nginx-error.log
├── media                        <= User uploaded files folder
├── run
│   └── gunicorn.sock 
└── static                       <= Collect and serve static files from here

Uninstalling the Django application

If time comes to remove the application, follow these steps.

Remove the virtual server from Nginx sites-enabled folder:

$ sudo rm /etc/nginx/sites-enabled/hello_django

Restart Nginx:

$ sudo service nginx restart 

If you never plan to use this application again, you can remove its config file also from the sites-available directory

$ sudo rm /etc/nginx/sites-available/hello_django

Stop the application with Supervisor:

$ sudo supervisorctl stop hello

Remove the application from Supervisor’s control scripts directory:

$ sudo rm /etc/supervisor/conf.d/hello.conf

If you never plan to use this application again, you can now remove its entire directory from webapps:

    $ sudo rm -r /webapps/hello_django

::...

免责声明:


当前网页内容, 由
大妈 ZoomQuiet
使用工具:
ScrapBook :: Firefox Extension
人工从互联网中收集并分享;



内容版权归原作者所有;


本人对内容的有效性/合法性不承担任何强制性责任.


若有不妥, 欢迎评注提醒:









    或是邮件反馈可也:


    
    askdama[AT]googlegroups.com
    






订阅 substack 体验古早写作:
    







点击注册~> 获得 100$ 体验券:





关注公众号, 持续获得相关各种嗯哼:
    








自怼圈/年度番新



关于 ~ DebugUself with DAMA ;-)








    粤ICP备18025058号-1



公安备案号:

    44049002000656



...::