A Production Webserver from Start to Finish with PostgreSQL, Django, and Apache

Tags: tech django webdev

I love the Django web framework. I find it to be extremely well designed and making changes to an existing server is typically a pretty pleasant experience. However setting up a new server can be very daunting for a number of reasons:

Django, while providing many sensible defaults, still has a lot of options that need to be configured
Answers to problems on stackoverflow can be for older versions of Django
Many problems you may run into while setting up a Django server will have wildly-varying community-suggested solutions but often many of them are quite "hacky" and can cause problems down the line.
Most complete tutorials are specifically for serving Django with nginx and gunicorn, Apache is less frequently utilized.

In this post I want to record the key points in the process of setting up a new server along with solutions to common issues that I personally find to be sensible and wont break other Django components (or at least the components I utilize). This isn't intended to be a 100% guide but should provide enough to get most of the way to the foundation of a full production server.

I want to focus on the steps that I found unclear or hard to find consistent answers for, since part of the reason this document exists is for my own reference. Even while writing this post I found myself referencing it for a project.

Setting up/installing/configuring the Ubuntu system will be considered outside the scope of this post.

To clarify, the tech stack for this server will be:

Component	Name	Version
Database	PostgreSQL	16.1
Server	Apache	2.4.57
Framework	Django	5.0
Operating System	Ubuntu Server	23.10

There are... quite a few steps.

Getting set up

Python

Ubuntu already comes with python3 (mine came with version 3.10), but I installed python3.10-venv in order to create the virtual environment that Django will run in. I also installed python-is-python3 so I don't have to write python3 in a world where Python 2 is being phased out, as well as python3-dev since that is needed for building mod_wsgi from source. In one command:

sudo apt install python3-dev python3-venv python-is-python3 -y

PostgreSQL

I setup PostgreSQL using the setup instructions.

A simple step that I always forget when setting up PostgreSQL for the first time is "how do I log in?". Its amazing how confusing this can be, with stackoverflow giving me a variety of answers, most of which don't work for this setup on Ubuntu. This is the command that worked for me to get into psql with user postgres:

sudo -u postgres psql

Create a database for the app:

CREATE DATABASE app_db_1;

Create a new role:

CREATE ROLE appuser LOGIN PASSWORD '<password>';

Create the PostgreSQL password file:

touch ~/.pgpass

I see many people online recommend placing the .pgpass file inside the project and then pointing to that, but that seems like a major security risk and an easy way to accidentally upload raw credentials to your version control system. Placing this in the home directory seems like the better choice.

Set permissions

chmod 0600 ~/.pgpass

Figuring out what to put for the values of the password file was hard to understand. What port should I use? Is localhost good enough? The documentation mentions that that the wildcard character * can be used for every field except password. So I opted for:

*:*:*:appuser:<password>

I also created a ~/.pg_service.conf file:

[appsvc]
host=localhost
user=appuser
dbname=app_db_1
port=5432

In order to avoid definition of service "<SERVICE>" not found errors, make sure to also set permissions on the pg_service.conf file:

chmod 0600 ~/.pg_service.conf

Now give this new appuser role full privileges over both databases:

PostgreSQL doesn't make this as easy as MySQL does. In PostgreSQL, default privileges have to be set per-schema for tables and sequences. In this case, we only have the default schema public:

\c app_db_1
ALTER DEFAULT PRIVILEGES FOR ROLE appuser IN SCHEMA public
GRANT ALL ON TABLES TO appuser;
ALTER DEFAULT PRIVILEGES FOR ROLE appuser IN SCHEMA public
GRANT ALL ON SEQUENCES TO appuser;
GRANT ALL ON SCHEMA public TO appuser;

If there were already tables present in the database, privileges could also be provided for those by running these two commands:

GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO appuser;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO appuser;

This really makes me miss MySQL's GRANT ALL PRIVILEGES ON database_name.* TO 'username'@'localhost'

Note: I see some discussion online where people say to give the user that Django uses SUPERUSER permissions but that seems like an unecessary level of permissions. Giving it full permissions exclusively over the database it uses seems like the better idea.

I can then log in to psql with the new user with:

psql -d app_db_1 -U appuser -h localhost

Observe the presence of the -h localhost command line argument. By default PostgreSQL will attempt to use peer authentication over the Unix socket (configured on Ubuntu in /etc/postgresql/<version number>/ph_hba.conf) which will fail with the message:

FATAL: Peer authentication failed for user "appuser"

Also observe that when the above login command is run - if the ~/.pgpass file was configured correctly - no password prompt was given and immediate log-in occurred.

Python/Django/DRF

Steps:

Set up a virtual environment with venv in the directory which stores the Django project:

python -m venv .venv

Activate it in the terminal with

source .venv/bin/activate

Ensure pip is up to date with:

python -m pip install --upgrade pip

Install Django (v5.0), the Django REST framework, and psycopg (for use with PostgreSQL) with:

python -m pip install django Pillow psycopg[binary]

Create a new project with

django-admin startproject demoproject

Test it out by entering demoproject and running the following command, and then navigate to localhost:8000 in a browser to see the test page

python manage.py runserver

Initial database sync:

python manage.py migrate

Apache

sudo apt install apache2 apache2-dev -y

I confirmed that the Apache2 Default Page came up when I entered localhost into my browser in the VM, and I saw the same page when I entered localhost:8080 into my browser on the host machine.

To set up mod_wsgi I followed these instructions to build and install from source since that is the link provided from the Django docs, but alternatively one could install it on Ubuntu with:

sudo apt install libapache2-mod-wsgi-py3 -y

I configured my installation to use the python binary in my virtual environment, not my globally installed Python instance.

To enable mod_wsgi, create a new file in /etc/apache2/mods-available called wsgi.load with the following content:

LoadModule wsgi_module /usr/lib/apache2/modules/mod_wsgi.so

Then run

sudo a2enmod wsgi
sudo systemctl restart apache2

Configuring Django to use PostgreSQL

Its time to start editing the settings.py file. I like to start by creation a production_settings.py and development_settings.py file in the same directory as settings.py so that they can be dynamically selected depending on the context that Django is running in, and copying changes to production becomes much simpler.

Ensure that DEBUG = False in production_settings.py.

I will edit DATABASES to match what appears in the Django documentation as:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "OPTIONS": {
            "service": "appsvc",
            "passfile": "/home/<user>/.pgpass",
        },
    }
}

That passfile entry as written will cause issues later that will be resolved.

Important: To allow for development and production settings when running via wsgi, the Python files need to be available in the path. Add this to the wsgi.py file:

import sys
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent
sys.path.append(str(BASE_DIR))

Note: in production, one complexity may be determining where to locate your credentials and service information on the server. I've opted for different solutions but I don't feel comfortable providing a clear cut solution because I haven't found one. In the past I've used credentials stored in /var/www but that has some serious security ramifications so I wouldn't recommend it. Credentials stored as an environment variable is another potential option (/etc/apache2/envvars).

Django app groundwork

I wont go into too much detail here as this isn't meant to be a tutorial on the use of Django, only its configuration. I created a new app with:

python manage.py startapp app1

I also add "app1" to the INSTALLED_APPS list in settings.py.

Create a superuser:

python manage.py createsuperuser --username admin --email admin@example.com

For testing a brand new, large, and complex database I like to make a custom django-admin command to wipe and fill out my database with fake/test data customized exactly how I want it.

Potential Issues

makemigrations error

When you initially run python manage.py makemigrations app1 you receive the following error message:

connection failed: fe_sendauth: no password supplied

I could not find documentation from Django mentioning this issue at all, and Google didn't help much either. It seems like Django - with the current DATABASES configuration in this article - cannot locate the .pgpass file. I looked through the Django source code to see why this might be. Django takes the provided passfile value in the DATABASES dict and uses it to set the PGPASSFILE environment variable. This variable is then provided to the env argument of a call to subprocess.run. My suspicion is that the path passed to env is never canonicalized at any point so the ~ in ~/.pgpass is never converted to something like /home/<user>.

On the plus side, postgreSQL already expects the .pgpass file to be in the user's home directory, so the easiest solution is to simply remove the passfile entry entirely.

So the new DATABASES value looks like:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "OPTIONS": {
            "service": "appsvc",
        },
    }
}