Connecting Django to AWS RDS

This blog post is part of a series about how to deploy a Django app to AWS. If you haven't deployed your Django app to EC2 and S3 yet, I recommend reading the AWS EC2 and AWS Django posts first.

Databases contain the long-term memory of your app server. Out-of-the-box, Django gets you started with a SQLite database, which works great for local development and small web apps. However, when it's time to scale up your app, you may find yourself wondering whether it's time to move your database to a dedicated machine for database operations like AWS RDS.

How do you know when it's time to move to a dedicated database architecture? Well, if your website is receiving more than 5-10 simultaneous visitors, our single EC2 machine may struggle to keep up since it's balancing serving our web app with database operations. If that's the case, the solution is to offload your database operations to a dedicated machine. This architecture also enables scaling to multiple web servers in the future, where we might have multiple web servers talking to a centralized database which securely hosts our user data. In this blog post, we'll walk through scaling up from our SQLite database to a PostgreSQL database hosted in RDS.

1. Navigate to the AWS RDS console and select Create database.
2. Select:
   • Engine options: PostgreSQL
   • Master username: postgres
   • Templates: Free tier
   • DB instance identifier: my-database-name
   • Master username: postgres
   • Master password: some_secure_password
   • Connectivity: Don't connect to an EC2 compute resource
   • Public access: Yes (just for testing—we'll fix this later)
   • Database authentication: Password authentication
   • Additional configuration: Initial database name: tutorial_db
3. Select Create database.
4. Once created, copy the endpoint hostname (ends in.rds.amazonaws.com) and port number (usually 5432) from the console and store them somewhere safely. It's bad practice to store them in code, so we'll store them in environment variables in our EC2 instance for now:

   export RDS_NAME="my-database-name"
   export RDS_USER="postgres"
   export RDS_PASSWORD="some_secure_password"
   export RDS_HOST="<your hostname>"
   export RDS_PORT="5432"
   

Next, we need to edit the security rules for this database to allow incoming connections from our EC2 machine.

5. From the RDS console, select the security group corresponding to your RDS instance.
6. Select Edit inbound rules.
7. Add a rule that allows connections of type PostgreSQL from the security group of your EC2 instance.
8. Add another rule that allows connections of type SS from the security group of your EC2 instance.
9. Select Save.

Great! Now our RDS instance is setup. It has no tables or data yet, but we'll fix that soon.

Ideally, we want to continue using SQLite while developing our app locally but point to RDS in production. We can enable this in our settings.py file by replacing the default values for DATABASES with:

...

if DEBUG:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': BASE_DIR / 'db.sqlite3',
        }
    }
else:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql',
            'NAME': os.environ['RDS_NAME'],
            'USER': os.environ['RDS_USER'],
            'PASSWORD': os.environ['RDS_PASSWORD'],
            'HOST': os.environ['RDS_HOST'],
            'PORT': os.environ['RDS_PORT'],
        }
    }
...

This code will read values from the environment variables we created earlier and use them to connect to our RDS machine in production. Next, on your server, run:

pip install psycopg2 # or psycopg2-binary

We can now test the connection is working by using:

python manage.py dbshell

If everything worked, our EC2 machine should connect to RDS and we should get a PostgreSQL prompt allowing us to query the newly created database. But hang on, there's no tables or data yet!

Now that everything's connected, the final step is initializing our database with our tables and data. If we want to start fresh with an empty tables, simply run:

python manage.py migrate

However, if we want to bring data over from our old SQLite database, run these commands instead:

1. Temporarily point the code to the old SQLite database (e.g. by fudging the if statement we made in settings.py). Then, run:

   python manage.py dumpdata > data.json

2. Now, point the code back to the new RDS instance. Then, run:

   python manage.py migrate --run-syncdb # Creates all the tables without filling out the django_migrations table

3. We also need to manually delete some default data in our newly created RDS instance. Run:

   python manage.py shell
   > from django.contrib.conttentypes.models import ContentType
   > ContentType.objects.all().delete()
   

4. Finally, we can load the data from our data.json file into our new database. From the terminal, run:

   python manage.py loaddata data.json
   

To confirm our data has made it into the database, we can run:

python manage.py dbshell> dt

If all is well, we should get a list of all our tables. Feel free to query them using SQL statements to confirm all your data is there, such as: select * from django_migrations;

If everything looks good, you're done with your database migration. From here, we can make further optimizations in the future, such as using a read replica for read-heavy operations or using a multi-AZ setup for high availability, but those are reserved for another blog post. Congratulations, and happy coding!