In Part I of this tutorial we built the working environment and created the TaskBuster project.
Now, we will configure the different environments for testing, developing and production, editing different Django settings files.
Moreover, we will remove Django’s SECRET_KEY from these files in order to keep it Secret.
Next, we will create a new repository to keep our code in version control and uploaded it into Bitbucket.
Ready for the next part? Here’s the guideline:
- Virtual environments and requirements files
- Different settings.py for each enviroment
- Production settings.py – Debug False
- Django security and the Secret Key
- Initialize a Git repository and Commit
- Upload your project into Bitbucket
Virtual Environments and Requirements files
One important thing when working on a Project, is to control the version of your packages. For example, imagine that you’re developing in one computer that has Django 1.8 installed, and you are deploying in one server that has an older version of Django, let’s say 1.6. Your code works fine locally, but when deploying it, some incompatible errors may occur. And the same could happen if more than one developer is working in the same project, each of them with its own package versions installed.
The standard solution to this problem is to unify all the packages and save the versions used in a file named requirements.txt. This file will contain something like:
Django==1.8.5 selenium==2.48.0
which are the packages we have installed so far (don’t worry if the versions of your environment are different).
You can see the packages installed in a virtual environment by typing
$ pip freeze
You might see another package, wheel, that is installed by default in some versions (don’t worry if you don’t have it). Therefore, you can create automatically the requirements.txt file by saving the output of the previous command into the file:
$ pip freeze > requirements.txt
However, as you may notice, Selenium is only needed for the testing environment, so there is no need that the developing or the production environments have this package installed.
Let’s work out this issue by creating a requirements folder and creating a file for each environment. Go inside the taskbuster_project folder and type:
$ mkdir requirements
$ touch requirements/{base.txt,development.txt,production.txt,testing.txt}
Note: don’t add any extra spaces in the previous command or it won’t work 😉 And you can delete the previous requirements.txt file, as we won’t need it. Moreover, you can also define a staging.txt file if you are planning to run a semi-private version of your site on a production server.
Let’s edit first the file base.txt. This file will contain all the packages that are common for all the environments. Now, it should only contain the Django version:
$ cd requirements $ echo "Django==1.8.5" >> base.txt
If you have another version, write yours instead!
Now let’s make the three other files to inherit the packages of the common.txt file.
$ echo "-r base.txt" | tee -a development.txt testing.txt production.txt
Finally, we need to add Selenium into the testing file:
$ echo "selenium==2.48.0" >> testing.txt
again, write your version here.
Ok, we have everything ready now. When a new programmer joins our team, we will tell him to create two different environments, one for testing and another for developing (the production environment is for deployment).
Next, he only needs to activate each of these environments and install the packages saved in each requirements files:
$ workon tb_dev $ pip install -r requirements/development.txt $ workon tb_test $ pip install -r requirements/testing.txt
Different settings.py for each environment
Each environment defined previously has a different purpose, and therefore, they will need different configurations. For example, the database configuration for production and developing might be different, or the testing environment might use some Django apps that are not needed in the other environments (like selenium).
That’s why we will specify different setting files for each environment. First, we will create a folder to contain our setting files, inside the taskbuster folder:
$ mkdir taskbuster/settings
This folder will contain:
- __init__.py file to make this folder a Python package
- base.py will contain all the settings that are common in all environments. The other setting files inherit from this one.
- development.py is for local development.
- testing.py is for testing.
- production.py will be used in the production environment.
- staging.py if you want to run a staging version on the production server of your project.
Let’s create these files, all inside the taskbuster/settings folder:
$ cd taskbuster/settings $ touch __init__.py development.py testing.py production.py staging.py
And edit each of them (development.py, testing.py, production.py and staging.py) to make them inherit from the base.py file — we’ll create this file in a second :-):
# -*- coding: utf-8 -*- from .base import *
And finally, we have to move and rename the settings.py file created by Django, to be our base.py file inside the settings folder. Working inside the settings folder, you have to type:
$ mv ../settings.py base.py
After creating these files, we need to specify the virtual environment to work with the correct setting file.
With virtualenvwrapper, there is a way to configure different hooks that are sourced before or after activating the virtual environment, and before or after deactivating it. This means that we can define a set of statements that will be run at different stages of the virtual environment lifecycle. These hooks are saved inside the bin folder inside the virtual environment folder, and their names are preactivate, postactivate, predeactivate and postdeactivate.
In our case, we will set a postactivate script that will set the DJANGO_SETTINGS_MODULE variable just after activating the virtual environment, and a predeactivate that will clean it up before deactivating it.
$ workon tb_dev $ cd $VIRTUAL_ENV/bin
The last command will take you to the virtual environment folder, where the different hooks reside. Edit the postactivate file by adding:
export DJANGO_SETTINGS_MODULE="taskbuster.settings.development"
and edit the predeactivate file by adding:
unset DJANGO_SETTINGS_MODULE
Do the same with the testing environment, with the only change:
export DJANGO_SETTINGS_MODULE="taskbuster.settings.testing"
It’s time to check! Go back to the taskbuster_project folder and activate your development environment. Next, run:
$ python manage.py runserver
and in the output displayed you should see a line indicating that you are using the taskbuster.development_settings file, something like:
Django version 1.8, using settings 'taskbuster.settings.development'
Open another tab (and leave the previous server active), and activate the testing environment. Check that it’s using the taskbuster.testing_settings file. Probably it will complain saying that that port is already in use by the development environment, so specify another port with
$ python manage.py runserver 127.0.0.1:8001
Next, quit the server in the testing environment and run the functional test:
$ python functional_tests/all_users.py
Yes! We didn’t break anything 🙂
Production Settings – Debug False
One important thing to remember is to set the variable DEBUG to False in your production settings files.
Note: Previous to Django 1.8 you also need to set TEMPLATE_DEBUG to false. However, with the introduction of the new TEMPLATE setting, the old TEMPLATE_DEBUG is set automatically equal to DEBUG. If you really want to specify a value, check the official documentation.
First, cut the DEBUG variable from the base.py settings file and copy it into the development.py and testing.py settings files:
DEBUG = True
Next, add it on the production.py setting file and make it False:
DEBUG = False
This way, each environment will have the correct value of this variable. If you’ve also defined a staging.py file, copy it there too.
Django security and the Secret Key
If you open the file taskbuster/settings/base.py you will see a variable named SECRET_KEY . This variable should be kept secret, and therefore out of version control.
One option would be to add the base.py file into the .gitignore file, that is, remove it from the version control. However, during project development this file suffers many changes, and it’s quite useful to have it in version control, specially if you want to share it with your coworkers. Therefore, a better approach is to remove the secret key variable and import it from somewhere else. And this somewhere else is the one that should remain out of version control.
The approach we will follow here is to put the secret key inside our virtual environment configuration, and get the key from the environment by importing it in the base.py file.
Note: If you’re using Apache this method won’t work. The best option is that you save your SECRET_KEY in some file, and import it into the base.py file. The key file should be removed from version control by including it in the .gitignore file. I recommend you read the Two Scoops of Django 1.6 book, section 5.4. (You might want to check they new version, updated for Django 1.8!)
To include the secret key inside the virtual environment we will also work with the virtualenvwrapper’s postactivate and a predeactivate hooks.
Activate your tb_dev environment and go to its bin folder using the shortcut
$ cd $VIRTUAL_ENV/bin
If you type ls you will see that it contains the files we just described. Edit the postactivate file and add the secret key line
export SECRET_KEY="your_secret_django_key"
Note: don’t put any spaces around the = sign.
Next edit the predeactivate file and add the line:
unset SECRET_KEY
This way, if you type:
$ workon tb_dev $ echo $SECRET_KEY your_secret_django_key $ deactivate $ echo $SECRET_KEY -
Where the last line indicates that there is no output. This means that the variable SECRET_KEY is only visible when working in this virtual environment, as we wanted.
Repeat the same process for the tb_test virtual environment.
Next, deactivate and activate each environment to make these changes effective.
And finally, edit the base.py file, remove the SECRET_KEY and add the following lines:
from django.core.exceptions import ImproperlyConfigured
def get_env_variable(var_name):
try:
return os.environ[var_name]
except KeyError:
error_msg = "Set the %s environment variable" % var_name
raise ImproperlyConfigured(error_msg)
SECRET_KEY = get_env_variable('SECRET_KEY')
The function get_env_variable tries to get the variable var_name from the environment, and if it doesn’t find it, it raises an ImproperlyConfigured error. This way, when you try to run your app and the SECRET_KEY variable is not found, we will be able to see a message indicating why our project fails.
Let’s check that it all works as expected. Save the base.py, deactivate both environments and activate them again, in different terminal tabs.
Run the development server in the tb_dev environment
$ python manage.py runserver
and run the functional test in the tb_test environment
$ python functional_tests/all_users.py
Hope the test also works for you!! 🙂
Note: When deploying your app, you will have to specify the SECRET_KEY in your server. For example, if you are using Heroku, you can use:
$ heroku config:set SECRET_KEY="your_secret_key"
But don’t worry, we’ll cover Heroku latter in this tutorial!! 🙂
Initialize a Git Repository and Commit
Ok! now we are ready to commit our project into a new repository! Note: you can read a basic git tutorial here.
Go into the taskbuster_project directory and type:
$ git init .
to initialize the repository in the current folder. You will see a new folder named .git, containing the new repository.
Before adding files into the repository, we have to think if there are files that we want to keep away from version control.
Note that after running the development server, we have the extra files:
- db.sqlite3 – a database
- __pycache__ – a folder containing all the *.pyc files.
These two should be removed from version control. Create a .gitignore file inside the taskbuster_project foler and write:
db.sqlite3 __pycache__ TaskBuster.sublime-workspace
Next, let’s add all the files of the current directory into the staging area (except the ones in the .gitignore file):
$ git add .
And check the files added into the staging area with:
$ git status
You should see something like this, with all the new files added in green:
If you see some file that you don’t want to commit, you can remove it using
$ git rm --cached path_of_file
don’t forget to add it into the .gitignore file for subsequent commits.
Finally, let’s commit our changes:
$ git commit -m "Taskbuster project created"
The -m flag indicates that the following text will be used to describe this commit. If you simply type git commit, an editor will open to write the description of the commit (by default, this editor is VI).
You can see the commit made with
$ git log
Upload your project into Bitbucket
Create a Bitbucket account if you don’t have one, and create a new empty repository. We use Bitbucket because it allows for a private repository, but the steps described here will work almost equal with GitHub.
You will have to determine the URL of this repository, which will be something like
https://user_name@bitbucket.org/user_name/repository_name.git
You can find it in the Overview –> Command line –> I have an existing project.
First we need to add Bitbucket as a remote repository. Go to the taskbuster_project folder and type:
$ git remote add origin https://user_name@bitbucket.org/user_name/repository_name.git
where you should change the url with your repository url. Note: The previous command is a single line. This will create the alias origin to refer to your Bitbucket repository (using origin as an alias for a remote repository is a common convention).
Next, let’s push our existing repository into this new Bitbucket repository with:
$ git push -u origin --all
where the –all flag makes all the refs under refs/heads to be pushed, and the -u flag stands for –set-upstream (add a tracking reference). You will have to enter your password.
At the end, you will see the message:
Branch master set up to track remote branch master from origin.
You can see your active branches with
$ git branch -a * master remotes/origin/master
Okey! Now that we have our first project with a good working environment, and in version control, we can work on our Home Page!
But… I’m not talking about creating models yet, just configure the static files and templates to have a nice Home Page with some CSS — I hate developing without a basic CSS, so it’s one of the first things I usually include.
Find out this and more in the next part of the tutorial! Create a Home Page with TDD, Staticfiles and Templates settings
Please, share this Tutorial with your developer friends! 😉