Server Architecture and configurations

Third Party Services

Following third-party services are required in order to setup/deploy this project successfully.

Heroku

Heroku is platform as a service provider. We use to host the primarily web server along with different services required by this project like postgres database, newrelic, redis. See getting started docs here, you’ll require to create an account and install the cli-tool to successfully deploy this project.

Note: Alternatively, you should be able configure a linux instance to run the same project as well, heroku like settings can be added via .env file. (refer: settings/common.py). Just that, this documentation and project is focused more with Heroku as platform of choice.

Amazon S3

Amazon Simple Storage Service (Amazon S3) is used to store the uploaded media files and static content. It is a scalable and cost-efficient storage solution.

After signing up for Amazon S3, setup an IAM user with access to a S3 bucket, you’ll need BUCKET_NAME, and AWS_ACCESS_ID & AWS_ACCESS_SECRET of IAM user to setup the project.

Note: - Heroku doesn’t provide a persistent storage for uploaded content, best practise is to store the uploaded files in S3 buckets. - IAM user must have permission to list, update, create objects in S3.

Deploying Project

The deployment are managed via travis, but for the first time you’ll need to set the configuration values on each of the server. Read this only, if you need to deploy for the first time.

Heroku

Run these commands to deploy this project on Heroku

heroku create --ssh-git deepositive-workshop

heroku addons:create heroku-postgresql --app=deepositive-workshop
heroku pg:backups schedule DATABASE_URL --at '04:00 UTC' --app=deepositive-workshop
heroku pg:promote DATABASE_URL --app=deepositive-workshop

heroku addons:create mailgun --app=deepositive-workshop
heroku config:set EMAIL_HOST="\$MAILGUN_SMTP_SERVER" \
                  EMAIL_HOST_USER="\$MAILGUN_SMTP_LOGIN" \
                  EMAIL_HOST_PASSWORD="\$MAILGUN_SMTP_PASSWORD" --app=deepositive-workshop

heroku addons:create redistogo --app=deepositive-workshop
heroku addons:create redismonitor --url `heroku config:get REDISTOGO_URL --app=deepositive-workshop` --app=deepositive-workshop

heroku addons:create newrelic --app=deepositive-workshop
heroku config:set NEW_RELIC_APP_NAME=deepositive-w --app=deepositive-workshop

heroku config:set DJANGO_SETTINGS_MODULE='settings.production' \
DJANGO_SECRET_KEY=`openssl rand -base64 32` \
SITE_DOMAIN=deepositive-workshop.herokuapp.com \
SITE_SCHEME=https \
SITE_NAME=DJANGO_SITE_NAME_HERE --app=deepositive-workshop

git push heroku master
heroku run python manage.py createsuperuser --app=deepositive-workshop
heroku open --app=deepositive-workshop

The following configuration doesn’t allow you to “by default” upload the media on the heroku server as heroku does not support persistent storage. We use S3 for storing uploaded media. If you want to enable media upload:

Create S3 bucket and get AWS access key and secret that has access to this bucket.
Follow the instructions below to enable S3 upload configuration on heroku.

heroku config:set ENABLE_MEDIA_UPLOAD_TO_S3=true \
DJANGO_AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_ID_HERE> \
DJANGO_AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_KEY_HERE> \
DJANGO_AWS_STORAGE_BUCKET_NAME=<YOUR_BUCKET_NAME_HERE>

Note: - Use --app=deepositive-workshop if you have more than one heroku app configured in current project. - Update travis.yml, and add the deepositive-workshop to automatically deploy to this configured heroku app.

AWS/EC2

For deploying on aws you need to configure all the addons provided and use python-dotenv to store and read enironment variables.

Add the following to your ~/.ssh/config file.

Host workshop.com
    hostname <server_ip_or_>
    user ubuntu
    ForwardAgent yes
    identityfile <PATH_OF_SERVER_PRIVATE_KEY_HERE>

Add your github private key to your local ssh-agent, which will be used by ansible on remote server to fetch the code using ForwardAgent

ssh-add <PATH_TO_YOUR_GITHUB_PRIVATE_KEY>

Now you can run the ansible script to setup the machine.

fab prod configure

This will setup os dependencies, services like supervisor, nginx and fetch our code from github. Our production environment requires some environment variables in .env. So you can write a file prod.env locally and upload it to server with

scp prod.env workshop.com:/home/ubuntu/workshop-web/.env

You can also use fab to set environment variables one by one:

fab prod config:set,<VAR_NAME>,<VAR_VALUE>

Now that you have .env setup, you can deploy your code and start services:

fab prod deploy