CircleCI + AWS

= Happy Deployment
😃 🚀

Follow Along

Russell Heimlich
(like the maneuver)

Lead Developer at Spirited Media

We build software!

We need to get that software to other places

But we don't want to break things

Continuous Integration / Continuous Deployment

• Automated
• Predictable
• Stops if something breaks

Deployment Should Be Boring

What does CircleCI do?

• Listens for changes to a GitHub/Bitbucket repo
• Does stuff we tell it to do

Add a .circleci/config.yml file to the root of your project

Steps in the build process are called jobs

One or more jobs makeup a workflow

Workflows can run sequentially or in parallel
(fan in/out)

Our build process is straightforward

• Setup the build enviornment
• Download dependencies
• Compile static assets (SCSS → CSS, minifying JavaScript files)
• Lint files to make sure they conform to coding standards

config.yml

version: 2

references:
  # Default container configuration
  docker_config: &docker_config
    docker:
      - image: circleci/php:7.0-node-browsers

jobs:
  build:
    <<: *docker_config
    steps:
      - checkout

- restore_cache:
    keys:
      - v1-dependency-cache-{{ checksum "yarn.lock" }}
      - v1-dependency-cache
- restore_cache:
    keys:
      - v1-composer-cache-{{ checksum "composer.lock" }}
      - v1-composer-cache

- run:
  name: Remove previous Yarn
  command: sudo rm /usr/local/bin/yarn*
- run:
  name: Install Yarn
  command: sudo npm install -g yarn
- run:
  name: Install Grunt CLI
  command: sudo npm install -g grunt-cli

- run:
  name: Install NPM Dependencies
  command: yarn install
- run:
  name: Install Composer Dependencies
  command: composer install -o --no-dev

- save_cache:
  key: v1-dependency-cache-{{ checksum "yarn.lock" }}
  paths: node_modules
- save_cache:
  key: v1-composer-cache-{{ checksum "composer.lock" }}
  paths: vendor

- run:
    name: Building
    command: |
      if [ "${CIRCLE_BRANCH}" == "staging" ]; then
        grunt
      else
        grunt build
      fi

- run:
    name: Maybe Deploying?
    command: |
      if [ "${CIRCLE_BRANCH}" == "staging" ]; then
        .circleci/deploy.sh
      fi
      if [ "${CIRCLE_TAG}" ]; then
        .circleci/deploy.sh
      fi

workflows:
    version: 2
    build-and-deploy:
      jobs:
        - build:
            filters:
              branches:
                ignore: master
              tags:
                only: /^v[0-9]+(\.[0-9]+)*/

Deployment

• 🚀 We want to get our code changes out to servers FAST!!!
• 🍰 Only download the changes, not every single file
• 👍 Git is awesome for this

deploy.sh

#!/bin/bash
echo "Starting Deploy..."
git config --global user.name "CircleCI"
git config --global user.email "[email protected]"

# Make a new README.md with build status details
rm README.md
cat > README.md EOF
Build [#$CIRCLE_BUILD_NUM]($CIRCLE_BUILD_URL) by $CIRCLE_USERNAME at $TIMESTAMP
[$CIRCLE_COMPARE_URL]($CIRCLE_COMPARE_URL)
EOF

Build #6608 by kingkool68 at 2019-02-07 04:00 AM UTC
spiritedmedia/spiritedmedia/compare/fc0e1c7f394a...52af362adfde

# This should be ignored in .gitignore-build
# but let's try and remove it just to be safe
rm -rf node_modules/

# Find all .git/ directories and remove them.
# If we commit directories with .git in them then they are
# treated like sub-modules and screw that.
find . | grep -w ".git" | xargs rm -rf

# Remove all .gitignore files in
# the wp-content/ and vendor/ directories
find wp-content/ vendor/ -name ".gitignore" | xargs rm

rm .gitignore
mv .gitignore-build .gitignore

git clone [email protected]:spiritedmedia/spiritedmedia-build.git tmp/
mv tmp/.git .
rm -rf tmp/

# If no branch is set then assume master
if [ ! $CIRCLE_BRANCH  ]; then
  CIRCLE_BRANCH="master"
fi

# Switch branches... maybe?
if [ ! `git branch --list $CIRCLE_BRANCH` ]; then
  # Branch doesn't exist. Create it and check it out.
  # See http://stackoverflow.com/a/21151276
  git checkout -b $CIRCLE_BRANCH
else
  git checkout $CIRCLE_BRANCH
fi

# Add everything and commit
git add -A
git commit -m "Build #$CIRCLE_BUILD_NUM by $CIRCLE_USERNAME on $TIMESTAMP"
git push origin $CIRCLE_BRANCH --force

echo "Code changes pushed!"

🎉 Hooray!

🤔 Now how do we get this code to servers?

AWS CodePipeline is the glue between GitHub and AWS CodeDeploy

AWS CodeDeploy talks to our servers and tells them to update

CodeDeploy is also configured by a YAML file, appspec.yml.

version: 0.0
os: linux
files:
hooks:
  BeforeInstall:
    - location: bin/codedeploy.sh
      timeout: 600
      runas: root

codedeploy.sh

#!/bin/bash
# The appsec.yml file prohibits calling arbitrary scripts.
# This stub shell script downloads and executes
# the shell script specified in the
# EC2 instance User Data field.
# The user data shell script calls the deploy script
# baked in to the AMI running on the server.

# Fetch the user data script associated with this
# type of instance and save it into a temporary shell script
sudo curl http://169.254.169.254/latest/user-data > temp.sh

# Execute the shell script to run the update
sudo bash temp.sh

# Clean up
sudo rm temp.sh

“You can specify user data to configure an instance or run a configuration script during launch.”
🆒

deploy-production.sh

#!/bin/bash
# Shell script to update the app with
# the latest changes from GitHub (ex. when called form AWS CodeDeploy)
# Should be placed in /var/www/spiritedmedia.com/scripts/
# and run as root

cd /var/www/spiritedmedia.com/htdocs/

# Force git pull
git fetch --all
git reset --hard origin/master

# Reset file ownership
chown -R www-data:www-data /var/www/spiritedmedia.com/htdocs/

Tips & Tricks

Make a separate GitHub user and use that for managing credentials to different servers.

💰 Pricing

CircleCI

• 1st container = 1000 build minutes free per month
• $50/month for each additional container, unlimited build minutes
• Open source = 4 free containers

AWS CodeBuild

• $0.005 per build minute (smallest tier)
• First 100 minutes per month are free

📈 If your monthly build time is…

• Less than 168.333 hours → AWS CodeBuild
• Greater than 168.333 hours → CircleCI

CircleCI
vs
AWS CodeBuild

Pros of CircleCI

• Awesome debugging experience
• Much easier to set-up paralell jobs
• Pricing if you need 1000 build minutes or less per month

Cons of CircleCI

• Expensive once you need to move past the free tier

Pros of AWS CodeBuild

• Pay-for-what-you-use pricing
• Cost if you need more than 1000 build minutes per month
• Run as many concurrent builds as you need

Cons of AWS CodeBuild

• Breaking up one job into multiple concurrent parts is difficult
• More work to set up (you need CodePipeline to trigger builds for example)

Further Reading

• Walkthrough of a Multi-job Workflow on CircleCI
• Serverless CI/CD AWS CodeBuild Tutorial

In Conclusion

Continuous Integration / Continuous Deployment is like playing dominos with YAML files and shell scripts.

But it's so awesome when it all works! 🚀

Thank You!

@kingkool68

Questions?