Git Push Deployment for Big Commerce

One of the challenges we had working on a Big Commerce implementation was integrating it efficiently into our team’s work flow. At Gazelle, GitHub is a key part of how me move code from engineer to production. As is common, we branch and use pull requests as an opportunity for peer review. We wanted to insure we could take the same approach with the Big Commerce templates, and ensure individuals didn’t override each others work.

Code:
To get started, I created a private repo in our GitHub account. Next,I exported the templates folder into the new project. I chose to only export changed files to reduce the project size. Our GitHub project has two branches; dev and master. The dev branch is tied to the staging site. The master branch is tied to the production site. It was important we have a place to review changes internally before those changes move into production.

Continuous Integration:
We use CircleCI to run our tests. Circle is great. It’s dead simple to setup, very customizable, and they let you run tests in parallel. There are no tests to run in our Big Commerce project. It’s just HTML, Javascript, CSS, images, and fonts. I was particularly interested in the git commit hook as a mechanism to trigger branch deployment. The following is particular to Circle CI, but I imagine, could easily be adopted to a variety of CI alternatives. Below is the circle.yml file used in this project:

general:
  branches:
    only:
     - master
     - dev
test:
  override:
    - ruby ./run.rb

A quick overview of what’s going on.

general:
  branches:
    only:
     - master
     - dev

This sets up circle to only run on commits made to the master and dev branches. This lets us work on branches and review those changes before the changes find their way into the staging or production environments.

test:
  override:
    - ruby ./run.rb

We’ve overridden the default test call. Instead of running rspec or unit tests, we’ll run run.rb. run.rb handles the actual deployment. I’ll go into this file in more detail next.

Deployment:
Here is an overview of the script to handle the actual code deployment. I’ll step through the parts below.

require 'rubygems'
require 'bundler/setup'
require 'uri'
require 'curb'
require 'net/dav'
require 'dotenv'

Dotenv.load if defined? Dotenv

def master?
  ENV['CIRCLE_BRANCH'] == 'master'
end

def env_name
  (master?) ? 'PRODUCTION' : 'STAGING'
end

def url
  ENV["#{env_name}_URL"]
end

def password
  ENV["#{env_name}_PASSWORD"]
end

def create_directory(conn, path, arr)
  @current_folders ||= {}
  create_path = (path.empty?) ? arr.shift : "#{path}/#{arr.shift}"
  unless @current_folders.has_key?(create_path)
    conn.mkdir(create_path) unless conn.exists?(create_path)
    @current_folders[create_path] = 1
  end
  create_directory(conn, create_path, arr) if arr.length > 0
end

def create_path_if_missing(conn, file)
  create_directory(conn, '', File.dirname(file).split(/\//))
end

def changed_files
  all_files = Dir.glob('template/**/*.*')
  return all_files if !ENV['CIRCLE_COMPARE_URL'] || ENV['CIRCLE_COMPARE_URL'].empty?
  puts ENV['CIRCLE_COMPARE_URL']

  range = ENV['CIRCLE_COMPARE_URL'].split('/').last

  `git diff --name-only #{range}`.split("\n").reject{|file| file !~ /^template\//}
end

Net::DAV.start(URI("https://#{url}/dav/")) do |dav|
  dav.credentials(ENV['USERNAME'], password)

  puts "push to: #{url}"
  changed_files.each do |file|
    if File.exists?(file)
      create_path_if_missing(dav, file)
      dav.put_string(file, File.open(file, 'r').read) 
      puts "add/update #{file}"
    else
      dav.delete(file) if dav.exists?(file)
      puts "remove: #{file}"
    end
  end
end

Now, a quick run through what this code is doing. We need to include a couple of Ruby libraries, rubygems (so we can add gems), and URI. Circle uses Bundler to resolve gem dependencies. Bundler/setup loads the cached gems into this scripts scope. The net_dav gem is going to handle the WebDAV connection. It’s a bit light on documentation, but the code is easy to read. I’ve included to curb gem as well to take advantage of the speed of curl. The dotenv gem simplifies the environment variable setup while developing locally.

Some simple helper methods to figure out where to deploy the code and what passwords to use:

def master?
  ENV['CIRCLE_BRANCH'] == 'master'
end

def env_name
  (master?) ? 'PRODUCTION' : 'STAGING'
end

def url
  ENV["#{env_name}_URL"]
end

def password
  ENV["#{env_name}_PASSWORD"]
end

Unfortunately, the net_dav doesn’t have anything like mkdir -p to recursively create directories, so we need to make sure they can be created efficiently before we try to upload a file to that location:

def create_directory(conn, path, arr)
  @current_folders ||= {}
  create_path = (path.empty?) ? arr.shift : "#{path}/#{arr.shift}"
  unless @current_folders.has_key?(create_path)
    conn.mkdir(create_path) unless conn.exists?(create_path)
    @current_folders[create_path] = 1
  end
  create_directory(conn, create_path, arr) if arr.length > 0
end

def create_path_if_missing(conn, file)
  create_directory(conn, '', File.dirname(file).split(/\//))
end

We keep track of confirmed directories for the obvious performance benefits. Now we’re getting to the good stuff.

def changed_files
  all_files = Dir.glob('template/**/*.*')
  return all_files if !ENV['CIRCLE_COMPARE_URL'] || ENV['CIRCLE_COMPARE_URL'].empty?
  puts ENV['CIRCLE_COMPARE_URL']

  range = ENV['CIRCLE_COMPARE_URL'].split('/').last

  `git diff --name-only #{range}`.split("\n").reject{|file| file !~ /^template\//}
end

This method takes advantage of GitHub’s comparison view to only act on the files that have changed. Most of the changes we’re pushing are small, and we want this deployment to be as nearly real time as possible.

And that brings us to the final block of code:

Net::DAV.start(URI("https://#{url}/dav/")) do |dav|
  dav.credentials(ENV['USERNAME'], password)

  puts "push to: #{url}"
  changed_files.each do |file|
    if File.exists?(file)
      create_path_if_missing(dav, file)
      dav.put_string(file, File.open(file, 'r').read) 
      puts "add/update #{file}"
    else
      dav.delete(file) if dav.exists?(file)
      puts "remove: #{file}"
    end
  end
end

Here were connecting to our Big Commerce account and upload (or remove) the files add, changed, or deleted as part of the commit.

That’s it. Sixty four lines of code and the power of CircleCI lets us deploy our Big Commerce templates with:

git push origin dev

This entry was posted in big commerce, ruby and tagged , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *