Integrating Capistrano, MediaWiki, and Bugzilla

Ryan Schenk

ruby -e "p 'ryan'[0...1] + 'schenk' + '@gmail.com'"

At work, we needed to keep two running logs on our Wiki every time we deployed new code with Capistrano. Not wanting to do this manually, I decided to automate it with Capistrano, and was pleasantly surprised at how easy the entire process was. Let's take it from the top.

Due to various complexities, and a mildly custom deployment structure, it became desirable to keep a running deployment log on my company's Wiki, which is powered by MediaWiki. After some quick Googling, I found MediaWiki Interface on RubyForge, which is a fantastic API that wraps MediaWiki. After finding and fixing a bug, I was soon editing pages on our wiki flawlessly from Ruby.

For our internal processes, we wanted to keep a running list of the code release version (such as "0.9.1"), the SVN repository revision that was deployed, the timestamp Capistrano used as the folder name in the Releases directory, and a more human-readable date and time of deployment. The first field, the release version, is an arbitrary string given by the person making the deployment; Capistrano prompts the deployer for this string as the first step in the deployment. The repository revision and release timestamp are both provided by Capistrano, and the human-readable date is provided by Media Wiki.

This process of updating the wiki is quite straightforward, as long as you make a few assumptions and stick to them. The most important assumption is that when the Capistrano goes to update the page on the Wiki, the last element on that page is a table which already has one or more rows. The last assumption is that we are inserting a new row at the bottom of the table. If such is the case, this task is a simple matter of lopping off the closing 'tag' of the table, injecting a new row, closing the table again, and saving the page. Note that we are using the wiki markup syntax "~~~~~" to insert a human-readable date and time.

task :update_wiki do 
  get_code_version unless respond_to? :code_version

  wiki = MediaWiki::Server.new("http://wiki.company.internet.com/~wiki/index.php", port = 80, username = "CapistranoBot", password = "secret")
  log = wiki.page('RailsDeploymentLog')

  log.contents.chomp!
  log.contents.chomp! '|}'

  log.contents << "|-\n"
  log.contents << "| #{code_version} || #{real_revision} || #{release_name} || ~~~~~ \n"

  log.contents << "|}\n"

  log.save
end

This works quite well, and produces a handy table for keeping track of our deployments, as you can see below.

As we progressed into the internal Beta phase of our project, we decided that it would be helpful for our Beta users (who were employees from other departments) to not only be able to see when new code had been pushed up, but also see what had changed in each deployment. Because our application is composed of other components in addition to the Rails web front end, we decided that this "readme" page needed to be on a separate Wiki page than the Rails Deployment Log above, so that changes to the entire system (not just the Rails component) could be viewable on one screen.

The prospect of filling in this new "readme" page manually, by prowling through the SVN commit messages and paraphrasing them, seemed incredibly unpalatable. Then it hit me -- why not automatically pull the commit messages out from SVN, and post them to the Wiki in bulleted format? Fabulous! And, turns out, this is very easy to do with the SVN log command. Using the -r flag, you can give two revision numbers, and it will return all commit messages between the two. And it can give them to you in XML. How Very Handy.

Even better, Capistrano has already encapsulated the log functionality from SVN in its Subversion class. Unfortunately, the Subversion.log method returns results in raw text, and I wanted XML, so I wrote my own extension that tacks on the --xml flag and monkeypatched it into the Subversion class. (Note: if you know a better way to do this, please let me know!)

require 'capistrano/recipes/deploy/scm/subversion.rb'
module Schenk
  module Deploy
    module SCMExtensions

      # Returns an "svn log" command in XML for the two revisions.
      def xml_log(from, to=nil)
        scm :log, repository, authentication, "-r#{from}:#{to || head} --xml"
      end

    end
  end
end
Capistrano::Deploy::SCM::Subversion.send :include, Schenk::Deploy::SCMExtensions

Once I could get the SVN commit logs into XML, I simply used REXML to burn through the logs, appending each <msg> onto a string, and inserting that string into our Wiki. This worked extremely well and was very easy, the problem was that our commit messages were too programmery for the intended audience of our "readme" wiki page -- seeing "New implementation of bound C libraries for both big and little endian storage" was a hindrance to our internal Beta users, not an aid.

# Dump all SVN commit messages into the Wiki page
wiki_readme = ''
xml = REXML::Document.new(`#{source.local.xml_log(previous_revision.to_i + 1)}`)
xml.elements.each("//msg") { |msg| wiki_readme << msg.text unless msg.text.nil? }

We decided that our commit messages should contain a programmer-only section, and an optional external "Beta friendly" section. If an external section was found in the message, it would appear in the "readme" Wiki page; commit messages without an external section were gracefully ignored. Then a stroke of genius hit me: if someone mentioned a Bugzilla bug number in the external commit message, why not link them to that bug in Bugzilla?

The implementation turned out to be very easy for both. I figured that it would be most important to have the programmery part of the commit message at the beginning, and the fluff at the end; this also has the nice side effect of not requiring a closing tag for the external message, so less opportunity for typos. Simply type [ext] in the commit message, and everything following will be included in the external part of the commit message. I also found myself referring to Bugzilla bugs most often in the form "Bug #nnn", so I decided everything in the commit message that was in that form would be translated into a hyperlink to Bugzilla. The implementation of this simple parser is below.

def format_message(msg)
  if msg && msg =~ /\[[Ee]xt\](.*)/

    external = $1.strip

    # Bugzilla magic keywords
    external.gsub!(/([Bb]ug #([0-9]+))/, '[http://bugzilla.company.internet.com/show_bug.cgi?id=\2 \1]')

    '* ' << external << '<br/>' 
  else

    ''
  end
end

That's really all there is to it. I added the "readme" updating functionality back into the update_wiki task that I provided at the beginning of this article. I probably should refactor it out into its own task, but it seems to work fine where it is.

desc "Updates RailsDeploymentLog on the wiki."
task :update_wiki do 
  require 'rexml/document'
  get_code_version unless respond_to? :code_version

  wiki = MediaWiki::Server.new("http://wiki.company.internet.com/~wiki/index.php", port = 80, username = "CapistranoBot", password = "secret")
  log = wiki.page('RailsDeploymentLog')

  log.contents.chomp!
  log.contents.chomp! '|}'

  log.contents << "|-\n"
  log.contents << "| #{code_version} || #{real_revision} || #{release_name} || ~~~~~ \n"

  log.contents << "|}\n"

  log.save

  external_readme = ''
  xml = REXML::Document.new(`#{source.local.xml_log(previous_revision.to_i + 1)}`)
  xml.elements.each("//msg") { |msg| external_readme << format_message(msg.text) }

  releases = wiki.page('Release_Readme')

  releases.contents.chomp!
  releases.contents.chomp! '|}'

  releases.contents << "|-\n"
  releases.contents << "| [[Presentation Layer (PL)|PL]] || [[RailsDeploymentLog|#{code_version}]] || ~~~~~ || development || #{external_readme} \n"

  releases.contents << "|}\n"

  releases.save  
end