SVN and Squid

We run a number of servers behind the scenes; Version Control (SVN), Trac, ColdFusion (dev and staging), Database (SQL Server, MySQL, PostgreSQL) ...

We've now deployed a proxy server in front of it all. In this case we've deployed Squid to act as a reverse proxy.

So, basically everything now goes through the Squid server, and it routes our requests based on the domain name requested.

It was all fairly straight forward to set up, except we encounted a slight problem when it came to committing changes back into SVN.

svn: Commit failed (details follow):
svn: MKACTIVITY of '/path/to/svn/repo/!svn/act/20b34b89-e624-0410-bfb9-b7cf8f28e335': 400 Bad Request (http://mydomain.com)

For the life of me I couldn't figure out what was going on. As well as the introduction of Squid, we had also upgraded SVN and Trac and I assumed it was the SVN upgrade that was causing the issue.

As it turns out, it was Squid. You have to make a slight change when you're using a proxy with SVN.

As they say, Read the F*%king Manual!!! http://subversion.tigris.org/faq.html#proxy


Updating the Status of a Trac ticket via a SVN commit

One of the really nice things about the SVN/Trac combo is that when you commit changes to SVN, you can have the changeset update the status of a Trac ticket.

So, say the changes you have been working on, have tested, and are now committing back to SVN fix a specific ticket within Trac, and once you've made your commit, you would then log into Trac and close the ticket there.

You could also do this via the superb Mylyn integration if you're using Eclipse - which you should be :)

Well, how about updating the actual status when you do your commit? One action to rule them all, so to speak.

Well, this is possible, and here's how to do it.

The main reason for posting this, is that I've recently upgraded our Trac environment from 0.10 to 0.11 and I noticed the trac-post-commit was now broken.

The following is for Linux, so you'll need to Google for Windows.

Grab the trac-post-commit script

You need to pull the latest working version of the script from the Trac SVN repository.

Next rename the file you've just exported by sticking a ".py" on the end - it's a Python script.

Next make sure your post-commit script file is named appropriately (post-commit funnily enough) and make sure that execute permissions are set on the file. You know, chmod ugo+x post-commit

Edit the post-commit script

Next, edit your post-commit file and add the following line - this is the reason ours was broken.

TRAC_ENV="/path/to/your/trac/env"

Now add the following line, and this is the one that calls the trac-post-commit script

/usr/bin/python /path/to/hooks/trac-post-commit-hook.py -p "$TRAC_ENV" -r "$REV

Just to be on the safe side of everything, just Apache a restart.

Commit Your Change

Now, from within Eclipse, when you want to commit your changes, you can now put the following in your comments (and you are filling in comments aren't you?????!!!???"

command #1
command #1,#2
command #1 & #2
command #1 and #2

The following commands are supported:

  • close
  • closed
  • closes
  • fix
  • fixed
  • fixes

The above will close the specified ticket number, in addition to adding the message you have specified.

Also, you can use the following commands:

  • references
  • refs
  • addresses
  • re
  • see

The ticket numbers specific here are left as is, but the contents of the commit messages are added to their notes.

A quick example would be something like:

The changes made here fix #100, #102 and refs #85

This commit message will close tickets 100 and 102, and add the commit message to ticket 85.


Installing and Configuring SVN and Trac on Windows

At the risk of doing my company (Fuzzy Orange) out of business, I thought I would blog about how to set up SVN and Trac on Windows.

There has been a lot of discussion on this at the ScottishCFUG Google Group, and as much as I'd love to just say "Use Linux, it's easier", some folks aren't in that position.

So here it is:

Installing and Configuring SVN and Trac on Windows

Please leave comments, suggestions, questions, problems, etc here and I'll do my best to answer you.


Version Control: You're a Fool if You Don't Use It

We've been having a right good old natter on the ScottishCFUG Google Group, in particular about why Version Control is essential to all software development.

One of the group members asked:

All that you mention sounds very cool but i want to see what benefits it can bring a sole developer.

A reply by Alan Williamson, in my opinion, was so important I wanted to repeat it here.

and there in is the trap that many lone consultants/developers face; stop thinking of yourself as a single entity, but as a member of a team that, at present has only one member.

Alan also followed this up by stating:

I really don't know how anyone can work without version control and still claim to develop software. Bold statement? yes, but i stick by it.

I agree 100%. Any software development company who is without Version Control is ... well, it is appauling. Not only are they putting their clients at risking by not taking their business and indeed livihood seriously, but they are also inflicting damage to themselves by not making themselves as proficient as they could be, but they are also damaging their own reputation... I look on any software development firm who doesn't use Version Control as one who shouldn't be in this business. Calling them amateur, well I think I'm being rather generous.

In the past, when I've been the client, one stipulation I've also insisted on being in any contract is that the company doing the work has a Version Control system in place. If they didn't, end of conversation.

Even as a sole developer you should be using Version Control.

So, just to finish up, here's a couple of reasons why you should be using Version Control. Feel free to add more by leaving a comment.

  • Provides a history of all changes
  • Recover accidental deletions
  • Helps co-ordinate new releases by deploying a build corresponding to a specific revision.
  • Supports development by X number of developers
  • Enables remote working if your repositories are public facing (and secure).
  • It's not just for code. You can store all your contracts, legal documents, staff policies in Version Control.


Using SVN to assist in commenting your code

I'm quite heavy handed when it comes to commenting my code.

As well as inline comments, I always have a config comment at the top of every file.

In addition to basic information such as Author, Filename, when the file was created, I use Subversion to get information such as: the current revision, when was the file last updated, who updated the last file.

To automate the process, I have a CFEclipse snippet that is saved as a template (require separate ones for both CFM and CFC files). So when I create a new file, I create it from the template, and this runs the snippet automatically, prompting me to fill in certain information. You can see the snippet below.

<!---
Filename: $${CURRENTFILE}
Creation Date: $${DAYOFMONTH}/$${MONTH}/$${YEAR}
Original Author: $${author}
Revision: $Rev$
$LastChangedBy$
$LastChangedDate$
Description:
$${description}

Methods:

$${methods}

--->

Once I commit this file to Subversion, the Subversion properties are populated, meaning I end up with something like below.

<!---
Filename: weatherProxy.cfc
Creation Date: 18/July/2007
Original Author: Andy Allan
Revision: $Rev: 45 $
$LastChangedBy: allana $
$LastChangedDate: 2007-07-18 12:24:59 +0100 (Wed, 18 Jul 2007) $
Description:
This is our proxy CFC, which in turn calls the Yahoo weather CFC as part of Ray Camden's Yahoo API package.

The component works by first being init'd, recieving a list of towns to which we have to retireve weather. At the moment this is passed in, but in time we will retrieve this information from a database as it will match all the locations for the company.

On init, we call getWeather() passing it the list of towns, and this method loops through the towns returning the weather data for each and storing in an Application
variable structure.

We then use getMyWeather() which accepts a town as an argument to return the weather data specfic to that town.

Methods:

init: This is our constructor method, which MUST be called before any other method. It takes a single
argument "datasource" and returns itself.

getWeather: private function that loops through all the companies locations, getting their weather data and storing the data in an Application structure.

getMyWeather: method which returns the weather for a specific town.

--->

To get Subversion to actually populate your comments you need to assign a new property to every file/folder in your application.

You can either do this by running the following command:

propset -R svn:keywords "Rev LastChangedDate LastChangedBy" /pathToFiles/trunk

Or from within CFEclipse, simply right click your Project folder and choose "Team" > "Set property" and then add the new property as indicated in the image below.

New SVN Property

With every commit, you'll now see the most up to date Subversion information directly in your comments.


Woot - Trac up and running

Here at work we've been using Subversion since I joined. There was no version control in place and it was the first change I made.

For some time now I've wanted to get Trac up and running as I've seen it in action, and it's integration with Subversion is nothing short of astounding.

Due to lack of time and resources, and because I heard Trac was a killer to set up on Windows, we went with Ray Camden's Lighthouse as our bug tracker until I could find a spare day or two.

Well today I finally decided to bite the bullet and install Trac. And you know what, in about 30 mins, it was up and running... and even then most of that was downloading files.

In fact, the only complication I had was forgetting to edit trac.cgi to find the python.exe file.

So, whilst not an extensive guide to getting Trac installed here's what I did.

Files to Download

Bear in mind I already had Subversion up and running with Apache on Windows 2003, so I won't be covering this step

  1. Python
  2. Subversion Python Bindings
  3. PySQLite
  4. Clearsilver
  5. Trac

Installation Notes

  • Install Python - You must use Python 2.3.x - the Subversion bindings are not available for Python 2.4
  • Copy the libsvn and svn folders in the Python Bindings to <python>\lib\site-packages
  • Install PySQLite
  • Extract neo_cgi.pyd from your Clearsilver download to <python>\lib\site-packages
  • Install Trac

Configuration

  • Copy <python>\share\trac\cgi-bin\trac.cgi to <apache>\cgi-bin
  • Edit the trac.cgi file so that the first line points to your python.exe file

Configure Trac DB

d:\python23>;python scripts\trac-admin d:/svn/trac.db

Initialise DB

Trac [d:/svn/trac.db]> initenv

Trac will now ask you for a Project Name, Database Connection String, Subversion Repository, and the Path to Templates

So as an example:

Project Name: Creative

Connection String: (just hit enter to accept default)

Subversion Repository: d:\svn\creative

Path to Templates: (just hit return)

Add Administrative Permissions

Trac [d:/svn/trac.db]> permission add admin TRAC_ADMIN

Add Trac to Apache

Edit httpd.conf:

# TRAC
Alias /trac "d:/Python23/share/trac/htdocs"

<Location /cgi-bin/trac.cgi>
SetEnv TRAC_ENV "d:/svn/trac.db"
SetEnv PYTHONPATH "d:/Subversion/bin"
# if you are running Apache as a user other than System, the TMP variable
# needs to be set to a place where that user can write scratch files. Make
# sure that this directory is created and writable by that user.
# SetEnv TMP "d:/svn/trac.db/tmp
</Location>

<Location /cgi-bin/trac.cgi/login>
AuthType Basic
AuthName "
Creative"
AuthUserFile passwd
Require valid-user
</Location>

<Directory "d:/Python23/share/trac/htdocs">

Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>

Now you want to restart Apache.

Finally, hit http://yourdomain/cgi-bin/trac.cgi and log in via the username/password combinations you have set up in "passwd" for accessing Subversion.

Final Words

I know I've not covered everything, and I've certainly not mentioned any error messages you may potentially come across, but if you do have any issues, leave a comment here and I'll try and help out.


Scotch on the Rocks 2008