Skip to main content

Offline Subversion: Building a Trac of your own

trac_svn.png

I have recently, like many other folks, become a fan of my friends Daniel Jalkut and Manton Reece's new Mac developer podcast, Core Intuition (especially the whimsical theme music). In their latest episode, Traveling Luddites, among other things, they discuss distributed version control systems such as Git, Mercurial, and Bazaar.

I'm kind of in the same boat in that I've spent little or no time trying these systems out compared to what I've been using, Subversion, and am a little hesitant to invest a lot of time into migrating and learning a new system. Don't get me wrong -- I'm a hacker and I love to try new technology. However, I don't really have any problems with Subversion and in fact, some time ago I came up with a really sweet setup for offline development, offsite backup, and integrated project management using Trac that I've been meaning to blog about, and this episode was just the catalyst I needed.

Following the directions here, you will get:

  • Offline access to your versioning, ticketing, and documentation systems.
  • Support for multiple projects (though one project is just fine, too).
  • A multi-user setup, if you need it.
  • Integration of your code repository with web-based ticketing, GUI changesets and diffing, activity timelines, and project roadmap planning.
  • Complete, incremental offline backup whenever you like (and have a connection).
  • Very little intrusion into your Mac OS X environment.
  • Complete flexibility in getting your data out at any time.

Just have a look around the Trac demo to get a feel for this system. It was a tremendous thrill during the development of Meerkat to see the roadmap progress meter get ever closer to the 1.0 milestone as I closed tickets and definitely helped me organize the project much better than before I adopted Trac.

And this isn't rocket science -- if you're already technical enough to be using version control, I'm pretty sure you can handle this and have a great setup in a short amount of time. I've found that the parts that trip most people up are Trac installation and Apache configuration. Relax, I've got you covered.

A quick historical note

I've been using Subversion in both user and administrator capacities since 2001 and before that, like Daniel and Manton, I too had a painful coexistence with CVS (and still do sometimes with projects like Drupal). I was introduced to SVN by a former co-worker, Garrett Rooney, back in the days when he was one of the few people who knew about and worked on SVN. Since then, he literally wrote the book (Practical Subversion) on SVN and through the years, I've come to really love the project and have introduced my fair share of folks to it as well.

While, as I said, I'm generally open minded about new technology and systems, when it comes to version control I'm in the camp right now of "if it ain't broke, don't fix it". I'm using Trac on some other client projects, including one with a team of over 30 people all over the world, and I use SVN for the development of both Pukka (over two years of commits) and Meerkat (about a year of commits) as well. It works well for me, I don't have hassles, and since SVN has been integrated with Trac, Apple's Xcode, and the Apache web server that ships on OS X, it's going to take free pony rides or something comparable before I seriously look elsewhere for my version control needs. Of course, YMMV.

What you'll need

In order to replicate this setup, you'll need:

  • A Mac running Leopard and administrator access to it.
  • Familiarity with SVN, but you don't need an existing repository.
  • Some Terminal experience.
  • About an hour to spare (or less, and not counting some background compilation time in which you can do other things).

If you already have MacPorts installed, you'll save even more time and you won't even need to manually download anything else.

Ready? Let's do it.

Install MacPorts and Trac

I'll admit it, Trac can be a little intimidating to setup because of dependencies. But if you get MacPorts, this can all be reduced to a fraction of the time and very little hassle.

  1. Install MacPorts if you don't have it already. The latest installer package for Leopard is here.
  2. Make sure that /opt/local/bin is in your path so that the commands provided with MacPorts are available to you easily in the Terminal.
  3. Open up Terminal and start the Trac installation process by typing sudo port install trac.
  4. Let it roll while you grab a $BEVERAGE or move on to the next steps. When it finishes, you should be able to access a command called trac-admin in Terminal. Depending on what ports you already have installed, you may need to answer some basic questions during the install process, so keep an eye on the window. Brian Cooke's Growl When Done is a good solution here.

Setup Subversion

You'll need to setup a parent directory for your Subversion repository or repositories. I like /usr/local/svnroot, so if you are using someplace else, you'll need to alter the directions below as appropriate. Keep in mind that stuff under /usr/local gets backed up by Time Machine by default, even if you exclude "System Files and Applications". This is separate from the backup mechanism that I'll talk about in just a little bit.

  1. Make a top level Subversion folder using sudo mkdir /usr/local/svnroot and change into it.
  2. Either move an existing repository into a subfolder of this folder, or create a new repository folder using sudo svnadmin create MyProject in Terminal.
  3. Make the whole setup owned by Apache, which on Leopard can be accomplished with sudo chown -R _www:_www . in the Terminal.

Setup Trac

Again, I prefer a similar convention here and put stuff under /usr/local/tracroot. Adjust as necessary.

  1. Make a top level Trac folder using sudo mkdir /usr/local/tracroot and change into it.
  2. Create a new Trac project using sudo trac-admin MyProject initenv.
  3. Edit the MyProject/conf/trac.ini file to configure it. You'll likely just need to change the following items:
    • max_size if you want to upload larger files to the wiki
    • downloadable_paths if you have a non-standard repository structure
    • link to something like http://localhost/trac/MyProject
    • descr, footer, name, and url if you wish
    • repository_dir if you didn't specify it during the initialization step
  4. Give your account full project permissions using sudo trac-admin MyProject permission add <username> TRAC_ADMIN in Terminal (replacing <username> with your own username).
  5. Make it owned by Apache using sudo chown -R _www:_www . like you did with SVN.

Configure Apache

This is perhaps the trickiest part, but I've done the work for you. I did some poking around and figured out a good way to get this working with the Apache that is built into Leopard. Although I haven't tested the other parts of this guide for Tiger compatibility, Apache is pretty much the part that requires Leopard, since it ships with Apache 2 and has some of the modules necessary for SVN integration built in. If you have your own Apache 2 going with MacPorts or something else, perhaps this will still work for you.

First, you'll want to create a password file for authenticating your Trac and SVN users, even if it's just you.

sudo htpasswd -c /etc/apache2/other/htpasswd <username>

Be sure to replace <username> with your own username. This command will create a new password file at the chosen path and add your first user. If you have other users, repeat the command above with their usernames but leave off the -c option since the file already exists and you don't want to clobber the previous commands.

Next, you want to create a configuration file for SVN to make your repository available over the (local) web and to password protect it. Create a file at /etc/apache2/other/svn.conf that looks like this:

LoadModule dav_svn_module libexec/apache2/mod_dav_svn.so
LoadModule authz_svn_module libexec/apache2/mod_authz_svn.so
<Location "/svn">
    DAV svn
    SVNParentPath /usr/local/svnroot
    AuthType Basic
    AuthName "Subversion Repository"
    AuthUserFile /etc/apache2/other/htpasswd
    Require valid-user
</Location>

After this, make another file at /etc/apache2/other/trac.conf that looks like this:

ScriptAlias /trac /opt/local/share/trac/cgi-bin/trac.cgi
<Location "/trac">
    SetEnv TRAC_ENV_PARENT_DIR "/usr/local/tracroot"
    AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName "Trac"
    AuthUserFile /etc/apache2/other/htpasswd
    Require valid-user
</Location>

Remember in both of these instances to change the paths as necessary if you put your SVN or Trac environments someplace other than the paths I mentioned above.

Once these files are set, go to Terminal and issue the command sudo apachectl graceful to let the configurations take effect. Once you do, you should be able to reach your Subversion repository at http://localhost/svn/MyProject and a list of your Trac projects at http://localhost/trac. Both will be password protected using the account(s) you created above.

Backing it up

You're almost there! The last bit is some scripting that lets you back both of these systems up to a remote server using rsync over SSH.

  1. Edit a new text file at /usr/local/bin/ts (or $HOME/bin/ts, if that's how you roll) and fill it with the following:

    #!/bin/sh
    echo "Backing up Subversion repositories..."
    /usr/bin/rsync -e ssh -avz --delete /usr/local/svnroot/ <username>@<hostname>:svn/Repository.sync
    echo "Backing up Trac databases..."
    /usr/bin/rsync -e ssh -avz --delete /usr/local/tracroot/ <username>@<hostname>:trac/Trac.sync

    Be sure to replace the <username> and <hostname> values with whatever username and hostname your SSH account has.

  2. In Terminal, use chmod 755 /usr/local/bin/ts to make the script executable.

Now, whenever you want to incrementally backup your SVN and Trac databases, just type ts (think: "trac sync") at the shell. Everything that has changed since the last time you performed this command will get copied up to the remote server, keeping a handy backup of both your SVN repositories and your Trac installations should you need to move to another server. And even without the Apache configuration, the SVN repository will be useful as a standalone. Just keep in mind that it is for backup purposes and there is not any two-way syncing going on. That is, your local copy is the authoritative copy and is the one that you should use from day to day, and ideally over HTTP and not using local filesystem access to /usr/local/svnroot directly.

Conclusion

This system has been really useful for me during the development of Meerkat and through the continued development of Pukka, since I've been able to easily track feature requests, bugs, wiki-based notes for future reference, and have it all tied together with Subversion commits. In addition, I've been able to take advantage of offline time while at airports or otherwise on-the-go to actually get code committed, issues resolved, ideas jotted down, and processes documented.

I hope that this guide has been useful to you. Of course, feel free to leave a comment if you have any questions or clarifications or find any typos and I'll try to correct them.

Happy Trac-ing!

Trackback URL for this post:

http://codesorcery.net/trackback/131

Justin, Great tutorial,

Justin,

Great tutorial, thanks for putting it together. The one problem that I'm seeing is that the trac.cgi script referenced in the trac.conf file does not seem to exist in the release that I just installed from MacPorts. I did a search for it in my /opt directory with no luck.

Any ideas? Can you just send me yours?

Thanks for the help,
Michael

I just ran into this problem

I just ran into this problem myself last week when I tried to upgrade my Trac to 0.11. I actually reported a bug on MacPorts here. You can see the resolution was that they moved the CGI. I didn't have a lot of time, so I ended up using Time Machine to restore my /opt after I reported this bug, and as soon as I upgrade, I'll update this guide to reflect it. But I think you could just update the Apache config like so:

ScriptAlias /trac /opt/local/lib/python2.5/site-packages/trac/web/cgi_frontend.py

However, there may be something involved with getting .py to run, so as I said, I'll update the guide when I give it a shot.

Modifying the ScriptAlias

Modifying the ScriptAlias definitely gets me farther, but I get an error about clearsilver not being installed in the apache logs. Installing clearsilver (port install clearsilver) seems to get it installed, but it fails to activate it. I'll be curious to see what you find when you give it a go. Thanks for the response.

I'm still plugging away at

I'm still plugging away at this -- see ticket update. I think that MacPorts should be including some build scripts to setup the CGI under the new layout, and I can't figure out how to do it myself because Trac's own documentation (including the internal wiki) isn't fully updated for 0.11 yet.

I've got an update regarding

I've got an update regarding the problematic upgrade to 0.11. Until I figured this out, I was running the following as

root<code>: 
 
<code>sudo tracd -r --port 8000 --basic-auth *,/etc/apache2/other/htpasswd,Trac -e /usr/local/tracroot/

Then, the installation was accessible at http://127.0.0.1:8000/.

However, after reading this ticket on the Trac project, I see I'm not the only one with this problem. The upgrade to 0.11 leaves much to be desired.

Here's what I did to fix (for now):

  1. Install the CGI files as so:

    sudo trac-admin /usr/local/tracroot/&lt;MyProject&gt; deploy /usr/local/share/trac

    I use /usr instead of /opt here since 0.11 installs stuff at /opt/local/share/trac/contrib and deploy won't allow overwriting or adding to this directory.

  2. Change permissions on the resulting CGI script:

    sudo chmod 755 /usr/local/share/trac/cgi-bin/trac.cgi

  3. Edit the CGI script's first line to point to MacPorts' Python:

    #!/opt/local/bin/python2.5

  4. Update the Apache configuration in /etc/apache2/other/trac.conf to point to /usr instead of /opt.
  5. Restart Apache with sudo apachectl graceful.

Seriously? This could be a shade easier. I'll be watching the ticket to see what they do about this.

Well now I'm a bit confused.

Well now I'm a bit confused. If I run sudo trac-admin /usr/local/tracroot/Bezipped deploy /usr/local/share/trac I get the folowing error: *** Unknown syntax: deploy '/usr/local/share/trac'

Running trac-admin help doesn't indicate that it knows about the deploy command. Also, trac-admin -version tells me Welcome to trac-admin 0.10.4

Yet, if I run port installed trac it tells me trac @0.11_0 (active) It seems that trac-admin is a version behind the trac installation.

I know this is probably outside the realm of this tutorial, but if you have any ideas I'd love to hear them.

@Michael: Hmm, only thing I

@Michael: Hmm, only thing I can guess is that perhaps 0.10.4 is still installed, but partially deactivated or something like that. Does it show up in port list installed | grep ^trac?

Also, check which trac-admin to make sure you're using the MacPorts one.

Lastly, you could uninstall Trac entirely and start from scratch :-/

Justin; This site came up on

Justin;

This site came up on MacSB and I've tried to follow it and I think things have gone pretty well...
Using 10.5.5 and sqlite trac backend...

ran the sudo port install python_select and sudo python_select python25 BUT

sudo trac-admin myProj initenv threw an error:
...
WikiStart imported from /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/trac/wiki/default-pages/WikiStart
Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap
the directories appear in /usr/local/tracroot and svnroot...

I went on anyway. When I do http://localhost/svn/myProj all I see is:
Revision 0: /
Is this to be expected? Huh? What do I do with that?

When I do: http://localhost/trac I get a 404 error....

Sigh... Any clues hints/ tips.
I don't have the bandwidth or time for major trip into the weeds....

Thanks for any help!
Steve

@Steve: I'm wondering if it

@Steve: I'm wondering if it has to do with Trac being updated in MacPorts since I last installed? I'm not a python guy, doubly so when it comes to which version to use, but it sounds like you are selecting 2.5 properly. When I get a chance, I will try this on a fresh 10.5 install and see if it happens to me...

@Justin/@Steve Same error....

...I'm working through this right now, and am also getting the same "Interpreter not initialized" just after importing WikiStart.

I know nothing about Python, so wouldn't know where to start debugging this one. If only it were a subversion error! ;p

Incidentally, Steve that SVN page you see is the start of an empty repository (Revision 0) and is as expected :)

SOLUTION! ? Fatal Python error: ... (version mismatch)

I ran into the same issue as many of the above had.


WikiStart imported from /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/trac/wiki/default-pages/WikiStart
Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap

I got this working with a little help from the guys @ macports. Turns out that I had installed the subversion-python25bindings port BEFORE running python_select python25. This caused the bindings to use the native python installed w/ Leopard.

Anyway, to cut to the chase, the solution was:
1- sudo port uninstall trac
2- sudo port uninstall subversion-python25bindings
3- sudo port uninstall subversion (not sure if this was really necessary)
4- python_select python25
5- which python (should report /opt/local/bin/python )
6- sudo port install trac
7- with any luck trac-admin will now run w/o a hitch.

-- Moof!