Code Sorcery Workshop

I have recently, like many other folks, become a fan of my friends Daniel Jalkut and Manton Reece's new Mac developer podcast, Core Intuition (especially the whimsical theme music). In their latest episode, Traveling Luddites, among other things, they discuss distributed version control systems such as Git, Mercurial, and Bazaar.

I'm kind of in the same boat in that I've spent little or no time trying these systems out compared to what I've been using, Subversion, and am a little hesitant to invest a lot of time into migrating and learning a new system. Don't get me wrong -- I'm a hacker and I love to try new technology. However, I don't really have any problems with Subversion and in fact, some time ago I came up with a really sweet setup for offline development, offsite backup, and integrated project management using Trac that I've been meaning to blog about, and this episode was just the catalyst I needed.

Following the directions here, you will get:

• Offline access to your versioning, ticketing, and documentation systems.
• Support for multiple projects (though one project is just fine, too).
• A multi-user setup, if you need it.
• Integration of your code repository with web-based ticketing, GUI changesets and diffing, activity timelines, and project roadmap planning.
• Complete, incremental offline backup whenever you like (and have a connection).
• Very little intrusion into your Mac OS X environment.
• Complete flexibility in getting your data out at any time.

Just have a look around the Trac demo to get a feel for this system. It was a tremendous thrill during the development of Meerkat to see the roadmap progress meter get ever closer to the 1.0 milestone as I closed tickets and definitely helped me organize the project much better than before I adopted Trac.

And this isn't rocket science -- if you're already technical enough to be using version control, I'm pretty sure you can handle this and have a great setup in a short amount of time. I've found that the parts that trip most people up are Trac installation and Apache configuration. Relax, I've got you covered.

A quick historical note

I've been using Subversion in both user and administrator capacities since 2001 and before that, like Daniel and Manton, I too had a painful coexistence with CVS (and still do sometimes with projects like Drupal). I was introduced to SVN by a former co-worker, Garrett Rooney, back in the days when he was one of the few people who knew about and worked on SVN. Since then, he literally wrote the book (Practical Subversion) on SVN and through the years, I've come to really love the project and have introduced my fair share of folks to it as well.

While, as I said, I'm generally open minded about new technology and systems, when it comes to version control I'm in the camp right now of "if it ain't broke, don't fix it". I'm using Trac on some other client projects, including one with a team of over 30 people all over the world, and I use SVN for the development of both Pukka (over two years of commits) and Meerkat (about a year of commits) as well. It works well for me, I don't have hassles, and since SVN has been integrated with Trac, Apple's Xcode, and the Apache web server that ships on OS X, it's going to take free pony rides or something comparable before I seriously look elsewhere for my version control needs. Of course, YMMV.

What you'll need

In order to replicate this setup, you'll need:

• A Mac running Leopard and administrator access to it.
• Familiarity with SVN, but you don't need an existing repository.
• Some Terminal experience.
• About an hour to spare (or less, and not counting some background compilation time in which you can do other things).

If you already have MacPorts installed, you'll save even more time and you won't even need to manually download anything else.

Ready? Let's do it.

Install MacPorts and Trac

I'll admit it, Trac can be a little intimidating to setup because of dependencies. But if you get MacPorts, this can all be reduced to a fraction of the time and very little hassle.

1. Install MacPorts if you don't have it already. The latest installer package for Leopard is here.
2. Make sure that /opt/local/bin is in your path so that the commands provided with MacPorts are available to you easily in the Terminal.
3. Open up Terminal and start the Trac installation process by typing sudo port install trac.
4. Let it roll while you grab a $BEVERAGE or move on to the next steps. When it finishes, you should be able to access a command called trac-admin in Terminal. Depending on what ports you already have installed, you may need to answer some basic questions during the install process, so keep an eye on the window. Brian Cooke's Growl When Done is a good solution here.

Setup Subversion

You'll need to setup a parent directory for your Subversion repository or repositories. I like /usr/local/svnroot, so if you are using someplace else, you'll need to alter the directions below as appropriate. Keep in mind that stuff under /usr/local gets backed up by Time Machine by default, even if you exclude "System Files and Applications". This is separate from the backup mechanism that I'll talk about in just a little bit.

1. Make a top level Subversion folder using sudo mkdir /usr/local/svnroot and change into it.
2. Either move an existing repository into a subfolder of this folder, or create a new repository folder using sudo svnadmin create MyProject in Terminal.
3. Make the whole setup owned by Apache, which on Leopard can be accomplished with sudo chown -R _www:_www . in the Terminal.

Setup Trac

Again, I prefer a similar convention here and put stuff under /usr/local/tracroot. Adjust as necessary.

1. Make a top level Trac folder using sudo mkdir /usr/local/tracroot and change into it.
2. Create a new Trac project using sudo trac-admin MyProject initenv.
3. Edit the MyProject/conf/trac.ini file to configure it. You'll likely just need to change the following items:
   • max_size if you want to upload larger files to the wiki
   • downloadable_paths if you have a non-standard repository structure
   • link to something like http://localhost/trac/MyProject
   • descr, footer, name, and url if you wish
   • repository_dir if you didn't specify it during the initialization step
4. Give your account full project permissions using sudo trac-admin MyProject permission add <username> TRAC_ADMIN in Terminal (replacing <username> with your own username).
5. Make it owned by Apache using sudo chown -R _www:_www . like you did with SVN.

Configure Apache

This is perhaps the trickiest part, but I've done the work for you. I did some poking around and figured out a good way to get this working with the Apache that is built into Leopard. Although I haven't tested the other parts of this guide for Tiger compatibility, Apache is pretty much the part that requires Leopard, since it ships with Apache 2 and has some of the modules necessary for SVN integration built in. If you have your own Apache 2 going with MacPorts or something else, perhaps this will still work for you.

First, you'll want to create a password file for authenticating your Trac and SVN users, even if it's just you.

sudo htpasswd -c /etc/apache2/other/htpasswd <username>

Be sure to replace <username> with your own username. This command will create a new password file at the chosen path and add your first user. If you have other users, repeat the command above with their usernames but leave off the -c option since the file already exists and you don't want to clobber the previous commands.

Next, you want to create a configuration file for SVN to make your repository available over the (local) web and to password protect it. Create a file at /etc/apache2/other/svn.conf that looks like this:

LoadModule dav_svn_module libexec/apache2/mod_dav_svn.so
LoadModule authz_svn_module libexec/apache2/mod_authz_svn.so
<Location "/svn">
    DAV svn
    SVNParentPath /usr/local/svnroot
    AuthType Basic
    AuthName "Subversion Repository"
    AuthUserFile /etc/apache2/other/htpasswd
    Require valid-user
</Location>

After this, make another file at /etc/apache2/other/trac.conf that looks like this:

ScriptAlias /trac /opt/local/share/trac/cgi-bin/trac.cgi
<Location "/trac">
    SetEnv TRAC_ENV_PARENT_DIR "/usr/local/tracroot"
    AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName "Trac"
    AuthUserFile /etc/apache2/other/htpasswd
    Require valid-user
</Location>

Remember in both of these instances to change the paths as necessary if you put your SVN or Trac environments someplace other than the paths I mentioned above.

Once these files are set, go to Terminal and issue the command sudo apachectl graceful to let the configurations take effect. Once you do, you should be able to reach your Subversion repository at http://localhost/svn/MyProject and a list of your Trac projects at http://localhost/trac. Both will be password protected using the account(s) you created above.

Backing it up

You're almost there! The last bit is some scripting that lets you back both of these systems up to a remote server using rsync over SSH.

1. Edit a new text file at /usr/local/bin/ts (or $HOME/bin/ts, if that's how you roll) and fill it with the following:
   

   #!/bin/sh
   echo "Backing up Subversion repositories..."
   /usr/bin/rsync -e ssh -avz --delete /usr/local/svnroot/ <username>@<hostname>:svn/Repository.sync
   echo "Backing up Trac databases..."
   /usr/bin/rsync -e ssh -avz --delete /usr/local/tracroot/ <username>@<hostname>:trac/Trac.sync

   Be sure to replace the <username> and <hostname> values with whatever username and hostname your SSH account has.

2. In Terminal, use chmod 755 /usr/local/bin/ts to make the script executable.

Now, whenever you want to incrementally backup your SVN and Trac databases, just type ts (think: "trac sync") at the shell. Everything that has changed since the last time you performed this command will get copied up to the remote server, keeping a handy backup of both your SVN repositories and your Trac installations should you need to move to another server. And even without the Apache configuration, the SVN repository will be useful as a standalone. Just keep in mind that it is for backup purposes and there is not any two-way syncing going on. That is, your local copy is the authoritative copy and is the one that you should use from day to day, and ideally over HTTP and not using local filesystem access to /usr/local/svnroot directly.

Conclusion

This system has been really useful for me during the development of Meerkat and through the continued development of Pukka, since I've been able to easily track feature requests, bugs, wiki-based notes for future reference, and have it all tied together with Subversion commits. In addition, I've been able to take advantage of offline time while at airports or otherwise on-the-go to actually get code committed, issues resolved, ideas jotted down, and processes documented.

I hope that this guide has been useful to you. Of course, feel free to leave a comment if you have any questions or clarifications or find any typos and I'll try to correct them.

Happy Trac-ing!

Just wanted to pop in and mention a few appearances of our apps around the web recently since the Meerkat launch. Things are really moving this summer!

Again this year, I'm happy to announce that I'll be supporting Seth Dillingham in his efforts to raise $10,000 for cancer research and treatment. Last year, I was able to donate some Pukka licenses and this year, I'll be doing the same, plus adding Meerkat into the mix as well! I'll be blogging more about this as the time gets closer. How can you help? If you are a software user, check out his auctions, and if you are a Mac developer, please consider donating some licenses to your application(s) to help in his fundraising. Seth is riding his bike across the entire state of Massachusetts and all you have to do is continue to use great Mac apps to help!

Next up, I was happy to learn that a fun podcast, NeatLittleMacApps, reviewed Pukka recently. If you like, well, neat little Mac apps, you should check out this podcast and especially Pukka's episode. Thanks, NLMA!

Lastly, stay tuned soon for an update to Meerkat. I'm happy to report that thanks to some great beta testers, there are no known bugs in the 1.0 release, so I'm focusing on some great feature enhancements and additions for the next release. Have an idea? Contact me and let me know so that I can get it on the roadmap. And if you like Meerkat, please consider bookmarking it on del.icio.us (I know a great app you can use), adding it on iusethis, and posting a review on MacUpdate or VersionTracker.

After many months of development, I'm pleased to announce that Meerkat is ready! As the tag line goes, UNIX power, Mac style: SSH tunnels made easy. Meerkat is an easy to use SSH tunnel manager built specifically for the Mac. A lot of blood, sweat, and tears has gone into this release and I'm happy (and relieved) to get to this point.

If you're reading this on the actual site, you may also notice that I've redesigned the website. I hope that it makes more information available, while staying uncluttered and more easily manageable.

Lastly, I've also moved from WordPress to Drupal as my content management system. There are many reasons why, and I hope to blog about them in the near future, but for now, please do drop me a line if anything seems to be out of place.

And now, go get Meerkat! :-)

Thanks to the fine folks at TUAW, I've been alerted to an interesting start to a series at Apple Matters called Using SSH: Secure Tunnels for the Common Man. There's a nice bit of explanation on the need for and the basic concepts behind SSH tunneling.

Of course I'd be remiss to not mention Meerkat, Code Sorcery Workshop's next application, built specifically to address the need for easy and feature-rich management of SSH tunnels. Even for folks like me who know the Terminal commands for SSH and the concepts behind rather arcane uses such as dynamic forwarding as a SOCKS proxy, it's very useful to have menu bar access, Growl integration, Bonjour capability, and other Mac-like features for a classic UNIX tool like SSH.

However, I think that even with a tool like Meerkat, it's important to have some sort of forum or knowledge base for various uses of SSH, which is something that I hope to start when Meerkat launches. It's tricky because part of an SSH tunnel is inherently personal (e.g., your account details), but the basic concept can be shared for other people -- whether it's tunneling to your home iTunes collection, accessing your colocated Mac server securely via screen sharing, or securely browsing the web while on guest networks.

Anyway, give the article a look. I look forward to the upcoming installments -- SSH is a dark art to many and it's good to shed some light on it!

I'm happy to report that a rather longstanding issue with posting in Pukka has been resolved -- on the del.icio.us side of things. That is to say, the version of Pukka that you are using right now will now work properly with no action required on your part.

I had been getting some reports from users of periods of inability to post for a few months now, but only very occasionally. Users would get a message like this when trying to post:

This would happen many times in a row in a short period, but then eventually go away after a few minutes. It turns out that the resolution may have been due to users finally getting a different del.icio.us backend server, one without the problem, but without knowing otherwise on the frontend of things.

Unfortunately, the condition that was producing the posting problem was indistinguishable at a network level from being offline when trying to post (which would obviously fail). So it was difficult to determine if this was a sporadic networking problem on the user side or something deeper. However, in the last week or so, the frequency of folks seeing this problem went up, so I began to hand out some debugging builds of Pukka that could supply me with more information about what exactly was happening.

Once I had this info, I posted to the del.icio.us developer Yahoo! group about the problem and within a day or two, they were able to track it down to a problem in their system.

So in review, I'm happy that this is resolved, but by all means, please do let me know if you continue to see it, especially if you are using Ma.gnolia or another non-del.icio.us service as I have never had a report of a problem under those conditions.

I'd also like to extend an apology to users of Pukka who have been dealing with this for a couple months. As I've mentioned before, it can be difficult when core functionality of your application relies on a third-party, network-based service, so it's important that if you are having a problem that seems to be related to the service to let the developer know so that they can get to the bottom of things as quickly as possible. More votes means more time with my eyes on the problem and hopefully a speedy resolution!

Congratulations to Environmental Working Group for launching Pets for the Environment yesterday! From the announcement:

According to the latest research from EWG, I'm the canary in the living room, soaking up more chemicals than you or your children.

New tests confirmed that I'm full of toxic industrial chemicals, and I'm barking mad. You should be, too. I grow 7 times faster than humans, so what happens to pets like me - increased cancer rates, for one - might be happening to people soon.

Did you know that the humans' government doesn't make companies test our toys, furniture, or even our food for safety?

I did a little work on this site (built in Drupal, by the way), but the bulk of the credit goes to the EWG team and Mike McCaffrey.

Clearly, this is a big problem and I look forward to more of EWG's research as this is something that is affecting our cat, too. Code Sorcery Workshop and Macy are definitely on board!

First off, I realize that I'm a little late to blog about this, but this past weekend I've been in the process of moving. I'll certainly be posting more on that in the not-too-distant future, but suffice it to say that I've been a little busy these past couple days...

Anyway, last week, I was thrilled to hear from my friend Daniel Jalkut of Red Sweater Software that Leo Laporte singled out and mentioned Pukka on MacBreak Weekly, his weekly podcast, as a Pick of the Week. Ever since I learned that both Leo and productivity geek and frequent show guest Merlin Mann had used Pukka, I was hoping that they would mention it. A few months ago, my friend Jim alerted me to the fact that Pukka's posting sound effect appeared in the middle of a MacBreak Weekly episode (Episode 65, about 10:20 into the show) but no mention was made of Pukka -- though obviously someone was using it during the show! That's why in this past week's episode, when Leo and the guys spent a good chunk of time talking about Pukka, I was really excited! I found the spot in the show (Episode 82, about 1:36:18 into the show) and my wife and I sat down to listen.

On the whole, I was quite happy with the feedback -- after all, Leo said "I love Pukka" and "Pukka is simple -- it just does what it does." But pretty quickly, though the feedback about the app never went negative, I heard something that made both my wife and I take pause -- "They shouldn't charge for it, but they do."

I'm not going to rehash arguments for and against this, as Daniel wrote eloquently on this and received 87 comments on his blog post before he closed commenting. I really wanted to get my word in there, but the time for that conversation has passed (however, you should still go over there and read both Daniel's excellent analysis as well as the many thoughtful comments if this is a topic of interest for you.) Other folks also got their say on their own blog posts: Michael McCracken, Tom Armitage, Matt Johnston, and Baron VC, for starters.

One of the commenter's thoughts, about Pukka being no more than two hours of work, is ludicrous enough that I'm not even going to address it save mentioning a few nuggets of Pukka's programming that could keep you busy for more than a couple hours each:

• AppleScript support
• Spotlight support with Core Data
• Scalability testing on up to 25,000 bookmarks
• Custom managed object deletion policy in Core Data (Hint: when deleting a bookmark, only delete each of its tags if it was the last bookmark for that tag, and when deleting a tag, only allow deletion if all of its bookmarks are deleted, but maintain a consistent and responsive data set during these transactions. Also, see above re: 25,000 bookmarks.)
• Sparkle support: Cocoa code, but also a distribution and appcast feed update process.
• Working out fun API issues like inconsistency and client throttling.
• Trying to implement dockless mode while maintaining future compatibility with Leopard code signing. Also, taking the time to blog publicly about it as a way to help other programmers as well as to open your own thinking to analysis & criticism by others.
• Dealing with a Leopard API instability that is core to your application. Also, taking the time to blog publicly about it for your users and for other developers.
• Doing the above with an installed user base on Tiger and Leopard.
• Doing the above, while providing what I feel is an above average level of support, for two years.

I'll leave that argument as is -- blame the above list on a lot of driving in the past few days while thinking about that specific comment ;-)

It's true, at first glance, Pukka may not look sexy (it doesn't produce smoke effects, after all) and may seem to be nothing but a simple form-like frontend to a free, public service, but as many kind commenters on the above-mentioned blogs have pointed out, and as Leo himself summarized when he said, "It doesn't do much -- I just love it!", Pukka represents hours and hours of thought and planning and attention to the user experience -- that is what is king in the Mac software business. Mac software for me is like a good relationship -- the more time you invest in the software, the more richly you will be rewarded with little surprises and delights along the way. You can see the kernel of this idea in my first post here over two years ago:

I set out to write a program that does the actual posting and I had it working that night. It grew from there as I lovingly tweaked it and worked with beta testers to make it better. I hope you like it. What’s more, I hope that you’ll support my efforts so that I can keep the creative juices flowing.

I followed that up a few months later when I started charging for Pukka:

If you look at my blogroll, you’ll see over 30 Mac developers or products that I admire, nearly all of which I use and have paid for, if they charge. I’ve easily spent several hundred dollars on Mac shareware [...] The Mac has shown me what it has undoubtedly shown you — that software is art, and Mac users have deeper loyalties than price points.

On the topic of pricing, it's true that there is some voodoo to determining the price of a piece of software that you've created, and I say that even though I do not make my entire living off of Mac software (though I will point out that Pukka enabled me to bootstrap my own business, including starting out on my own and enabling me to attend developer conferences and meet like-minded entrepreneurs.) I don't typically like to quote the same blog when discussing two different points, but again Daniel Jalkut nails this topic in his post The Price is Wrong from several years ago (there's a reason Daniel attracted a Daring Fireball link and 87 commenters on his latest post.)

I'm all on board with free software if it works for the developer -- in fact, the day the MacBreak Weekly episode was published, I released a little freeware application called Snarf for icon designers. I maintain and build Drupal modules. And before that, I released a number of open source programs like Ticketsmith, spliff, and purgeimap.

But for me, it all comes down to this: what will the market support for an application that, for the right user, adds at least that amount of value to their workday? For me, right now, that's what I'm charging for Pukka. I hope that you agree, but if not, I fully support your right to make a better application, at a price point that works for you, or to use the free services that Pukka works with yourself directly. And I'll even help you out if I can, because others have certainly helped me along the way.

A friend of mine asked me today if I knew of a quick and easy way to preview an ICNS file as it would look on an application in Mac OS X's dock. I didn't, aside from opening an existing app's bundle and overwriting its icon or something like that.

Late tonight I decided that I could probably whip up a tiny app to accomplish this sort of thing in far less than an hour, and I did! So I give you: Snarf (requires Tiger or Leopard). Snarf allows you to drag an ICNS file either onto its main window or onto its dock icon and will instantly update to use the dragged file as its new icon.

There's not much more to say than that, so enjoy!

Another conference update -- I'm at SXSW in Austin, Texas. If you'd like to follow my whereabouts, hear what I'm up to, and/or would like to meet me, you can follow me on Twitter. I can't promise that it will always be coherent as time is limited and my sub-iPhone-quality SMS messaging is not the easiest, but it seems to be what all the kids are using and it's helping me out in a big way to figure out where people are headed and what sessions are good or bad.

So again, meet me at South By Southwest -- I hope to see you here!

Though I'm headed to SXSW in Austin later this week, I'm in Boston right now for the twice-annual Drupal conference, aptly named Drupalcon. If you're around, say hi -- this is a fairly good representation of what I look like, though sadly I do not have my hat, or at least a non-winter hat. It's kind of cold up here!

Today, the first day, was pretty engaging and inspiring. I'm excited to see the roadmap for the future of Drupal, and I'm always on the lookout for knowledge that will benefit me both in the Drupal services that I provide as well as some cross-pollination of ideas between both web services and Mac desktop software, be they data storage, user interface, industry trends, or any number of other issues. Not to mention meeting & hanging out with some great folks!

Probably the most interesting session so far for me was the last, on Drupal in China, covering issues as varied as outsourcing & off-shoring, governmental technology initiatives, software piracy, language barriers & internationalization, community involvement, and lifestyle & free-time activities -- which of course, if you look at them, are all integral to China's current standing and future trajectory in the technology scene. Really interesting and fascinating stuff!

Though it's of course easy for me to hang out with the large DC representation up here, I do need to meet some other folks, so again, don't be shy -- come say hello. And enjoy Drupalcon!