Drupal

In honor of this week's Open Source Bridge conference, as well as in recognition of the role that open source software has played in the development of our business, we're pleased to announce that today, June 16, 2009, Code Sorcery Workshop is offering any open source contributor a free license to Meerkat, our SSH tunnel management application. We are also giving away a $250 gift certificate to the legendary Powell's Books. Read on for the details.

If you'd like a free copy of Meerkat, just leave a comment on this post linking to an open source project that you've worked on with a brief mention of what you did. It could be coding, but doesn't have to be -- it could also be documentation, helping new users, anything that contributes to the common good of the project. We'll collect all the info and send each contributor a full, unrestricted license to Meerkat, a $19.95 USD value.

However, if you'd like to instead try for the $250 USD gift certificate to Powell's Books, a purchase of Meerkat will make you eligible for this drawing. Just register Meerkat today and you will automatically be entered for the drawing. The winner will be announced in a followup post.

In both cases, you must take action by midnight Pacific Daylight Time tonight to qualify.

Meerkat is an application that adds a lot of Mac-specific value to SSH, an open source tool that ships with every Mac (as OpenSSH). And Macs themselves are built on a ton of open source software such as Apache, Postfix, CUPS, Perl, PHP, Python, Ruby, sudo, unzip, zlib, and many others. You can read more about Apple's commitment to open source as well as open source releases pertaining to Mac OS X.

I began knowingly using open source software in the mid-90s and started contributing by releasing my own projects on freshmeat in late 1999. I've always looked for ways to contribute to open source projects when I can, whether it's by bug fixes, new feature patches, documentation, or just community help. Most recently, I've been involved with the Drupal content management system.

Open source is the lifeblood of the internet. So many of the tools that we take for granted everyday have been developed in this way, by generous folks giving their time for the greater good. I am extremely thankful for the many ways that open source has enabled me to teach myself a lot of what I know today about technology, to provide economical solutions for clients who need it, and to make software better and better by degrees.

So, here's to open source!

I'm currently at the biannual Drupal worldwide conference, Drupalcon, in Washington, DC. I hope to write more later about the amazing people and innovative technologies that I'm seeing here this week, but for now I just wanted to post an entry for folks arriving here looking for information about the talk I gave yesterday: Beyond The Web: Drupal Meets The Desktop (And Mobile).

I will be posting more information later on the resources & projects mentioned in the talk, but for now, watch this page for that info as I'm able to post it. I covered a number of technologies that can connect Drupal websites to desktop and mobile applications, be they in Cocoa (for the Mac & iPhone) or any other language that you might want to integrate with a PHP-based Drupal site.

Hope to see you at Drupalcon!

I wanted to take a moment to point out two items of mention related to Drupal from the past week.

First, congratulations to the United Nations World Food Programme, which recently relaunched their website in Drupal! Congratulations as well to both Phase2 Technology and Workshop friend Development Seed for the outstanding job on this beautiful and functional website.

WFP's new Drupal-based website

I had the privilege of working with the WFP a few years ago in preparation for Walk the World 2005. The website for that project was one of the very first Drupal sites that I programmed on (having administered a few before that). I'm glad to see that Drupal has made inroads into the WFP's structure all the way to the top since then, allowing them to maintain this great new website and further their humanitarian efforts around the world.

The other bit of info is about Drupalcon this March in Washington, DC. Today, the final ticket was sold, and the attendee count will be 1,300 people! The tentative session list, based on submitted proposals, is now out. I hope to be involved in presenting two sessions, my own proposed Beyond The Web: Drupal Meets The Desktop (And Mobile), which is likely to become a panel presentation on the topic, and Miglius Alaburda's Introducing a new File Framework, which is likely to become a joint session on file handling in Drupal. I also hope to be releasing a new Drupal module in the next week or two in these areas, so stay tuned.

The amount of momentum around Drupal right now is pretty astounding!

Today is the eighth birthday of Drupal, the PHP website platform that I'm most involved with. In Drupal creator Dries Buytaert's own words from the news release:

When I started work on Drupal as a graduate student, Drupal was just a little hobby project grown out of my own interest in the web. As you can tell from the original release notes, being the only programmer certainly had its charms. ;-)

Fast forward 8 years, and we're a global project with hundreds of thousands of users, thousands of active contributors and a healthy ecosystem. Along the way, I've always tried to listen to the community, and to trust my own instincts and moral compass. We built an amazing community together, and because of that, working on Drupal continues to be a labor of love. Even after eight years.

A big project can't always do what a small project can; there is more legacy and overhead, but nonetheless, I think what is important is that we stayed true to our initial values: innovation, collaboration and a healthy desire to keep the code as small and simple as possible.

I'm very much looking forward to where the next eight years takes Drupal and the web at large!

If you are interested in finding out more about Drupal, have a look at groups.drupal.org to see if there is a group near you or head to DC in March for Drupalcon. I just attended my first Portland Drupal meeting last night and hope to be more involved with the community here going forward.

Viva la Drupal!

At last, I'm happy to announce the Code Sorcery Workshop support forums! These forums will gradually become the official support channel for our Mac products Meerkat and Pukka, as well as a place to discuss what's on your mind with regard to our website, potential future products, our services, or happenings in the Mac & Drupal communities.

The forums have been open for a week or two in unannounced form, but have quite expectedly not garnered much activity, so consider this the official "word". Feel free to go to it!

Feature Run-Down

We are using Drupal for the forum solution, which is what is used for the rest of the website as well. I'd like to take a moment to go over some of the features that this provides. In the near future, I also hope to make another post about the more technical details, such as which modules were used, what kind of custom solutions were implemented, and what administrative features are provided on the backend.

Main Page

The main page gives an at-a-glance view of the latest topics, much like any forum software. Posts are organized into containers, such as Mac OS X Products, and below that, forums addressing a particular product or group of topics, such as Pukka. When new topics are posted under a forum, they bubble up to the top.

Forums main page (click to enlarge)

Your Account

To participate in the forums, you must register for an account. With this account, you can maintain a unique identity across all of the posts. You can include as much or as little information as you like, currently including real name, photo or avatar, physical location, and website. This information is only available to other forum users -- only your username is available publicly.

Account page (click to enlarge)

In addition, you get a box in the right sidebar with easy access to My forum posts (posts created by you) and My forum votes (posts you've voted on).

User box

Search

Just like the blog archives and all of the pages on the site, forum topics are searchable. And these searches are able to be bookmarked, so you can easily check back frequently for updates related to a topic you are interested in.

Topic Voting

Aside from easy access to any forum topics that you may have created yourself, you can also vote on anyone else's topics using a zero- to five-star rating system. Perhaps the best use for this feature is that you can use this to flag topics that you are interested in periodically checking back on. Another use might be a tip that you really want others to see or a feature request that you'd like to weigh in on.

The popular topics get aggregated to a special page called top forum topics where they can be easily tracked. I'm hoping that this can be a useful way to chart the future direction of our applications, as well as to more easily resolve important issues affecting many customers.

Topic voting

Feeds, Feeds, Feeds

One of the strongest features of the forums is easy and plentiful RSS feeds. Currently, you can access feeds for:

• All forum topic activity: codesorcery.net/forums/feed
• Container activity: For example, all posts about Pukka. Just add /feed to the end of any container URL.
• Topic activity: If you make a post, you subscribe to all comments on the post by clicking the link on the topic page or adding /feed to the URL.

Forum feeds (click to enlarge)

Conclusion

In conclusion, I'm happy to launch the forums and I hope that they will be of benefit to users of our products, Mac, iPhone, and Drupal enthusiasts, and folks interested in our services, for starters. Please, if you have any suggestions or feedback, consider using the General Discussion forum topic.

Enjoy the forums!

I'm happy to help announce that the next Drupalcon will be March 4-7, 2009, in Washington, DC! As a former member of the awesomely rocking Washington, DC Drupalers, it will be a real thrill to get back to the city as part of our biannual pilgrimage to the hub of all things Drupal.

Expect this to be the biggest Drupalcon ever -- the first 100 tickets sold out in ten minutes, but others are still available. I've attended Drupalcon Barcelona and Drupalcon Boston in the past, but I'm looking forward to this one especially as DC is where Code Sorcery Workshop got started. Plus, we'll have a new president and the whole town will be abuzz.

In addition, I'm proposing a session entitled Beyond The Web: Drupal Meets The Desktop (And Mobile). If you're interested in seeing this session, please head over to the Drupalcon DC site and vote for it. Here's the abstract:

We all know that Drupal is a robust self-contained system for running a social website, but what about when you want to hook it up with the desktop or mobile devices? This session will take a look at the capabilities built into Drupal core, as well as contributed modules, for combining Drupal with desktop and mobile clients. The presenter is a Mac & iPhone developer and the co-maintainer of the DAV API, File Server, Boost, and Trace modules, and inadvertently had code in Drupal core in 2001.

Whether you are a Drupal veteran, a web developer who's been meaning to try it out, or a newbie (technical or not) who's wondering what all the talk is about, Drupalcon is a great (and relatively affordable) way to tap into the vibrant Drupal community. There's no better place to be if you are, or want to be, involved with Drupal in any way.

Hope to see you there!

I recently had the pleasure of participating in the excellent Mac Developer Roundtable podcast, where the featured topic was open source software. I've been involved in open source software myself for about ten years, so it was great to, first, be asked onto the show, but also, to share my experiences and knowledge -- not to mention, to "meet" and discuss software with some great fellow Mac developers!

This podcast was particularly satisfying because it brought together the two main halves of my business -- Cocoa and Mac development, where I tend to use open source software, and Drupal and web development, where I tend to contribute open source software.

So, have a listen to the show and if you'd like to check out my main open source efforts, I have an info page with more details.

Like many other people, I received an email this morning announcing that Mollom, a project of Drupal founder Dries Buytaert, is out of beta. I use Mollom on this website, as well as another personal one, to reduce and sometimes outright eliminate blog comment spam. In my configuration, Mollom works by using heuristics against blog comments and if they are considered spammy enough, the user is challenged to a simple CAPTCHA system to prove that they are not an automated spammer.

Mollom is a great system, since some studies (direct PDF link) have shown that as much as 75% of all attempted comments on blogs are spam!

I've been using Mollom for a few months and have been very happy with it. If you run a blog, even one that isn't in Drupal, I would encourage you to check it out. It's free for many uses, so you lose nothing by at least giving it a shot. There is a developer API and Mollom has already made downloadable integration available for Drupal, WordPress, and Joomla, as well as language-specific libraries.

Congratulations to Dries and the Mollom team! I look forward to the system only getting better as more and more folks start to use it.

Update: A couple folks asked if I'd mind posting my Mollom statistics graph, so you can find it here. There were many hundreds of spams stopped in the past month!

I've finally been able to make a public, alpha release of the File Taxonomy Server for Drupal 6, allowing integration of a Drupal site's category structure into a drag-and-drop WebDAV interface. I blogged about this software a bit previously, but am finally getting some code out there. If it sounds like something you'd be interested in, then I encourage you to check out the project and give it a spin.

I've fixed a couple minor bugs and now the install instructions mention the latest tested versions of the DAV API and File Framework projects, so it should be pretty easy to get going.

You can create and edit categories and upload and tag files to your site, all from the comfort of your WebDAV client, including Mac OS X's Finder, Windows' Explorer, or other applications like Transmit, Coda, and Cyberduck.

Feedback is welcome, but please report bugs over on the project page, not here. I welcome comments and feedback, but please try to keep official feature requests and bug reports over there.

I wanted to mention a few technologies that I've been working with and contributing to lately. This post will be pretty tech-heavy, so I'll throw out some keywords ahead of time so that you can decide if this is something that you'll be interested in reading further: WebDAV, desktop-web integration, RDF, and content-addressable storage, all in the context of my content management system of choice, Drupal.

My work specifically has been in porting and updating some work started by Arto Bendiken (whose projects, it seems, I just start to wrap my head around, and appreciate the foresightedness of, about two years after he himself conceives of them). Aside from writing a guide to his high-traffic caching module, Boost, and porting his debugging tool, Trace, to the latest version of Drupal some time ago, I've been porting his Drupal-WebDAV content bridge, File Server, fixing some bugs in the underlying DAV API, and integrating with the updated File Framework that Arto and my other colleague Miglius Alaburda have been working on.

I'll tackle each of these technologies in layers, starting from what the user sees on down to the gory details under the hood in how the files are stored and queried.

File Server

File Server lets you take a WebDAV client, such as Mac OS X's Finder or Windows Explorer or, better yet, a richer program like Transmit or Cyberduck, login to your Drupal-based website with it using the account that you already have, and drag and drop the files and folders presented there to you. Assuming your Drupal site has file nodes (essentially, chunks of content in the form of file uploads) on it, this dragging and dropping can be used to re-categorize the files, upload new files, and change your site's category structure.

Here's an example:

1) I have a Drupal-based website.

2) I connect to http://my-website-url/dav in a WebDAV client and login with my Drupal user credentials. I'm then presented with a view of my tags:

3) I drag a file from my desktop into a folder corresponding with the tag that I'd like it to be categorized under:

4) When I go to my Drupal site, the file exists as a node and has been tagged appropriately:

This happens via File Server, which I've recently ported to Drupal 6, and Arto's DAV API, which lets you hook all kinds of Drupal facilities into a standardized DAV interface. In this case, we're connecting Drupal's taxonomy (i.e. tags and categories) and in turn, its file nodes, to DAV. It makes a lot of sense here because the files are going into Drupal as file nodes, with automatic conversion into other formats, automatic metadata extraction (as seen in the sidebar above), and automatic indexing into the search system.

File Framework

File Framework is the link between the actual files and the node structure in Drupal. In short, File Framework takes the default facility in Drupal for file uploads and replaces it with a more robust system for backend storage, exporting of info, and conversion into alternative formats. For example, you can upload a PowerPoint presentation and automatically get related nodes out as PNG, PDF, Flash slideshow, and more.

File Framework is a lot of under-the-hood stuff and is in active development for Drupal 6. It also builds upon two other frameworks, RDF and Bitcache.

RDF

RDF stands for Resource Description Framework and is not Drupal-specific, though it embodies a concept that Drupal is trying to move towards. RDF is a step in the direction towards the semantic web, where computers can understand what data is about, not just what it contains.

The most common application for this technology is search engine technology. Today, when trying to find a picture, we search for web pages that contain the words picture of a sunset and only turn up hits if those words are found. In the semantic web, this info could be found because the search engine can understand that there is a person who has a name of Joe, who has a profession of professional photographer, and who has a website at http://joephoto.com, which in turn has a file which is a digital photograph, which has a description containing the word sunset. As a result, some context can be gleaned by the computers indexing all of this stuff, such as the fact that this likely to be a good photo of a sunset since Joe is a professional photographer, and can provide much richer info than the search engine merely looking for words that the content author may or may not have written near the object in question.

Anyway, RDF is the stuff that stores these triples, the idea of subject-predicate-object, e.g., website has a file. Arto has hacked together an API for RDF storage in Drupal and the File Framework now uses it. This RDF storage facilitates not only descriptiveness on the site, but also helps with cross-site searches when this data is needed.

None of this stuff is useful if people have to focus on creating the RDF, so this RDF API combined with Drupal makes it easy for people to keep doing what they were doing before and have the system take care of all of this context stuff.

Bitcache

Bitcache is a means for content-addressable storage (CAS), which means that, unlike most filesystems that we deal with today, the address or URL pointing to a file is based merely on the content, not on the set location. To put it another way, when you put a file on your hard drive, it gets an address like file:///users/justin/myfile.txt that is assigned arbitrarily (well, actually by you based on how you name it and where you put it, but it's arbitrarily related to the actual content). In Bitcache, a unique string of letters is calculated when the file is put into the system and as long as the content remains the same, the pointer URL to that file will remain unchanged. When the file changes, a new copy is created which necessarily has a different address; however, the old file remains in the system as well. So, a benefit is that a given file that exists in a given state is never put into the system twice, since it can be continually referred to by its content address. Another benefit is that the old versions are necessarily retained for archival purposes as well.

A quick hypothetical example: a computer server is in use by an office of people to store their MP3 collections on. When someone puts a new song on the server, it gets a content address. When a second person later puts the same song on the server, a second copy is not created -- instead, the second person gets a reference to the original file's address, since the file is the same anyway. That way, half the storage is used. As more people add the same song to their collections, the storage benefit increases. This is a simplified and contrived example, but it's the basic gist of things.

The File Framwork for Drupal makes use of Arto's Bitcache project. So, when you are using the stack of tools I've been talking about here, under it all, you also get the benefits of CAS behind the scenes.

Conclusion

I'm going to stop there, as I'm sure I've done some grave injustice to some of the complexity involved, left out some of the cons that come with the pros of these systems, and I've probably mangled some of the descriptions too. But then again, that's why I'm working on a piece of it and not the whole thing ;-)

I get excited about this stuff not just because of the Cool Factor™, but also because this sort of thing, when combined with Drupal 6's capabilities in the workflow department with triggers and actions, can lead to some powerful publishing and conversion capabilities for file uploads. And all of this paves the way for Drupal to be the most forward-thinking content management system out there.

Questions? Comments? Corrections? Leave a note below and let's sort it out.