Boblet Learned things and Japan diary, by Oli Studholme

URL cruft, and how to remove it

By Oli
At 2:19 PM · Monday, 29 December · 2003
To Coding · Unix · Weblogging

I just posted a comment on non-crufty URLs that turned into a novella over at Whitespace, so here it is rewritten as an article for future reference:

An important but overlooked aspect of websites is the URL, or Uniform Resource Locator. This is the ‘web address’, usually of a page on the internet. While it seems minor, this is part of the page’s interface, and some simple rules can make a big difference in ease of use. Important ease-of-use elements for people are the URL should be:

• easy to type
• easy to remember
• short if possible
• ‘hackable’ ie predictable enough to guess
• and permanent

For search engines there’s really only one important thing:

• the URL shouldn’t contain troublesome characters, usually ? and &. ? usually means the URL came from a script, so search engines won’t index it. & is the HTML delimiter for escaped characters eg & is written in HTML as &. This makes URLs with & in them not validate (insignificant but annoying).

Big sites using a CMS usually have a programmer using mod_rewrite or something to do URL mapping — this is definitely the way to go. Unfortunately mod_rewrite is hard (for us mere mortals anyhow). For sites that don’t use URL mapping, here are some links that might be interesting:

Background Information:

Cool URLs don’t change by Tim Berners-Lee

This article advocates making sure a URL will exist in 2, 20 or even 200 years. URLs only change when some information in them becomes outdated, so the important thing is not to put in anything that might change. This basically leaves you with the date of creation and the page title. Not using an extension is also good, as you can just set up extensions to check in order of preference in .htaccess.

URL as UI by Jakob Neilsen

Of course Jakob has something to say about this :-) He advises on domain name choices & avoiding link-rot, and also states that URLs should be shorter than 78 characters. This is the typical line length where email will break a URL over two lines.

The Perfect Blogging System by Matthew Thomas

Matthew lists what he thinks the requirements of a perfect weblog are. His ideal permalink is something like http://domain.name/2003/07/26/keyword.

How to recognize a Weblog tool by its permalinks by Matthew Thomas

This shows the cruft in a variety of weblog software default URLs.

When Good Interfaces Go Crufty by Matthew Thomas

This article is about software and OS interfaces, but is relevant to many things. Highly recommended. Cruft is used to mean “extra stuff that isn’t necessary”.

How to improve URLs:

…for MovableType

Cruft-free URLs in Movable Type, by Mark Pilgrim

Mark details how he changed from MT’s default URLs to cruft-free date+title ones. You can also check out the Dive Into Mark templates. Note that to remove the index filename from category and date archives, you need to replace all links to these archives with manual ones that don’t include index. eg instead of using the <MTArchiveList archive_type="Monthly"> and <$MTArchiveLink$> tags, you’ll have to hard-code the links, eg
(<a href="http://domain.name/year/month/">December 2003 archive</a>)

Howto: Future-proof URLs in Movable Type, by Már Örlygsson

This achieves a similar result, but Már uses regex and hacks the MT source code to do it. For the hard-core ;-)

…General Methods

Making “clean” URLs with Apache and PHP, by stef*notabene on eVolt

This article explains how to turn query string-type URLs into clean ones, and is aimed at people using PHP (eg /index.html?cat=coding&story=00056 to /coding/00056.html, or even to /coding/goodurls). You’ll need PHP, Apache (.htaccess and ForceType), and some way of storing the lookup tables (maybe a database).

Clean url’s, by Thijs van der Vossen

Thijs details a mod_rewrite rule that forwards URLs lacking a .html extension to the filename+.html, if the file exists.

Apache Content Negotiation and Multiviews

For the hard-core! You could also consider going super-hard-core with the Apache 1.3 URL Rewriting Guide by Ralf S. Engelschall (very scary :-)

The method I use is based on using Mark Pilgrim’s code with Brad Choate’s MT-KeyValues plugin. I use a KeyValue (if present) to replace the article title; instead of a dirified title (I like long, informative ones) or using keywords like Mark does (I sometimes actually use keywords, for things like UK/USA spelling differences), I can specify exactly what I want. You can see the code I’m using in Read the funky manual. Note the title of the post is “Read the funky manual” (not RTFM ;-) It gets the URL slug via a KeyValue of “url=rtfm”. The title for this page is “URL cruft, and how to remove it”, but using a KeyValue of url=urls I end up with http://oli.boblet.net/2003/12/29/urls.