Most Fridays we hold talks known as 'osmotics' at Last.fm. Their purpose is to share understanding about the various things that are going on in different departments; an open session which anyone can use to present recent work. It's a response to our growth, as it's now impossible to know what's happening throughout the company at any given point in time.

A few weeks back I presented some of the work we've been doing on interaction path analysis. As web developers we're completely spoilt for usage information on our apps, and interaction histories are sitting in our apache logs the entire time. We've been using tools to mine these and better understand real interaction flows on our site, in order to improve our understanding of common use cases.

You'll often find that your users don't use your software exactly as you designed for. That's natural, and as web developers we need to be equipped to observe these flows and adapt/optimize our software accordingly. As Stewart Brand observed of buildings, so too good software should learn from usage patterns rather than working against them. A good app should flow.

We use a tool suite from Omniture - the pricing may well be out of reach of most startups, but anyone can mine apache logs to get similar information. Bear in mind the presentation assumes no technical knowledge and was prepared in about 20 minutes - it's more of a lightning talk than anything. I've blacked out some sensitive bits of information.

→ Interaction Path Analysis - slides with annotations (1.1MB)

If a building is allowed to fail small, early and often, and be corrected, the building as a whole can succeed

- Stewart Brand, How Buildings Learn (1995)

I'd heard of Stewart Brand's book How Buildings Learn before, but didn't know there was an accompanying TV series (with music by Brian Eno). The TV series is available in full online (episodes 1-6). By focusing on what happens to buildings after they're built, Brand articulates lots of principles that we use in software today. Brand's own notes even refer to it:

Most of the 27 reviews on Amazon treat it as a book about system and software design, which tells me that architects are not as alert as computer people.

It's interesting to see so many insights that tie together the two kindred fields of architecture and software design in one place. The fact is architects still struggle with the seeming permanence of their design decisions - even big memes of recent years like parametric design haven't laid the path for conscious design of adaptive buildings yet. And as Brand points out, it's not neccessarily a technological question, simply a re-definition of the architect's role and attitude.

We, on the other hand, have got it easy, working in a fairly pliable medium of pixels on screens, with tight feedback loops and the opportunity to evolve our design over time both available at a low cost.

This talk is worth a look. Forgive the title (which is a misnomer) and the way it frames REST; equating it to feeds designed to be consumed by polling clients - Newtonian physics to PubSub's quanta (poorly formed analogy). REST and RPC simply are suited to other types of services (I fail to see how RPC over XMPP is useful unless the time to process a request is long and callbacks are required). The talk goes on to use concrete examples that illustrate how hijacking Jabber servers and XMPP for generic push messaging using a pubsub architecture is far more efficient for lots of web services outside of IM that are currently pull-based. They even manage to fudge oAuth in for protected resources. It's the kind of pragmatism that smacks of real life problems solved (I salute).

I'd be very surprised if Last.fm's 'now playing' notifications didn't switch over to pubsub very soon.

Boriska, a young boy over-seeing the casting of a bell, in Tarkovsky's Andrei Rublyov

Most developers aren't paid to think about what they build. This "divorce of head and hand", as Richard Sennett puts it, is counter-productive. The more pronounced this divorce, the less likely you are to innovate. Why? Because programming is a craft, and innovation happens in the midst of practising this craft. Sennett's latest book, The Craftsman, is a study of craftsmanship through the ages. I'm writing this as a collection of my after-thoughts on the book (which I'd recommend, despite its grammatically inept first edit).

The workshop ethic

Sennett discusses the history of the workshop and explores what he calls the 'material consciousness' of a craftsman who knows his tools and working materials intimately. In a workshop, authority is equivalent to skill. Historically, the most skilled craftsmen (masters) have had authority over journeymen and apprentices in their workshops. We could learn from a closer modelling of development teams as workshops - most developers are told what to build (and in some cases even how to build it) by managers who don't possess a 'material consciousness' relevant to software engineering, which often leads to problems.

On Engineering

Why does our industry separate out the 'lead engineer' from 'engineering management'? Dealing with software engineering as craft would indicate that a hybrid works better - craftsmen are motivated to work for better craftsmen, in order to improve their skills. In the traditional workshop, the master craftsman would interact with clients (read: product managers/executives) and manage the rest of the craftsmen whilst running the workshop.

By this logic the most effective software engineering manager is an experienced, highly-skilled programmer (hello Mr. Gates). By definition they should still be programming ("If you stop coding, you stop learning" - Beck), for the same reason a master craftsman never actually stops practising his craft. Because to stop practising is to stop evolving the material consciousness that makes you a great craftsman in the first place.

The arguments against this approach are obvious - how's a lead engineer also going to manage developers? Surely he can't do two jobs? Let's take a step back. In order to manage great developers effectively you need to gain their trust and respect. Without this you'll struggle to get anywhere. Upon which we come to a programming truism:

In the work-place, the only way to get the full respect of a skilled programmer is to prove yourself at least as skilled as them at programming, and the only valid evidence is actual working code.

I don't think I've ever met a developer for whom this isn't true. Bear in mind that your average software developer, not known for their humility, will often not dispense this kind of respect until you've coded alongside them (or in a visible forum such as that of an open source project) for a while.

Given this, you can't feasibly manage great devs without being one yourself. I mean you can (it happens all over), but it's a fruitless, unproductive and frustrating operation for all involved compared to what happens when you get it right. In this regard, I'm assuming you agree with the notion that programmer productivity is a completely non-linear phenomenon (see Paul Graham). If you address the issues of trust and respect at the core of a programmer's productivity, you open up a lot of potential. This overrides (by an order of magnitude) the fact that a hybrid lead/manager doesn't have as much time as two separate people, because it gives the individual developer a higher degree of agency in their work and a clear reporting line to a developer that a) speaks the same language as them, and b) which they are motivated to work for and with.

Pretty much all successful startups naturally possess someone who fulfills this hybrid role. It's only when companies grow that this 'workshop' ethic is lost.

Another argument against letting a lead actually manage anything is that the lead has lousy social skills. You'll hear a lot about skilled programmers that are completely socially inept. In application development, I don't even buy this notion. To be a great application developer you have to have superb natural language skills as the entire craft is equal parts natural language and logic. Algorithm-driven programming may well be a different kettle of fish, but app devs absolutely are measured by their capacity to communicate in natural language.

Design & engineering: hybrid minds

This kind of logic about craftsmanship leads to some more questions - particularly around the separation of 'front-end engineering' from 'web design'. Why? The syphoning off of these two disciplines is a symptom of this separation of head and hand; one that stifles the feedback loop of discovery and product evolution at the heart of craftsmanship (the 'designer' only has to think about the final product, not actually craft it). This design/engineering split is obviously evident across sectors (take architects and civil engineers), but rarely do you find a practice like software, where results can be near immediate and design/build feedback is quick. If designer and engineer are to be separated then they will need to work as a hybrid mind in order to innovate in any case (e.g. if Cecil Balmond, a lead at ARUP, and architect Rem Koolhaas hadn't developed this kind of relationship over decades, you wouldn't get results like this).

On technique and expression

Sennett explores the craftsman's worldview that technique "is intimately linked to expression". I believe this to be true. There's an obvious difference between a purely technical innovation (such as the evolution of object-oriented design paradigms in the design of Smalltalk and its dev community in the early eighties) and a software product innovation (e.g. Joshua deciding book-marking the web should be a social activity), but I intuit that the two are linked. Both these types of innovation (indeed, all types of innovation) are highly expressive acts, and it's impossible to be expressive without cultivating expertise in technique.

On 'theoretical' practice

This leads me to think about so-called 'theoretical' disciplines such as pure maths and quantum physics that, whilst being highly abstract, are expressive and require technical expertise. Can such theoretical practices display a 'material consciousness'? Physicist Richard Feynman is a good example of how they can. In Feynman's anecdotal autobiography, he talks about his need to come up with clear material examples when discussing any kind of physics problem, no matter how abstract. His grasp of mathematical techniques was rigorously practical (and as a result, unorthodox) as he was always trying to solve 'real world' problems. He was as skilled in his ability to grasp the right problems (it's no coincidence that the word 'grasp' alludes to the hand and the handling of materials) as he was in his ability to solve them. You can't put it better than Feynman when he says "What I cannot create, I do not understand".

The pyschology of the craftsman

Lastly, a note on what I think might be one of the most illuminating insights into the psychology of the craftsman in the history of art, Tarkovsky's epic film Andrei Rublyov, in particular the penultimate vignette, which depicts a young boy that takes on a commission to cast a huge bell for the Russian Prince in the 13th Century. Those 30 minutes of cinema express a lot more than can be summarised here, depicting in detail the process of founding and casting the bell, and the young boy's descent into a craftman's obsession (his life depends on the bell ringing true). I'd simply ask you to watch it. There are other observations on the artisan/craftsman and their relation to religion and art (a foreign concept in the period the film is set in) throughout the film, which is the (mainly fabricated) biopic of a famous Russian icon painter.

PS: Whenever i've used the term 'craftsman' or 'craftsmen' I actually mean men and women, obviously.

Glad to have been a part of this in some way (I handle all the API's). Now all we need is for Apple to approve the app for their store.

Update: It's been approved now so you can download it for free from the store. Toby has the full scoop.

AS#Enumerable is a port of the Ruby Enumerable API for Actionscript 2.0. It provides the following enumerable methods for use in your flash applications:

• all,
• any,
• collect,
• detect,
• each,
• find,
• findAll,
• include,
• inject,
• map,
• max,
• member,
• min,
• pluck,
• reject,
• select,
• size,
• sortBy

It's provided as a subclass of the native Array object, but if you want all native Arrays to inherit the enumerable interface (a la Prototype) just add:

Array.prototype = new Enumerable();

var enum = [];
enum.push(0.2);
enum.push(0.3);
enum.each(function(item,index) {
  trace(item + ':' + index);
});

var enum = [];
enum.push({value:'foo'});
enum.push({value:'bar'});
enum.push({value:'camp'});		
var item = enum.find(function(item,index) {
  return item.value == 'bar';
});

var enum = [];
enum.push('foo');
enum.push('bar');
enum.push('camp');	
trace(Math.floor(enum.inject(0,function(strlen,string) {
	return strlen + string.length;
})/enum.size()));

Go ahead and grab the source code, which will find its permanent home at /code, should you want to check back.

See the prototype enumerable API doc for further documentation, as the usage examples are identical in Actionscript 2.0. Thanks to Sam Stephenson of Prototype for the JS port.

[7:02pm] alex: Haskell users are all pretty nuts.
[7:02pm] johan: when the revolution comes they're first up against the wall
[7:02pm] alex: johan: Actually, they won't.
[7:02pm] alex: Haskell does not guarantee execution order.

it's not enough to just instruct a machine to do something that you need also to think about what will people read in what I write here, so it's programming more like a writer would write, than just thinking of a set of instructions for a machine.

- Kent Beck, OOPSLA interview

Kent has been the biggest single influence on my programming.

Sennett describes craftsmanship as "an enduring, basic human impulse, the desire to do a job well for its own sake". It's a great topic, which has come to my mind a lot with respect to software design. I have always viewed programming as a craft, amongst other things. There are reviews from The Times & The Guardian. I've ordered my copy. Grab yours.

This is the year in which hAudio ushers in the era of distributed music publishing and playback on the web. As enthusiastic as I am about OpenId, oAuth and DataPortability (the emerging protocols of the distributed social web), it's microformats that are going to be the building blocks of this shift in the short term. Microformats can be seen as "the nanotech of the semantic web" (Jeremy Keith) and the metaphor for usage is one of proteins and surface-binding. My hope is that this year microformats will have a big impact on media publishing. It's more than a hope because I will be dedicating some of my time to help make it happen.

Download the free AJAX Control Toolkit which includes over 30 AJAX controls including rounded corners...

- Microsoft Visual Web Developer Express website.

Uncanny similarity in conduct between Microsoft, Scientology and Neo-con Republicans here, all going ahead and creating their own new realities. Conclusion: positions of such power allow organisations to disregard the actual state of the universe. By this logic, Microsoft is purely an ideology machine and no longer a technology company.

- 'Silverlight Architecture' schematic, Microsoft

This illuminating 'architecture' drawing from Microsoft manages to pack six (awful) logos and two pieces of packaging onto one stupid schematic. If you're in Microsoft's marketing department, can you please do us all a favour and take extended leave? Please? And leave Visio alone. It's not for you.

How others compared you recently: "Who is hotter", you won 0 and lost 1 time.

- 'Compare People', Facebook App Email Notification

Thank you Facebook application platform, for all that you have done for me in 2007.

Scant time for any kind of thought pieces on here this year. I offer this post as a snapshot of my thoughts on sustainability - a running theme for me of late.

Problem Statement/ Few organisations have managed to create sustainable work environments. Modern industry has largely failed to create sustainable processes.

Thesis/ Corporate models need to be re-invented because people do their best work in sustained group environments. This is true in all co-operative forms of expression (e.g. in most forms of design and engineering). Sustainable industrial processes are key to ecological sustainability and the health of the biosphere.

Myopia/ The 20th Century as a catalogue of short-sightedness. Industry rewards myopia as 'success'.

Percolation/ Sustainability is a timeless design challenge. Its largely unattended problems penetrate all of social life, from the individual through to the corporation and the body politic.

Capital/ Long-term profitability comes from sustainable development. Historically, only 1 in 10 public companies last longer than four decades. [3]

Current/ The notion that market success comes from organisations that hold sustainability as a core principle at all levels, from product design to organisational structure to development processes, is gaining currency.

Child/ The modern corporation, with a history of no more than 150 years or so, is a child in terms of the history of production.

Delight/ Delight as continual learning. No sustainability without some measure of delight.

Diversity/ Diversity as a key ingredient of sustainability. Preservation of diversity in the biosphere as a moral, social, and political goal as well as a commercial one. Preservation of intellectual diversity in a workplace as a purely commercial goal.

Settler/ A sustainable financial, social and intellectual living set-up over a nomadic and erratic one.

Fractal/ Sustainability has a fractal relationship to the Nietzschean notion of 'eternal return' (the ability to live your life as you would wish to live it out an infinite number of times marks the ultimate affirmation of life).

Performance Metric/ Team health. Relative happiness of those expending the effort.

Perspective/ Contrast the modern Perspex industry - dependent on petrochemicals and with a product impossible to recycle without releasing pollutants - to that of Murano Glass - a non-toxic, largely recyclable product that has been produced since 1291.

Growth/ Through the lens of sustainability, Head count in an organisation is not a valid marker for success.

HR/ Modelling people on resources is bad design. People are too complex to be modelled on any contemporary metaphor.

Fossilization/ Vertical power structures fossilize products.

Happiness/ Sustainability as a key ingredient of happiness. We want to be involved in activities that are fundamentally sustainable, because we can then afford to engage in them fully.

Cultivation/ People embrace sustainable forms of product development because everything it implies on a human level - a serious cultivation of teams, retention of people, corporate independence, building products people have a personal investment in - is desirable to the individual. This observation is in sharp contrast to the nomadic form of professional working that has become popular in the western world.

Rhythm/ Keeping motivated as a team over a period of not just months, but years. Preserving momentum.

Consultants/ Companies are unable to cultivate specialized knowledge in all their interest areas since they are not modelled on self-organizing, adaptive organisms. Cue the age of consultants.

Abundance/ What industry calls 'innovation' is creative abundance. How to sustain creative abundance whilst meeting short-term needs (i.e. not going bust).

Democracy/ Democracy itself is a sustainable (political) process. Its principles can therefore be used as building blocks to other sustainable (commercial) processes.

Specialization/ Mechanisms by which specialized knowledge can be spread across organisations without people needing to leave: meet-ups & co-working locations. A consultant eco-system is ultimately problematic for product, departmental & organisational development.

Catalysis/ A horizontal structure hybridized with vertical paths of responsibility elected and erected on-demand. Encourage natural catalysis, intervention.

Co-operation/ Distributed ownership of an organisation feeds accepted responsibility.

Cycle/ Cycles are natural rhythms. Intelligence is a function of feedback bandwidth and quality. Sustainable development as an iterative cycle.

Command/ Companies continue to be based on military-inspired structural models. Tyranny over democracy. Command/divide (and conquer) is still the primary metaphor for conduct. It's an uncompetitive model for a free market.

Atomization of intellect/ An explosion in freelance working has been brought about by the problem statement.

Adaptivity/ How to create a team and process with the capacity to consistently develop not just one great product, but any great product in a given field or even multiple fields.

Self-Organization/ The people that compose a company should be able to shape their organisation's structure, with built-in review processes.

Retention/ Sustainable systems absorb nutrients and release others to nourish wider cycles that are indirectly beneficial to both the system and other co-dependant systems.

Autonomy/ Building a framework of services and tools that allow any new project in an organisation to get off the ground quickly.

Flourish/ Success as an execution of the model abiding by the principles of sustenance. A company as an adaptive organism that doesn't just survive but flourishes.

Accepted Responsibility/ "No single action takes the life out of people more than being told what to do." [4]

Nutrients

[1] Cradle to Cradle: Remaking the Way We Make Things, William McDonough & Michael Braungart, North Point Press

[2] The Ecology of Commerce, Paul Hawken, Collins

[3] Leading By Omission, Ricardo Semler, MIT School of Engineering Talks

[4] Extreme Programming Explained, Second Edition (accepted responsibility), Kent Beck with Cynthia Andres, Addison-Wesley

[5] Scrum Et Al (iterative development), Ken Schwaber (Google Talks)

[6] Small Giants: Companies That Choose to Be Great Instead of Big, Bo Burlingham, Portfolio Hardcover

[7] Birth of the Chaordic Age, Dee W. Hock, Berrett-Koehler Publishers

[8] A Thousand Years of Non-Linear History, Manuel De Landa, Zone Books

[9] Cybernetics: or the Control and Communication in the Animal and the Machine, Norbert Wiener, The MIT Press

If you stop coding, you stop learning.

I have a long dream to build my own house in a very energy-efficient approach. That's going to be very soon. It uses the right kind of wood that serves as a heater and as an air conditioner, combined with some other techniques in how the wood is assembled to operate energy life pressure. You don't have to add energy into a house after you build it. I love that concept. It's like the way I used to make computers. I want to build it myself.

Steve Wozniak, 'Wozniak's New Goal is energy efficient housing'

The time to make a decision is a function of the possible choices he or she has and can be expressed as t = b log₂(n + 1)

Where t is the time taken, b is a constant and n is the total number of choices.

Hick's Law, from Haacked's, '19 eponymous laws of software'.

New export options in Zim.

Live demo of Photosynth (Microsoft Labs) from Ted.

Just another 13 argument constructor from Adobe's native Actionscript API. I dub thee an 'Adobe Constructor'.

As much as I love remembering the order of all 13 arguments (btw there's no defence for a 13 argument anything), I'm stumped as to why a language that implements ECMA-262 doesn't take a leaf out of Javascript/Ruby and go for parameter hashes as object literals.

My wrapper for the above now has client code like:

Text.Format({size:6,font:'Arial',align:'left'});

Obviously.

The 'Bundles' view in Zim gives you a simple way of grouping outlines.

Developers, developers, developers just found a new platform. It's a pretty major breakthrough for social software on the web. Credit to Facebook for their innovative and forward-thinking implementation. Overview here.

Yahoo! unveil PIG. It's up against an elephant. Also, what a great icon? The dungarees are a stroke of genius.

The last.fm toilets have become a battle-ground. Left to right - Textmate reference, Vim cheat sheet, Emacs cheat sheet. Barely legible on the Textmates sheet, someone has scrawled 'tab triggers ftw'. I use textmate and vim over emacs nowadays. My observation on Emacs (through 18 months of use about two years ago) is that Stallman must have very large hands. Not a very useful take-away but I thought i'd share nonetheless. Allan Odgaard on the other hand, he has my kind of hands.

Well, it's taken a while, but Google Reader's social features are finally taking shape, though their sharing feature is essentially a 'reblog' feature, simply publishing all the items you want to share on a public page - here's mine. The feature would be better off targeting your gmail contacts and allowing you to share an article with a specific person/group (either via email or via the reader) - after all, 'share' and 'republish' are very different things.

The obvious tagging feature is there, but no powerful tag navigation method is on offer and the search facility is ironically non-existent. So there's still a lot of work to do on the navigation front - the most crucial design problem for a newsreading app. I think the navigation itself needs to have a social dimension (tag-driven content drawn from 'pools' of articles, not just your own explicit subscriptions).

Interesting to note the attention data is now pretty much collected real time, and a lot less crudely than in the first generation of browser based RSS readers - as you browse a particular feed, scrolling through the articles makes the feed 'articles read' count go down article by article, meaning that the reader knows if you really have set eyes on an article or not. Google's concerned with capturing accurate attention data - and it's beamed back at you on the reading habits page; right now an interesting set of trends, but pretty useless until the reader starts recommending articles/feeds based on the data. Kind of like our last.fm profile pages - pre-radio, pre-recommendations. So with a pinch of collaborative filtering and more interface nous, Google Reader is set to be quite a big product. As it is, Gmail still seems to be losing the interface wars against Yahoo! mail (i find Gmail incredibly efficient myself), so it'll be interesting to see how the newsreading attention is distributed amongst the big players. It also depends if they can ever recover from their disastrous first impression (the first version of the Reader was truly a shocker and put lots of people off, myself included).

Whilst i'm not using Google Reader (I use Clippr and NetNewsWire), I've become a regular user of google notebook of late. Very quick to use, and comes with a firefox extension. May have something to do with using 4 computers on a daily basis.

I'll be speaking along with my esteemed colleague and budding musician Matt Ogle (profile here) at the upcoming Future of Web Apps conference in London (Feb 2007). The line-up is frankly intimidating (Khoi Vin in the house) but we'll do our best. And if that fails, we have some pretty good dance routines up our sleeves.

We'll be talking about ponies, unicorns and um, other last.fm obsessions, so if you're a coder, bug your employer/mum for some cash to come to this - it should be pretty ace.

One of our talented data miners, Martin, put these chart arcs up on our data playground last week. They show chart positions and movements of artists in our profile, along with popularity information. Turns out i probably have the least 'mainstream' taste of any of the last.fm staff. For further explanation and all the staff chart arcs, visit the last.fm chart arcs page on our playground.

Well we're up for the 'best music community' award at the upcoming BT Digital Music awards - it's people's choice so vote for us!

Reasons why we might be the best candidate in the shortlist:

• We're not owned by Rupert Murdoch
• We're not a glorified download store
• We're passionate about social software in the widest sense
• We've been commited to our community for years now (we're not hopping on the social network bandwagon)
• We actually innovate
• We're nice

Update: We won this! Thanks to anyone who voted through this here post.

I've been pushing for the adoption of scrum practices at last.fm of late (with some success!). Julian pointed our team to this Google Tech Talk from Ken Schwaber over at google video. It's a great talk - my personal highlight:

Scrum works with idiots... uniformly they will produce crap every increment... everyone knows where you are all the time...

For startups exiting the frenetic growth phase and trying to settle into an environment of sustainable software development, i can't recommend Scrum highly enough - it's used throughout Amazon, Google and Yahoo!, as well as smaller firms. Scrum comes from Schwaber and Sutherland, two advocates of the original agile manifesto.

I hadn't noticed that Google provide a range of ace tech talk videos, well worth browsing through.

This news that Universal is going to offer its catalogue for free and move to an advertising model is a small step in what i consider an inevitable economic shift in the culture industry (and yes, i'm repeating myself). As we move towards embedded advertising (this announcement doesn't detail the nature of the advertising in this particular venture), DRM is going to be cast aside for purely economic reasons. It's simply not viable when the main goal of a publisher is for a piece of media to be shared amongst as many consumers as possible (in order to gain advertising revenue proportionate to attention metrics).

In other news i'm knee deep in actionscript 2.0 again and actually quite liking elements of the language (to my dismay). Despite it's shortcomings, it's still the best way to deliver browser-based audio/video content. If i've got a moment i'll detail some of the new (Flash-specific) design patterns i'm using in my code. I've switched back to Textmate to handle my AS project though, and i'm finding real simple stuff missing, like caret based matching bracket highlighting(!).

If you're in London, go to the Frieze art fair gig this month. It will be ace. Sunn o))), Florian Hecker and Liars all showing up.

UPDATE: Seems like Universal's tunes will come in PlaysforSure-protected WMA (via Wired) :( . So no embedded advertising, more likely intrusive online ads. And DRM all over the place.

Well, working with php is colourful at the best of times, but today i encountered a bug so big, so fundamental to the design of the language, that my face almost fell off in disbelief. In short, if you're calling a method statically in PHP 4, within the static method, the '$this' keyword will point to the calling object, as opposed to being 'null' (there should be no object context in a static method). Here's an example:

class Static
{ 
  function method()
  { 
    echo get_class($this);
  }
}

class Container
{ 
  function Container()
  { 
    Static::method();
  }
}

$Container = new Container();

The $this pseudo-variable is not usually defined if the method in which it is hosted is called statically. This is not, however, a strict rule: $this is defined if a method is called statically from within another object. In this case, the value of $this is that of the calling object.

Basic principles of addressability for web applications:

• Always tie your URL's to your data models.
• Use mod_rewrite as sparingly as possible. It's too flexible.
• Don't punish plurals - provide predictable redirects.
• Go for human readable URL's if at all possible.
• Provide namespaces for RESTian web services. Have an xml-based (e.g. rss) representation of every model in your application.

As always, delicious is a model of good design in this area. We're working at last.fm to improve addressability with what is a more complex data model (artist/album/track/label/user/tag).

So a couple of weeks back we got all the programmers together to have a meet about coding conventions. During the meet (which involved much shouting, an intermittent hyena-like wailing and finally a placid burnout calm) I decided two things:

1. Good coders don't need house styles.
2. Good coders don't need coding 'standards'.

Like cubicles, compaq stress balls and water coolers, these are signs you're working in the wrong place. House styles are an attempt to instill consistency by force. Coding standards are put in place whenever management decides staff expansion is a valid strategy for software development (wrong) and start recruiting mediocre developers on a monthly basis from websites like planetrecruit. "Oh you're on planet Earth? And you want to recruit? Boy do we have a specialised service for you." Oh so wrong. Wronger than left-over meringue.

No one should infringe on your programming style. The thin line between convention and style is crossed when someone stipulates what syntax to use for particular expressions in your code, or dictates it's layout beyond basic editor settings.

This is why, for example, i find stipulating tab-style (a text editor setting) acceptable but bracing style (1TBS Vs BSD) unacceptable. Given the holy wars incited by bracing styles i'd like to point out that it really is ok to disagree on this point, it's just part of your programming style. I personally feel my screen has enough vertical real estate to warrant the latter (Kernighan & Ritchie only had 24 vertical lines to work with when they adopted 1TBS), but it's no big deal. If i'm refactoring code, i'll change the layout to my favoured style. If not, i'll leave it be. Not worth wasting your time on arguing over style, because there's rarely a conclusion to be had, only difference in taste.

Good programmers are incredibly fussy about their code. Try and instill stylistic rules on a set of these programmers and you'll cause immeasurable pain. Consistency is overrated - as long as the code quality remains high (you shouldn't need 'standards' for this), it's good to house a mix of styles. A rich mix is also useful for discerning authorship over any given piece of code when you're all just too damn fast-lazy to bother with javadoc @author (or equivalent) tags.

So here's a call to slimming conventions to those essentials that enable you to work with others on one codebase (file naming/class naming/function naming) and less dogma on things like control structures, declarations, , assignments, braces and the like. Here's a call to conventions that don't take longer to read than a coffee break (like sun's amusing java conventionpedia). Here's a call to making your dev environment as open as possible to a variety of coding and development styles (IDE VS vim/emacs) and instilling a little diversity in your codeplace.

To harmony. Group hugs. And code peace.

From songbird.

Blog looks like a link wreck right now, sorry. It's been a little over a month since i started showing myself at last.fm hq, things happening:

• Redesign - gradients, drop shadows, multiple themes, ponies. Still being tested but soon to be unveiled. If you're a subscriber try it out.
• We decided to make friendships mutual going forward and this 'friends' thread kicked off in response. We are thinking about categorising more than one type of relationship (delicious has friends/fans, flickr has family/friends/contacts, we need a more finely grained model)
• New last.fm desktop app to replace old radio. OS X version coming real soon.
• Lots of Dr. Pepper going on.
• Launching in Japan. Hopefully means we'll get some more J-Pop in our catalogue. Ace.
• Discovered synergy for multiple box convenience. Simple setup. Just works.

Sony research group demo an 'augmented surface'

Spurred by Miranda's reference to Malcolm Gladwell's article, The Social Life of Paper and other articles on the subject ( MIT Press' The Myth of the Paperless Office, Passion for Paper and Bill Gates' paperless working methodology), i thought i'd throw in my $.02.

I really can't see a case for paper in the workplace outside of enhancing co-located group sessions/workshops (i've described some below - Gladwell points one out as well). I'm trying, but i don't see it. At least for software development, though i believe this view is defensible for a range of industries.

One of paper's great advantages lies in its physicality. As a result of its portability it can potentially radiate information to multiple people in different locations without any technology requirements other than a pen/pencil. It's spatially flexible and provides a low barrier to authorship.

When you access paper-based information, subtle authorship traits are clear (handwriting and drawing styles communicate myriad things to us which digital typography and rigid outlines do not). Paper media retains these traits so well because the paper/pen combination provides one of the greatest interfaces mankind has ever come up with. It allows for much self-expression. From an accessibility perspective, the printed press gives you high resolution fonts you can take to bed with you. No eye-strain included.

So it's not all bad for paper. But the paper-less office isn't a myth. I don't use any paper at work. The only pieces of paper i have touched involve HR processes such as holiday forms (arcane admin rituals).

I use OmniGraffle or a whiteboard for quick visualization of ideas. Whiteboards beat paper hands down for radiating information to groups in the workplace (see Cockburn). Textual collaborative, dynamic information resides in the web browser - accessible by all, manipulable at all times. Wikis, chat-rooms and project management tools sit there. The browser is our core communication medium.

For notes, I have OmniOutliner which i also use to deconstruct large problems into identifiable tasks. Technical designs can be drawn in applications pretty quickly. These designs can then be uploaded for others to comment, in their own time. I can re-organise and re-prioritise my tasks at a keystroke. I can take notes on a meeting with others, during a presentation (backchannel). Put all this together and paper is starting to look a little ragged.

In the processes of archiving and organising (search/sort), paper also makes a weak case for itself. Digital alternatives exist for tracking ideas, dialogues, designs, tasks and most other everyday information. The notion that paper is a more secure information storage mechanism is also often mistaken and i can only assume comes from a misunderstaning of modern technology (stories of giant electromagnetic guns and hackers making people nervous).

In contrast examine the issues that arise from using paper instead of digital tools, and accumulate these over time (no versioned history of events, unsearchable, restricted ability to re-organise, closed access). It's a cludge. Moreover, it's potentially harmful; paper is, after all, the medium of bureaucracy. Solutions to everyday design problems require agile, accessible information. You need to be able to mix and match approaches, sculpt them in a group environment and change them on the fly. Your chosen media should reflect this.

The only place i see for paper in software dev is in CRC cards and planning sessions (blitz planning, sprint planning or xp planning games) - specialized techniques peculiar to the field (though kaizen-blitz sessions are widespread in the manufacturing industry). Broadly speaking, these are group processes taking place in one location, in which members pursue a single goal of identifying and organising many discreet chunks of information in real-time. Paper's spatial properties come to the fore here. Note that the paper is often thrown away after the session is complete and results captured.

All of these group operations could be carried out on a screen, but screens are interfaces to personal computers, which are designed with the individual in mind. Personal Computers are not for groups (as the name suggests), and a network of such computers does not constitute a true group interface. Turkle has alluded to the solipsistic design of the modern computer in the past. I think as better groupware interfaces are commoditized, such as augmented surfaces (sony research video), the use of paper even in these group processes will become optional and ultimately deprecated.

From the ibm archives photo gallery.

My father worked with the IBM 1620, the Control Data Corporation's CDC 6600 & the Cray 1 supercomputer, which looks like a monolith.

These last two were designed by Seymour Cray.

The 1620 took punchcards and required the entire operating system to be fed in every time it was switched on, since it lacked anything but 60,000 digits of volatile storage. It got the nickname CADET (Can't Add Doesn't Even Try) through its use of lookup tables instead of adders to perform arithmetic.

Extension Object Pattern implementation for php - uses call overloading (on which i mistakenly called bs a while back) and the reflection API:

• skeleton code here

In reality it's an Extension Object implementation as opposed to a Façade, because façades need provide a high-level interface to a low-level subsystem, whereas this does nothing of the sort.

Façades are used all over the place, including sessions and web services. Extensions are a way of getting round the fact that PHP doesn't implement either multiple inheritance or mixins. They are also dynamic, so you can shape the capabilities of an object to fit its context.

I'm joining social music service last.fm as a full-time developer in a couple of weeks over at their new office in old st (it still looks like a mess - story + pics). As those who know me will confirm, i have views on social software and how culture can suck less under new classification, distribution and publishing models, so i'm really looking forward to getting stuck into moderation strategies, web services, folksonomy and data mining. Particularly psyched to find individuals i respect (e.g. Joi Ito) among the investors.

It should be a tremendous year for last.fm. Hopefully i can play my part in that.

Capistrano* is a lightweight web application deployment tool suited to all codebases. It's written in ruby and hooks into the rake gem, but you don't need to be deploying a Rails app for it to come in useful. Ruby only needs to be installed on the box doing the deploying. Check out the other assumptions it makes. It's very practical and i highly recommend it for web shops with a cluster of web/db servers.

Having come out of the rails community, it sticks to some of the design principles of said project, including 'convention over configuration'. However, the creator's conventions aren't neccessarily your team's conventions - there was one core convention that didn't hold true in my deployment scenario: Capistrano deploys the latest revision of your repo and uses timestamps to identify the release on the participating servers. However, we use subversion and we deploy 'tags'.

Tags are named snapshots of your repository (more on tags). We typically use version numbers as tags, e.g. 'v1.2.2' or some such scheme. This allows us to say with certainty what is deployed at any one time, without having to scan logs etc. Tags give context to a release name (you can easily associate the release with project milestones) that revision numbers and timestamps do not. Tags are usually kept in a 'tags' subfolder, in the top level of the repository.

I've produced a small task library which allows you to deploy and rollback subversion tags with this kind of syntax:

rake remote:deploy_tag [tagname]

• Your 'releases' folder will hold your releases by tag name, not timestamp.
• The library adds a capistrano variable, 'svn_tag_dir' which defaults to 'tags', to describe your tag subdirectory.
• You can customize the deploy_tag script for your particular deploy scenario.

To install just place the task library in 'lib/tasks' in your particular rails app, and add the following to your 'config/deploy.rb' script:

require 'tasks/capistrano_svn_tags'

You can then use the new deploy_tag task. Assumptions made:

• Your :repository variable will now have to point to the top level of your application's repository.


• Your tag names follow an enumerable convention like v1, v2, v3 (ie sortable by Array#sort)

If you want to use the default capistrano deployment convention again (last revision, timestamps) then just comment out the require statement and use the default deployment tasks.

Download the task library:

• capistrano_svn_tags.rb

* Wins my award for Most Emo Sounding Dev Tool of All Time.

As Deputy Chairman at ARUP, Balmond's approach to large scale projects has single-handedly shifted the role of engineering in the architectural process. The principled miscibility of design and structural engineering he encourages is inspirational and refreshing. It serves to break down the perception of engineer as dry, technical doer and promotes the idea of creative engineering and intertwined design/build phase. It's a great parallel to certain agile approaches to software engineering. Pick up Informal for more.

role: model.

For those of you using PHP's Flexy (Pear HTML_Template_Flexy) templating engine, this patch provides better subtemplating support. Adds a new 'partial' tag to the flexy:xxx namespace and allows for variable mapping. The word 'partial' is borrowed from the ERB templating engine (ruby). You can define variables in the subtemplate context as attributes of the new tag. This feature is pretty important for modularising templates. Usage example:

<flexy:partial src="subtemplate.html" varNameInSubtemplateScope1="#String Literal#" 
varNameInSubtemplateScope2="currentPage.var" varNameInSubtemplateScope3="var" />

My changes were made in PHP 5.0.4 on Flexy v1.2.1.

Add the function linked here to Compiler/Flexy/Flexy.php to get the 'partial' tag working. I'll see if the project integrates this feature in the future.

Japanese for continuous and incremental improvement, a business philosophy about eliminating waste in working practices.

Kaizen is a daily activity whose purpose goes beyond improvement. It is also a process that when done correctly humanizes the workplace, eliminates hard work (both mental and physical), teaches people how to do rapid experiments using the scientific method, and how to learn to see and eliminate waste in business processes.

- Wikipedia

Japanese for continuous and incremental improvement, a business philosophy about eliminating waste in working practices.

Kaizen is a daily activity whose purpose goes beyond improvement. It is also a process that when done correctly humanizes the workplace, eliminates hard work (both mental and physical), teaches people how to do rapid experiments using the scientific method, and how to learn to see and eliminate waste in business processes.

- Wikipedia

It's been a strange week for me & the GOOG. First, Rob pointed this out in the new Adwords API T&C:

3. Inspection for Compliance.

1. User-Interface Inspection. You agree that Google may inspect your AdWords API Client user interfaces up to 3 times per calendar year. Any such inspection must be during normal business hours. You must allow Google to visit your place of business, or inspect your AdWords API Client in some other manner agreed between you and Google, within 7 days after notice from Google that Google desires to inspect your AdWords API Client interfaces...

2. Systems Inspection. In addition to Google's rights under Section IV(3)(a), at Google's expense (except as set forth below) Google may appoint (subject to the below) a Third Party inspector (the "Inspector") for a systems inspection, only as set forth in this section, of your AdWords API Use...

Que? For the pleasure of talking to you we get a day trip from an 'inspector'? Whilst i like catching up with GOOG folk to talk falling stock and vint cerf's breakfast habits, i think this is over-stepping the mark a little. Both parties using an API can pretty clearly discern who's playing nice and who isn't without visiting each other's premises and inspecting products.

This came with the news that the Adwords API has a new 'feature'. Usage is not free anymore! The old quota caps system has been replaced with a pay-as-you-go type deal. Coupled with the fact that google doesn't provide test accounts, this could mean that if you're developing an Adwords client for your app you'll be spending every step of the way. Still got to wait on this, but it sounds both more flexible and more costly.

Finally, Google Calendar came along, with great desktop UI touches like cell dragging to define event duration:

And google map integration and all sorts of other treats.

And suddenly everything is ok again with me & the GOOG and i forgive them and they can send inspectors to my house to question my family about my API usage and i'll even make them continental breakfast. With fresh croissants. On a Sunday morning.

Each week the four of us sit down and throw patterns in each others' faces. Last week i threw adaptor. Things got messy - the pdf presentation the only remaining evidence. UML is not evil btw. Big revelation. Growing pains.

Well i've used the term mashupware a bunch of times, but Zimbra's online demo coins the phrase 'enterprise mashups'. The business model could mean a new age of SLA's between inter-dependent services, meaning folks like QOOP could ask for legal agreement from folks like Flickr, replete with uptime guarantees and service availability/quality guidelines.

From the article, Dignity is Dead

Not everything needs a software solution. Sometimes the human brain is the solution. I can capitalize my own words and decide when something is a bullet point.

- Jason Fried, 37 Signals

First off - if i was looking to hire, the answer would be

No. A brain and an attitude are the only pre-requisites.

About 18 months ago i decided to apply for an MSc in Computing at Imperial College, London. It's the best thing i've done in my short career programming.

I'd fought against the urge for post-graduate education for a couple of years because, well, i wasn't the academic sort. Not at this level at least. I didn't care for formal logic and proofs and such. I wanted to make things. I wanted real-world projects and constraints, because they were what made me tick. And anyway, if you're a good coder, you're a good coder, right? You teach yourself.

I'd taught myself upon leaving my bachelors degree (Chem Eng). I'd been freelancing on and off for about two years, but my formal programming concepts were messy, confused. I knew i could code, but i knew i was missing guidance - what technologies to pick up, which books even. So i thought - enough hacking, time for some proper stiff upper lip formal programming kicks.

I walked into Imperial 20 days before the term was supposed to start. They'd stopped taking applications 4 months ago. They were like, "you want to what?", and i was like "YES. LET ME IN.". I think i spooked the guy because after some mumbling he was all like "ok, see you in a couple of weeks" just to get me out of the office.

When I showed up I was afraid. Very afraid: I had a bunch of previous clients and a bunch of juicy contracts lined up. Now i had this full-time course on my plate and i'd only previously done web programming and matlab. We walk in and on the first day my tutor says,

don't work. if you work and study here you will fail. we are that hardcore. people think they are hardcore, but we are imperial. we are complete xxx.

Or words to that effect.

Gulp.

Imperial stuck true to their word. Some courses were taught at a frenetic pace (all fundamental SQL syntax taught in 2 hours - i mean pretty much everything) and pushed me to the limit, even with a couple of years of web development experience to lean on. But in the end i managed to pick up easily the best commercial work i'd ever done, whilst doing the whole academia thing. It was tiring. Nights were short (add to the commitments a 1+ hour commute to college). But in the end it was worth it.

There's some stuff you just won't force yourself to do. You won't force yourself to learn Prolog from strict first order logic fundamentals. You won't teach yourself finite state process models or assembler programming (probably. unless you're wozniak. or plain bored.). You sure as hell won't teach yourself to formally prove the logic of your code is correct, because you don't need to prove it to anyone. But doing all this informs you as a coder. I'm not saying i implement all the techniques i learnt - what i'm saying is i've got perspective on my code. For example, stack structures are fundamental at all levels of coding - but what better way of ramming home memory allocation concepts than the assembler stack pointer?

Our best tutor was Will Knottenbelt. He would belt out C code infront of us and debug live to reinforce concepts. I found out the practical approach is always the best to learn programming concepts. Darlington was also great - he's a long-standing contributor to the design of the Haskell language, which is what he taught us. He also has a 80 node fujitsu AP3000 super computer poking out of his office. Not many academics can say that.

OK so today i got my results through and it appears i did the near impossible - work commercially on a bunch of software projects whilst doing the full-time MSc thing. The nice folk at IC even let me do my dissertation in Ruby on Rails, which was great experience for me:

These results tell you that i make a pretty lousy sysadmin, but a pretty good high-level coder. And that i can code OK in real-time (labs). And that i can juggle. Juggling is pitching for projects 10 days before your finals and working off your tuition fees as they come in. Juggling is on all the job adverts so i guess juggling is in.

So that's it - MSc done and dusted and after all my apprehension to go strict on my code it was worth it.

Did it help me get my current job? The answer is no. Not one of the questions on the entry test would have been answered through knowledge picked up solely through the MSc. In other words, if i'd graduated fresh with a distinction but with no added personal experience, i would have flunked my interview completely and utterly.

Caught this whilst browsing the trac timeline at work and almost fell off my chair laughing.

Context beats consistency. Reuse only works well when the particular instances are so similar that you're willing to trade the small differences for the increased productivity. That's often the case for infrastructure, such as Rails, but rarely the case for business logic, such as authentication and modules and components in general.

A framework is a framework, no more no less. Its downfall is the inclusion and creep of business logic - no matter how implicit (e.g. authentication) - into its domain. On the broader question - is re-use overrated? - i am always wary of speculative generality and counter-productive reuse in code. As one commenter to David's article points out,

Programmers have an irrational fear against throwing code away.

Cron is great for servers, but for your dev box (be it (Power|i)Book), which may be switched off or sleeping for long periods of the day, you really need anacron. More info here. Install once, then just put symlinks to tasks in,

/etc/periodic/daily
/etc/periodic/weekly
/etc/periodic/monthly

If you want more (cron-like) power over execution intervals and conditions, you'll need to create a launchd plist file. Lingon provides an interface.

Twelve weeks with geeks is on google video - trailer here, feature-length for purchase here.

I particularly like the google vid query syntax,

geeks is:forsale

Will give you the feaure-length version. It's more intuitive (natural language) than some of the queries you have to come up with on google search proper,

"parent directory " MP3 -xxx -html -htm -php -shtml -opendivx -md5 -md5sums [artist] [title]

On the other hand, google video frames its credit card form, forcing you to think twice about the SSL nature of the transaction.

Boo google.

In case you'd missed it - Google Earth is now on OS X. I managed to locate my grandparent's place in New Delhi. That's something.

Note: Google video purchases are still US only. Complete bummer.

I"m convinced the next big step for web browsers is implementing offline mode.

So what's required? A local caching framework that server-side applications can hook into, allowing you to, for example, read and compose gmails whilst in transit, browse your rss archive in your browser, etc. This will lead to the inevitable. What's the inevitable? The inevitable is the day when home users will swap most of the applications on their desktop for their web browser.

Paul Graham voices about 80% of everything i think about Business. The other 20% goes on this blog. Check his latest speech.

Splitting work and life is one of the key tenets of professionalism. This, i am convinced, is one of it's main mistakes.

The semantic root of 'work' in French ('travail') is the Latin word for torture. The Latin root of 'amateur' ('amator') means 'lover'.

If you don't love what you do, don't do it.

I can't stress the importance of Sergei Dmtriev's words enough. What he's saying doesn't sound particularly revolutionary to the average developer, and sure, the idea of DSL's is not so new, but their importance in the near future - one in which programming becomes more and more of an everyday activity for a larger portion of the population, in which project lifecycles shorten and software development activity multiplies - can't be overestimated. I know that since taking a look at Rails i've had the feeling that domain specific languages are critical to my future in software development.

It all boils down to the fact that programming languages strong in meta-programming (can you say Ruby?) are going to gain strength in the kind of phase we're in - one where programming is seeking the human, for productivity's sake.

OK so google have announced Google Reader, which is an RSS client for your browser. The best way to introduce Clippr is as an RSS reader i built that shows up Google Reader in most departments. It feels good to be ahead of the google this time ;)

I built Clippr as part of my dissertation over at Imperial. It's social software for the web. I use it everyday - it's become one of the most important apps in my browser, which i guess is enough reason to tell you a little more about it.

First off, a one-line description of Clippr:

Social bookmarking meets RSS feeds. A great way to discover, archive, create and share articles online.

OK, now for the details.

Overview

Clippr bears some similarity to the social bookmarking tool Del.icio.us, in that it allows users to tag and archive content from the web. However, where del.icio.us revolves around the concept of the 'bookmark', Clippr's main building block is a 'clipping'. A clipping is either a textual excerpt taken from a webpage or an article from an RSS feed. Both these types of clippings are treated in the same way by Clippr.

Why would a user want to take clippings?

Clippings are like a fluid bookmarking strategy for the web 2.0. They're a replacement of the bloated 'temp' folder which sits in everyone's browser bookmarks. They acknowledge the kind of transitory interest in numerous topics that the web encourages in users, as well as the fact that users are increasingly browsing by RSS feed. Clippings are a replacement of the bookmark per sé, in that they are represented as RSS articles and not simply as hyperlinks. Put simply,

Clippings tie a user's bookmarking activity to their RSS reading habits and provide a richer definition for archiving web content than a bookmark.

What's the big deal about folksonomy? It's just labels.

Um, yeah, like what's the big deal about computers, they're just big calculators. OK.

Folksonomy is a faceted, emergent classification strategy. It's the natural classification strategy for networked media, in that it addresses directly the fact that,

In an information retrieval system, there are at least two, and possibly many more vocabularies present. [14]

One of the main advantages of Folksonomy is that,

Since the organizers of the information are usually its primary users, folksonomy produces results that reflect more accurately the population’s conceptual model of the information. [35]

For more on folksonomy - and the sources of these references - check my dissertation.

What's implicit social software?

Social Software can perhaps be summed up as software that recognises the most important component of it's system is the user-base. If social software wants to do smart stuff, it does so by aggregating the intelligence of it's user-base, in terms of the data captured by the application during usage.

Implicit social software means getting rid of user profiles and user visibility in general and just letting the app use the aggregated information to work out relations between data. Like this the application functions like a single-user desktop application without all the distractions of social networking, but draws on community intelligence to help the user navigate the data architecture created by the user-base.

Text-analysis is rubbish, it gives me google adsense ads about detergent when i'm reading about the SOAP protocol

True. Clippr acknowledges the user-base as the central component in it's classification strategy. The system performs text analysis on incoming articles from user subscribed feeds, which it uses to extract keywords and top stories from the fresh crop of articles. Then, whenever a user's tag coincides with a keyword extracted by the system, all articles with that keyword are shifted over into the tag-space. This is a crude automated classification strategy which reconciles text analysis and tagging - it works pretty good.

Moreover, Clippr analyses tag intersections to recognise related tags, and presents these to the user as they search for articles. It's more subtle than del.icio.us in this respect, in that it quantifies the similarity between tags, in terms of intersection magnitudes.

So text analysis needs to be combined with context analysis to be effective at picking out similarity between documents, and that's what Clippr does, by letting users provide contextual information and analysing it.

Folksonomy is an uncontrolled mess. Jack says 'web', whilst Jill says 'internet'

True. Wordnet can be used to reduce the tag space by recognising synonyms, hypernyms, hyponyms and compound terms, but it has limited performance with a completely uncontrolled vocabulary. Acronyms and neologisms abound in tagging. A lot of this is still unresolved and impacts the performance of search in folksonomy.

Show me the money

Here's the feature list:

• OPML import/export of feed subscriptions. Folders are flattened to tags and imported automatically into Clippr.
• Firefox plug-in and bookmarklet to facilitate clipping stuff from your web browser.
• Tag clippings, tag feeds, tag like a demon.
• A community oriented article base formed through user subscriptions, refreshed periodically. Full text RSS/RDF/ATOM formatted feeds are supported.
• Text analysis (article clustering) on incoming articles, in order to extract Top Stories and article keywords.
• Context analysis (tag clustering) used to recognise related tags.
• Change your tags whenever you want. Clippr handles merging/splitting of tag-spaces.
• Power editing using batch actions thanks to a gmail style dynamic dropdown.
• Tagging combined with keyword extraction to produce automated classification of articles. Text analysis and folksonomy reconciled.
• A search engine supporting a query syntax for folksonomy - search by tag (intersection/union), feed, keyword or any combination of these. Implemented as Live Search for desktop style responsiveness (it behaves like Apple Mail search - wipe the search field and return to where you were)
• RIS export for using web references in bibliographies
• RSS export of your Clippings archive.
• Mail an article to a friend or recommend it to a fellow Clippr user.

Ok, let's try it out

You can't. It's not public right now because it needs a dedicated server to run and i can't afford to run and maintain one. If you want to help me setting one up, please get in touch with me.

More information

Here's a quick presentation and here is my dissertation in full. Source code (ruby/javascript) to come.

The dissertation format in Computer Science completely ignores the nature of software development today.

The academic establishment needs to change this.

A dissertation comprises an individual piece of work, authored, documented and executed by an individual.

This scenario is neither a fruitful nor desirable one in software development today. The scenario, in short, does not exist outside the academic context.

Kernighan & Ritchie's original C manual was under 300 pages long.

PHP declares over 3,000 built-in functions.

Take a guess at which is more powerful.

Ergo:

array_uintersect_assoc -- Computes the intersection of arrays with additional index check, compares data by a callback function
array_uintersect_uassoc -- Computes the intersection of arrays with additional index check, compares data and indexes by a callback functions
array_uintersect -- Computes the intersection of arrays, compares data by a callback function

Design by commitee is tough.

Being a large open source project is tough.

A programmer's main activity is design. His tools should reflect that.

The distinction between native 'types' and objects makes for an amorphous pool of native functions:

There is inconsistency in the naming, arguments and return types. Most of these criticisms are documented by wikipedia.

Most don't think twice about these issues until they start with C and Ruby; both masterclasses in good language design.

Matz designed Ruby. He says,

One way is by looking at what can be done with that language. The other is by looking at how we feel using that language—how we feel while programming.

Dennis designed C. He says,

C is not a big language, and it is not served well by a big book.

Rasmus invented PHP.

He is coming to our local pub for a drink this week.

Wow. I had plans to post a lot of things on here, but for now, general busyness has overloaded all these into bullet points in an OPML file somewhere. Speaking of which, OmniOutliner is the most critical productivity application ever made, by virtue of it's complete open-endedness and masterful key binding intuition. It's a work of art.

So what's changed round here? My dissertation on social software and the World Live Web is done & dusted (more in subsequent posts), my love affair with Ruby has blossomed (more - so much more - in later posts), i'm now working as an application developer in PHP5 and my ears are filled with podcasts (Open Source & PHP related), courtesy of itunes 5, making my commute to work almost bearable.

Elsewhere the gang of four, kernighan, knuth and hofstadter are illuminating the corners of my days.

I coded this quick del.icio.us plug-in for Movable Type publishing software. More information here.

PHP5 is going to be the focus of much of my time in the coming year, this much seems certain. Ruby on Rails will take years to reach PHP's level of ubiquity with web hosts, who still seem in the process of upgrading to this new version of the language.

PHP5 features a complete rewrite of the object model. It's OOP support is so broad it seems more pertinent to point out what common OO features it doesn't implement, as opposed to which it does:

• No C++ style templates
• No metaclasses
• No namespaces
• No operator overloading

Other than this PHP5 seems to have the works. Plus a lot of MySQL/XML features i haven't gotten around to yet.

Adam Trachtenberg is the man to spill about PHP5, here's an overview of the language's new features, and here's his blog, which should be a mandatory RSS feed for any PHP developer. I also had a chat with Ed Lecky-Thompson last week, who's contributed to the Professional PHP5 title. Looks solid too, though i haven't read it yet.

Hopefully this'll all help move PHP developers away from the procedural and into strict OOP for the bigger projects, where frameworks can be built, design patterns effected, and many hours browsing tangled procedural code & rough testing methods avoided. That said, it's important to retain the rapid development capabilities that procedural PHP encourages - this sets it apart from languages like Java, after all.

And if there's one thing PHP does not want to be, it's Java.

Lately i've been keeping an eye on the job market. I've got agencies ringing me up the whole time, asking me to quote buzzwords down the phone. I keep saying 'ruby' instead of Oracle/.NET/whatever they want to hear. The agency guys go quiet at the mention of ruby. Something like, 'hang on a second, let me jot that down - how do you spell that?'

I've got job alerts coming at me from all over, most targeting the keyword web. I did a quick and dirty text analysis of ten of these job alert mails, each one holding circa 100 IT jobs. Below are the results, with term frequencies expressed in font size. They give an interesting idea as to recruitment buzzwords right now.

Conclusions:

• Time to learn some Java everyone :(
• Intelligence was mentioned just once. No need for it. Same with 'nice'. No need for nice. Meanwhile degree, creative, experienced and corporate came up loads.
• Juggling helps. I must learn to juggle.

Continuous Partial Attention is both a drainer & enhancer of productivity for web developers. Linda Stone brought it back onto the agenda at Supernova 2005. It's an important behavioural skill - particularly applicable to those working directly with the Web - which needs to be honed and controlled in order to get on with the job(s) at hand. Microsoft's approach,

Bill Gates has three types of meetings: free-for-all, mixed (sitting at back indicates paying half-attention), and full (if you're sitting at the table, you focus on what's going on).

I haven't yet gone down the email-free Fridays route myself...

Been working with Ruby On Rails on & off for about a fortnight. A few observations for those starting to pick up this framework,

1. Keep Dave Thomas' excellent text by your side at all times

Dave Thomas, co-author of the 'Pragmatic Programmer' series, is a worthy guide to Ruby. This text is excellent - Programming Ruby: The Pragmatic Programmer's Guide. Also keep Matz's original Ruby manual to hand. It's fun.

2. Use Rake

Rake is a ruby makefile-style utility. It's useful. Try rake --tasks to check the available tasks. You can define your own rake tasks in the Rakefile sitting in your application's root directory. You can define dependencies between tasks and files. It's useful for deployment, sql dumps, code stats, documentation generation etc. In addition to the built-in tasks, you can find :deploy and :rollback tasks out there if you root around, allowing you to version your application formally.

3. Use Gem

Gem is the CPAN for Ruby - it fetches 'gems' - ruby modules/extensions. You'll need to use it to get your database adapter and other utilities.

4. Use Rdoc

Rdoc creates documentation from Ruby classes and spits it out in HTML format. I have a bookmark group in my web browser made up of Ruby and Rails standard libraries. Cmd-click in Safari makes all the tabs load in one go. I'm sure Firefox has an equivalent.

Note: All three of these command line utilities (rake/gem/rdoc) are for Ruby in general, and exist independently from Rails. However, they will come with your Rails install.

5. Use the logs

Use the logs in the ./log directory. I always keep a unix screen open with

tail -f ./log/development.log

running in it. This is the best way to get a feel for how ActiveRecord behaves in terms of database calls. To write to the log, use

logger.info("message here")

6. Use Rexml for XML parsing

Most projects have some XML parsing going on - REXML is the most complete XML parser for ruby. You'll need to install it - it's not stdlib. Google it.

7. Take the time

RoR is encouraging a whole load of web developers to enter application development. This can be a good thing. However, it's a different ball game, so to speak. For starters, understanding the Model View Controller (MVC) design pattern is crucial to RoR. It's pretty simple, but you need to be able to apply it smartly in order to keep things neat as your application progresses. Understanding the role of testing and code versioning is also really important. Take the time to learn common application development and design techniques/patterns if you're coming from a web background. Your test code should usually outnumber your production code.

8. Get the Ruby syntax highlighting module for BBEDIT

When developing locally i like to use the BBEDIT text editor, because the cli editors start hurting my eyes. You can set up a project using New > File Group. The ruby syntax highlighter is here.

9. Check the irc channel

#rubyonrails at irc.freenode.net - nice people on the whole.

10. Using Webrick/Apache

First off, use the Webrick server while testing. More here. That's another unix screen i have running continually. If you want to test under Apache, turning on fast-cgi is explained here.

Finally

Feel confident. Rails is an enterprise level framework taking it's first steps - don't let the J2EE crew drown you out. Ruby > Java in spirit and in tone, even if Ruby's gem pool still has a long way to go to match the colossal Java API or Perl's extensive CPAN libraries. In terms of RoR: The framework is there. It will grow by virtue of the agility it encourages.

I'm still yet to find any particularly great apps in Rails since the 37Signals ones, but here's a simple code snippets library produced in RoR - Snippets. Keep an eye on the OS X and Ruby sections as they grow. I like the simplicity of this tool.

That little girl is Ruby. She is hacking a PHP corpse to pieces. She is on Rails. It does that to you.

Today i woke to the tune of nausea. It didn't subside for about an hour. I think it's lack of sleep.

Just a couple of days with Ruby on Rails (meme of the month) and i must say: wow.

Unlike most coders i'm an aesthete. I believe strongly in an 'art' of programming. This came down from my father, who in the early eighties used a multi-national's resources to stage a Fractals exhibition. That was in S.America and Fractals were (and possibly remain) the big daddies of computer art. My dad believes in spiritual programming. I'm not sure what this means yet. Then again i am young, dumb and full of CRUD.

Which brings me onto Ruby. And Rails. This framework has style. It feels great. I can't remember the last time i looked at code and thought that. Ruby's syntax is a breath of fresh air in a world of Java tangles and Perl obscurity. And it's powerful. It's not a designer-wants-to-make-a-website kind of tool. It has real depth. Downside: The Ruby libraries look a bit thin. And no webhost is running it.

It was a close call between going with this or Perl's Maypole (a similar MVC framework). But for now, I'm sold. More info on my project soon.

Jon Aquino's YubNub is a social command line for the web. Try it here. At base level it's a customisable search query form. Much like existing web browser plug-ins. However, any user can add functionality to it, and there's lots of potential in adding a formal 'piping' (as in unix piping) framework, whereby you can envisage search queries like:

amazon hasselhoff | aws_get_titles | technorati_array_query | technorati_get_authors  

To give you a list of anyone who's blogged about any films david hasselhoff has ever been in (for possible contact and formation of Hasselhoff co-cast fanclub). Or simply:

google_img david hasselhoff | upload_to_flickr

Any web script triggered by a GET request can be declared as a YubNub command.

YubNub is a crucial, simple idea (cooked up as a 24 Hour challenge), and it's wholly in step with the net's slow morph from a set of interlinked documents to a global operating system running distributed, networked applications and services. It faces serious customer abuse/spam/security issues. It'll be interesting to see how it evolves in the coming weeks.

Also check Jon's Introspicious - intermediate tag analysis on the bookmarks you index at Delicious.

Sure, several of these plug-ins already exist. Mine has the following advantages:

• You don't need to know any code to get it working.
• It uses the RSS feeds, as opposed to the potentially volatile Delicious API.
• It caches feeds locally so you don't suck Delicious' bandwidth.
• You can easily filter by tag and username.
• I wrote it

OK, so i did it as a small exercise in Perl, but if you use Movable Type it's a great
way to keep all your links upto date and in order, since the numerous Delicious browser plug-ins make it so easy to store bookmarks as you browse. The links on the left-hand right-hand side of the index page here at quotesque.net are all tagged 'sidebar', for example.

Download

• MT-Yummy-1.0.0.tgz (unix)
• MT-Yummy-1.0.0.zip (un-unix)

If you find the plug-in useful or even like it, then a simple 'thank you'
is always appreciated. If you use the code then include a reference to me.

Requirements

This plug-in is tested on Movable Type 3.16/3.17.

Should work in 3.0, 3.01D but haven't checked.

You need the XML::RSS::Parser perl module. It's available to download
free.

Installation

To install, place the Yummy.pl file in your Movable Type 'plugins'
directory. If you do not have a Movable Type plugins directory, create one,
then put the Yummy.pl file in it. Refer to the Movable Type documentation
for more information regarding plugins. So make sure Yummy is here:

MT_DIR/plugins/Yummy.pl

The functionality in MT-Yummy requires the use of the LWP::UserAgent,
HTTP::Request and XML::Parser modules, so you'll need to make sure they are
installed on your web host. These are very common modules.

Yummy additionally requires the XML::RSS::Parser module. This is less common.
Get it from the URL above (cf. Requirements). Unarchive it and place the 'RSS'
directory within into

MT_DIR/extlib/XML/

The first part of the package name (in this case, XML), indicates the directory.
If the 'XML' directory doesn't exist in your 'extlib' directory, then create it
before uploading the "RSS" directory.

Last but not least, you need to make a new directory called 'yummy' in your MT root directory:

MT_DIR/yummy/

And you need to set the permissions on the directory to 0777. Your ftp software
should give you a way of setting permissions. 0777 means anyone can read/write/execute in that directory.

Using Yummy in Movable Type Templates

Three usage examples. From basic to advanced:

Basic:

<MTDeliciousLinks username="Hasselhoff">
<a href="<$MTDeliciousLinkURL$>"><$MTDeliciousLinkTitle$></a>
<br />
</MTDeliciousLinks>

Including this code in your MT template will display a list of links from
Delicious user "Hasselhoff", with one link on each line. Because "lastn" has
not been specified, it will by default retrieve only the last 20 links posted.

Pretty Basic:

<MTDeliciousLinks username="Hasselhoff/groovy" lastn="15">
<a href="<$MTDeliciousLinkURL$>"><$MTDeliciousLinkTitle$></a>
<br />
</MTDeliciousLinks>

This looks like the last example, except it retrieves links which user
"Hasselhoff" has tagged with the keyword "groovy". Like this you can display
categorised link lists. It also specifies how many links - the last 15
posted by date.

The Good Stuff:

<ul>
<MTDeliciousLinks username="MyUsername/kraut">
<li><a href="<$MTDeliciousLinkURL$>"><$MTDeliciousLinkTitle$></a> 
posted by <$MTDeliciousLinkAuthor$> on <$MTDeliciousLinkDate$></li>
</MTDeliciousLinks>
</ul>

Ok this one has the works. It prints a bulleted list of links tagged "kraut"
by user "MyUsername", including the person who posted the link and the
date it was posted on.

Hopefully these illustrate how you can use Yummy to manage your link
lists, blogrolls etc on your Movable Type site. Coupled with the numerous
Del.icio.us tools, this is a very convenient way to keep your categorised links fresh.

See the readme in the downloader for more info.