Yahoo! is the Xerox of Web 2.0

- Someone (ok me) at the Last.fm kitchen table, whilst discussing Ian Rogers' presentation, Winners Leverage Scale

Ian Rogers is VP of Product Development at Yahoo! Music.

We had a power outage so last.fm migrated to 'The Diner' on Curtain Road, Old Street. They were great to us. Pop in there if you get a chance.

Photo from the Last.fm rehearsal room by Dekstop.

We were joined by the lovely folks from MOO and Trampoline Systems for a BBQ on the Last.fm roof on Friday. I was particularly thrilled to meet Stefan, whose work on UpMyStreet, TheyWorkForYou and WriteToThem has been inspirational to all those interested in social software on the web over the last decade. Thanks to everyone who came for what was a fun night.

Yahoo! unveil PIG. It's up against an elephant. Also, what a great icon? The dungarees are a stroke of genius.

Flickr phonecam shot from govan depicting me fumbling with a presentation remote

First of all thanks to everyone who gave their support at the Future Of Web Apps conference in London last week. I had an amazing time and met many awesome people - absorbed lots of energy and enthusiasm and gained some interesting insights. Thanks again to Carson Systems for having us on stage and even letting us drag our last.fm sound-system over to the conference to stream FOWA group radio in the foyer of the Kensington Conference Hall...

Matt & I spoke on the first day, on the topic of 'Lessons from the Building of the World's Largest Social Music Platform'. Corante has written up some notes, and someone even posted a great mindmap of the talk.

2 slides I particularly enjoyed presenting:

• Demonstrating osmotic communication through our internal wonderbot, irccat. In response to one of the questions on the day, our CTO RJ has open-sourced our implementation of irccat so you can start playing with it. To clarify - we use irc as our internal comms channel, and all aspects of our company absorb information through it.

• Attention as filtering tool for user-generated content, in the context of tag cloud moderation. The talented heiress Ms. Hilton was used as an example.

Other bits and pieces - I referenced this Fred Wilson blog post on the future of media with respect to the monetization of attention data. Also someone in the audience brought up the principles of the AttentionTrust, set up by Ed Batista and Seth Goldstein. If attention data is going to be critical to your web app (and I would argue that it should), then I encourage you to read Seth's entire blog archive on the subject of attention. I know I pretty much have.

My personal highlight was hearing Bradley talk on Flickr interestingness and Yahoo! Pipes (which caused quite a stir in the last.fm office last week). I was particularly bummed out about missing Simon's openID presentation on the second day, as it was ace by all accounts.

A podcast of our presentation will be up soon, but in the meantime download the slides - 'Lessons from the Building of the World's Largest Social Music Platform' (pdf)

I'll be speaking along with my esteemed colleague and budding musician Matt Ogle (profile here) at the upcoming Future of Web Apps conference in London (Feb 2007). The line-up is frankly intimidating (Khoi Vin in the house) but we'll do our best. And if that fails, we have some pretty good dance routines up our sleeves.

We'll be talking about ponies, unicorns and um, other last.fm obsessions, so if you're a coder, bug your employer/mum for some cash to come to this - it should be pretty ace.

It's vince's van!

Basic principles of addressability for web applications:

• Always tie your URL's to your data models.
• Use mod_rewrite as sparingly as possible. It's too flexible.
• Don't punish plurals - provide predictable redirects.
• Go for human readable URL's if at all possible.
• Provide namespaces for RESTian web services. Have an xml-based (e.g. rss) representation of every model in your application.

As always, delicious is a model of good design in this area. We're working at last.fm to improve addressability with what is a more complex data model (artist/album/track/label/user/tag).

This is a man - bluntly - whose only contact with Web 2.0 that I can find is a pretty humiliating set of pictures on Flickr of him on a private jet and ogling at half-naked dancing girls

- Tom Coates, on Ashley Highfield, the BBC's head of New Media

Much talk about RoR and will it go mainstream and blah.

I was recently asked by someone managing a RoR project why it was moving so slowly. When i inquired as to the skill-sets of the programmers involved the manager replied, "they used to be our dreamweaver guys".

OK, i thought, looking worried.

I suggested the quality of the programmers may be a factor. He looked at me puzzled, offended even. THESE GUYS ARE PERFECTLY FINE. THEY ARE WEB PROFESSIONALS. He said this with his eyes. Not with his mouth.

Matter of factly, i said, "You may need better programmers. Ruby is a powerful language."

It was like breaking bad news to a sick child. I felt really bad afterwards.

Right now i program in PHP5 with Zend 5 and the IDE, coupled with the vast array of third party libraries, cancels out the productivity boosts of Ruby the language (which is more powerful than php) and Rails the framework (which is more powerful than our homegrown php framework).

Third party libraries play an important role in all this. GEM is under-populated right now. Though gems seem to be consistently of a higher quality than a lot of the things you'll find on PEAR or CPAN, there's just not enough out there.

Will RoR become mainstream?

All i can say is:

Talent does not scale well.

A good ruby developer has demonstrably better OOP skills than a good php developer. The php developer may be better, you just have no way of knowing, because he hasn't access to as many OOP features. To run a successful RoR project you have to hire high quality ruby programmers, there's no middle ground. IMO a lot of web programmers will not migrate productively to python/ruby. Their heads will hurt and they won't program to the strengths of the language and their design will be off and it'll be PAIN PAIN PAIN for other developers on their team.

More here and here and here.

From the article, Dignity is Dead

Not everything needs a software solution. Sometimes the human brain is the solution. I can capitalize my own words and decide when something is a bullet point.

- Jason Fried, 37 Signals

Gigr.net is a community driven gigs portal built on Ruby on Rails. It already talks to audioscrobbler & itunes (to pick up your listening habits) and exports iCal calendars / RSS feeds with shows involving the bands you're currently listening to. It's heading in the right direction by learning to talk early on. What i mean is that this kind of service gains value by playing well with others and that's precisely what Gigr has set out to do. Solid import/export options already.

Anyone can upload information about shows/artists/venues. Username/email is all that's required to sign up for free. Do it. It's just getting off the ground but there's already a ton of shows in there.

Note: If you're a Rails developer then get talking to Ernest about access to the SVN repository. This is just the kind of service that could wind up embedded in something like the last.fm service.

Caught up with Corante head, social network expert and organiser of the Symposium on Social Architecture, Stowe Boyd last weekend on Skype to discuss Clippr following his great post detailing a frustration with current RSS readers.

More interest like this might even convince me to deploy and open up clippr to the public in a more scalable manner.

Must. Find. The. Time.

I"m convinced the next big step for web browsers is implementing offline mode.

So what's required? A local caching framework that server-side applications can hook into, allowing you to, for example, read and compose gmails whilst in transit, browse your rss archive in your browser, etc. This will lead to the inevitable. What's the inevitable? The inevitable is the day when home users will swap most of the applications on their desktop for their web browser.

Been waiting about 18 months for this announcement from Microsoft. The beast, as they say, is stirring from it's slumber. Do they get the new landscape though? Does it even have space for the big boys? I would say that yes, services will nucleate around hubs, but those hubs will only grow by implementing Openness better than their competitors, and by shifting from a product to service paradigm. Not sure even MS can pull off a shift that radical.

If you're looking for a flatshare in London in the near future make sure to bookmark Move Flat, which really just gets it. Julian, creator of Moveflat.com, understands the process. He's been there.

Here's what he says on his site:

This site is about not making journeys across London to look at something that turns out to be a complete waste of time.

That is what the site is for.

I'm not interested in running a property site where anyone can post anything.

That's because it really isn't very interesting for me to do it.

What's interesting for me is to use the internet to change things.

So I'm interested in ads that are interesting for people that are looking on the site.

That means the ads have to contain useful information for people that are looking for somewhere to live.

That means I am refusing between 20 and 30 ads a day on this site, every day.

... what has become important about all internet sites is trust.

Couldn't have said it better myself.

The only other internet service for Londoners that comes close to Move Flat is CraigsList UK. CraigsList is number 1 in NYC but still relatively unknown in London, so spread the word - it's like an internet Loot that works 10 times better and is run by nice people.

I'm posting this entry from within the all new hyped up OSCON'ed to hell and back Flock browser. It's got integrated blogging, bookmarking (del.icio.us) and flickr services. But the best innovation in Flock (most of it is just web services) is the 'shelf', a container for dragging and dropping images  and text during browsing. It acts like a halfway house for all these services. It's duh obvious, but good ideas always are.

Flock is built on the mozilla code base. So get XUL'ing kids.

OK so google have announced Google Reader, which is an RSS client for your browser. The best way to introduce Clippr is as an RSS reader i built that shows up Google Reader in most departments. It feels good to be ahead of the google this time ;)

I built Clippr as part of my dissertation over at Imperial. It's social software for the web. I use it everyday - it's become one of the most important apps in my browser, which i guess is enough reason to tell you a little more about it.

First off, a one-line description of Clippr:

Social bookmarking meets RSS feeds. A great way to discover, archive, create and share articles online.

OK, now for the details.

Overview

Clippr bears some similarity to the social bookmarking tool Del.icio.us, in that it allows users to tag and archive content from the web. However, where del.icio.us revolves around the concept of the 'bookmark', Clippr's main building block is a 'clipping'. A clipping is either a textual excerpt taken from a webpage or an article from an RSS feed. Both these types of clippings are treated in the same way by Clippr.

Why would a user want to take clippings?

Clippings are like a fluid bookmarking strategy for the web 2.0. They're a replacement of the bloated 'temp' folder which sits in everyone's browser bookmarks. They acknowledge the kind of transitory interest in numerous topics that the web encourages in users, as well as the fact that users are increasingly browsing by RSS feed. Clippings are a replacement of the bookmark per sé, in that they are represented as RSS articles and not simply as hyperlinks. Put simply,

Clippings tie a user's bookmarking activity to their RSS reading habits and provide a richer definition for archiving web content than a bookmark.

What's the big deal about folksonomy? It's just labels.

Um, yeah, like what's the big deal about computers, they're just big calculators. OK.

Folksonomy is a faceted, emergent classification strategy. It's the natural classification strategy for networked media, in that it addresses directly the fact that,

In an information retrieval system, there are at least two, and possibly many more vocabularies present. [14]

One of the main advantages of Folksonomy is that,

Since the organizers of the information are usually its primary users, folksonomy produces results that reflect more accurately the population’s conceptual model of the information. [35]

For more on folksonomy - and the sources of these references - check my dissertation.

What's implicit social software?

Social Software can perhaps be summed up as software that recognises the most important component of it's system is the user-base. If social software wants to do smart stuff, it does so by aggregating the intelligence of it's user-base, in terms of the data captured by the application during usage.

Implicit social software means getting rid of user profiles and user visibility in general and just letting the app use the aggregated information to work out relations between data. Like this the application functions like a single-user desktop application without all the distractions of social networking, but draws on community intelligence to help the user navigate the data architecture created by the user-base.

Text-analysis is rubbish, it gives me google adsense ads about detergent when i'm reading about the SOAP protocol

True. Clippr acknowledges the user-base as the central component in it's classification strategy. The system performs text analysis on incoming articles from user subscribed feeds, which it uses to extract keywords and top stories from the fresh crop of articles. Then, whenever a user's tag coincides with a keyword extracted by the system, all articles with that keyword are shifted over into the tag-space. This is a crude automated classification strategy which reconciles text analysis and tagging - it works pretty good.

Moreover, Clippr analyses tag intersections to recognise related tags, and presents these to the user as they search for articles. It's more subtle than del.icio.us in this respect, in that it quantifies the similarity between tags, in terms of intersection magnitudes.

So text analysis needs to be combined with context analysis to be effective at picking out similarity between documents, and that's what Clippr does, by letting users provide contextual information and analysing it.

Folksonomy is an uncontrolled mess. Jack says 'web', whilst Jill says 'internet'

True. Wordnet can be used to reduce the tag space by recognising synonyms, hypernyms, hyponyms and compound terms, but it has limited performance with a completely uncontrolled vocabulary. Acronyms and neologisms abound in tagging. A lot of this is still unresolved and impacts the performance of search in folksonomy.

Show me the money

Here's the feature list:

• OPML import/export of feed subscriptions. Folders are flattened to tags and imported automatically into Clippr.
• Firefox plug-in and bookmarklet to facilitate clipping stuff from your web browser.
• Tag clippings, tag feeds, tag like a demon.
• A community oriented article base formed through user subscriptions, refreshed periodically. Full text RSS/RDF/ATOM formatted feeds are supported.
• Text analysis (article clustering) on incoming articles, in order to extract Top Stories and article keywords.
• Context analysis (tag clustering) used to recognise related tags.
• Change your tags whenever you want. Clippr handles merging/splitting of tag-spaces.
• Power editing using batch actions thanks to a gmail style dynamic dropdown.
• Tagging combined with keyword extraction to produce automated classification of articles. Text analysis and folksonomy reconciled.
• A search engine supporting a query syntax for folksonomy - search by tag (intersection/union), feed, keyword or any combination of these. Implemented as Live Search for desktop style responsiveness (it behaves like Apple Mail search - wipe the search field and return to where you were)
• RIS export for using web references in bibliographies
• RSS export of your Clippings archive.
• Mail an article to a friend or recommend it to a fellow Clippr user.

Ok, let's try it out

You can't. It's not public right now because it needs a dedicated server to run and i can't afford to run and maintain one. If you want to help me setting one up, please get in touch with me.

More information

Here's a quick presentation and here is my dissertation in full. Source code (ruby/javascript) to come.

Seth Goldstein explains the nature of a new venture, the AttentionTrust, one of whose goals it to impede applications attempting lock-in through personalization. The AttentionTrust provides a framework for personalized services based on recording information about your web browsing. It's an ambitious project that attempts to tackle two big issues of the web 2.0 - Attention and Trust. Seth has some insightful observations on the nature of attention and the value associated to it in the new economy.

Lots of exciting applications unveiled at the Web 2.0 launchpad this week. Stowe Boyd's Get Real blog has concise descriptions of much of this javascript (cringe) heavy software.

It seems like every other day a new piece of web software is announced - from Zimbra to Jotspot to Writeboard to Google Reader. All of this is social software, in one way or another.

So, online accounting software is going to start pouring out. So far, financial application powerhouse Intuit have Quick Books Online which doesn't work on a browser running outside of Windows. This oversight astonishes me - Web Apps are by their very nature cross-platform, all that's required is a sensitive approach to javascript and CSS and a slightly longer Test cycle. A lot of small businesses are using Mac. Among the other contenders we have Bill My Clients which integrates with Paypal and Blinksale which wins the GUI war but has a restricted feature set (it's an invoicing tool above all else). They all come in at about $12/month. All these services wind up being cheaper for Brits than our American counterparts. Needless to say they all support Pounds Sterling.

Surveying this software i get the distinct impression i'm beating around the bush. What we need is a comprehensive collaborative spreadsheeting package for the web. Winner takes all.

I went to visit a web firm and they were talking about the development of 'Broadband' content. They had some neat interactive video pieces that worked well under a variety of connection speeds. On the whole, the 'broadband' talk sounded a little nineties to me. I'll tell you why.

The Web 2.0 second annual conference is held this October. The term 'Web 2.0' refers to an Internet focused on applications and services. Full definition here. Sure, connection speeds are increasing, but that's not news. News is the way the semantic web is kicking in, and how a breed of social applications are leveraging their vast user-base to provide services that can't exist on the desktop. News is how collaborative web-based tools are improving rapidly. News is how web services are beginning to connect to form process pipelines.

Big challenges in web 2.0 are in the realm of information/knowledge management, security & authentication and distributed application development.

For a developer, the most enticing thing about the web right now is the sheer pace of it. Web application technology is moving at a pace that will soon put business-critical desktop applications under immense pressure.

So when i'm shown rich-media ('broadband') content for the entertainment sector i think - neat. But it doesn't get me twitchy with excitement. The principles of Web 2.0 do.

Been working with Ruby On Rails on & off for about a fortnight. A few observations for those starting to pick up this framework,

1. Keep Dave Thomas' excellent text by your side at all times

Dave Thomas, co-author of the 'Pragmatic Programmer' series, is a worthy guide to Ruby. This text is excellent - Programming Ruby: The Pragmatic Programmer's Guide. Also keep Matz's original Ruby manual to hand. It's fun.

2. Use Rake

Rake is a ruby makefile-style utility. It's useful. Try rake --tasks to check the available tasks. You can define your own rake tasks in the Rakefile sitting in your application's root directory. You can define dependencies between tasks and files. It's useful for deployment, sql dumps, code stats, documentation generation etc. In addition to the built-in tasks, you can find :deploy and :rollback tasks out there if you root around, allowing you to version your application formally.

3. Use Gem

Gem is the CPAN for Ruby - it fetches 'gems' - ruby modules/extensions. You'll need to use it to get your database adapter and other utilities.

4. Use Rdoc

Rdoc creates documentation from Ruby classes and spits it out in HTML format. I have a bookmark group in my web browser made up of Ruby and Rails standard libraries. Cmd-click in Safari makes all the tabs load in one go. I'm sure Firefox has an equivalent.

Note: All three of these command line utilities (rake/gem/rdoc) are for Ruby in general, and exist independently from Rails. However, they will come with your Rails install.

5. Use the logs

Use the logs in the ./log directory. I always keep a unix screen open with

tail -f ./log/development.log

running in it. This is the best way to get a feel for how ActiveRecord behaves in terms of database calls. To write to the log, use

logger.info("message here")

6. Use Rexml for XML parsing

Most projects have some XML parsing going on - REXML is the most complete XML parser for ruby. You'll need to install it - it's not stdlib. Google it.

7. Take the time

RoR is encouraging a whole load of web developers to enter application development. This can be a good thing. However, it's a different ball game, so to speak. For starters, understanding the Model View Controller (MVC) design pattern is crucial to RoR. It's pretty simple, but you need to be able to apply it smartly in order to keep things neat as your application progresses. Understanding the role of testing and code versioning is also really important. Take the time to learn common application development and design techniques/patterns if you're coming from a web background. Your test code should usually outnumber your production code.

8. Get the Ruby syntax highlighting module for BBEDIT

When developing locally i like to use the BBEDIT text editor, because the cli editors start hurting my eyes. You can set up a project using New > File Group. The ruby syntax highlighter is here.

9. Check the irc channel

#rubyonrails at irc.freenode.net - nice people on the whole.

10. Using Webrick/Apache

First off, use the Webrick server while testing. More here. That's another unix screen i have running continually. If you want to test under Apache, turning on fast-cgi is explained here.

Finally

Feel confident. Rails is an enterprise level framework taking it's first steps - don't let the J2EE crew drown you out. Ruby > Java in spirit and in tone, even if Ruby's gem pool still has a long way to go to match the colossal Java API or Perl's extensive CPAN libraries. In terms of RoR: The framework is there. It will grow by virtue of the agility it encourages.

I'm still yet to find any particularly great apps in Rails since the 37Signals ones, but here's a simple code snippets library produced in RoR - Snippets. Keep an eye on the OS X and Ruby sections as they grow. I like the simplicity of this tool.