With luminaries like Tim O'Reilly posting on the new Yahoo! Pipes facility you hardly need someone like me to say how neat it is, but nevertheless, let me try. As someone who cut their tech teeth on the Unix computer operating system years ago at Bell Labs the idea of being able to hook up the output of one program easily into the input of another is certainly not a cutting-edge concept, but how it's applied in the Yahoo! Pipes toolkit is downright brilliant.

Anyone can use the Yahoo! pipes to develop a range of sophisticated filters on content that allow one to feed outputs from any number of sources such as an XML feed, Yahoo! search query, Google Base or other key source, filter it and wind up with surprising combinations of content. The popular example program takes a headline feed from The New York Times and finds photos on Flickr that match up with headline keywords. The resulting feed is intriguing, if a little raw. All of a sudden the content of one source is given instant context based on filters applied from another source.

As in Unix's pipes the power is in the library of filters that one can apply to content streams. You can take an input and apply a library of logical operators and filters, you can extract location information, you can count items, de-duplicate items, join items, perform text string substitutions, and so on. The interface that provides this capability (above) is a easy to use visual interface that allows one to connect and adjust sources and filter modules easily - a whole lot nicer than the shell scripts command files we used to have to build on Unix systems and resembling the ease offered by a high end product like Reuters' Kobra object-oriented desktop solution for financial analysis.

This is sophisticated stuff in a very friendly package that encourages people to develop their own ways to look at content that they can share with others or use to a personal or institutional advantage. We've seen a lot about widgets that users can drag and drop onto desktops or embed in weblogs, but Yahoo! Pipes takes this concept one step further and allows non-techies (well, at least people who can't be too bothered with tech stuff) the ability to do their own light programming against content sources to come up with a wealth of interesting combinations that can be fed via XML feeds or JSON widgets. It's also the first truly interesting thing technologically to come out of Yahoo in ages - something that Google could have and should have (and may yet) come up with to put publishing in a whole new light. If you thought mashups were exciting before this, get ready for a wealth of new toys for viewing content that will make Yahoo! Pipes a very popular destination for creative content producers of all sizes who want to create unique content in unique contexts.