Apps fever is sweeping across the content industry, spurring hopes amid content providers that software applications development toolkits available for mobile devices like Apple's iPad and iPhone and Google's Android phones will allow them to define new channels for revenues. Certainly "apps" that can be downloaded from online storefronts provided by these and other platform providers are taking off in a big way.
There are more than 160,000 apps available for Apple devices that have been developed over the past two years, while in the six months since the introduction of the Android Marketplace there are already more than 42,000 Android apps available. The lure of having a little icon on the desktop of these devices for apps that can add engaging features to content - and, many hope, premium revenues - is hard for most publishers and services developers to resist.
And why not? After all, mobile phones come equipped with all sorts of new sensors and services that make the integration of content with mobile services very intriguing. People are "checking in" to hot spots via geolocation apps like Foursquare and Godwalla, pinching and zooming their way through layers of data in mobile Google Maps, as well as downloading movies from Netflix and steering airplane traffic via Flight Control HD, not to mention reading news from magazines and newspapers. It's all a bit reminiscent of the PC-based consumer software revolution of twenty years ago, when store shelves were lined with all sorts of packages to make use of that generation's emerging technologies.
Go to a tech-oriented store today, though, you'll find that packaged software is pretty scarce. Along came the Web, making both software downloads an easier way to get a hold of zippy applications as well as Web sites that made content like CD-ROM references seem like stale stuff. Apps are in part an attempt to reclaim the glory days of premium packaged software, as well as an attempt to shove content services into Web-proof cans that will "protect" them from all of that nasty Web content that would otherwise be rubbing up against it. If you doubt this, try using the default search tool on the new iPad; you'll be directed to apps-only selections for your content, forcing you to go to your browser to find content from the Web via the search engine of your choice (by contrast, Google's Android-equipped Nexus One's default search looks at content on that device plus Web content, with a separate search for apps via Android Marketplace).
There are pluses and minuses for Web-based content versus apps-based content - thanks to Jill O'Neill of NFAIS for a link to this nice tech summary by Richard Padley - but the largest minus of all for content producers seduced by apps mania is findability. Although many apps consume Web-based content - or are, in many instances, just lightly reskinned versions of Web content - apps exist largely in a netherworld of darkness when it comes to search engines. That's just fine by many publishers that are more eager to reproduce the print experience on devices like iPad via premium apps than they are eager to get their apps content discoverable via the Web. In hopes of offering their advertisers and shareholders new value via apps through old software and publishing models, the presence of findable options for their content via the Web is a given, or, for some, perhaps, something that they wish would go away.
Yet, curiously, neither the Web nor the power of search engines to get good content in context at the point of demand show any serious signs of going away. In fact, with the continuing expansion of HTML 5 Web standards, Web-enabled applications are starting to interface with many of the mobile sensors that today's apps toolkits enable software developers to exploit. Publishers may be looking to apps as an alternative to the Web for advanced functionality, but the Web itself is becoming increasingly functional and extensible into sensors on mobile devices. Even in today's apps on Apple and Google Android devices, most links in both editorial and ads in these apps lead typically to Web content. The notion that apps are going to make the Web disappear by the desire of publishers willing it to be so is a myth. There is no substantial "there" in apps without the Web.
Nevertheless, apps are going to be with us increasingly as combinations of information and experiences that provide value to audiences in new contexts. As such, apps fit Shore's definition of content, content that still needs to be discovered as Web pages do, even if, perhaps, in different ways. In a sense search engines traverse some apps already by querying databases that drive some Web sites. But the broader question is what happens when unique content gets delivered via apps and not via their Web page equivalents, be it via HTML 5-enabled apps or via apps using proprietary toolkits such as Apple's. There's the strong chance that some sources of content will sink permanently into the "dark Web" again, not to mention new sources of content that will never be discoverable via the Web.
Great minds are thinking about this, of course, but not necessarily equally. One of the great neglected opportunities of the apps era is creating search utilities that can place emerging apps into the right context via search alongside more traditional page-based Web content. Already we get video clips, images and widgets delivered up via search engines that match particular queries or metadataclusterings; why not apps also? Some apps providers may balk at this notion, preferring to keep content consumers corralled into can-like containers that limit their options for cross-pollinating with rival apps platforms. The gaming console industry has certainly managed to keep stores that used to stock software well-lined with CDs that are in essence apps for those devices, so perhaps publishers have reason to hope. But my sense is that it's largely a false hope.
I believe that it's a false hope because browsers aren't going away any time soon. In fact, Web browsers are becoming only more powerful, with ever more technology packed into them to launch advanced applications as well as run-of-the-mill Web pages. Thinking of the rapidly developing Chrome OS operating system, browsers are, in their own way, even becoming devices themselves. If you thought that the iPad was slick, imagine what happens when you get an instant-on device that you can log into once and be enabled for both everything that the Web offers and everything that premium apps offer from one Web-driven touchscreen device? Now imagine one step further - imagine that it's all discoverable via one search utility. Game over, content industry friends.
The same discoverability issues will exist within enterprise firewalls, of course, if not moreso. Most organizations cannot afford to have their content locked into proprietary apps if they are to build business intelligence dashboards from multiple sources rapidly and effectively. Few will have patience for publishers wanting to sell them independent apps "cans" - you may as well tell them to go back to the era of CD-ROM products. No chance. As more enterprise-ready apps make their way to the marketplace, their day-to-day utility to individuals in businesses on mobile platforms will clash more and more with the need for those businesses to break open those cans to increase productivity amongst collaborators. Images of jolly executives toting touchpads to board meetings with print-friendly digital documents are largely mythical representations of how businesses really need to work today. It's not about individual convenience as much as getting teams productive as rapidly as possible. In a corporate world that's trying to break out of its own silos constantly, tight-as-a-can apps for content consumption are silos that few will be able to afford.
With all this said, the new generation of software and content services developed via emerging apps offer tremendous promise as platforms that can deliver real functional value to audiences. However, that functionality in and of itself cannot replace the need to find all of the relevant content that's needed to accomplish personal or organizational goals, be it through an app or any other number of useful content consumption tools. It's the ability to integrate content from multiple sources with multiple sensors that makes apps most valuable; using apps as a short-cut DRM tools based on proprietary standards shuts down most of the value that they have to offer in the first place. So, as you approach your apps strategy, remember at least these three simple rules:
Don't use apps as an excuse to ignore the power of the Web
Use apps to extend functionality that integrates content, not as a tool to segregate it
Design your apps with content discoverability via search in mind - even if your current app store search tools may not warrant it
This is all a way of saying that although the current interest in apps has grabbed a lot of headlines, there will be plenty of other trends grabbing headlines in the months ahead. Brace yourself for an emerging, complex landscape that will be integrating the world of apps and Web pages into a cohesive whole of services, with search engines playing a key role in gluing these together rapidly into on-demand services that individuals and enterprises will be craving. If you thought that apps were going to line up your content problems into neat little packages, it's time to break out the can opener.
It's always been fun to be a part of ASIDIC events, so I was very pleased to have been invited to moderate a Q&A period at this year's Spring ASIDIC get-together at the offices of Lyrasis in Philadelphia. It's a bit more low-key venue than for previous ASIDIC events, which reflects in some ways the challenges that many enterprise-oriented publishers have faced these days, but also the degree to which their business models are trying to catch up to the value points in publishing that revolve around metadata and search technologies. The good news is that the ASIDIC meeting pulled together some excellent case studies demonstrating how publishers are moving away from "pull up a document" styles of electronic publishing towards using sophisticated semantic processing to get their content ready for battle for use in contexts driven by metadata. Here are some links to the panel-by-panel posts that I recorded on Google Buzz (no login required to view, login required for comments):
IDC's Sue Feldman on the New Search Architecture Sue was in good form, I really enjoyed her insights. Key stats from IDC's 2010 enterprise user survey: 21 percent use colleagues as their first stop for information, 61 percent go to the Web first, only 1.8 percent to their subscription database services. My take: if you're not using the open Web and social media as marketing channels, you're missing more than 80 percent of your opportunities to be relevant in the "go-to" source for people who need your enterprise content.
Thane Kerner, Silverchair - A Primer on Semantic Technologies A good overview of today's semantic technologies and terminology. One of the nice things about this ASIDIC meeting is that it got pretty deep into the implementation of semantic technologies without lapsing into endless "geek speak."
Case Studies - IEEE and SciTech Strategies, Inc. This was a very interesting study of how the IEEE used domain mapping as a tool to reveal expertise appearing at the intersection of subject domains not usually associated with one another. By using taxonomies and domain mapping they revealed opportunities at the intersection of information technologies and medical science - the type of opportunities that innovation professionals are focusing on to build out new markets for products and services.
Case Studies - Enhancing the user's experience with semantic "smart linking." McGraw-Hill highlighted work that they are doing using metadata and XML-formatted content to build out new editorial content for their premium Aviation Week and Platt's enterprise services rapidly. These technologies are enabling them to generate "topic pages" rapidly that can be destinations for links embedded in their news coverage and archives. Metadata can also enable opportunities at the intersections of their publishing properties - for example, it would be interesting to see how information on commodities such as jet fuel prices from Platt's could be made useful in Aviation Week content.
Case Studies - Collexis and the American Association for Cancer Research This was an excellent example of how deep taxonomies and semantic technologies solved a very crucial problem for a scholarly publisher. Collexis enabled AACR to identify a much broader range of topic experts to be available for peer-reviewing scientific research articles and to filter out people who may have a conflict of interest. At a time when scholarly publishers are trying to position their assets more effectively against Open Access competitors, being able to demonstrate superior methodology for peer review via advanced technologies is a great idea.
Case Studies - Getting references right - how semantic technology helps linking, findability and analysis Interesting example of how the American Psychological Association went from a "square zero" in Smart Content to state-of-the art infrastructure to help it begin to build rich and powerful search experiences on Mark Logic's XML server. One of the real stories about semantic technologies today is that although it's not effortless to make the transition to Smart Content, today's technologies can enable publishers to make that transition much more rapidly and cost-effectively. Harder, though, is getting business models up to speed.
Closing keynote - Steve Sieck, SKS Advisors Steve always has powerful and thoughtful insights delivered with a good dose of understatement, a combination that makes him well worth listening to at events. Steve did a good job highlighting some of the key "what's next" themes for semantic content, including social media integration, "linked data" - enabling data to "talk to other data" on the Web in ways that enable semantic APIs - and the extension of semantics into marketing and branding.
All that and much more made the trip down to Philadelphia for the day well worth it. As I was discussing with an attendee afterwards, this is still the early days of semantic implementation for many publishers, with many high-value products and services only beginning to emerge for enterprise use. For example, what happens when you start applying semantics to newly released scientific research that puts previous research about a company's drugs or medical technologies in a negative light? All of a sudden technologies that were intended primarily as search interface tools then become powerful technologies for building real-time news and intelligence that can move securities markets rapidly. We're in the early days for these technologies, indeed, offering publishers opportunities to "leapfrog" their way into new value propositions.
Yet looming above all of these opportunities is the Web itself, that vast collection of human insight that most people still use as their primary reference so often. Precious little was said at ASIDIC about how to use Smart Content to built more Web-aware content. There was also an interesting interchange that I had at the end of the meeting with a long-time indexing expert who mused about how in many ways the metadata that was adding the most value in many of the examples discussed at the event were not necessarily those tried-and-true indexing tools that have been used for years. Yes, the truth about metadata is that much of what has been considered useful "information about information" is just the starting point for adding value to content today.
Here, also, the Web points the way. While Google is not thought of as a service that uses semantic tools in its presentation of content, in fact its content is rich with semantic inferences from Web page links, analysis of use statistics, evaluation of geo-tagged data and other content to derive useful information and experiences. These happen mostly "behind the scenes" in Google services, but they are there nevertheless, aiming towards the very "accuracy" that was discussed at the day's sessions. Ultimately Smart Content is the content that transforms what was previously thought of as just a publication or a search result into the input for sophisticated content-serving applications, whether they are presented as a publication or a problem-solving tool or a workflow service.
Thanks again to the ASIDIC team that put together a very interesting event with great attendees. Hopefully better times will enrich us with more events like this.
With more publishers of scholarly and learned professional journal articles trying to build revenues through improved marketing, the search, display and sales tools being developed by DeepDyve are finding stronger interest than ever 2010 among publishers. DeepDyve exposes free and premium scholarly content through its own search engine and through the search tools of partners and makes it available through its read-only viewing tool embedded in Web pages. This allows people finding articles to "rent" them on a once-off basis in read-only mode for as little as 99 cents. This can be particularly handy for people who would otherwise have little occasion to purchase a full subscription to a premium scholarly journal, thus opening up this premium content to markets that would otherwise not provide opportunities for new publishing revenues.
How much more revenue? In a recent discussion with DeepDyve CEO Bill Park, he indicated an estimate in the low billions USD for the total market available for "rental" pay-per-view style access to scholarly content. While this is certainly not enough to float the boats of scholarly publishers in general, it's largely found money that will increase their total revenues at a time when revenue growth is a challenge. That's a concept that attracts partners large, small old and new to DeepDyve's services, including newly announced alliances with De Gruyter, one of the oldest and most respected scholarly publishers, and CiteULike, a Springer-sponsored social boomarking service for scientific researchers.
For De Gruyter, an established brand still requires new marketing techniques to reach researchers who do not have access to paid collections in institutional libraries, while CiteULike, a venue that attracts researchers both in institutional and independent settings, provides a way for people in cross-disciplinary research to sample collections that may eventually be a part of their more permanent interests. In both instances the services of DeepDyve are well aligned with the needs of people involved in innovation management as they probe their own adjacent markets and test out new ideas that may be worth research and product investments.
Scholarly publishers are having to adapt to research markets that are increasingly moving beyond traditional academic boundaries, prompting both alliances with organizations such as DeepDyve and their own repackaging efforts to make topic-based slices of content available from a broad selection of their journals. While the topic-based repackaging has its merits, the DeepDyve approach to ad-hoc access on a read-only basis is an essential component of this repositioning of premium scholarly content, allowing publishers to test out what kinds of content are attracting premium access far more quickly than traditional marketing cycles are likely to capture.
So not only is "rental" content valuable in terms of its direct and ad-supported revenues, but also valuable because it is, in effect, "live" market research into "willingness to pay" habits in specific market sectors. It is then up to publishers, of course, to respond to the insight that they can gain from this sales data to consider new slices and titles that can respond to premium opportunities more rapidly. The more partners that a company such as DeepDyve gets, the more insight they are likely to have available to their partners via use and sales metadata to determine such trends. Should Google Scholar join the many established publishers already using DeepDyve, their metadata on content usage could become more interesting yet.
To some degree these concepts are "Publishing 101" ideas, but the speed with which research markets are shifting are changing the ways in which they need to be applied. With permanent collections of well-established journals constantly under the pressure of institutional budget cuts, the pressure is on scholarly publishers to define "must-have" collections that are really responsive to the needs of their customers. DeepDyve's content discovery and "rental" tools can help publishers to respond to both opportunities and threats to premium revenues more rapidly, even as they build premium revenues on an on-demand basis. Yes, this may seem like ancillary revenues to some publishers, but it is revenue that is both sorely needed and which can be a guide to where best to grow broader revenues that are more easily defended in challenging times.
In a typical game of chess, there are three distinct phases of play: the opening, in which a handful of chess pieces stake out strategic territory on the chessboard, the middle game, in which the positions of many pieces are used to jockey for control of the chessboard, and the endgame, in which the pieces are traded and moved rapidly into a reduced and final push for ultimate control of the board and the strategic goal of the game - capturing the king. It takes both logic and passion to excel at chess, but at the end of the day it's a well-executed plan that wins the day.
You might say that Google has been in the process of introducing its own endgame for online publishing, quietly moving dozens of initiatives into strategic positions which in and of themselves may seem inconsequential to the game as a whole - until its ultimate position begins to evolve rapidly. As in a chess endgame, Google's recent moves are swift, monumental in their impact and, potentially, decisive in determining the outcome of how content becomes valuable on the Web. Media critics like Ken Auletta have quipped that Google needs more "Kirks" and fewer "Spocks" to succeed, mistaking the crowded middle game of media posturing against Google for an ongoing battle, when in fact Google has been keeping its well-reasoned eye on the pieces that will be most important for the outcome of the game.
What's the king that needs to be captured in this endgame? The Moment. Media companies continue to churn out outdated moves such as media players serving up magazine-like renditions of their own content, thinking that quality that reflects the last game that they won is what will win the day. In the meantime, Google's intense concentration on processing power in cloud computing, Web-standardized applications and search dominance have revealed a strategy that is quickly eliminating viable moves for many B2B and consumer content and technology companies. After the September introduction of The Second Web via its Google Wave preview platform for real-time collaboration, Google has in recent days extended its dominance of The Moment via three new initiatives: expanded personalization of search results, real-time search results and voice, location and sight-activated mobile searches, including Google Goggles, a point-and-click camera-activated search feature.
Danny Sullivan at Search Engine Land has an excellent analysis of how Google's debut of personalized searching that doesn't require a Google login is introducing a "new normal" for its search environment, in which the content presented in search results will by default be different for different people based on their last 180 searches on Google. What is The Moment for these people? Where their interests have been most recently. Instead of waiting for editorial boards to decide what The Moment should be, Google is yet again trumping traditional editorial functions and allowing people's own behavior to have a seat at the editorial table automatically.
The introduction of content from real-time Web sources such as Twitter, Facebook and other status-oriented messaging services in Google search results extends The Moment into content sources that have split-second relevancy to online content seekers. Klipp Bodnar points out that this stream of tweets and postings means that B2B companies can no longer ignore real-time in favor of traditional SEO strategies if they're going to get people's attention. It's a broader scope than that, of course: nobody can afford to ignore real-time social media content generation now any more than a securities trader can ignore real-time stock tickers. All brands must enter the real-time conversation of The Moment to keep in touch with their markets and to define their markets.
Google's mobile search initiatives, introduced last week at the Computer History Museum, are perhaps the most profound in their potential impact, even if their ultimate powers are years away from being felt. Voice-activated and GPS-activated Web search is being perfected rapidly at Google and through other outlets, but the Google Goggles initiative, previewed in its development phases on MSNBC recently, brings a point-and-click element to The Moment that promises to give Google a real leg-up in mobile search markets. Using the camera in mobile phones, Goggles enables searches for information on things such as landmarks, stores, products and text simply by filling the camera's viewfinder with the item and clicking. Remember all of those fussy infra-red applications that were supposed to get us "beaming" business cards to one another? Now, just take a photo of someone's card and it will be uploaded into a contacts record. In just those few capabilities already targeted, whole content markets are about to develop as people capture content in The Moment.
And who will have all of the search data and metadata regarding all of these Moments? Yep. Yet again, Google is positioning itself to be the cloud-empowered master of what people are interested in right now, giving them the ability to bring people closer to their interests and passions simply by asking for them. And, yet again, by including as much content as possible in serving their customers, Google doesn't second-guess what people consider to be valuable in The Moment. If the stock and news tickers of the 20th century distributing content from central markets and publishers were the gold mines of Moments in that era, Google's absorption and distribution of content from anywhere to anywhere in The Moment has enabled it to enlarge its unique databases far more broadly and rapidly than any other publisher on earth. And, like a chess endgame, the speed with which other players are losing effective counter-moves against Google's strategic position in The Moment is only quickening.
No small wonder, then, that the U.S. Federal Trade Commission is scrutinizing Google's acquisition of AdMob, a leading mobile ad network. Markets thrive when there are still a good number of pieces on the board to keep competition high. But perhaps it's time for the FTC and companies in the content industry to look beyond this rapidly emptying game board and to consider what the next round of content industry chess is going to look like. If The Moment is the new center of the publishing industry, how does content become most valuable in this context? The answer to this question is, in part, to acknowledge that the companies who collect the most input about the world most rapidly become the most knowledgeable about what is happening in The Moment.
It's a phenomenon that I call "the Sensor Society," a world in which our corporate awareness and memory becomes a valuable through common access in a way that reverses the "information is power" equation. Certainly having private information will continue to empower people and organizations in select circumstances, but for the average person or business having access to all information in the right context is becoming a more powerful resource for decision-making. To borrow a concept from my book Content Nation, some portion of the DNA of society is migrating into the Google-dominated cloud, with each of us feeding that part of our collective consciousness through our voices, our camera "eyes" and our fingers touching screens and keyboards. That may be a good thing for society as a whole, but it will be an enormous challenge for institutions who are not ready to accept that migration as a beneficial development.
What does this mean for publishers? It means good things for those that can manage to get their content into these personally defined Moments more effectively. But it also takes an acceptance that "the first draft of history" that many in the media business cherish as their mission is taking on a radically new form. Like the "playback" feature in Google Wave, everyone will have access to who did what where and when soon enough. The question is, who edited it the best? Google has staked its claim as the world's dominant editorial resource for displaying billions of histories a day, sweeping away front pages across the Web into a stream that assembles Moments that matter most to audiences.
We will spend time with content in any number of spaces thanks to this editorial resource, as we have on the Web for many years. But Google has accelerated the endgame radically in the past few months for those not tuned into The Moment. 2010 is going to be a year of momentous change in the content industry. Publishers that are tuned into The Moment will be in good shape to take on all of the inputs of The Sensor Society and to trigger astounding growth in cloud-based content markets. For those that aren't tuned in, well, you better get used to the idea that you're playing a two-dimensional game of chess against a 3-D chess master. Set up the chess pieces again, Spock. It's a whole new game.
A recent press release from Autonomy hailed an IDC report that gave them the leading market share for the search and discovery technology market. While congratulations are no doubt in order for Autonomy, which has thrived as other major competitors have struggled to gain momentum in general enterprise search markets, there's a wrinkle to this boast that should give one pause to wonder. Sue Feldman's indicating in the report that Autonomy has a 14.4 percent share of the search and discovery market in 2008, which is certainly nothing to downplay but also not a crushing dominance of this market. In other words, even the world's dominant enterprise-oriented search technology provider is little more than a niche player.
This is in part because there really isn't "a" search technology marketplace in any strict sense of the term. That may sound strange at first, but it's certainly true that search as a content location tool can only measure its success against very specific needs. Each enterprise, each publisher and media outlet, each marketplace has specific needs for content that determine whether a particular technology has been well tuned to its needs. We can use tech terms such as precision and recall to define in general terms how effective a search technology may be in returning useful information, but if a technology can't deliver editorial value very specific to an enterprise, it's just a general tool that is rapidly and easily commoditized rather than a powerful content tool.
The importance of catering to very tailored content delivery needs was underscored in my mind by a recent chat with Craig Carpenter, Vice President of Marketing for Recommind, a company providing content categorization and discovery tools that are finding particular success in legal and corporate compliance markets. Recommind has focused its capabilities on supporting functions such as e-discovery processes that enable an organization to understand what documents relate to a particular legal matter in the early phases of assessing a case. Going through emails, word processing and other unstructured enterprise documents rapidly to determine which ones relate to key figures in a legal matter or or compliance issue is a good stress test for any search technology. With recent U.S. government rules encouraging the use of electronic tools to accelerate content discovery, Recommind is one of a few companies that are well positioned to both accelerate compliance with those expectations and to eliminate legal expenses associated with the discovery process.
Certainly companies like Autonomy may be competitive in such situations, but when companies such as Recommind are focused more deeply on the needs of specific market sectors, they become, in effect, like subscription enterprise information services, delivering highly relevant content rapidly and reliably. There are, in truth, fairly few ways to attack search from a technology standpoint, so the most profitable victories in enterprise search and discovery technologies tend to go to the companies that have technology that is highly tuned to the very specific needs of a given market or client. That doesn't necessarily make one technology better than another in attacking those problems, but oftentimes only better tuned and one step ahead of other technology providers. So the fact that a company like Recommind is down in the depths of tuning their technologies to legal discovery and corporate compliance can offer them better margins for solving more focused, high-value enterprise problems - often the same kinds of problems that many enterprise publishers are trying to solve.
I do think that companies like Recommind that have done the heavy lifting on difficult enterprise search problems in specific sectors or problem sets can turn out to be double threats in enterprise content markets. Not only do they get to solve higher-value problems that are easier to measure for ROI, they also get to redefine market opportunities into other adjacent markets that may be difficult for others to attack. For example, when you look at the technology issues behind legal discovery, corporate compliance and more general high-value enterprise problems such as records management and knowledge management, there's a lot of overlap with a whole different range of technology services providers. On the other side of the spectrum, being able to categorize and organize content for the legal sector very effectively also begins to nibble at the opportunities for subscription enterprise services such as Thomson West and LexisNexis, which are also focusing more on semantic content organization but not necessarily with the deep technology focus of niche players such as Recommind.
Of course, the opposite forces of two-sided competition from large rivals can push back at niche-oriented technology players, but in general today's markets seem to be favoring specific solutions that make specific pains go away quickly in enterprises, with more general solutions with bigger tickets and fuzzier ROI being strung out on longer sales cycles. I don't think that we'll be seeing many new players like Recommind entering enterprise markets any time soon, but I do think that those that were able to get launched and cash-positive in the past few years are going to be tough competitors in the two-prong fight for content and technology dominance in the enterprise. Individually they may not take up anything like a 14 percent share of search and discovery markets, but when you look at their ability to respond to the best revenue opportunities within those markets, you can pretty much forget about the pie as a whole and start looking for the plums inside the pie that matter most.
If I had a dollar for every opportunity over the past few years to blog about the ins and outs of Yahoo's present and future, I could take you out for a pretty good dinner. The soap-operatic saga of how the leading but beleaguered Web portal lost many opportunities for greater industry dominance are well-chronicled, but now a completing deal for Yahoo to use Microsoft's new Bing search engine in exchange for Microsoft using Yahoo's ad network appears to set the stage for a new assessment of Yahoo's place in the online content industry that rises above the the usual cult of obsession with Silicon Valley personalities. More importantly, this deal is not the only step that Yahoo is taking to strengthen its position as an online destination that solves problems for people with engaging content.
On at least one level the deal appears to be a no-brainer. Yahoo's search capabilities are quite good for consumer search, but they lack Microsoft's investments in the engineering mojo of its Powerset-enhanced Bing search engine to accelerate the maturing of search results into rich, contextual content. Yahoo has good ad technology and brand marketing, but needs both more inventory and more overall market share to get a more serious share of advertisers' budgets. Each organization will be able to take capital out of competing for their common but smaller pieces of the online search and ad pies and concentrate more on drawing market share away from Google and other sites using Google services. In doing so they will be able to build online and mobile revenues more effectively through their combined audiences.
This is all good, and probably well-needed competition for Google to strengthen the online breed. It also puts Yahoo's efforts to re-engineer its future as a direct competitor to Google comfortably in the past: Yahoo's greatest growth came during its earlier technology partnership with Google, which allowed Yahoo to concentrate on user experiences and content partnerships more effectively. Different partners, now, but similar opportunities await. So in spite of the "Yahoo has thrown in the towel" rhetoric floating around - or worse - there's reason to believe that this alliance is a good step towards Yahoo using its more limited assets to do what most successful Web companies do anyway: use alliances to do what you do best and to leave the rest to others. Bing will kill the Yahoo brand no more than Google's search and ad alliance killed the AOL brand; there's plenty of room for Yahoo to be a strong aggregator and services provider through and around Bing's capabilities. It may also, of course, be a way for Microsoft to absorb the benefits of a Yahoo one step at a time while avoiding regulatory issues that an acquisition might raise, but given the iffy online future for both companies individually it's probable that a trial marriage through this deal that strengthens the assets of both companies is a more realistic step at this time than risking capital on a merger.
Yahoo is also not relying simply on Microsoft to reposition its strengths in the Web marketplace. In today's world of virtual aggregation, Yahoo's recent home page redesign beta, which includes links to major online Web sites such as Facebook and eBay is an indication that they have finally accepted that Yahoo's strength as a brand can't grow exclusively on traditional content licensing deals. If Yahoo is to be the "starting point" of using the Web, as suggested by Jerry Yang, Yahoo’s co-founder and former chief executive, then it has to do as the Web itself does and become more adept at using links as a form of powerful brand endorsement. A media cynic may look at this and say, "Well, it's nothing more than a big Huffington Post with some extra ecommerce features," but if it does what people want it to do and they come back for more, then, well, who's going to laugh last? A successful product is first and foremost about meeting the needs of your markets cost-effectively, after all.
There are still many hurdles for Yahoo to overcome before it can be labeled a truly "hot property" again, but the new Microsoft alliance and the home page redesign are both key indicators that Yahoo is focusing increasingly on the things that will keep people coming back for more. The days of walled gardens filled with licensed content built one deal at a time are a waning phenomenon, but that leaves many hopeful days ahead for those who help people make the most of their online experience in whatever garden suits them best. Hopefully Yahoo will remain a key player in those efforts through their latest moves.
There was some scuttlebutt buzzing around last week's duel between the Google I/O developer's conference and the All Things D conference to the effect that perhaps Google had some intelligence about the Ballmer announcement of the Bing-flavored preview of Microsoft's new Live Search search engine that prompted their announcement of their Wave messaging and collaboration technology. Somehow that doesn't ring true, given the breadth of the Google Wave announcement, which is a pretty encompassing technology initiative. By contrast, Ballmer didn't have anything nearly as broad to offer the ATD crowd, but at least he had something to put up against I/O to keep people buzzing about Microsoft, most of which was catch-up to counter announcement's at Google's earlier Searchology event.
If you can find any significant differences between Bing and the earlier Kumo-labeled version of Microsoft's Live Search preview, you have sharper eyes than I do. That's not necessarily a bad thing; there's a lot to be said for Microsoft's leveraging of their new Powerset technology that helps to dress up search engine results with related content and faceted navigation features. But in several forays into Bing searches, I cannot say that I am finding all that many melds of information that are truly impressive. Yes, it's nice to be able to to have comparison shopping data, reviews and related links embedded in searches such as "Samsung LCD TVs," but that's not so different than, say, a search on Google for "JFK to SFO" with the "related searches" option turned on that has comparison flight shopping tools in the search results. Bing is good, perhaps even state-of-the-art, but hardly a game-changer for the state of search in general.
What the maturing Bing search results do seem to indicate is that the lines between destination sites and search engines will continue to blur as content providers and search engines both go in search of more valuable and engaging contexts for high-quality content. For search engine providers, being able to increase engagement time on a given page of search results is good for ad revenues and overall user satisfaction and brand value. For online publishers, the melded results offered in Bing, Google's Universal Search and other evolving search portals represent opportunities to engage audiences at the point of demand with solutions that enhance their own brand value while building revenues from advertising alliances with search engine portals. You might say, even, that the Bing/Google Universal Search approach is like dialing up a custom magazine/shopping guide/newspaper, with increasingly slick and well-organized content that begins to mimic the editorial capabilities of traditional specialty publications.
The parallel between traditional media and on-demand publications assembled by search engines is underscored in Bing by the rich and engaging photographs that appear on the home page of the Bing site. Squint a little bit and you can imagine the cover of a National Geographic magazine or other glossy high-quality publications. The visual promise of Bing's home page is that what you're about to experience is really, really good at a visceral level. The guts of this "magazine" don't yet match the cover, but you can tell that over time both Bing and other search engines are headed in the direction of getting search results to be as engaging and visually rewarding as traditional magazine publications, albeit with lots of the Web-savvy functionality that keeps people coming back.
With these evolutions in mind, publishers need to be prepared to make their content brands resonate in the online pages of whatever on-demand context appeals to their audiences - including increasingly sophisticated search engines that are aiming to keep people hanging around their pages as long as possible. Initiatives such as Journalism Online will help to make search engines more profitable aggregation venues for traditional publishers, but they need to be ready to accept more willingly the idea that search engines can be great publishing partners that help them to get their content to their audiences in the contexts that they value most. Certainly Bing will help to convince some publishers of this, but it's still early days for publishers recognizing that The New Aggregation is not a mere thought piece but instead a key component in the future of profitable publishing.
While the concept of the content organization features found in the Powerset search application was always compelling, the original content in the demo application set up for the early version of Powerset was not the most powerful presentation of its strengths. Now in the hands of its acquirer Microsoft, the Powerset features appear to be ready to take on a much-improved content set and interface in the guise of an internal project at Microsoft labeled "Kumo." As revealed by Kara Swisher at All Things Digital, an internal Microsoft memo is encouraging staff to play with the prototype search engine to get some initial feedback.
In spite of some scathing negative reviews from the search engine intelligentia, the screen grabs provided by ATD of the Kumo interface look to be pretty competent. Gone is the over-busy Powerset interface, replaced by and interface that is at once Google-esque and yet unique. The top five web results are followed by results that match different facets of a search term. For example, results for the recording artist Taylor Swift return groupings of content available for her songs, her lyrics, her bio and her music downloads and her albums. On the left are possible searches by related artists and categories, as well as the ability to initiate new searches in video collections, bios and so on.
It's unclear at this point whether Kumo will be just a project name - it's apparently a word that means both "cloud" and "spider" in Japanese - or whether it's just an internal marker that may disappear at its features get absorbed into Microsoft's Live Search engine. For that matter, it's unclear that the features will make their way into production at all, though they are certainly useful enough. What is clear, though, is that Microsoft is going to continue to search for new ways to make alternatives to Google palatable in a way that might appeal to both enterprise and media audiences. I don't think that too many people harbor illusions about the ability to crack Google's dominant market share in search any time soon, but competition is good for the breed, they say.
I suppose the most intriguing aspect of Google's success that challenges the challengers such as Kumo is how Google has attained its success without explicit content categorization features. One can go to dozens of knowledge management and search conferences every year and hear about how important good content categorization features are for the success of search engines - and then look at the nearly naked search results on Google to contemplate just how true that may be. The assumption that categorization specialists have is that having categories makes it easier to browse content collections. Well, that may very well be true if you are in fact interested in browsing relatively finite and well-organized collections of content, but in general search engines have become less about browsing and more about delivering specific answers for most people. The average searcher seems to be trained now to refine their own searches via the "white box" rather than to traverse through browsing categories.
This isn't to say that content categorization isn't useful: it's more a matter of where it turns out to be most useful. Where it does seem to help most is in portal solutions where someone has come to a specific page of content and may want to explore that site or database from different facets. Where people understand that there's a finite, well-curated collection at their disposal, categorization seems to do quite well. Where it's a matter of sifting through billions of pages for the needle in the haystack, most folks are getting used to typing in the best search string that they can think of. With that said, the features in Kumo do provide an interesting and engaging alternative to Google search results, but they'd probably be better off either in specific content portals that need enrichment or in creating an on-demand portal from its results sets, so that it will be a more browsable set of content in its own right - and then, perhaps, attract a higher breed of advertising, if that's the goal. Instead of trying to out-Google Google, perhaps challengers such as Kumo need to think about how to out-aggregate the aggregators to build better revenue margins for smaller search operations. Something to wrestle with, perhaps.
The shameless self-promotion division of Shore is proud to announce that I'll be amongst the speakers at next week's SIIA Brown Bag Lunch panel presentation on Wednesday, 11 June focusing on how to attract, monetize and retain audiences and clients through search technologies. The panel will be moderated by Leslie Kues, Senior Director at Microsoft's FAST with my distinguished co-panelists Kate Noerr, Founder, Chairman & CEO of MuseGlobal, Stephen Baker, Chief Revenue Officer for EveryZing and Barbara Kroll, Director, Corporate Strategy for Wolters Kluwer. It promises to be a great panel, including both publishers using search in enterprise and media markets as well as two leading technology companies helping publishers and enterprises to get more value from search as a publishing platform. Registration information is here, it's going to be available as a live event at the McGraw-Hill Building in New York as well as an online video event.
As for myself, I will be emphasizing how search is a publishing tool that is not just about the "white box" and a list of results but a technology that can enable content to be aggregated in a "just in time" publishing environment to support a wide variety of content applications for media and enterprise markets. If you're planning to come you may want to catch my earlier entry "Beyond Search Engines: The Database is Now" to get a feel as to how search engines are starting to replace databases as the primary content gathering mechanism for content applications and its implications for publishing. Long story short, the way that financial markets thought about stock tickers and trading room system middleware is how more advanced publishers are beginning to think about search engines.
Hope to see you at the brown bag - no food but plenty of beverages and great cookies - trust me.
There are rocket scientists, then there are rocket scientists - and then there's Barney Pell, long-time Silicon Valley startup maven and currently the Founder and Chief Technology Officer at Powerset. Barney is one of those rare people who has been a rocket scientist via both the NASA side of the term and the software industry side, an outlook that has helped him to assemble many teams through the years that have developed advanced search and language processing technologies. Powerset has unveiled its first effort recently at a new technology to provide rich content from semantic searches, an interesting look at how one can completely reshape the face of a content product via enhanced search technologies.
Using Wikidpedia as its primary target content, Powerset technology analyzes search phrases to come up with search results that match natural language phrases as well as keywords. This being a very early stage debut of technology some search targets work better than others and overall I'd have to say that it's a technology that seems to do best with people and things as opposed to concepts. For example, if you type in "Who is Bill Gates?" you get the screen similar to the top of the above screen grab, which includes a top deck of biographical information from the Freebase reference database followed by Powerset's sets of semantic analysis called "Factz" that focus on what the Wikipedia article says about this prominent figure. One of these sets, for example, tells us that Gates gave testimony, a speech, an address, a demo, a presentation and a deposition. You can click on any of these terms to get more details from the underlying article.
Below the initial bio and Factz information is a set of search results for the initial query, including the best-match article on Microsoft founder Bill Gates. This is in essence the straight Wikipedia article with links mapped over to Powerset's version of this content, along with a handy visual presentation of the article's outline on the right or another listing of key Factz organized within the article outline. I like some of the inferences that it's come up with in the Wikipedia definition of Content that I contributed a while back: "information provides value; experiences provide value; content provides value." True enough.
I like how Powerset prefixes organic search results with federated content, taking a best stab at results on very focused topics that enable people to obtain knowledge more quickly and effectively. The automatically generated Factz, though, suffer from the same problem that most semantic tools experience when they examine a very small data set: spotty inferences. For example, in the Factz about Bill Gates Powerset inferred that he founded Cher, an inference drawn from the fact that biographer Howard Johns was known for revealing the addresses of these and other celebrities. Hmm. Don't think that I'd put that info down on my "final Jepoardy" slate. I am also not so crazy about the organic search results, which tend to err on the side of word proximity. Again, with a relatively narrow data set such as Wikipedia it's not always easy to tune content analysis well to the capabilities of semantic text analysis in search engines.
The big picture for this early-days release of Powerset is that it is a great demonstration of how one particular source of content can be transformed through search and content federation technologies into an altogether different kind of publication. Oftentimes I talk these days about search technologies being similar to datafeed technologies, but in this instance it's important to recognize that search technologies are also end-publishing technologies in and of themselves that can aggregate, filter and organize content in altogether new ways that enhance the value of one or more core publications. Using free content from Wikipedia and Freebase the Powerset technology does a good job of demonstrating this concept simply, albeit with some early growing pains. Publishers wanting to stay in the forefront of content markets are turning in droves to content federation technologies as a solution to add value to existing product sets, so expect to hear more from technologies such as Powerset that help publishers to add value rapidly.
The announcement of Adhere Solution's partnership with MuseGlobal to launch the "All Access Connector," a federated content integration solution for the Google Search Appliance, is one of those situations where an event is both obvious and profound in its potential impact on the marketplace. As enterprises today face an explosion of internal and external content sources that they need to integrate to create insightful content services there is a huge gap that has arisen between what most content platforms can do to unify that information and what enterprises really need. This is particularly true in enterprise search, where many search services fail to provide access to all of the sources that a person typically needs to access.
Federated search solutions have been one route to address this problem, querying interfaces to multiple searchable sources and assembling the results "on the fly" to yield a combined search result. Instead of trying to shoehorn all of the needed information into a single database or search index federated search enables content to live wherever it has to and to come together when needed via multiple queries into integrated search results. Some do this better than others, and some have been at it for longer than others. MuseGlobal falls into both camps pretty handily, having been providing federated content solutions for more than a decade which has allowed them to hammer out an infrastructure that will pull together thousands of different types of content sources together via federated queries.
All well and good, but the question is, how do you make this sing in the eyes of enterprise users? MuseGlobal's support of Adhere Solutions, a company that includes Googlephile Steven Arnold's son Erik Arnold as a Director, points towards a very powerful possible answer to that question: the Google Search Appliance. While the GSA is a popular search tool in many major enterprises it's not been deemed the "go-to" search interface when it somes to getting all the right content from the right places all in one place in many instances. Federated content capabilities from MuseGlobal united with the GSA seem to fill that gap very handily. Capable of searching any number of search engines, internal and subscription databases and feeds as well as harvesting content via its own site crawlers, the MuseGlobal platform turns GSA into a clearing house for all of the content sources than an enterprise user might want - all delivered on the highly popular Google interface that provides access to Web content as well.
Combine this with both Google's programming interfaces for applications development and MuseGlobal's own extensive library of content integration tools and all of a sudden the GSA looks like a lot more beefy competitor for expanded use within the enterprise. And since the MuseGlobal library of source connectors includes many interfaces to subscription content services as well it's a platform that can put subscription database providers on a new footing with their users as well. All of a suddent the GSA looks less like a user-friendly also-ran and a lot more like a growing hub for enterprise and online content resources.
We hear lots of talk about workflow as the key solution that's going to enable value-add enterprise content services to build new revenues, but the ability to pull together a comprehensive set of sources that their customers' users really need to do the job is a slow and laborious process oftentimes for many subscription database providers to accomplish. At the same time enterprise portal providers are stymied oftentimes by users who refuse to use their solutions to any great degree because they're used to getting the answers they want from the search engines they rely upon as ther real "go-to" workflow solutions. The All Access Connector solution offered by Access Solutions and MuseGlobal offer both camps a lot to think about as they ponder how best to ensure that they are delivering the content that their users want in the applications that drive their productivity the most. The era of The New Aggregation's ability to deliver more content value from more content sources more rapidly than ever is upon us in full, indeed.
I really love Rafael Sidi's Really Simple Sidi weblog, it's a great compilation of insights into sciences publishing that is easy to read and is in my daily bookmarks of news sources to monitor. Turns out that Rafel is a big fan of ContentBlogger also, so I was pleased to get a preview briefing from him on Elsevier's new Illumin8 product making its debut today. While it's hard to draw major conclusions on the significance of any product Day One, it appears that Elsevier has enabled Rafael's team to come up with what promises to be a real breakthrough in STM workflow solutions focused on getting the right insights into emerging solutions to scientific problems effectively.
The problem in big-stakes scientific research and development fields is that most search tools are oriented towards topical approaches to research that don't necessarily focus on relating problems and the organizations and people focusing on them with the solutions and benefits that they provide. For example, if one were to look for research, news and Web content relating to the HIV virus, the typical search engine is going to look at a search centered on that term and come up with documents that relate to this topic - but not necessarily focus on the solutions and benefits being provided by specific research studies for available new products.
This is a critical factor when trying to select a new line of scientific research or to understand how to position a new product based on that research. How quickly can one define what solutions are in play for specific types of scientific problems by specific companies or universities? Who's delivering the most beneficial solutions? Illumin8 addresses these kinds of questions by adding an important semantic twist to search processing. Instead of focusing just on nouns to define how content relates to a topic Illumin8 clusters results based on how they fall into verb categories that align topic groups such as organizations, products, experts and technology with problems and benefits associated with those topics. Using this tool one can discover easily not just recent research, Web postings and news stories but the items that the real problems being addressed by that research and the real benefits being revealed very rapidly.
Illumin8 has a very simple search interface thus far, a "white box" approach that will move from topics to problems and benefits mapping automaticaly or the ability to define more sophisticated queries using special keywords. You can choose from news, research and Web content or any combination of these via a checkbox interface and adjust your precision/recall balance for getting lots of results or just of few of the best matches with a slider bar. Search results come with graph bars and totals to make it easier to see which keywords and clusters of topics, problems and solutions are coming up most frequently in results.
While lacking some of the interface sophistication of a more mature product like Collexis that focuses deeply on helping people navigate expert network relationships and still needing to address some entity mapping issues the fundamental power of Illumin8 is quite evident even in its early introduced form. More sophisticated analysis of verbs as valuable tools in semantic processing is in part behind the proliferation of "sales triggers" intelligence products such as Generate and InsideView, which enable sales professionals to understand when news and other content sources are pointing towards companies involved in activities that impact their sales processes. Applying this type of processing to scientific studies and product development is likely to help scientific, medical and technical companies and organizations to get a similar leg up on understanding who's moving towards revenue-impacting insights more quickly.
It's an approach that can probably yield tangible benefits for many types of business information as well as consumer information. It would be nice, for example, to see a semantic engine such as Illumin8's applied to product and catalog sites. To some degree many existing search engines factor these kinds of semantic issues into their processing behind the scenes, but Illumin8 demontrates that when one focuses on the problem-solution relationship from a product standpoint instead of a straight topic approach the benefits can be dramatic.
I am skeptical oftentimes when new products claim to be "workflow solutions," but Illumin8 seems to be pointing towards a pain point that people in R&D departments encounter often enough without real effective solutions being offered elsewhere that it probably qualifies as such a tool. It's another way of saying that there just might be some significant ROI in there if someone can do the research to tease it out from an early adopter community. Hats off to Rafael for a nifty product launch - helps to have that blog - and to the folks as Elsevier for giving Rafael a chance to strut his stuff. Hopefully Illumin8 continues to grow in scope, substance and quality.
Steven Arnold writes a thoughtful post on his Beyond Search blog about the inadequacy of traditional databases and search engines to deal with organizing and delivering content when the Web and many private content collections measure in petabytes and exabytes of information. Steve hints at a "next generation" database management system that can start to leapfrog over these problems, but the greater question is perhaps unasked in his article. Namely, as the problems that people need to solve with content technologies become increasingly complex and increasingly fleeting, why is it that we really need permanent unified databases to solve those problems? There is an important need for data normalization, but if normalization can be achieved "on the fly," as leading content federation services can provide, do people need a database or instead data objects that solve specific problems in the moment?
When data normalization was associated with creating massive databases that would be used for repeated functions such as payroll management or publishing functions such as newspapers or directories permanently structured databases made a lot of sense. But as market advantages gained through content publishing fall increasingly to those who can mine unstructured content, aggregate content from disparate sources and enable people normally confined to consuming content to create it and organize it, the traditional database is being relegated to one of many silos from which advanced content services can develop on-demand content solutions. Search engines, which rely on databases that can be queried in a standard format to provide standard answers, are beginning to fall into this same role of specialized answer tools. If you look at the typical search results page today from major providers you're looking at federated content from multiple sources, logically related to a greater whole but residing in separate storage environments and coming together in the moment as the answer to a specific question or need.
In short, what we have called a database is no longer a storage and indexing device. Rather, the database is now, the content sets that we assemble in a given moment to solve the moment's problem. Its structure is consistent thanks to XML standards, data dictionaries and data mining normalization tools, it can be stored as needed for time series analysis or corporate compliance, it can be shared with others to develop collaboration services or new forms of content and analysis. But in the next moment our needs may shift, sources may change structure or become unavailable or be replaced by different sources.
Market advantages tend to flow from institutions who can take advantage of content most effectively, and in the markets we can see how this concept already impacts business in a large way. In financial markets profits are shifting from public securities exchanges, whose transactions are built around highly normalized databases and data formats, to private transactions on highly complex financial instruments, whose underlying complex calculations on financial risk and return may apply to only a single transaction at a time. There is structure in such transactions, yes, and lots of normalized data, but the uniqueness of the content's structure at the moment that a deal is executed is far more important than its standard components.
Search engine providers such as Google understand this paradox explicitly and work hard to provide value-add interfaces that enable people to use search engine content as one of many feeds that can power "mashup" consumer and enterprise content applications. The Google search engine may be one of the world's largest databases but if other content in a form that's more usable in a specific context can come along and complement it in the moment, it becomes rather moot beyond a certain point whether or not it's in Google's index or another index. This federated approach to content value becomes at least as important as the quality of the individual sources. In a "the database is now" world, quality is as quality does - and it may mean something else a moment from now.
The implications of this concept for content publishers is enormous. Long used to building their standardized databases, the long-promised New Aggregation is on the verge of becoming the value leader for both enterprise and media publishers. Through the on-demand federation of content sources into aggregated content solutions the uniqueness of insights for small audiences is becoming a much more important method for creating value in aggregation than the pervasiveness of standardized insights.
Make no mistake, we'll be using today's search engines and databases for a long time as building blocks for federated content services, but we'll be less fixated on owning databases and more focused on owning the contexts in which they provide solutions. This is likely to change the pricing structure of content aggregation services significantly and to force traditional publishers into becoming on-the-fly aggregation services pulling in content agnostically from many sources that may not be under their direct control for more than a few moments. Subscription databases will yield, sometimes gradually and sometimes very rapidly, to subscription contexts, services that can assemble content from anywhere consistently and reliably for workflow and lifestyle applications. Yesterday's email inbox is becoming today's content inbox via feeds and social media: tomorrow's federated inboxes will be even more rich and complex through databases that live in the moment.
Social media and enterprise content federation services have already pressed many of these changes forward, but expect 2008 to be the year in which more than one company will begin to recognize the value of databases in the moment. The database is now - and so is the opportunity for publishers and enterprises to move beyond isolated content solutions.
Sometimes two distressful situations can combine to create relief, rare though that might be. Such seems to be the lucky break that both Microsoft and FAST Search and Transfer caught in the recent acquisition of FAST by Microsoft. FAST needed fast relief from crippling cash flow problems generated in part from a sales strategy that reached beyond their ability to deliver on ambitious promises. Microsoft on the other hand had failed to create any significant sales momentum behind its own enterprise search efforts, with players such as Google beginning to breathe down their necks more warmly with each passing day. So a mere USD 1.2 billion in cash works quite nicely to bring together two impressive partners that promise to dominate enterprise platforms for some time to come.
FAST's rapid growth over the past few years into an increasingly dominant position in enterprise search markets is just the ticket that Microsoft needs to position itself in increasingly competitive enterprise platform markets. With ever more content being consumed in enterprises via non-Microsoft platforms, domination requires a more agnostic approach to assembling on-demand content than Microsoft has been able to manage recently. FAST offers both solid enterprise search technology and an installed base of global corporate clients that Microsoft can leverage very effectively with the combination of FAST search capabilities to gather content and Microsoft's Sharepoint servers to store and aggregate content.
This last point is especially important for Microsoft's future revenues. With its Vista operating system rendered a ho-hum at best by most enterprise users and panned widely in consumer markets Microsoft needs to shift the center of its profits to platforms sy uch as search engines that are more central to what drives internal publishing in today's enterprises. Each page of search results can become in effect a purpose-built portal: in effect, the database is now, the content that's required to solve immediate business problems. Search technology such as that offered by FAST holds out the promise of search engines becoming the focal point for Microsoft's enterprise publishing strategy, offering Microsoft more opportunity to have offerings that scale effectively to both global and mid-sized corporations. That $1.2 billlion make look like relative pocket change today, but in terms of the market share secured and the future market positioning that will be required to counter slowing sales on its aging operating systems it's a major investment in securing Microsoft's future cash flow.
Attendees voiced the value of the range of tracks from strategic management of knowledge to the practical aspects of selecting and living with search software and applications, down to the nitty-gritty of taxonomy implementations. Traffic was good in the vendor booths of the Expo area, as technologists and content managers mingled over receptions, meals and seminars.
The opening keynoter for ESS was Susan Feldman, Research Vice President, Content Technologies, IDC. describing a market in flux with many competing technologies. Search is the missing piece for enterprise software, and large software vendors are entering the market. SaaS options are good solutions due to the complexity of search technology, and need to have the latest version.
The keynote was a nice lead into the session that I chaired on "Solving the Multiple Search Engine Problem" addressing approaches to the proliferation of departmental search vendors within organizations. Rennie Walker, Wells Fargo, described "waking up one morning with the multi-search engine blues", resulting in creating a Search Center of Excellence (COE). Swetswise uses a federating search software, Museglobal, to deliver a subscription delivery product incorporating multiple search indexes. Miles Kehoe, New Idea Engineering, identified the challenges of maintaining distributed search engine indexes--a practicality not addressed by vendors.
Security, ediscovery and regulatory compliance were themes in other presentations. Search across multiple repositories brings the thorny problems of access control to the underlying content. Depending on the application, different levels of security may be necessary, down to the sub-document level. Choices include "early binding" vs. "late binding" options for access. Additional challenges include the changes in Federal Rules of Civil Procedure of 12/1/2006, making risk management of the enterprise search environment more critical.
Steve Arnold, highly regarded industry expert on search engines chaired a keynote panel originally entitled "Giants Do Stumble: Are Google and Microsoft in Decline?" modified in the final program to "What's Next for the Search Engine Giants", questioning product managers from Google and Microsoft, who provided little new insight. Both companies are relative newcomers to the enterprise search space, and had vendor booths in the expo, joining traditional vendors. Arnold, in a later session, honed in on Google and his analysis of their patents to predict new directions.
Findability is more than keyword search in full text documents, a message which came through in both the sessions and vendor presentations. Sessions on semantic search indicate progress in actual implementation, which is closely tied to classification and taxonomy systems. Improved navigation, particularly faceted search, are another approach to improve the user experience, and improve findability.
Niche software vendors on the exhibit floor, demonstrated other approaches to improving findability. Siderean uses a relationship approach which intuitively fits research and discovery processes, to improve findability. Cognition was demonstrating their linguistic search software with great promise for in depth research, particularly in scientific and technical literature, with a plethora of potential search terms. Deep Web Technologies showed the power of federating search software, as implemented at science.gov and scitopia.org.
Enterprise search and management of organizational intellectual capital have become mission-critical. The challenge is finding the right approaches for the organization, then the technical tools for implementation. Increasingly, behavioral and linguistic aspects are being recognized as essential factors in the process of adding value to the organization. Search is not easy, and delivering answers to people is not straightforward. It's finding the right combination of solutions that challenges the attendees at these conferences..there is no one-size-fits-all!
I have enjoyed using the Compete.com traffic analysis service, which provides some useful data to compare Web site traffic performance more accurately and finely than the oft-bashed Alexa statistics. While Compete offers a more limited range of sites for analysis and only a year's worth of data to mull through it's able to track real visitors, audience engagement and growth with more meaningful data. On the Compete blog recently was a post that looked at how major search engines are performing in comparison to one another for both traffic and performance. While Google leads Yahoo and Microsoft with 67 percent of market share, the Compete stats claim that Yahoo comes out on top in terms of search fulfillment - the percentage of searches that actually result in someone clicking on a link in a search results page. Compete claims that Yahoo's search fulfillment rate is 75 percent, compared with Google's 64 percent and Microsoft's 61 percent.
Does this mean that Yahoo's search results are more "clickable" than Yahoo's? Maybe so, but it's a rather ambiguous claim to make. One has to assume that with only 20 percent of people using Yahoo for searching to start with that a minority find its search results to be more useful than Google's. So for that minority they seem to use them more effectively. Overall, Yahoo searches are more optimized for people in a purchasing mode than Google search results, which tend to be optimized more for people seeing general information. With this in mind it could be that Yahoo tends to lead shoppers somewhat more specifically to product information that they're seeking - a factor that's likely to attract the brand advertisers that are at the core of Yahoo's marketing strategy.
Yahoo search benefits from doing fewer things better for fewer people, but Compete also shows that Yahoo as a whole performs far better than Google in the total attention that it gets from audiences:
While Yahoo's strong destination content helps to bolster its attention ratings it's losing ground to Microsoft in total page views as Microsoft bolsters its Live.com search engine:
In the middle of this is Google, still the overall search leader but beginning to stagnate as a destination as other search-oriented sites bolster content that transforms search portals more into destination content sites. Google has these abilities also but focuses more on solving a broader array of requirements for a broader search audience. Google also has more partners using its search technology as well as mashups and other API-based services so to some degree the Compete statistics are not revealing the full strength of Google's market presence. Google's growth as a destination search engine may have slowed, but its presence as a technology platform that influences where and how people find content in valuable contexts is growing in highly profitable directions.
All of this should serve to remind us that there is no longer one clear answer to how to create marketable value through search. You can focus on becoming more portal-like, you can focus on being more embeddable, you can focus more on a specific function such as ecommerce or you can focus on a range of functions - but regardless of the focus it's no longer a matter of just having great ranking algorithms or great server farms. Search has become just one of many tools for contextualizing Web content effectively on demand, one that will continue to grow in importance but just one tool in an arsenal of methods to be used for more effective audience engagement.
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=537a6dc9-bddc-478a-aa73-e224af317ae1)
