The company unveiled the new service at the O'Reilly Open Source Developers Conference. It's called Project Hosting, and like all Google betas, it's free to use, though you do need to have a Google account. The service is supposed to give open source software developers a web-based ability to track bugs and other issues with their software, collaborate, and otherwise handle the many details involved in working on and coordinating an open source project.

The search engine wars are fought with strategies, alliances, and robots. As Yahoo! primes itself to be the number one contender for market share after Google, websites that want to optimize for Yahoo must study how Yahoo ranks pages and how it indexes pages. The Yahoo web crawler SLURP should be studied; your site server logs should have recorded visits from various robots, including SLURP. If you do not have records of SLURP visiting your site, then this article will give tips on how to get SLURP to crawl (hopefully deep crawl) your site.

The Preamble

Yahoo SLURP evolved from Inktomi SLURP. The Yahoo SLURP robot is an upgrade from Inktomi’s SLURP. Yahoo used Inktomi’s search engine to replace Google, which used to take care of its search results. This officially triggered the second search engine wars (the first was won by Google without it declaring hostilities).

Yahoo has at least 130 million registered users on its network. Granted, Google is the definitive search engine, but Yahoo is large enough that it should not be ignored.

Behavioral marketing has actually been around for a few years. In its crudest form, pre-Internet, it may have consisted of special mailings to repeat customers. Today, behavioral marketing involves serving up ads to a particular individual based on his or her previous online behavior. It is not to be confused with contextual advertising, which serves up ads that are related to the content of the web page on which they're appearing.

This means that two people seeing the same web page could see completely different ads. For example, let's take two surfers, one an outdoorsman who likes to visit hiking-related websites, another a big theater buff. They're both planning a visit to South Florida. Maybe they both end up on the same site with general information about the area, but the outdoorsman sees ads for the local parks or hiking groups, while the theater enthusiast sees ads for tickets to upcoming theater performances.

You can probably see why many online advertisers are very excited about behavioral marketing's potential. It promises the ability to reach an even tighter audience with more relevant ads than contextual marketing can achieve. In theory, this means advertisers can show (and pay for) fewer impressions of their ads, while enjoying a higher click-through rate - and more importantly, a higher conversion rate.

With this kind of potential, you would expect to see a lot of advertisers eager to try out this new form of online marketing. The truth is a little more mixed. According to figures quoted by Search Engine Watch, only eight percent of all online advertising is behaviorally targeted. That number seems a little low, but there are reasons for this, as I'll go into shortly. Some of them will become obvious when I explain more about how behavioral targeting actually works.

And if you do have hits, how long will your traffic grow? How far can you expand? How big can you become? What is the difference between a small, profitable operation like www.internetmarketingsecrets.com, or www.freewebmasterhelp.com, and a large household name like www.devshed.com. I mean, they all offer great content, they all optimize their sites for the SERPs (Search Engine Ranking Pages) but only one of them is a household name, relatively speaking, on the Internet.

The Bad News

In the real (as opposed to virtual) world of traditional marketing, it is the biggest brands that have the largest market share. In fact, market share is not determined by quality of service or excellence of product. It is a question of branding. Does Daimler Chrysler make the best cars? Or does Nokia really make the best phones? If they do, will they still make such great products tomorrow? I answer, who cares? I have my mind made up. Don't confuse me with the facts!

On the Internet, Google has branded itself as the search engine. The terms "google" and "googling" even made it into the Merriam-Webster dictionary recently to signify using the Google search engine to search online. Despite the distinction of using a lower case letter and the fact that it specifically refers to just the Google search engine, Google risks its name becoming like Levis, which I found in the Oxford Advanced Learners dictionary defined as "jeans." Amazon has branded itself for books, and eBay for auctions; devshed has branded itself through its network of sites for content -- specifically, for information about open source software.

A brand will beat a better product or service ALL the time, not once, not nine out of ten times, but all the time. In traditional marketing, if the customer wants to choose any other product apart from the brand, he will easily find an alternative (a second brand that competes in that space, like Coke and Pepsi). In the online world, however, there usually is no second. Second in a category can spell death (read as "dependence on SERPs").

As a historian by training, I try to look at things in chronological order. So I’ll start with what happened first chronologically, even though it wasn’t the first point to come to light. In July 2004, a young software engineer named David Braginsky wrote in his blog that he “became a techlead of project Platypus at Google,” among many other things happening in his life. Sadly, his next update wasn't for another year, and only mentioned that he “really wants to focus on work.” There are no entries newer than August 2005 and no hints as to what project Platypus actually is.

In September 2005, Garett Rogers posted some speculation in his blog about a new project of Google’s called GDrive. After revealing that Google owns the gbrowser.com domain, he suggests that it could stand for “file browser.” Then he figures that “if google was smart, they would provide some sort of online storage medium that can be accessed from anywhere… a similar 3rd party application had been developed called ‘GDrive’ which utilized GMail as it's storage. This application has suspiciously been discontinued.” After doing a little digging, Rogers discovered that the gdrive.com domain is owned by the same parties who own the gbrowser.com domain, which leads back to Google. He saw that as convincing evidence that Google is working on a GDrive.

In December 2005, Google bought Writely.com, an online word processor that lets you store your documents securely online. It was by no means the first service that Google made available to its users that allowed them to store content online, nor would it be the last; Gmail, Google Calendar, Google Spreadsheet, Blogger, Google Base, and Picasa, just to name a few, allow users to store and share different kinds of content in a variety of ways. But there’s nothing to tie it all together into one drive, or at least not yet.

Search engines are continuously attempting to refine and streamline their results to fit the user’s search parameters.  Semantic search has made some advances, taking the context of a web page and delivering ads to target that particular audience, but it's a long way from being perfect.  We are seeing trends in speech recognition software, and it keeps improving.

The next logical product of Internet search is voice activated search.  What’s amazing is that it is already being done on a far broader scale than you might imagine. The next logical step for search engine results is to not have to speak at all, but rather to “know” just what the user wants. The only way to do this is either to develop a psychic machine, or to plug directly into the human brain and read our thoughts.  You might be surprised to find out that experiments have already begun.

Accessibility options for those with hearing impairments, disabilities, and poor vision have existed for a little while now, and these have already evolved well into the world of semantics from speech recognition, of which one group of these accessibility options is referred to as alternative input devices.

These can include systems that operate by inhaling and exhaling (called sip and puff systems), touch screens, and electronic input devices that can be worn on the head and operated by eye movement, ultrasound, infrared and other light systems, or even sheer brain waves. Brain waves are the very weak electrical signals given off by the firing of millions of nerve cells in the brain, which can even be measured outside of the human head.

You might remember that Google agreed to a $90 million settlement last March to clear up litigation in a class action lawsuit led by Arkansas-based Lane’s Gifts and Collectibles. Yahoo’s settlement, at least initially, isn’t nearly as large; the venerable search engine will pay $4.95 million in cash to plaintiff’s counsel. In Google’s case, the plaintiff’s counsel received $30 million, while credits to advertisers for the class action were capped at $60 million. Yahoo’s settlement has no such cap on advertiser claims.

Yes, you read that correctly. Yahoo has said that it will not put a cap on the refunds if it finds evidence of click fraud. The company is even quite optimistic that its claims payments will not be overly onerous, even though it made billions of dollars in revenue from pay-per-click advertising during the period covered by the class action.

Does Yahoo have a good reason for this optimism, or is it just bravado? Well, it’s worth remembering that Yahoo Search Marketing came out of Yahoo’s purchase of Overture, the company that created the pay-per-click search advertising model back in 1998 –- so by definition, Yahoo has more years of experience with this particular business, and its pitfalls, than anyone. It notes on its web site that “one of the very first challenges we identified was the potential for people with bad intentions to click on listings with the sole purpose of generating a charge to the advertiser. To address this challenge, we quickly established a system of proprietary technologies and people dedicated to protecting our advertisers from click fraud and other traffic quality related issues.”

Indeed, during the litigation process between Yahoo and Checkmate Strategic Group, Yahoo invited the plaintiff’s counsel and their experts to take a look at the search engine’s proprietary click through protection system in action. Yahoo even gave the plaintiff’s counsel access to team members and filtering data. After this review, Yahoo proudly states that “the Plaintiff’s Counsel and their experts determined that Yahoo! has in the past and continues to operate a click protection system that goes above and beyond what is necessary to address recent industry estimates about click fraud.” With some estimates of click fraud quoted as high as 20 percent, that is no mean feat.

The Verdict

One of the most maligned search engines is the British based http://www.ask.com/; it have been accused of sins ranging from not taking paid search listings (pay per click) seriously, to making search uncomfortable for browsers. Also, in contradiction to some of this, it has been accused of having too many paid listings on top of its site's listings.

Why would I now ask if a search engine that took literally forever to get into the paid listing market, and whose branding is confused in the mind of the audience (is it askjeeves.com, ask.com, teoma.com or askgive.com), will be the next big search engine? I mean, in terms of market share, Ask is at best fourth, behind Google, Yahoo and MSN. It is losing its top staff to the competition (Steve Berkowitz to Microsoft, for example). Ask is, to put it simply, not that important right now. But it has several things going for it that could turn it into a top contender for a large market share of the search engine industry.