What Is A Specialized Search Engine

What is a search engine, anyway?

If you think a search engine exists as an index to the internet, it's time to update your thinking.

Mike Elgan By

Contributing Columnist, Computerworld |

This column is not about politics. It makes no political judgments and takes no political positions. No, really! Stay with me here.

President Trump this week slammed Google, claiming that the company "rigged" Google News Search results to favor stories and news organizations critical of the president.

To drive home his claim about bias, Trump posted a video on Twitter this week with the hashtag #StopTheBias (which, at the time I wrote this, had 4.36 million views), claiming that Google promoted President Barack Obama's State of the Union addresses, but stopped the practice when Trump took office.

In a statement issued to the press, a Google spokesperson said that the company did not promote on its homepage either Obama's or Trump's first "State of the Union" addresses because technically they are considered mere "addresses to a joint session" of Congress, the idea being that brand-new presidents are not in a position to reveal the "state of the nation." Google also claimed that it did promote Trump's second and most recent State of the Union, a claim that screenshots found on social media and pages captured by the site Wayback Machine appear to confirm.

The facts around this incident are being funneled into ongoing, rancorous online political debates, which, in my opinion, isn't particularly interesting.

What is interesting is the Big Question this conflict brings to the surface.

What is a search engine?

A search engine can be four things.

An index to the internet

When Google first launched its search engine in 1996, it was clear what a search engine was: an index of the internet.

Google's killer innovation was its ability to rank pages in a way that was supposed to reflect the relative relevance or importance of each result.

Both the results and the ranking were supposed to be a reflection or a snapshot of the internet itself, not an index to the information out there in the real world.

An arbiter of what's true

In this view, Google Search would favor information that's objectively true and de-emphasize links to content that's objectively untrue.

An objective source of information

The objective source idea is that Google makes an attempt to present all sides of contentious issues and all sources of information, without favoring any ideas or sources.

A customized, personalized source of information

The personalized source concept says that a search engine gives each user a different set of results based on what that user wants regardless of what's true, what's happening on the internet or any other factor.

This is all pretty abstract, so here's a clarifying thought experiment.

When someone searches Google to find out the shape of the Earth, how should Google approach that query? It depends on what Google believes a search engine is.

(Note that it's likely that flat-Earth proponents generate, link to and chatter about the idea that the Earth is flat more than people who believe it's spherical. Let's assume for the sake of argument that, objectively, the content and activity on the actual internet favors the flat-Earth idea.)

If a search engine is supposed to be an index to the internet, then search results for the shape of the Earth should favor the flat-Earth idea.

If a search engine is supposed to be an arbiter of what's true, then search results should favor the spherical-Earth idea.

If a search engine is supposed to be an objective source of information, then search results should provide a balanced result that equally represents both flat- and spherical-Earth theories.

And if a search engine is supposed to be a customized, personalized source of information, then the results should favor either the flat-Earth idea or the spherical-Earth idea, depending on who is doing the searching.

I use the shape of the Earth as a proxy or stand-in for the real search results people are conducting.

For example, searches about your company, product, brand or even yourself are still subject to the same confusion over what a search engine is supposed to be.

When your customers, prospective business partners, employees or future prospective employees and others search for information about your organization, what results should they get? Should those results reflect what's "true," what's false but popular, or what's neutral between the two? Or should it depend on who's doing the searching?

The truth is that Google tries to make Google Search all four of these things at the same time.

Adding to the complexity of the problem is the fact that search engine results are governed by algorithms, which are trade secrets that are constantly changing.

If you were to ask people, I suspect that most would say that Google Search should be Model No. 1 — an index to the internet — and not get involved in deciding what's true, what's false or what's the answer the user wants to hear.

And yet the world increasingly demands that Google embrace Model No. 2 — to be an arbiter of what's true.

Governments won't tolerate an accurate index

Trump has claimed repeatedly that, in general, news media coverage is biased against him. If that's true, and if Google News Search was a passive index of what the media is actually reporting, wouldn't it be reasonable for Trump to expect anti-Trump coverage on Google News Search?

By slamming Google News Search as "rigged," Trump appears to reveal an expectation that Google News should reflect what's happening in the real world as he sees it, rather than what's happening on news media websites.

Or it reveals that, regardless of the weight of activity in favor of news sources Trump believes are biased against him, Google News Search should provide a balanced and neutral representation of all opinions and sources equally.

The rejection of the search-engine-as-internet-index model is common among governments and political leaders worldwide.

One famous example is the "right to be forgotten" idea, which has been put into practice as law in both the European Union and Argentina. The idea is that information on the internet can unfairly stigmatize a person, and citizens have the right for that information to be "forgotten," which is to say made non-existent in search engine results.

Let's say, for example, that a prominent person files for bankruptcy, and that 100 news sites and blogs on the internet record the fact. Twenty years later, well after the person has restored financial solvency, the old information is still available and findable via search engines, causing unfounded stigmatization.

A successful right-to-be-forgotten petition can remove reference to those pages from search results. The pages still exist, but the search engines don't link to them when anyone searches for the person's name.

The advocates of right-to-be-forgotten laws clearly believe that a search engine exists to reflect the real world as it is, or as it should be, and does not exist to reflect the internet as it is.

Google was recently caught in a controversy over an assumed return to the Chinese market with a custom China-only search engine that censors internet content in the same way that domestic sites are required to by the Chinese government. Hundreds of Google employees signed a letter in protest.

Google wants to "return" to the Chinese market. The Chinese government would not allow Google to operate a search engine accessible to Chinese citizens that accurately reflected what's actually on the internet.

The examples go on and on.

What governments tend to have in common is that in political circles, it's very difficult to find people advocating for the index-to-the-internet conception of what a search engine should be.

Why the search-engine-as-index idea is dead

Google's self-stated mission is to "organize the world's information and make it universally accessible and useful."

Nebulous, yes. But for the purposes of this column, it's telling that Google says that its mission is to organize, not the internet's information, but the "world's."

The reality is that people search Google Search and other search engines because they want information about the world, not because they want information about what the internet collectively "thinks."

And, in any event, the point is growing moot.

What the internet "thinks" is increasingly being gamed and manipulated by propagandists, bots, fake news, trolls, conspiracy theorists and hackers.

Accurately reflecting all this manipulated information in search engines is valuable only to the manipulators.

Also: With each passing day, more information "searching" is happening via virtual assistants such as Google Assistant, Siri, Cortana and Alexa.

In other words, virtual assistants are becoming the new search engines.

With augmented reality glasses and other highly mobile sources of information, search engines such as Google will have to increasingly become arbiters of what's true, or supposed to be true, because the public will increasingly demand a single answer for its questions.

That's why the old initiatives for your company's presence on the internet — SEO, marketing, social media strategy and all the rest — have new urgency.

With each passing day, search engines exist less to index the internet and more to decide for us all what's "true" and what's "not true."

It's time to redouble your efforts to make sure that what Google thinks is true about your company really is true.

Mike Elgan is a technology-obsessed journalist, author, blogger, podcaster and digital nomad. He writes a column for IDG's Insider Pro. Learn more at his website: elgan.com.