Search Results Clustering Demystified

Clustering may mean to have two or more computer systems working together or multiple servers linked together for the purpose of handling variable workloads as well as to provide continued operation in case one fails. It may also refer to data clustering which is a technique used for data analysis by dividing a data set into subsets whose elements share common traits. Search result clustering aims to change the way people search online by organizing search result into folders that group similar items together.

Why Clustering is Needed

The use of the vast information available online cannot be maximized unless an effective means of organizing it can be provided. Clustering engines put search results together based on textual and linguistic similarity. This basic similarity is supported by heuristics which are coded by programmers using as basis the users’ preference on what they want to see on clustered documents. Clusters are presented using the style of folders and sub-folders.

When a search engine provides millions of results for a particular query, the searcher can either sift through the endless pages of results or depend on the search engine’s judgment as to the most relevant results. Neither can ensure that the targeted information can be accessed as it may remain buried under pages of results or it may not meet the search engine’s criteria. In the same way that all other things are clustered or organized, the world of web searching would be more useful once given the benefit of organized search results.

Clustering engines automatically cluster results into categories that have been intelligently selected from words and phrases contained in search results. Categories are intended to reach human-level accuracy and to offer hierarchical drill doom capability in a familiar folder-style interface. Mind-numbing lists need not be scrolled through or ignored as the main themes are viewed in the first 300 – 500 results right on the first page. A quick overview of the types of information available on a particular topic is made available so that the area of interest can be immediately put into focus.

With the great improvement of search engines’ capability to return a large number of relevant results, it became more difficult to navigate meaningfully through all the results. A typical searcher does not take the time to view results beyond the first page which makes it very probable to miss results that would have been relevant and useful to his/her search or query. Clusters make it possible for results found on the tenth page to be just a click away. Related items can also be viewed together without much effort. It even reveals unexpected relationships between words, ideas and concepts.

A good cluster is considered such if it possesses a readable description. It should be able to assist in narrowing down a search to find exact results. A clustering engine queries multiple search engines and combines the results to be clustered and displayed on one screen. Each result list comes with information regarding the total number of results clustered and retrieved. The clustering engine’s own heuristics shall determine the pages to be favored. Search engines sometimes return multiple copies of the same page with slightly different URLs but this is minimized in search result clustering. This is because clustering engines does not reproduce results with similar descriptions. Clusters are specific enough that repeated documents are very rare. Some are able to offer advanced search features which allows searchers to specify which sources should be searched, the number of results desired, allowable waiting time, the desired language to be used and the filtering out of offensive contents.

Search Engines that Clusters

Google Sets do not provide results but rather helps in finding similar terms to the ones entered. This allows the user to create more complex queries in one area and brainstorm on how to put a search together. Google Sets is Google Labs’ clustering agent.

Wisenut is a full-text search engine which provides for related topics aside from a number of results for any search item entered. This is called the WiseGuide. Some results would have subtopics which will show underneath the clustered results. A link can be found next to each of the clustered results whose keywords can be used to run another search. A different set of clustered results shall be produced in addition to the web page results. This search engine has been bought by LookSmart.

Teoma has been dubbed as the “Google Killer” due to its very interesting clustering technology. A single search run will produce four sets of results. Those found at the top left are sponsored results, those found at the bottom are website non-sponsored results, those at the top right are the suggestions for refining the result and those at the bottom right are link calculations from experts and enthusiasts. The link collections are suitable for general information needs while the suggestions are for more specific searches. A click on any would signal the search to run again where a different set of site results shall be provided. Teoma has been purchased by AskJeeves.

Infonetware.com is more of a demonstration of Infonetware’s Real Term Technology than a search engine. The results page is framed where the area on the left provides topics related to the search term while the web page search results are found on the right frame. It works with full searching.

Oingo uses the open Directory Project as its search source. The search results page gives a drop-down list of potential meanings. The list of categories in order of relevance to the search can be found beneath it as well as the site results from the directory itself. It is more useful for general term searches or search terms that are in a broad category.

Vivisimo is a meta-search engine that clusters its results. It provides a very simple front page with search results that are organized in groups. The page design makes it easy to explore several categories without having to “lose your place”. Clusty is the consumer search destination powered and owned by Vivisimo. It queries results from Ask, MSN, Open Directory, LookSmart, Gigablast and WiseNut. These sites were chosen because of their accurate results and quick return speeds.

Query Server offers several types of search on the left side of the front page. Each search has more or less the same interface and all cluster results. Search results are presented in a frame at the right side of the site.

Surfwax offers both subscription based and free services. A focus link can be seen in the upper left corner after a search is entered. These focus words can be used in addition to the search term. They are divided into narrower or broader categories and contain generic words and not links to specific people or places.

Northern Light News search requires a search to have a certain number of results in order to be clustered into folders. However, folder listing does not provide information about the contents of a particular folder although there are subfolders provided for broad topics. Search results are listed by order of date.

Clustering search engines break up several hundred results into manageable packages. Suggestions are provided so that the use of information is maximized and the search itself a lot easier. A search query cannot always be specific enough to target the right information at once.