Tuesday, March 19, 2019

How Search Engines Work?

World around has turned Digital making everyone now open a digital device and search for something daily, actually many times a day but have you ever thought of knowing how these search engines work and how are results that best answers your search query thrown on your screen. Here is the answer for your question. 

With no search engines there would be no access to the world of information. So let see how these search engines work..

Before proceeding further lets understand what a search engine is? A web search engine is a software system that searches the world wide web for the requested search query and gathers all the information relevant and throws them in an order, sorted based on the respective search engines algorithms.
So, How do they perform this job of finding results? For this question to be answered, lets move to the main point of How do Search Engines work? 
Search Engines usually perform 3 different activities as Crawling, to discover content over the web; Indexing, to track and store the content crawled; and Retrieving, fetch relevant content for the search query of a user in the search engine.

Crawling:
The whole work of a Search Engine starts with Crawling which is the all about acquisition of data about a website. 
As a part of this step, websites are scanned and details of each page like titles, descriptions, images, keywords, other linked pages, etc.is collected. Some other crawlers also collect details about page layout, if the links pointing are crammed in, where the advertisements if any are to be placed etc.
A Search Engine's automated robots also called Spiders visit page after page as quick as possible by following various links available. 
How Do Search Engines Work? web crawler diagram
A web crawler visits a page and then collects all the links in the page and adds them to the queue of links to be crawled and then do the same for the next links in queue. Crawlers also visit back old sites to look for any updates of the content. With that said, any site or page linked to an already existing indexed page will be crawled eventually. The rate at which sites are crawled varies from site to site, some are crawled more frequently while some others are crawled to a great depth.

Indexing:
Indexing is the step that falls after crawling. A site or a page when crawled is further processed and stored in a database.
Imagine making a list of all house addresses in your town, their owners details, tenants details, floor count, etc. Crawling is going through each house whereas indexing is listing them in your storage space.
This is wrt a small town, now apply the same task to the whole world. That's a simple easy to understand version of what search engines like Google does, storing all the data collected in their huge and vast data centers in the thousands and thousands of petabyte worth of drives.

Retrieving and Ranking:
Retrieving is the process of returning the most relevant page results for your search query. Each Search Engine has its own search criteria based on which the results are displayed to match your search query at best after going through the billions and trillions of pages in their database.
Search Engines are now keeping their search algorithms in mystery to avoid web creators from unfairly climbing the top as in the early days of Search Engines usage.
If you feel still confused a bit or want to have more understanding of a search engine works, here, go through this video below.


How Are Search Engines So Fast? 

Have you ever come across the problem searching your computer for a file that has been saved in an unnoticed location by mistake and closed. Remember how slow your system works for searching that file from an unknown location from the storage of just a few hundreds of files.Now go to a search engine like google or bing or any other and search for a query, you get the results at the blink of an eye. How do you think is this possible for search engines to work so fast?Because search engines unlike our computers won't look for each and every file every time you do a search. They have the pages crawled and indexed and when searched for a query, it takes very less time to retrieve the stored results on to the screen. 

How do search engines rank websites?
That used to be a very easy job for search engines to rank websites in the beginning. It just is the matter of finding the pages with keywords used in the pages and build back links. Thus making the job of an SEO simple by just filling the page with keywords multiple times and building huge links in whatever and every means possible. But this turned out the search results spamy.
Now Search Engines turned far smarter that they developed Algorithms that use various factors to rank the web pages. Few of the vast list of factors include but nor limited to keywords in the page content, on-page factors, off-page factors, social popularity, content quality, backlinks quality and quantity, etc.
Websites that rank better get positioned in the best places in the search results page.

Why do search engines give different results?
Yes, Google search results are not the same for everyone and vary from person to person and different from computer to computer. This is to better facilitate the user with their interests. Few reasons behind seeing different results are your physical location, search history, etc.

How do search engines make money?
While every search results so accurately and perfectly delivered at free of cost, what do you think will run these search engines to pay for its employees? Its obviously their ad network that serve the purpose.
Image result for search engines ads
When you search for a query along with the results, search engines like google also some most relevant ads to the right and on the top of the SERP. Those ads that you see around the search results are one of the money earners.
About 85% of the money that Google makes is from advertising. Even Gmail is a way for advertising that forms a part of the earnings. Records show that for every search made Google earns around $50.

What Search Engines exist?
There are lot of search engines available but as best always stands out, here are a few mostly used search engines Google, Yahoo, Bing, Ask.com, AOL.com, Baidu, Wolframalpha, DuckDuckGo, Archive.org, Yandex.ru

Which is the oldest search engine of the Internet?
Although the web was made public in 1990, until 1993 there is no search engine and all indexes are manually made. The search king Google has arrived in 1998 from then, indexation has started to be taken care by software programs. In the 5 year gap of web search availability and Google's birth around 20 search engines were gloriously enjoying, while many failed later.
webcrawler is the oldest still surviving search engine, Lycos, Altavista, Excite, Yahoo, Dogpile, Ask Jeeves, etc are few familiar ones.

Is Google a search engine or library database?
Google can be counted on as a Search Engine as its free and need no user account to search. Also, Databases have information that is credible and already verified. Whereas Google wont stand that way as a publisher can write any thin of their choice and post in the web.

1 comment:

Your thoughts and suggestions are valuable for me.