Yahoo Search Wants to Be More Like Google, Embraces Hadoop

by admin February 20, 2008 at 8:20 pm

Yahoo is following in Google’s footsteps again in search. Today, it is shifting a crucial part of its search engine to Hadoop, software that handles large-scale distributed computing tasks particularly well. Hadoop is an open-source implementation of Google’s MapReduce software and file system. It takes all the links on the Web found by a search engine’s crawlers and “reduces” them to a map of the Web so that ranking algorithms can be run against them.

Yahoo is replacing its own software with Hadoop and running it on a Linux server cluster with 10,000 core processors.

Read more: techcrunch.com