mj12bot.com
mj12bot.com is the home of MJ12Bot, a web crawler operated by Majestic, a UK based specialist search engine. The bot crawls websites to map the link relationships between pages, and the crawled data feeds a public backlinks search engine index called Site Explorer available at Majestic.com. The crawler identifies itself, obeys the robots.txt standard and crawl delay directives, and does not cache web content or personal data.
Websites by Majestic
Open source ↗