{"id":1136,"date":"2011-11-09T14:47:20","date_gmt":"2011-11-09T14:47:20","guid":{"rendered":"https:\/\/www.uk-cheapest.co.uk\/support\/?p=1136"},"modified":"2024-01-14T13:00:29","modified_gmt":"2024-01-14T13:00:29","slug":"how-do-i-block-the-yandex-ru-bot-from-crawling-my-site","status":"publish","type":"post","link":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/","title":{"rendered":"How do I block the Yandex.ru bot from crawling my site?"},"content":{"rendered":"<p><strong>Re: How to stop Yandex, Blocking Yandex.RU<\/strong><\/p>\n<p><strong><\/strong>Yandex is the most popular search engine in Russia. Your bandwidth can go through the roof if this bot targets your website.<\/p>\n<p>Unfortunately for many, the robots.txt file is ignored so blocking Yandex using the official method is not an option.<\/p>\n<p>If you have a busy forum or website with hundreds of pages you may find that the Yandex Bot is starting to take up more of your site resources by indexing up to 90 pages every 15 minutes often leaving connections open or failing to close them properly.<\/p>\n<p>You can easily disable the Yandex.ru Bot by placing the following in your .htaccess file:<\/p>\n<pre>SetEnvIfNoCase User-Agent \"^Yandex*\" bad_bot\r\nOrder Deny,Allow\r\nDeny from env=bad_bot<\/pre>\n<p>Using this method saves you the trouble of having to find blocks of Yandex IP addresses and block them individually which would only work for a limited time.<\/p>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-learn-more-about-the-yandex-bot\">Learn more about the Yandex bot<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-what-is-the-yandex-ru-bot\">What is the Yandex.ru bot?<\/h3>\n\n\n\n<p>The Yandex.ru bot, also known as YandexBot, is a web crawler and indexing bot used by Yandex, a leading Russian search engine. It scans websites, indexes web pages, and provides data for Yandex&#8217;s search results.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-can-i-check-if-the-yandex-ru-bot-has-visited-my-website\">How can I check if the Yandex.ru bot has visited my website?<\/h3>\n\n\n\n<p>You can check if the Yandex.ru bot has visited your website by examining your website&#8217;s server logs or by using tools like Yandex.Webmaster, which provides detailed information about bot activity.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-is-it-necessary-to-optimise-my-website-for-the-yandex-ru-bot\">Is it necessary to optimise my website for the Yandex.ru bot?<\/h3>\n\n\n\n<p>Yes, if you want your website to appear in Yandex search results, it&#8217;s advisable to optimize it for the Yandex.ru bot. This includes following Yandex&#8217;s webmaster guidelines for indexing and ranking.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-are-there-specific-rules-or-guidelines-for-yandex-ru-bot-optimisation\">Are there specific rules or guidelines for Yandex.ru bot optimisation?<\/h3>\n\n\n\n<p>Yandex provides webmaster guidelines that include recommendations for optimising your website for the Yandex.ru bot. This includes creating a sitemap, using proper meta tags, and ensuring mobile-friendliness.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-how-can-i-block-or-allow-the-yandex-ru-bot-from-accessing-my-website\">How can I block or allow the Yandex.ru bot from accessing my website?<\/h3>\n\n\n\n<p>You can control bot access through your website&#8217;s robots.txt file. To allow Yandex.ru bot, include &#8220;User-agent: Yandex&#8221; in your robots.txt. To block it, use &#8220;Disallow: \/&#8221; for all Yandex user-agents.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Re: How to stop Yandex, Blocking Yandex.RU Yandex is the most popular search engine in Russia. Your bandwidth can go through the roof if this bot targets your website. Unfortunately for many, the robots.txt file is ignored so blocking Yandex using the official method is not an option. If you have a busy forum or &#8230; <a title=\"How do I block the Yandex.ru bot from crawling my site?\" class=\"read-more\" href=\"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/\" aria-label=\"Read more about How do I block the Yandex.ru bot from crawling my site?\">Read more&#8230;<\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","footnotes":""},"categories":[7,61],"tags":[229,227,228,226],"class_list":["post-1136","post","type-post","status-publish","format-standard","hentry","category-search-engines","category-support","tag-block","tag-robots","tag-robots-txt","tag-yandex"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk\" \/>\n<meta property=\"og:description\" content=\"Re: How to stop Yandex, Blocking Yandex.RU Yandex is the most popular search engine in Russia. Your bandwidth can go through the roof if this bot targets your website. Unfortunately for many, the robots.txt file is ignored so blocking Yandex using the official method is not an option. If you have a busy forum or ... Read more...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/\" \/>\n<meta property=\"og:site_name\" content=\"UK-Cheapest.co.uk\" \/>\n<meta property=\"article:published_time\" content=\"2011-11-09T14:47:20+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-14T13:00:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.uk-cheapest.co.uk\/support\/wp-content\/uploads\/2023\/12\/IMG_3064.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"768\" \/>\n\t<meta property=\"og:image:height\" content=\"439\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"UKC\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@ukchelpdesk\" \/>\n<meta name=\"twitter:site\" content=\"@ukchelpdesk\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"UKC\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/\"},\"author\":{\"name\":\"UKC\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#\\\/schema\\\/person\\\/135cf981b8a3a120603fcbc027eed746\"},\"headline\":\"How do I block the Yandex.ru bot from crawling my site?\",\"datePublished\":\"2011-11-09T14:47:20+00:00\",\"dateModified\":\"2024-01-14T13:00:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/\"},\"wordCount\":374,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#organization\"},\"keywords\":[\"block\",\"robots\",\"robots.txt\",\"yandex\"],\"articleSection\":[\"Search Engines\",\"Support\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/\",\"url\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/\",\"name\":\"How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#website\"},\"datePublished\":\"2011-11-09T14:47:20+00:00\",\"dateModified\":\"2024-01-14T13:00:29+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"UK-Cheapest.co.uk\",\"item\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How do I block the Yandex.ru bot from crawling my site?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#website\",\"url\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/\",\"name\":\"UK-Cheapest.co.uk\",\"description\":\"Cheap Domain Names, Web Hosting, Site Builder &amp; Web Security\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#organization\",\"name\":\"UKC\",\"url\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/wp-content\\\/uploads\\\/2023\\\/12\\\/Screenshot-2023-12-14-at-17.40.07.png\",\"contentUrl\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/wp-content\\\/uploads\\\/2023\\\/12\\\/Screenshot-2023-12-14-at-17.40.07.png\",\"width\":217,\"height\":55,\"caption\":\"UKC\"},\"image\":{\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/ukchelpdesk\",\"https:\\\/\\\/www.instagram.com\\\/ukc_hosting\\\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/#\\\/schema\\\/person\\\/135cf981b8a3a120603fcbc027eed746\",\"name\":\"UKC\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g\",\"caption\":\"UKC\"},\"sameAs\":[\"https:\\\/\\\/www.uk-cheapest.co.uk\"],\"url\":\"https:\\\/\\\/www.uk-cheapest.co.uk\\\/support\\\/author\\\/colin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/","og_locale":"en_GB","og_type":"article","og_title":"How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk","og_description":"Re: How to stop Yandex, Blocking Yandex.RU Yandex is the most popular search engine in Russia. Your bandwidth can go through the roof if this bot targets your website. Unfortunately for many, the robots.txt file is ignored so blocking Yandex using the official method is not an option. If you have a busy forum or ... Read more...","og_url":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/","og_site_name":"UK-Cheapest.co.uk","article_published_time":"2011-11-09T14:47:20+00:00","article_modified_time":"2024-01-14T13:00:29+00:00","og_image":[{"width":768,"height":439,"url":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-content\/uploads\/2023\/12\/IMG_3064.jpeg","type":"image\/jpeg"}],"author":"UKC","twitter_card":"summary_large_image","twitter_creator":"@ukchelpdesk","twitter_site":"@ukchelpdesk","twitter_misc":{"Written by":"UKC","Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/#article","isPartOf":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/"},"author":{"name":"UKC","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#\/schema\/person\/135cf981b8a3a120603fcbc027eed746"},"headline":"How do I block the Yandex.ru bot from crawling my site?","datePublished":"2011-11-09T14:47:20+00:00","dateModified":"2024-01-14T13:00:29+00:00","mainEntityOfPage":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/"},"wordCount":374,"commentCount":1,"publisher":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#organization"},"keywords":["block","robots","robots.txt","yandex"],"articleSection":["Search Engines","Support"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/","url":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/","name":"How do I block the Yandex.ru bot from crawling my site? - UK-Cheapest.co.uk","isPartOf":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#website"},"datePublished":"2011-11-09T14:47:20+00:00","dateModified":"2024-01-14T13:00:29+00:00","breadcrumb":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/how-do-i-block-the-yandex-ru-bot-from-crawling-my-site\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"UK-Cheapest.co.uk","item":"https:\/\/www.uk-cheapest.co.uk\/support\/"},{"@type":"ListItem","position":2,"name":"How do I block the Yandex.ru bot from crawling my site?"}]},{"@type":"WebSite","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#website","url":"https:\/\/www.uk-cheapest.co.uk\/support\/","name":"UK-Cheapest.co.uk","description":"Cheap Domain Names, Web Hosting, Site Builder &amp; Web Security","publisher":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.uk-cheapest.co.uk\/support\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#organization","name":"UKC","url":"https:\/\/www.uk-cheapest.co.uk\/support\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#\/schema\/logo\/image\/","url":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-content\/uploads\/2023\/12\/Screenshot-2023-12-14-at-17.40.07.png","contentUrl":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-content\/uploads\/2023\/12\/Screenshot-2023-12-14-at-17.40.07.png","width":217,"height":55,"caption":"UKC"},"image":{"@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/ukchelpdesk","https:\/\/www.instagram.com\/ukc_hosting\/"]},{"@type":"Person","@id":"https:\/\/www.uk-cheapest.co.uk\/support\/#\/schema\/person\/135cf981b8a3a120603fcbc027eed746","name":"UKC","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/afbfe91204852778e8698de79f854348bec9aea508bef650c084d0c6daf2588d?s=96&d=mm&r=g","caption":"UKC"},"sameAs":["https:\/\/www.uk-cheapest.co.uk"],"url":"https:\/\/www.uk-cheapest.co.uk\/support\/author\/colin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/posts\/1136","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/comments?post=1136"}],"version-history":[{"count":7,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/posts\/1136\/revisions"}],"predecessor-version":[{"id":4872,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/posts\/1136\/revisions\/4872"}],"wp:attachment":[{"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/media?parent=1136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/categories?post=1136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.uk-cheapest.co.uk\/support\/wp-json\/wp\/v2\/tags?post=1136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}