# If the Joomla site is installed within a folder such as at # e.g. www.example.com/joomla/ the robots.txt file MUST be # moved to the site root at e.g. www.example.com/robots.txt # AND the joomla folder name MUST be prefixed to the disallowed # path, e.g. the Disallow rule for the /administrator/ folder # MUST be changed to read Disallow: /joomla/administrator/ # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/orig.html # # For syntax checking, see: # http://tool.motoricerca.info/robots-checker.phtml User-agent: * Disallow: /administrator/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/ Disallow: /joomlatools-files/ Disallow: /ifile_index/ Disallow: /fscn/ Disallow: /centrora/ Disallow: /dev/ Disallow: /cgi-bin/ Disallow: /tmp/ User-agent: Googlebot Allow: / Allow: https://translate.googleapis.com/translate_static/js/element/main.js Allow: https://translate.googleapis.com/translate_static/css/translateelement.css Allow: https://unredd.net/modules/mod_bt_contentslider/tmpl/images/pagination.png Allow: http://www.unredd.net/images/stories/slide2.jpg?1530168203391 Allow: http://www.unredd.net/modules/mod_accordeonck/themes/simple/mod_accordeonck_css.php?cssid=accordeonck153 User-agent: MauiBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Cliqzbot Disallow: / User-agent: YandexBot Disallow: / User-agent: Yandex Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SeznamBot Disallow: /