# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # User-agent: SemrushBot User-agent: Baiduspider User-agent: YandexBot User-agent: Baiduspider User-agent: Baiduspider-image User-agent: Baiduspider-video User-agent: YandexBot User-agent: YandexMedia User-agent: YandexImages User-agent: YandexBlogs User-agent: AhrefsBot User-agent: Googlebot Disallow: /stock_page/download/