User-agent: * # applies to all robots Disallow: /newsevents/public/ # disallow crawling of newsevents pages Disallow: /directories/