User-agent: * Allow: / # Block specific crawlers if desired: # User-agent: Amazonbot # Disallow: /