User-agent: * Disallow : /HTTPSRedirect.aspx User-agent: oBot Disallow: / User-agent: NPBot Disallow: / User-agent: ia_archiver Disallow: /