The user agent token is used in the User-agent: line in robots.txt to match a crawler type when writing crawl rules for your site.
確定! 回上一頁