问题描述
请问这个robots.txt文件只允许Googlebot的索引我的网站的index.php文件？警告，我有，人谁输入一个htaccess重定向
Will this robots.txt file only allow googlebot to index my site's index.php file? CAVEAT, I have an htaccess redirect that people who type in
http://www.example.com/index.php
将被重定向到简单
http://www.example.com/
所以，这是我的robots.txt文件的内容...
So, this is my robots.txt file content...
User-agent: Googlebot
Allow: /index.php
Disallow: /
User-agent: *
Disallow: /
在此先感谢！
Thanks in advance!
推荐答案
不是真的。
好机器人的
只有好机器人按照的robots.txt 的说明（不是所有的机器人和蜘蛛懒得读/遵循的robots.txt ）。这可能还不包括所有主要的搜索引擎的机器人，但它肯定意味着一些网络爬虫只会完全忽略你的请求（你应该看看使用的.htaccess或密码保护的，如果你真的要停止机器人/爬虫无法看到的部分您的网站）。
Good bots
Only "good" bots follow the robots.txt instructions (not all robots and spiders bother to read/follow robots.txt). That might not even include all the main search engine's bots, but it definitely mean that some web crawlers will just completely ignore your requests (you should look at using .htaccess or password protection if you really want to stop bots/crawlers from seeing parts of your site).
第二次检查的
谷歌让您的网站多次访问，包括出现作为一个浏览用户。这第二次访问将忽略的robots.txt 文件。第二次访问可能实际上并不指数（如果那是你的担心），但它确实检查，以确保你没有试图愚弄索引机器人（搜索引擎优化等）。
Second checks
Google makes multiple visits to your website, including appearing as a browsing user. This second visit will ignore the robots.txt file. The second visit probably doesn't actually index (if that's your worry) but it does check to make sure you're not trying to fool the indexing bot (for SEO etc).
话虽这么说你的语法是正确的......如果这就是你问，那么是的，它会工作，只是没有那么好，你可能希望。
That being said your syntax is right... if that's all you're asking, then yes it'll work, just not as well as you might hope.
这篇关于这是否会robots.txt的只允许Googlebot的索引我的网站？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！