Hello all, I recently upgraded to phpbb3.2 and decided to check the google web tools site, too my suprise when I clicked on the robots.txt checker it showed ...
Create my own robots.txt file to prevent google from wasting time indexing uninteresting routes on my website, such as 'memberlist.php?
This is a custom result inserted after the second result.
I have a general error in my log saying I'm missing robots.txt. I just read a 5 year old thread here that phpbb doesn't use robots.txt???
You can only disallow directories via robots.txt, not files. First, remove the asterisks from the directories. Disallow: /adm/ Disallow: / ...
i am considering creating robot.txt for my phpbb forum 3.2 because spiders are crawling almost all the areas of the forum.
I am looking at the source of Shredders ext and xml file from my site l, seems the same...Also, BING webmaster still accepts this XML no issues.
I just want to let the search engines see my categories, post, forums, user profiles and uploads. Block all those bots who can make my site go slow, ...
If I'm running the latest phpBB forum, which parts should I have included in robots.txt so they don't get spidered?
MOD Description: this file save your board from bots. MOD Version: Beta: 1.0.0. MOD Download: files in topic developed on: http://www.phpbbturkiye.net ...
It's just a text file, made in something like notepad with that text in and saved as robots.txt. It's nothing special, no coding knowledge is ...