WP Robots Txt

Sharraxaad

WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great, but how do you easily change the content?

Enter WP Robots Txt, a plugin that adds an additional field to the “Reading” admin page where you can do just that. No manual coding or file editing required!

Simply visit https://your-site.com/wp-admin/options-reading.php and you can control the contents of your https://your-site.com/robots.txt

Changelog

Sawir-shaashado

  • A view of the admin option

Rakibaad

  1. Download the plugin
  2. Unzip it
  3. Upload the unzipped folder to wp-content/plugins directory
  4. Activate and enjoy!

Or you can simply install it through the admin area plugin installer.

SBI

I totally screwed up my `robots.txt` file. How can I restore the default version?

Delete all the content from the Robots.txt Content field and save the privacy options.

Could I accidently block all search bots with this?

Yes. Be careful! That said, robots.txt files are suggestions. They don’t really block bots as much as they suggest that bots don’t crawl portions of a site. That’s why the options on the Privacy Settings page say “Ask search engines not to index this site.”

Where can I learn more about `robots.txt` files?

Here is a general guide by Google and here is the WordPress SEO documentation.

Dibu-eegisyo

21 Abriil, 2025 2 replies
I followed their instructions and it didn’t update the robots.txt file. It would be a great plugin if it worked!
7 Janaayo, 2025
Don’t know what WordPress does with robots.txt file and uploading manually to my server does not work. This plug-in fixed it quickly. Highly Recommend!
15 Janaayo, 2018
Good code, nice interface. It doesn’t have builtin dynamic replacement functionality (like, {siteurl}/sitemap_index.xml), but it can be added with a simple filter. Thanks for the great plugin!
Akhri dhammaan 20 dibu-eegis

Ka-qaybgalayaasha & Horumariyayaasha

“WP Robots Txt” waa softiweer il furan. Dadka soo socda ayaa wax ku biiriyay kaabahan.

Ka-qaybgalayaasha

“WP Robots Txt” waxaa lagu tarjumay 5 luqadood. Way ku mahadsan yihiin turjumaannada ka-qaybqaadashadooda.

Ku tarjun “WP Robots Txt” luqaddaada.

Ma xiisaynaysaa horumarinta?

Baadh koodka, fiiri bakhaarka SVN, ama iska qor diiwaanka horumarinta adigoo adeegsanaya RSS.

Isbeddellada

1.3.4

  • Compatibility with WordPress 6.7

1.3.3

  • Add Line Break to allow appending more rules. Thanks @flberger

1.3.2

  • Add translation template

1.3.1

  • Compatibility with WordPress 6.4
  • WP Coding Standards compliant

1.3

  • Ensure Compatibility with WP v6.3
  • Increase PHP minimum required version

1.2

  • Update the default robots.txt content
  • Include sitemap reference
  • Resolve code warnings/errors
  • WP Coding Standards compliant
  • Ensure Compatibility with WP v6

1.1

  • Moved the settings field “officially” to the reading page
  • General code clean up

1.0

  • Initial version