Hello, I want some changes in robots.txt file of my site. but I am confused in following commands Disallow: /wp-content/ Disallow: /wp-content/themes Disallow: /?comments=* Disallow: /trackback/ Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /comments Disallow: /category/*/* Disallow: */trackback Disallow: */feed Disallow: */comments Disallow: /*?* Disallow: /*? Allow: /wp-content/uploads Please suggest me the benefits of aforementioned commands.. plz plz help me I need desperately to know about these commands .
Those should keep crawlers from indexing those WordPress directories. "Comments" probably should not be on that list! It really depends on if you want them crawled.
