Sometimes we want to exclude a page from search engines. For example, repetitive pages that might lead to page rank penalties.

If you want to exclude pages with a specific name from several locations in your site – for example you might have a comments.php or a help.php in multiple parts of your site, and you want to hide it from search engines, you need to modify robots.txt.

Here is how you do it:
1) Create or edit robots.txt in the root of your site

2) Add these lines to robots.txt

User-agent: *
Disallow: */comments.php


Where comments.php is the name of the page and the * (in the Disallow: */comments.php) indicates every path.

This will exclude:
Yoursite.com/comments.php
But also
Yoursite.com/any_path/comments.php

Important tip, you can use Google web master tools to verify your robots.txt