Blogger Robots.txt Tips
Seriously, you cannot edit robots.txt in Blogger hosted blogs (a.k.a blogspot). What you can do is manipulate the meta robots tag using Blogger conditional tags. One of the common problems in Blogspot is content duplication, and effective use of meta robots tag can prevent this problem.But average blogger has almost no knowledge of blogger conditional tags. Although meta robots tag is very easy, blogger conditional tags is not simple. For example, if the following tag:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
Will be added to the html source code of blogger template, all blogger posts and archives will have these meta robots tag. In the end, your blog will be completely blocked in the search engines, like an equivalent:
User-agent: *
Disallow: /
Using robotst.xt command.
Rules of preventing duplicate content in Blogger:First rule: Of course all blogger post and the main page (front page) should be indexable, so it should have:
<META name="ROBOTS" content="INDEX,FOLLOW">
Second rule: The archived page should be placed with noindex and nofollow. Since you are not interested in search engines indexing these type of URLs since they contain just duplicates of the blogger posts.
Create implement blogger conditional tags
Blogger conditional tags are like HTML tags with opening and close tags. For example, if you like the command to work only in the blogger front page or main page, then:<MainPage> [HTML code here] </Mainpage
For archived page, it will be:
<ArchivePage> [HTML code here] </ArchivePage>
For post URL, it will be:
<ItemPage> [ HTML code here </ItemPage>
So how are you going to formulate the complete syntax that will be able to put meta robots noindex nofollow for archived, but put meta robots index follow for meta robots index follow???
The code is shown below:
<MainPage> <META name="ROBOTS" content="INDEX,FOLLOW"> </Mainpage><ArchivePage> <META name="ROBOTS" content="NOINDEX,NOFOLLOW"> </ArchivePage><ItemPage> <META name="ROBOTS" content="INDEX,FOLLOW"> </ItemPage>
How to go implement the above code
Simply delete the default meta robots tag if there is and replace it with the above code. It should be placed between the … tag. For example:<head>
<MainPage> <META name="ROBOTS" content="INDEX,FOLLOW"> </Mainpage><ArchivePage> <META name="ROBOTS" content="NOINDEX,NOFOLLOW"> </ArchivePage><ItemPage> <META name="ROBOTS" content="INDEX,FOLLOW"> </ItemPage>
<title>Your title tag<title>
[ THE REST OF THE HEAD HMTL CODE HERE]
</head>
Note .
If you find this article useful, please feel free to link to this page from your website or blog.
Posts
0 comments: