Typically, the bulk of the index contains content
from pages generated by channel rendering scripts and postings. They are
after all, the parts of the site that are visible to the end users. To
tell search engines which pages to index and which pages to ignore and
whether to follow further links from the current page you can add a
special META tag to your page named ROBOTS. Most (but not all) search
engines honor this META tag and evaluate its content property to see if
the page should be added to the index and if links from this page should
be followed.
To allow MCMS authors to benefit from this feature, channels and postings have two additional properties:
IsRobotIndexable
IsRobotFollowable
These two properties can be
used to generate the ROBOTS META tag. As already mentioned, be aware
that some search engines choose to ignore the ROBOTS META tag.
The table below summarizes the behavior of the different property settings:
IsRobotFollowable | IsRobotIndexable | META tag to render | Remarks |
---|
False | False | <meta name=“ROBOTS” content=“NOFOLLOW, NOINDEX”> | Search engines that recognize the robot META tag will not index the page nor follow its hyperlinks. |
False | True | <meta name=“ROBOTS” content=“NOFOLLOW, INDEX”> | Search engines that recognize the robot META tag will index the page but not follow its hyperlinks. |
True | False | <meta name=“ROBOTS” content=“FOLLOW, NOINDEX”> | Search engines that recognize the robot META tag will not index the page but will follow its hyperlinks. |
True | True | <meta name=“ROBOTS” content=“FOLLOW, INDEX”> | Search engines that recognize the robot META tag will index the page and follow its hyperlinks. |
For more information on the ROBOTS META tag, check out this resource:
http://www.robotstxt.org/wc/exclusion.html#meta
The RobotMetaTag Control and Channel Rendering Scripts
MCMS ships with a control named RobotMetaTag, which is automatically added to all template files created from the Visual Studio .NET “MCMS Template File” template. The RobotMetaTag control is used to generate the ROBOTS META tag based on the IsRobotFollowable and IsRobotIndexable properties of the page.
Unfortunately this control
does not work on channel rendering scripts. If you add the control to a
channel rendering script, despite setting the IsRobotFollowable and IsRobotIndexable
properties of the channel, the ROBOTS META tag does not get generated.
To get around this problem let’s implement an enhanced version of this
control that works with both channel rendering scripts and templates.
In the WoodgroveNet sample site that comes with MCMS, the RobotMetaTag control is wrapped in a user control called HeadTags.ascx.
First, open the Tropical Green solution. Add a new class file to the TropicalGreenControlLib project with the name EnhancedRobotMetaTag.cs.
The control requires the use of methods from the namespaces highlighted below, so add them to the class file.
using System;
using System.ComponentModel;
using System.Web.UI;
using Microsoft.ContentManagement.Common;
using Microsoft.ContentManagement.Publishing;
using Microsoft.ContentManagement.WebControls;
namespace TropicalGreenControlLib
{
. . . code continues . . .
}
Also, as we are creating a custom server control, the class will inherit from the RobotMetaTag class (in the Microsoft.ContentManagement.WebControls namespace).
namespace TropicalGreenControlLib
{
/// <summary>
/// Summary description for EnhancedRobotMetaTag.
/// </summary>
public class EnhancedRobotMetaTag : RobotMetaTag
{
public EnhancedRobotMetaTag()
{
}
}
}
Above the EnhancedRobotMetaTag constructor, declare four constants, Index, Follow, noIndex and noFollow, which store the strings we will use later for generating the ROBOTS META tag. We will also remove the constructor, EnhancedRobotMetaTag(), as it is not required.
public class EnhancedRobotMetaTag : RobotMetaTag
{
private const string Index = "INDEX";
private const string Follow = "FOLLOW";
private const string noIndex = "NOINDEX";
private const string noFollow = "NOFOLLOW";
}
In addition to rendering the ROBOTS tag, the original RobotMetaTag control has a different feature: it provides an option to render a <base> tag, which ensures that relative links added to the template files work correctly. Here’s an example of a <base> tag:
<base
href="http://localhost/TropicalGreen/Templates/GardenPublic.aspx?NRMODE=Publis
hed&NRORIGINALURL=%2fTropicalGreen%2fGardens%2fMembers%2ehtm&NRNODEGUID=%7bB95
DD11B-2954-4F84-AB60-AC0A3D39FC49%7d&NRCACHEHINT=NoModifyGuest">
When using the Word Authoring Connector and documents that use internal bookmarks, the <base> tag can cause an entire page refresh.
|
As a single template file
can be used by multiple postings, the URL varies from posting to
posting. Links relative to the template file will not work in this
situation as the browser does not know that postings and channels do not
exist as complete entities on the file system. The base tag tells the
browser what should be treated as the current location and allows the
browser to calculate the final link based on the base tag and the
information in the relative link. For example, should the template file
contain a relative link, say, <a href="MembersOnly/SomePage.htm">A Relative Link</a>, the browser will treat the URL as if "MembersOnly/SomePage.htm" was pre-pended with the URL stored in the href attribute of the <base> tag.
As we are going to implement a control that extends the RobotMetaTag control, we need to ensure that this feature is also included in our new control.
We will write the ROBOTS META tag and <base> tag by overriding the Render() method of the control. Notice we look at the properties of a ChannelItem object. This ensures that the code works for both postings and channel rendering scripts.
protected override void Render( HtmlTextWriter writer )
{
ChannelItem currentItem = CmsHttpContext.Current.ChannelItem;
if( currentItem != null )
{
string followKey = (currentItem.IsRobotFollowable)?Follow:noFollow;
string indexKey = (currentItem.IsRobotIndexable)?Index:noIndex;
// write the robot META tag
writer.Write("<meta name=\"ROBOTS\" "
+ "content=\""+followKey+","+indexKey+"\">\n");
if( this.RenderBaseHref )
{
// write the Base tag
writer.Write("<base href=\""+Page.Request.Url.AbsoluteUri+"\">\n");
}
}
}
Save
and build the solution. Add the control to the toolbar as you would
other custom controls and it’s ready to be dropped into channel
rendering scripts or template files between the <head> tags.