Automated Testing and Sitecore – Part 7

In the previous part of this series we covered testing the static output and behaviors of Sitecore presentation components such as rendering, layout and sublayout. But what about testing the dynamic behaviors of a component, such as “what happens when I push the button?”, or “did my javascript make the ajax call to grab the content from the CMS?”. In this part of the series I’ll take you through how I use WatiN to test the dynamic aspects of my Sitecore presentation components.

WatiN is a browser driver that instantiates a new browsing window and provides access directly to the DOM to allow manipulation and testing. WatiN only currently supports IE. If you need cross browser testing you will have to use one of the other browser drivers such as Selenium.

Because WatiN needs to interact with applications on your desktop (IE) I would not suggest running your WatiN tests in the custom test runner. In fact, I haven’t yet been able to get WatiN to run in the custom test runner due to the threading apartment of the ASP.NET requests.

The first thing we need to do is to create some test content programmatically using the techniques which have been covered in previous parts of the series. The gist of this technique is to have a layout which does item creation and cleanup and we call an item with this layout across HTTP, so our windows app can control the lifetime of the test items.

Now that we have some content to test against we can create a WatiN.IE object to make the request and verify the results.

const string DOMAIN = "http://localhost";
using (WatiN.Core.IE ie = new WatiN.Core.IE(DOMAIN + "/page1.aspx"))
{
  ie.WaitForComplete();
  Assert.AreEqual(ie.Paras[0], "expected text");
}

We can find elements in the DOM in a few ways using WatiN. Firstly as shown above, you can access a kind of element, such as a paragraph above, by index. Generally this isn’t adequate and you’ll want to be finding by ID. So in that case we can use the method on the IE object rather than the property.

ie.TextField("txtUsername");

Keep in mind the ID we’re searching for will be the clientID in ASP.NET controls which may be of the format “ctlxx_name”. Make sure you get the right ID.

We can also make use of LINQ for querying elements by other parameters. In the below code snippet I’ll locate the first div with a class attribute of “breadcrumb”.

var match = from WatiN.Core.Div d in ie.Divs where
  d.ClassName == "breadcrumb" select d;
WatiN.Core.Div div = match.ElementAt(0);

I can also use a WatiN constraint to find an element by an attribute.

ie.TextField(WatiN.Core.Find.ByClass("username"));

So now that we know how to find elements in the DOM, we can start testing the dynamic behaviors of our presentation components. First we’ll find a textbox, put a value in it, then click the submit button of the form and validate the text in the span.

const string DOMAIN = "http://snd.localhost";
using (WatiN.Core.IE ie = new WatiN.Core.IE(DOMAIN + "/wt.aspx"))
{
  ie.WaitForComplete();
  WatiN.Core.TextField tf = ie.TextField("username");
  tf.Value = "Al";
  WatiN.Core.Button b = ie.Button("btnSubmit");
  b.Click();
  ie.WaitForComplete();
  WatiN.Core.Span s = ie.Span("lblMsg");
  Assert.AreEqual("Please provide your password", s.InnerHtml);
}

Note the WaitForComplete call in there which blocks our test thread while IE does what it needs to. The span above may get populated either through javascript or a postback. It really doesn’t matter. But this also shows that we can use the exact same techniques to test both javascript logic and server side logic.

Footnote for Vista users: make sure the site you’re testing is in your trusted sites list so WatiN can interact with the IE window properly. Check out http://codebetter.com/blogs/james.kovacs/archive/2008/06/18/running-watin-tests-on-vista.aspx for details.

Automated Testing and Sitecore – Part 6

In this installment we’ll cover testing the static behaviors and presentation of our other Sitecore presentation components; layouts, sublayouts and renderings. These components are more difficult to test as they need to be hosted inside a Sitecore page to allow testing them. Because these components will be hosted in a page, we will be making web requests to the server to get the page output, then perform our assertions on the markup. This is almost as far from a unit test as you can get. But remember the name of this series. We’re interested in automated testing, not in unit testing.

Being that we’ll be making standard HTTP requests to the server, these tests do not have to be hosted inside the custom test runner, although they can if you wish. In fact, these tests can even be run from a different machine than that running Sitecore. There will still be some testing code required on the Sitecore server though, to handle the creation and cleanup of our test content.

Firstly we’ll need to create a test page which will host the Sitecore component under test. When we define presentation for an item, all we’re doing is editing the XML data in the “__renderings” field of the item. This is the field which Sitecore uses to work out the presentation for the item.

The easiest way to set presentation for an item programatically to include a presentation component to test is to set the presentation in the content editor, then turn on “raw values” and copy the XML data. We can then set the __renderings field the same as any other field.

Item home = Sitecore.Context.Database.GetItem("/sitecore/content/home");
TemplateItem template = Sitecore.Context.Database.Templates["sample/sample item"];
Item myItem = home.Add("My Item", template);
using (new EditContext(myItem))
{
  myItem["__renderings"] = "<?xml version=\"1.0\" encoding=\"utf-16\"?>
    <r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{BC88973F-D93B-47A4-85A0-D51034C457CF}">
    <r ds="" id="{7F2B13A6-8A2C-4A4F-93B2-582A043368CF}" par="" ph="page-columns" uid="{AF72D0AF-A0DC-4669-929D-278698200811}" />
    </d>
    <d id="{46D2F427-4CE5-4E1F-BA10-EF3636F43534}" l="" />
    </r>
}

The above code creates a new page and defines a simple presentation to use a specific layout and bind a single sublayout into the correct placeholder. If we look more closely at the XML of the renderings field we can start to see properties that we manipulate through the UI and would need to be adjusted to affectively perform testing against the component.

Let’s format that XML a bit nicer here and have a look at it.

<?xml version="1.0" encoding="utf-16"?>
<r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{BC88973F-D93B-47A4-85A0-D51034C457CF}">
    <r ds="" id="{7F2B13A6-8A2C-4A4F-93B2-582A043368CF}" par="" ph="page-columns" uid="{AF72D0AF-A0DC-4669-929D-278698200811}" />
  </d>
  <d id="{46D2F427-4CE5-4E1F-BA10-EF3636F43534}" l="" />
</r>

The “r” element is for rendering, either the whole rendering definition or a single presentation component such as a rendering or sublayout. The “d” element is for a device such as “print” or “default” which are both displayed above. In the device element, we can see the attribute “l” for the ID of the layout. In the rendering element, the “id” attribute indicates the ID of the presentation component, “ds” is the data source, “par” are the parameters and “ph” is the placeholder key to bind into.

If we use the UI in the content editor to set some parameters and a data source we can reveal the format of that data.

<r
  ds="/sitecore/content/Home/News"
  id="{7F2B13A6-8A2C-4A4F-93B2-582A043368CF}"
  par="param1=val1&amp;param2=val2"
  ph="page-columns"
  uid="{AF72D0AF-A0DC-4669-929D-278698200811}" />

So we can see the data source is the full sitecore path to the data source item and the parameters should be provided in a query string style. Once we understand the format here we can manipulate it programmatically to adjust these properties to allow more complete testing.

Now that we know how to set the presentation for an item, we can continue onto creating the test setup page. To create this test page from outside of Sitecore we can use the techniques I spoke about in part 3 of this series. To summarise: create a page inside Sitecore which when hit with the correct query string, will use the above code to create the required items and set their presentation.

Now we have a known content structure to test against as well as a page hosting the presentation component we wish to test. We will use standard WebRequests to pull the output of the page back over HTTP, then perform our assertions on the output.

using System.Net;
using System.IO;

namespace Testing
{
  public class PresTest
  {
    [Test]
    public void MyTest()
    {
      const string DOMAIN = "http://localhost";
      WebRequest request = WebRequest.Create(DOMAIN + "/test-page-1.aspx");
      WebResponse response = request.GetResponse();
      StreamReader reader = new StreamReader(response.GetResponseStream());
      string output = reader.ReadToEnd();
    }
  }
}

We can perform our assertions using any of the techniques described in part 5 of this series. These are string comparison, regular expressions and HtmlAgilityPack. The presentation components being tested in these scenarios are usually more complex than a few string literals placed into the output. They will contain structure so I would suggest HtmlAgilityPack would be the easiest to use to test these components.

HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(output);
XPathNavigator nav = doc.CreateNavigator();
XPathNodeIterator div = nav.Select("//div[@class='breadcrumb']");
Assert.AreEqual(1, div.Count);
div.MoveNext();

XPathNodeIterator liLinks = div.Select("ul/li/a");
Assert.AreEqual(5, liLinks.Count);
liLinks.MoveNext();

Of course, if we’re using HtmlAgilityPack we don’t have to worry about executing the web request ourselves. Instead we can use the HtmlWeb component provided by HtmlAgilityPack to do the loading for us.

HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(DOMAIN + "/test-page-1.aspx");

Note that we’re only testing the static output of the presentation components. We can’t interact with the markup or test the javascript. Those topics will be covered in the next part of this series.

Make Sitecore host WCF

For a long time I’ve been a huge fan of .net remoting. Once you got your head around the architecture, it is a really nice technology to use when you need to make calls across applications. In the past I have done this straight out of Sitecore with no problems.

The general scenario involves 2 Sitecore servers setup in a master / slave configuration where the master server is the content authoring platform and publishes content out to the slave server which only serves content out to the web. On any Sitecore server, we already know if we need to create items which are to be retained, we don’t create them in the web database as they will be lost on the next publish operation. We need to create the items in the master database. But given the master / slave dual server configuration, we can’t even do that; we need to create the item on a different server.

.net remoting has served me well to achieve this outcome. But recently I thought I should pull myself and my cross AppDomain communication techniques into 2008 and do the same with WCF. I also had an additional desire to make use of the new “OneWay” attribute which would allow me to “fire and forget” from the slave server so I didn’t have to wait for confirmation of item creation (again, it was a requirement to not wait for the response. You probably wouldn’t normally do that in your app).

So, with Visual Studio 2008 in hand, and a glimmer in my eye, I set off to create my WCF service to be hosted out of my Sitecore application. Through trial and error I stumbled upon the realization that I had to manually tell my ASP.NET application to host the WCF service. IIS 7 can host WCF services without invoking ASP.NET, so you have to manually do that. So the first thing I did after creating my plain-old-WCF service was to add the AspNetCompatibilityRequirements attribute to my service class and set the RequirementsMode to AspNetCompatibilityRequirementsMode.Required. That would tell my service that it required ASP.NET to run so I could get access to the Sitecore context. I also had to add the <serviceHostingEnvironment aspNetCompatibilityEnabled=”true”/> element to the <system.serviceModel> element in my web.config to tell the ASP.NET application (Sitecore) to host the service. I also added the <services> element to register my service as an endpoint.

OK, all going good. Let’s just hit the service manually to make sure the plumbing is right. So I fire up my browser, navigate to the svc file and get…

Download the svc file? Hang on ASP.NET, shouldn’t you process that file and tell me I need a client to invoke it? Why are you trying to pass me the file? I don’t want it, process it for me!

OK, so I tried many things and spent several hours / days on trying to get Sitecore to process the file instead of serve it. I tried creating a blank WCF project and hosting the service in ASP.NET: all good. I tried hosting the service in Sitecore but using the web.config from the previous blank WCF project: all good. Hmm. So Sitecore was doing something that was upsetting the processing of the svc file.

It got down to comparing the config files line by line. Tearing out the Sitecore stuff in the web.config to peel it back to the basics and try to pin point the element that was causing the problem.

It was late on a Thursday night, in Sydney domestic airport after my flight had been delayed that I finally stumbled upon the culprit element…the nexus httpModule. With that module commented out my service would be processed, not served. But with that module not active Sitecore would not work. What are you doing nexus? I dug around in the web.config for a while hoping to find another element which would allow me to control some of the processing nexus was doing. And I just so happened to find it. The FilterUrlExtensions processor of the preprocessRequest pipeline has a “Allowed extensions” parameter which controls which file extensions will be processed. I altered this element to include the svc extension as such:

<param desc="Allowed extensions (comma separated)">aspx, ashx, axd, asmx, svc</param>

And SUCCESS! Now when I hit the svc file in my browser the file is processed and I get that message telling me I need a WCF client to use the service.

Automated Testing and Sitecore – Part 5

In the last part of this series we started to write tests around code which uses the Sitecore API. These tests were run inside the NUnit custom test runner we wrote back in part 3 of this series. In this post, we’ll start testing our presentation components, starting with the easiest to test, which is a Sitecore WebControl. This is any class which inherits from Sitecore.Web.UI.WebControl.

The reason this type of control is so easy to test is because of the magical RenderAsText method which, as it’s name would suggest, renders the control and returns the string output. This means we don’t have to host the control in another container. We just have to instantiate an instance and start testing directly against it.

So let’s provide some context to this example. We’ll start by creating a simple Sitecore WebControl which just renders the current context item’s summary field in a div, with an optional CSS class. There will be a property to set the CSS class to apply. As always, we’ll develop this in a TDD style, so let’s start writing some tests!

We’ll create some test items in our test fixture setup method.

using NUnit.Framework;
using Sitecore.Collections;
using Sitecore.Data;
using Sitecore.Data.Items;

namespace Testing
{
  [TestFixture]
  public class WebControlTest
  {
    const string ITEM1_SUMMARY = "Item 1 has a summary";
    const string ITEM2_SUMMARY = "My summary";

    [TestFixtureSetUp]
    public void TestFixtureSetUp()
    {
      Database db = Sitecore.Context.Database;
      Item home = db.GetItem("/sitecore/content/home");
      using(new Sitecore.SecurityModel.SecurityDisabler())
      {
        Item item1 = home.Add("Item1", db.Templates["sample/sample item"]);
        using (new EditContext(item1))
        {
          item1["summary"] = ITEM1_SUMMARY;
        }

        Item item2 = home.Add("Item2", db.Templates["sample/sample item"]);
        using (new EditContext(item2))
        {
          item2["summary"] = ITEM2_SUMMARY;
        }
      }
    }

    [TestFixtureTearDown]
    public void TestFixtureTearDown()
    {
      Database db = Sitecore.Context.Database;
      Item home = db.GetItem("/sitecore/content/home");
      ChildList children = home.GetChildren();
      using (new Sitecore.SecurityModel.SecurityDisabler())
      {
        if (children["item1"] != null)
          children["item1"].Delete();
        if (children["item2"] != null)
          children["item2"].Delete();
      }
    }
  }
}

The complexity of the control will determine how you assert based on the string output of the control. For a simple control you’ll probably just assert on string comparisons or existence of substrings in the output. If the control is more complex, I would suggest HtmlAgilityPack. I’ll demonstrate both in these examples.

First, a simple string comparison test.

[Test]
public void Item1()
{
  Sitecore.Context.Item =
    Sitecore.Context.Database.GetItem("/sitecore/content/home/item1");
  MyControl control = new MyControl();
  string output = control.RenderAsText();
  Assert.AreEqual("<div>" + ITEM1_SUMMARY + "</div>", output);
}

We could also use a regular expression to account for whitespace either side of the text inside the div (new line, tabbing, etc). In this case our assertion would look like:

Assert.IsTrue(Regex.IsMatch(output, @"\s*<div>\s*" + ITEM1_SUMMARY + @"\s*</div>\s*"));

Now, we’ll adjust the CSS class property and use HtmlAgilityPack to assert the output structure.

[Test]
public void Item1WithClass()
{
  Sitecore.Context.Item = 
    Sitecore.Context.Database.GetItem("/sitecore/content/home/item1");
  MyControl control = new MyControl();
  control.DivClass = "myclass";
  string output = control.RenderAsText();

  HtmlDocument doc = new HtmlDocument();
  doc.LoadHtml(output);
  XPathNavigator nav = doc.CreateNavigator();

  XPathNodeIterator div = nav.Select("//div");
  Assert.AreEqual(1, div.Count);
  div.MoveNext();

  Assert.AreEqual("myclass", div.Current.GetAttribute("class", string.Empty));
  Assert.AreEqual(ITEM1_SUMMARY, div.Current.Value.Trim());
}

You might be looking at the code example above and saying to yourself “Well, string comparisons are heaps easier and require a lot less code”. The benefit of using HtmlAgilityPack is it’s resilience to whitespace. It you were doing string comparisons or regex you would have to take into account “<div>content</div>” and “<div> content </div>” and “\t\t\t<div>\r\ncontent\r\n</div>”. But HtmlAgilityPack hides you from these nasties and will handle small changes in your code better. For example, I may have generated the above just using string concatenation in my code:

string output = "<div>" + item["summary"] + "</div>";

Then I want to add some attributes and it starts to get ugly. So I do what I should have from the start, and I’ll start using the HtmlTextWriter and the output stacks, which will do this real easy for me:

output.AddAttribute(HtmlTextWriterAttribute.Class, DivClass);
output.RenderBeginTag(HtmlTextWriterTag.Div);
output.Write(item["summary"]);
output.RenderEndTag();

But the side effect of using the HtmlTextWriter is that it will also indent your markup properly. So output of:

<div>Summary</div>

Just changed to:

<div>
  Summary
</div>

HtmlAgilityPack will handle that change for you, but string comparisons will require tweaks.

We also need to write some more tests to cover other cases such as the item not having a summary, the item being null, the summary being too long, too short, text too long, too short, etc, etc.

After we’ve written our tests, it’s time to write our control.

public class MyControl : Sitecore.Web.UI.WebControl
{
  public string DivClass
  {
    get;
    set;
  }

  protected override void DoRender(HtmlTextWriter output)
  {
    if (!string.IsNullOrEmpty(DivClass))
      output.AddAttribute(HtmlTextWriterAttribute.Class, DivClass);

    output.RenderBeginTag(HtmlTextWriterTag.Div);
    output.Write(Sitecore.Context.Item["summary"]);
    output.RenderEndTag();
  }
}

And now we have a fully tested Sitecore WebControl.

In the next installment of this series we’ll cover how to test our static output presentation components such as layouts, sublayouts and renderings.

And Another One for Shared Source

Last night I uploaded the Quicklaunch toolbar which I created a few months ago into the Shared Source section of SDN. So hopefully more people will now be able to find it. You can check it out by following http://trac.sitecore.net/QuickLaunchToolbar/wiki. It is also listed on the SDN at http://sdn5.sitecore.net/Resources/Shared%20Source/Shell%20Extensions/Quick%20Launch%20Toolbar.aspx. You’ll find release 1.0 which I initially released on my blog up on the trac pages as well. Being that it’s shared source, all the source code and items required to rebuild the package are included.

I also have more plans for Shared Source, but I’ll keep those under wraps until I have something a bit more solid to announce.

Automated Testing and Sitecore – Part 4

So far in this series we have covered some of the theories and techniques required to automate testing of Sitecore components, creating a custom NUnit test runner which runs inside the Sitecore website and exposes the Sitecore context to the running tests and how to programmatically setup our test environment for repeatable testing against know content structures. In this post we are going to start testing some code.

The tests we’ll cover in this post will be any code written against the Sitecore API. To use the Sitecore API you generally require a Sitecore context. So the tests against this code will be run inside the custom NUnit test runner we created back in part 2 of this series.

The code I am going to test will be a simple content utility method which returns an item’s summary. Our item has a “summary” field, but if that is empty the method should return the first 100 characters of the “text” field. If there are more than 100 characters in the text field the method should append an ellipsis (…) to the end of the summary.

Let’s approach this in a TDD way. So first I’ll write some tests. The first case I’ll test will be with an appropriate item that contains both summary and the text fields. I created the test items which these tests use during my TestFixtureSetUp method as detailed in part 3 of this series. Note: the below code is NUnit tests as the custom test runner runs NUnit tests.

[Test]
public void WithSummaryPopulated()
{
  Item testItem = Sitecore.Context.Database.GetItem
    ("/sitecore/content/home/With Summary");
  Assert.AreEqual("The item summary", Util.GetSummary(testItem));
}

Next I’ll test with an appropriate item where the summary field is empty.

[Test]
public void WithoutSummaryPopulated()
{
  Item testItem = Sitecore.Context.Database.GetItem
    ("/sitecore/content/home/Without Summary");
  Assert.AreEqual("This is some text without summary", Util.GetSummary(testItem));
}

Now for an item which doesn’t contain any of the required fields.

[Test]
public void NoFields()
{
  Item testItem = Sitecore.Context.Database.GetItem
    ("/sitecore/content/home/No Fields");
  Assert.AreEqual(string.Empty, Util.GetSummary(testItem));
}

There’s a few basic tests. I would also want to provide tests for items without a summary and less than 100 characters in text, and above 100 characters in text, to test the cases for the ellipsis. You need to provide enough tests to ensure you have tested for known good and bad cases.

With the tests in place I can now start writing my code. If I try to compile now I’ll get complication errors due to the util method above not existing. So I’ll start by writing the stub for the method. After that if I run my tests they should fail as my method doesn’t yet do anything. And now after we’ve witnessed our tests failing, to prove that they can fail and they are testing against the cases we desire, we can fill in the body of the util method.

public static class Util
{
  public static string GetSummary(Item item)
  {
    if (item["summary"] != string.Empty)
      return item["summary"];
    else if (item["text"].Length > 100)
      return item["text"].Substring(0, 100) + "...";
    else
      return item["text"];
  }
}

From the examples given above, the first thing the detractors of TDD will point out is that we have written more test code than code under test. That may be the case in these examples, but note that a lot of the testing code is repetitive, and I actually copied and pasted much of that. But the clipboard does not make for a maintainable solution. So let’s refactor our tests. I’ll put out a method which encapsulates the common code.

private void TestItem(string path, string expected)
{
  Item testItem = Sitecore.Context.Database.GetItem(path);
  Assert.AreEqual(expected, Util.GetSummary(testItem));
}

[Test]
public void WithSummaryPopulated()
{
  TestItem("/sitecore/content/home/With Summary", "The item summary");  }

[Test]
public void WithoutSummaryPopulated()
{
  TestItem("/sitecore/content/home/Without Summary", "This is some text without summary");
}

[Test]
public void NoFields()
{
  TestItem("/sitecore/content/home/No Fields", string.Empty);
}

Wow! I saved 3 lines of code! Well, in the above examples I did. But applying this technique to real code I end up saving many more lines. The above example are quite academic and contrived.

This is actually the wrong time to be refactoring. When I started to write the second test and noticed it was quite like the first is when I should have started to refactor and saved the effort of all that copy-pasting and rework of the other test methods.

Using these techniques you can write tests that exercise your code which uses the Sitecore API. The main piece here is that your test is run within the custom NUnit test runner to provide the Sitecore Context. In the next post in this series we’ll start testing our controls and presentation components.