What to Test?


March 1, 2015 by Alistair Deneys

Recently I was having a discussion with someone around how to test certain scenarios in Sitecore. The scenario in question involved testing a sublayout on a page to ensure it was working properly, and the developer was thinking of using something like Selenium to do this.

This raised a flag with me. Although there are scenarios in which an automated web tester is the correct approach for this situation, we need to be sure of what we’re trying to test, and what kind of test we’re working with.

OK, here we go, opening the can of worms.

I often talk about testing in Sitecore in a general sense, not bothering with semantics of whether it’s a unit test or an integration
test. In any robust testing strategy, you’re going to need a mix of unit tests, to ensure all the small pieces work correctly, and also a
mix of integration tests, to ensure all the pieces work together. And for determining which approach you should take you kind of need to
answer that question.

Note to readers: please do not point out in the comments that all tests in Sitecore are actually integration tests. We’re using the spirit of a unit test here; testing a small bit of code, no matter how much baggage it brings to a test.

Are you writing an integration test (the page works as a whole) or a unit test (I’m showing the correct data on my component)? If you’re a developer you’ll likely be wanting a unit test. Trying to test a sublayout as a unit test is not the most straight forward thing. And it often helps to ask the simple question: “What am I trying to test?”. You’ll likely find a better way to test the functionality you’re trying to test.

So let’s consider a scenario. We have a sublayout that lists all child items that have a certain value in a field (it could be a tag or a checkbox). What do we want to test on this component? Do we want to test the structure of the HTML? Ensure we’re using UL/LI to list the entries? Probably not. The tag we use to output the list has no bearing on the logic that we want to test. The logic is that the correct items are shown.

This is where a good design helps achieve testing goals. In a badly designed sublayout the code that retrieves the items to show is written directly in the code behind of the sublayout. This makes the code hard to reuse and tightly couples the item location logic into the listing output. What if we wanted to change the logic to locate items based on different criteria? What if we wanted to use that same logic in another part of the application? It also makes it difficult to unit test as you’d need to instantiate the sublayout or have some other way of retrieving the markup produced by the sublayout, such as making an HTTP request for a page which has the component on it, then parsing the HTML to locate the items displayed.

Here’s an example of what this kind of sublayout might look like.


<%@ Control Language="c#" AutoEventWireup="true" Inherits="MyControls.ChildMenu" %>
<%@ Import Namespace="Sitecore.Data.Items" %>
<%@ Import Namespace="Sitecore.Links" %>
<div class="links">
  <asp:Repeater runat="server" id="Links">
        <div class="link">
            <a href="<%# LinkManager.GetItemUrl(Container.DataItem as Item) %>">
                <%# Eval("Name") %>


using System;
using System.Collections.Generic;
using System.Web.UI;
using System.Web.UI.WebControls;
using Sitecore.Data.Items;
using Sitecore.Web.UI.WebControls;

namespace MyControls
    public class ChildMenu : UserControl
        protected Repeater Links;

        public void Page_Load(object sender, EventArgs args)
            var ds = (Parent as Sublayout).DataSource;
            var dsItem = Sitecore.Context.Database.GetItem(ds);
            dsItem = dsItem ?? Sitecore.Context.Item;

            // This is an example of bad code. This logic belongs in a domain API        
            var items = new List<Item>();
            var children = dsItem.GetChildren();

            foreach(Item item in children)
                // filtering logic
                if(item["Primary Menu"] == "1")

            Links.DataSource = items;

Let’s consider the sub-optimal approach for testing the sublayout directly:

  • Execute test setup
    • Create an item with the sublayout to test in presentation
    • Create child items under the test item to show on the sublayout
    • Change field values of the child items so some should show and some should not (to test that the filtering is working properly)
  • Test the sublayout
    • From my test code make an HTTP request for the page
    • Use HtmlAgilityPack (which is included in Sitecore) to parse the returned HTML of the page
    • Use XPath to navigate through the parsed HTML locating all the links
    • Check the name of each link against what you know you created during test setup
  • Clean up after test
    • Delete the item with the sublayout to remove all test items

That’s a lot of work to validate some logic. Plus, it’s quite brittle and prone to errors, not to mention it will probably take longer to run this test than alternatives.

Just a quick side note on test performance. Typically I’m not too worried about how fast a test runs. If a single test runs in 250ms or 1 second it doesn’t bother me. But you’re not going to have just a single test. Consider a project where you have a good amount of test, like 600. If your tests are performing poorly then other developers are less likely to run the test suite as often as they should.

OK, back to it. The brittleness of this scenario. In the above scenario we’re pulling back the HTML and parsing it. If the structure of the HTML changes, that may change the XPath you need to use to find the elements you’re looking for. Also, we’re checking either the URL or title of the links. What if the structure of the site or location where you create pages under test changes? That would change the URL. Or perhaps the title of the test items changes.

In this case a better design is to extract the item location logic to a separate domain specific API and test that API, not the sublayout. The domain specific API you extract will end up being a method or perhaps a class. Whichever it is, it is much easier to instantiate that code inside a unit test. This makes the testing requirements much simpler than if we wanted to test the sublayout itself.

Here’s an example of extracting the item location logic from the previous scenario.


using System.Collections.Generic;
using Sitecore.Data.Items;

namespace ItemLocators
    public class ChildItemLocator
        public string IncludeFieldName { get; set;}

        public IEnumerable<Item> LocateItems(Item root)
            var children = root.GetChildren();

            foreach(Item item in children)
                // filtering logic
                if(item[IncludeFieldName] == "1")
                    yield return item;

And how you would use that in your previous code-behind:

public void Page_Load(object sender, EventArgs args)
    var ds = (Parent as Sublayout).DataSource;
    var dsItem = Sitecore.Context.Database.GetItem(ds);
    dsItem = dsItem ?? Sitecore.Context.Item;

    var locator = new ItemLocators.ChildItemLocator
        IncludeFieldName = "Primary Menu"

    Links.DataSource = locator.LocateItems(dsItem);

Let’s consider the optimal approach for testing just the logic using the example class above:

  • Execute test setup
    • Create a sub-tree of test items in the content tree to test against.
    • Change field values of the child items so some should show and some should not (to test that the filtering is working properly). This could be combined with the previous step if we’re creating the sub-tree from item XML.
  • Test the logic
    • Within the unit test, instantiate the locator class
    • Set any required properties
    • Call the ‘LocateItems’ method passing in the root of the sub-tree created above
    • Validate the ID of each returned item whether it’s expected or not
  • Clean up after test
    • Delete the test items

This scenario is much cleaner, faster and prone to less errors than the previous example.

The story for MVC is a little better than that of webforms and sublayouts. Due to the structure of MVC there is already a separation between the view and the model where the model would hold the links to be rendered. So for testing, we could just instantiate the model and test that, or the code we use to populate the model, such as the controller in the case of a controller rendering. Although this is a better situation than what we started with (it’s more easily testable), it’s still not ideal. If we wanted to reuse the item location logic on another part of the application we’d end up dragging along a bunch of redundant stuff. We still benefit from extracting the logic to a domain specific API.

I hope this advice helps you craft better, more stable tests.


One thought on “What to Test?

  1. mawkstiles says:

    Yeah, as I’m learning, anything that might be in a sublayout code-behind should probably look exactly like what would be in a test. All other code would be relegated to library functions that are testable. This makes it simpler to test the sublayout by only needing to check if the page is loading (through HttpStatus 200 check) and showing results (through html checks).

    Thanks btw. The TestStar unit test side was originally pulled from your test runner.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


The views expressed on this blog are solely my own and do not necessarily reflect the views of my employer.
%d bloggers like this: