Coffee => Coder => Code

Dynamic Roles in Sitecore

Alistair Deneys — Wed, 09 May 2012 12:24:44 +0000

So what is a dynamic role? A dynamic role is one where membership is dynamically determined instead of statically assigned. The normal Sitecore roles are statically assigned to a user. They never change. Sitecore also contain 2 dynamic roles: the Everyone role and the Creator-Owner role. Inclusion in these roles is determined dynamically. OK, the Everyone role may not be quite so dynamic, but the Creator-Owner role is. Membership in this role is determined by the value of the Owner field. You don’t assign users to the role.

Adding additional dynamic roles to Sitecore is quite easy. First we need the dynamic role to appear in the Security assignment dialogs so we can assign security permissions to the role. Then we need to assess whether a given user belongs to the dynamic role based on whatever logic we want to assess.

There are 2 classes in Sitecore that handle roles. One is the Sitecore ASP.NET role provider and the other is the roles in roles provider. The roles in roles provider is kind of a wrapper around the roles provider, extending it’s capabilities to support assigning roles within roles. It also already handles the existing Sitecore dynamic roles. Support for dynamic roles can be seen in the roles in roles provider in the fact it accepts a parameter to the GetAllRoles() method to determine if “system” (read dynamic) roles should be included in the results.

You don’t want to include the dynamic roles in all calls to the GetAllRoles() method as you don’t want to allow the user to statically assign one of the dynamic roles. If you opened the user properties dialog for an existing user and used the “Edit User Roles” dialog to assign roles to the user, you would not find the dynamic Everyone and Creator-Owner roles, but when using the “Security Settings” dialog to assign security to an item these roles do appear.

The dynamic roles I’m going to implement in this example come from a request I had from a client recently. The client wanted the ability to assign security to users who were not part of another role. In a default Sitecore site without customisation, an administrator would either need to assign security to all roles other than the role to be excluded, or create a role to group the other roles (roles in roles) and assign security to this role. This might be OK if you have a handful of roles, but if you have 50, this could become arduous. Not to mention the fact that every time a role was added you’d need to remember to update either security or the role membership for the role group.

In any case, this provided an interesting demonstration scenario for me and a solid example to use here .

So an appropriate class to override and extend with our dynamic roles is the roles in roles provider.

using System.Collections.Generic; 
using Sitecore.Security.Accounts;

namespace CustomSitecoreSecurity 
{ 
  public class DynamicRolesProvider: SqlServerRolesInRolesProvider 
  { 
  } 
}

And to have Sitecore use our custom roles in roles provider we need to update configuration to use our above class. Place the following in a configuration patch file in App_Config\Include.

 
   
     
       
         
          
             CustomSitecoreSecurity.DynamicRolesProvider, 
             CustomSitecoreSecurity

Now to have our dynamic role names returned so we can assign security to them. We need to override the GetAllRoles() method to include our dynamic roles when includeSystemRoles is true. To implement the scenario I gave above, we want a dynamic role returned for every normal role, prefixed with “not”. So in addition to sitecore\Author we want a dynamic role named sitecore\not Author.

public override IEnumerable GetAllRoles(bool includeSystemRoles) 
{ 
  var baseRoles = base.GetAllRoles(includeSystemRoles); 
  foreach (var role in baseRoles) 
  { 
    yield return role;

    if (includeSystemRoles) 
    { 
      var parts = role.Name.Split('\\'); 
      if (parts.Length == 2) 
      { 
        var domain = parts[0]; 
        var name = parts[1]; 
        yield return Role.FromName(domain + "\\not " + name); 
      } 
    } 
  } 
}

In the above method we call the base class implementation to get the roles, then iterate each role and yield return the role name plus the inverse dynamic role name (“not” version) if system roles are to be included.

The RolesInRolesProvider also contains a method to return just the system roles which is used by some parts of the Sitecore UI. The below method overrides the base implementation to add the inverse roles.

public override IEnumerable GetSystemRoles() 
{ 
  var systemRoles = base.GetSystemRoles(); 
  foreach (var role in systemRoles) 
    yield return role;

  // Get 'not' roles 
  var roles = GetAllRoles(false); 
  foreach (var role in roles) 
  { 
    var parts = role.Name.Split('\\'); 
    if (parts.Length == 2) 
    { 
      var domain = parts[0]; 
      var name = parts[1]; 
      yield return Role.FromName(domain + "\\not " + name); 
    } 
  } 
}

The last method to override is the IsUserInRole() method which is where we implement our logic to determine if the user is part of the dynamic role.

public override bool IsUserInRole(User user, Role targetRole, 
  bool includeIndirectMemberships) 
{ 
  var parts = targetRole.Name.Split('\\'); 
  var domain = parts[0]; 
  var name = parts[1];

  if (name.StartsWith("not ")) 
  { 
    // Check if user belongs to normal role 
    var role = Role.FromName(domain + "\\" + name.Replace("not ", string.Empty)); 
    var result = base.IsUserInRole(user, role, includeIndirectMemberships); 
    return !result; 
  }

  return base.IsUserInRole(user, targetRole, includeIndirectMemberships); 
}

In the above code we first determine if the role being assessed is one of the dynamic inverse roles by checking if the role name starts with “not “. Then if the role being checked for is a dynamic inverse role get the result for the non-inverse role (the real role the inverse role corresponds to) and inverse the result.

With the above dynamic role provider compiled and deployed, we can now assign security to a dynamic role and user security will be influenced by their dynamic membership within the role.

To test the provider is working properly let’s deny read access to a page for the sitecore\not Author role. This would result in any user not in the sitecore\Author role not being able to see the page.

Note in the above screenshot how each role has a corresponding inverse “not” role? The “not” roles are our dynamic roles returned by our dynamic roles provider.

You can see in the above screenshot I’ve denied read access to the sitecore\not Author which with our dynamic roles provider would have the affect of denying read permissions to anyone without the sitecore\Author role.

To test this out, we just need a user account that doesn’t have the sitecore\Author role assigned. That user won’t be able to see the above item.

And that’s it. The above customisation can be used to add any dynamic roles you like to Sitecore.

Mocking Sitecore

Alistair Deneys — Fri, 13 Apr 2012 12:19:07 +0000

Whenever I talk about unit testing and Sitecore, someone inevitably will ask about the practise of mocking. Mocking involves creating a simple object to replace a dependency in your unit tests so your test code doesn’t have to invoke that dependency. The issue I’ve had with mocking is that you have to deeply understand the object you’re mocking or your mocks won’t be a reflection of the dependency being mocked and your test will be useless.

Recently I was prompted to dive head first into mocks and investigate how I could mock Sitecore. This prompt was the first official Sitecore book “Professional Sitecore Development” which I can finally and proudly say I was a part of. After Dreamcore last year John West (CTO of Sitecore and author of the book) contacted me and asked if I could write a chapter for the book on testing, which I did.

My initial draft of the chapter (in fact, right up until the final submission to John) didn’t include mocking, but a reviewer of the chapter (Fellow Sitecore MVP Klaus Petersen) suggested I include mocking. Ultimately I thought “who am I to hold back this potentially powerful technique, just because I have concerns about it’s application?”. I mean, I wrote Revolver which used improperly could probably do a fair amount of damage to any Sitecore install (working directly on raw field values which could cause the UI tools to break). The reason I named Revolver as such was because it was dangerous. “Here’s Revolver, now don’t shoot yourself in the foot”. The same can be said about C and C++. You wanna cast that object to an incompatible type? Go ahead!

So I came to the conclusion that I should stop ignoring this technique and dive head first into it.

Now, this content didn’t end up making it into the book. I got it to John a little too late and there wasn’t enough time for it to go through the appropriate approval and editorial processes. So below is a slightly altered version of the content that didn’t make it in time for the book.

Introduction to Mocking

Mocking frameworks such as Moq (http://code.google.com/p/moq/) and Rhino Mocks (http://hibernatingrhinos.com/open-source/rhino-mocks ) can be used to create fake versions of objects during testing and orchestrated to behave as required to allow the test to run. The more popular open source mocking frameworks allow creating instances of an interface or class and providing an implementation for any virtual methods. The Moq framework will be used as the mocking framework in this post.

The steps required to create a mock object using Moq are as follows:

Create a new instance of the Mock object passing it the type to mock.
Define how each call of the object (method call, property access) should behave and any data returned.
Retrieve the mocked object from the Mock instance and use it in the test code.

The following code sample shows creating a mock of a ficticous IData interface.

// Create an instance of the Mock class to create a mocked IData instance 
var mock = new Mock();

// Set how the mocked object should behave 
// Setup mocked object to return 1 when "code1" is passed to the GetCount()
  method 
mock.Setup(m => m.GetCount("code1")).Returns(1);

// Setup mocked object to return 2 when "code2" is passed to the GetCount()
  method 
mock.Setup(m => m.GetCount("code2")).Returns(2);

// Setup mocked object to return "data" when the name property is retrieved 
mock.SetupProperty(m => m.Name, "data");

// Retrieve the mocked object 
var dataObject = mock.Object;

// Use the mocked object in tests 
Assert.AreEqual(1, dataObject.GetCount("code1"));
Assert.AreEqual("data", dataObject.Name);

In the above code an instance of the Mock class is first created, passing the type that must be mocked. The next 2 lines tell the mock object what to return when the GetCount() method is called with various different parameters. Then the Name property is setup to return the value “data” when retrieved. Finally the configured object is retrieved from the Mock and then used in the assertions.

A Word of Warning

When you mock a component, you are codifying many assumptions about how that component works and your understanding of how that component works. In a bespoke developed system where mocking is used to allow components to be developed ahead of their dependent components the mocks would typically be developed against a specification or common understanding of how the component being mocked should behave. And if at the end of development the component being mocked does not behave as expected it can be modified until it does. This is not the case with an external system such as Sitecore.

When you mock Sitecore components, you’re codifying those same assumptions about how the component works and your understanding of it, but if your assumptions are wrong you have no control to change Sitecore and make it work the way you thought it should. Your custom components have now been tested under false assumptions and may be harbouring bugs, which are hidden behind the incorrect mocks.

Consider Sitecore query. What would the following code return?

var items = Sitecore.Context.Item.Axes.SelectItems("../my-item");

Many developers would say the sibling of the current context item named “my-item” would be returned in the above code. Actually, the above code will throw a Sitecore.Data.Query.ParseException exception because the dash in the name wasn’t escaped. The query parsing engine is interpreting the dash as an operator instead of a name character.

This is the danger of mocking an external system. If you’re not aware of every facet of the component being mocked you risk creating mock objects which aren’t true representations of the components they mock, and the components being tested using these mocks may still contain bugs.

Mocking Sitecore Objects

Moq (and other mocking frameworks) can only create mock objects for interfaces and virtual methods on concrete classes. Unfortunately Sitecore version 6.5 and before doesn’t make use of interfaces or virtual methods for most of the objects which would need to be mocked during a testing scenario. (Sitecore 7 adds many interfaces that will greatly help in this regard.)

One technique which facilitates this kind of mocking is to update the code under test to use a different class which can be mocked, rather than the Sitecore types directly. This class can expose the same API as the Sitecore object, and pass the calls through to a wrapped Sitecore object. This kind of technique is referred to as the wrapper or adapter pattern. The adapter pattern is a software design pattern which allows wrapping one class to allow it to be compatible with another when it normally wouldn’t be. In this case the Sitecore API doesn’t need to change though the methods and properties being used in the test need to be virtual so the mocking framework can orchestrate a mocked version of the class. The methods and properties of the adapter (which copies the same API as the Sitecore component) would simply call the corresponding method or property on the wrapped Sitecore object.

The following utility method accesses fields of a Sitecore item and returns the data required. This method is going to be tested using mocks.

public static string GetTitle(Item item) 
{ 
  var fieldTitle = item["title"]; 

  if (string.IsNullOrEmpty(fieldTitle)) 
    fieldTitle = item.Name;

  return fieldTitle; 
}

To easily test the above code a mocked version of Sitecore.Data.Items.Item would be passed into the method, but because Sitecore.Data.Items.Item doesn’t implement any interfaces and the field properties aren’t virtual, Moq cannot create a mocked object for the parameter.

Instead, an adapter can be created to wrap the Sitecore.Data.Items.Item and make the methods virtual to allow Moq to create a mocked object for the tests. The following class shows an adapter which could be used in the previous utility method in place of the Sitecore.Data.Items.Item instance.

public class ItemAdapter 
{ 
  private Sitecore.Data.Items.Item m_item = null;

  public ItemAdapter() 
  { 
  }

  public ItemAdapter(Sitecore.Data.Items.Item item) 
  { 
    m_item = item; 
  }

  public virtual string this[string name] 
  { 
    get 
    { 
      return m_item[name]; 
    } 
  }

  public virtual string Name 
  { 
    get 
    { 
      return m_item.Name; 
    } 
  } 
}

In the above class, only the required properties have been implemented, after all, if a method or property isn’t required then there’s no need to add it to the adapter. Note also how each property is virtual which will allow Moq to create a mock object of this type for the tests. The parameterless constructor is also required to allow Moq to create instances of the adapter.

This adapter can now be used in place of a Sitecore.Data.Items.Item in the utility method.

public static string GetTitle(ItemAdapter item) 
{
  var fieldTitle = item["title"];

  if (string.IsNullOrEmpty(fieldTitle)) 
    fieldTitle = item.Name;

  return fieldTitle; 
}

Comparing the above implementation of the GetTitle() method using the adapter to the previous implementation using the concrete Sitecore.Data.Items.Item class, the only thing that needs to change is the type of the parameter. This is because the adapter is doing it’s job, by copying the interface of the class it acts as an adapter for. However the calling code needs to be updated to pass an instance of the adapter rather than a Sitecore.Data.Items.Item.

var title = ItemUtil.GetTitle(new ItemAdapter(item));

The GetTitle() method can now be tested using mock objects instead of real Sitecore objects.

Before Moq can be used in the test project, it must be added.

Download the latest release of Moq from http://code.google.com/p/moq/downloads/list
Extract the Moq.dll assembly for the .net version being used from the downloaded archive and place the assembly in the lib folder of the test project.
Add a reference to the Moq.dll assembly to the test project.

Now tests can be written for the utility method making use of mocks.

[NUnit.Framework.Test] 
public void GetTitle() 
{ 
  var mock = new Mock();
  mock.SetupGet(m => m["title"]).Returns("the title");

  var title = ItemUtil.GetTitle(mock.Object); 
  Assert.AreEqual("the title", title); 
}

In the above test method a mock is created for the ItemAdapter class. The mock is then configured to return a specific field value when the title field is accessed. The mocked object is then used in the method under test and the outcome asserted.

The above test will now run completely in memory without the need for Sitecore’s configuration or to connect to a physical database engine. Tests using the above technique will as a result run much faster than the techniques I’ve shown previously:

The downfall is you have to wrap all calls to the Sitecore API.

However, the adapter classes created can be written once and reused on other projects. They only need to be updated when the Sitecore API is updated.

Isolating Sitecore

If one can accept the risk of mocking Sitecore components, the above mocking technique only has one drawback; it requires all calls to the Sitecore API to be wrapped in adapter objects to allow a mocking framework to mock the adapter. There is another class of mocking frameworks which allow mocking all objects, not just interfaces or virtual methods. These frameworks are called isolation frameworks.

Isolation frameworks typically tap into core pieces of the CLR (Common Language Runtime) to orchestrate and fake any object. Two popular isolation frameworks are Moles from Microsoft Research (http://research.microsoft.com/en-us/projects/moles/) and Typemock Isolator (http://www.typemock.com/isolator-product-page).

I tried my hand at using Moles but ran into issues trying to mock field access on a Sitecore item. I was quite happy to see the Compiled Domain Model project from the Sitecore shared source library was using Moles in some tests. The Compiled Domain Model project uses Moles to mock Items but it doesn’t look like the tests access field values, so Moles works fine for these tests.

Typemock Isolator can be used in much the same way as Moq above, but it can mock a Sitecore.Data.Items.Item instance and not just interfaces and virtual methods.

To add Typemock Isolator to the test project:

Download the latest version of Typemock Isolator .NET from http://www.typemock.com/download/.
Install Typemock Isolator following the installation wizard of the MSI downloaded above.
Add references to the test project to the Typemock Isolator C# APIs and Typemock Isolator core DLL assemblies. These assemblies are installed to the GAC (Global Assembly Cache) by the installer and so appear in the .NET tab of the Add Reference dialog.

The following test makes use of Typemock Isolator allowing the GetTitle() method to use Sitecore.Data.Items.Item directly again without the need for an adapter.

[NUnit.Framework.Test, TypeMock.ArrangeActAssert.Isolated] 
public void GetTitle() 
{ 
  var item = TypeMock.ArrangeActAssert.Isolate.Fake.Instance(); 
  TypeMock.ArrangeActAssert.Isolate.WhenCalled(() => item["title"])
    .WillReturn("the title");

  var title = Web.ItemUtil.GetTitle(item); 
  Assert.AreEqual("the title", title); 
}

The above code first creates a mock that mimics the interface of Sitecore.Data.Items.Item. The mock is then configured to return a specific value when the title field is accessed. The mock is then passed to the original version of the GetTitle() method which accepts an instance of Sitecore.Data.Items.Item and the outcomes asserted.

The biggest benefit of using Typemock Isolator is that your project code doesn’t need to change to allow the tests to run as it did when using a mocking framework. There’s no need to use an adapter to wrap the Sitecore objects.

Conclusion

So there’s a few techniques for mocking with Sitecore objects. Now I’ve given you the gun, please be careful with it.

XSLT Inheritance

Alistair Deneys — Sun, 11 Mar 2012 02:27:24 +0000

I like XSLT. Although if you’ve ever visited my codeflood website and had a look at my SashimiCMS you’d have guessed that. SashimiCMS is a CMS I created many years ago, all based on XML and XSLT. In fact, I currently use that CMS to manage codeflood.

XSLT is a great tool for working with XML and transforming it. So it’s always a shame when I see Sitecore projects that avoid XSLT (renderings). XSLT can be quite foreign for C# developers. This is because unlike C# which is considered procedural (do this, then to this, then do this), XSLT is declarative (when this happens do this, when this happens do this, when this happens do this). It’s a very different mindset to get into. But once you tap it correctly, it’s quite powerful, for the right problem. XSLT is a very specific language and was never designed to do all things, which is why at times we need to drop out to extensions (C# methods) to get stuff done.

What it comes down to is, use the right tool for the right job. Use XSLTs for their strength, which is processing XML. Yes you could do it in C# code, but it may be more complicated than if you take the pill and use XSLT instead. Especially when working on hierarchical data.

This post is going to cover a few points to writing “good” XSLT stylesheets and make the maintainable. Let’s first take a look at how we can reuse common xsl:templates across multiple files. And no, copy/paste inheritance is not the answer.

xsl:import and xsl:include allow you to extract common templates to a separate file and reuse them. So what’s the difference between xsl:import and xsl:include? Just the importance of the templates that are brought in from the external file. When you use xsl:include, it’s as if the templates from the external file existed within the current file, so the included templates have the same importance as the templates of the current file. When you use xsl:import, the imported templates have less importance than the current file’s templates. If the imported file had a template the same as the current file, the current file template would be used, but if xsl:include was used the transform engine would throw an error that it saw two templates that match with the same importance.

Let’s take a look at an example of extracting common templates.

Let’s create a new XSLT file called DataUtil.xslt that contains the following code:

The above code contains a single xsl:template used to render the title of a Sitecore item. If the title field is empty it will fallback to use the item’s name instead.

Note how I’ve missed all that normal Sitecore boilerplate? That’s because this file is not a Sitecore rendering. I don’t intend to use it directly but instead it will be included in other renderings.

Now let’s create a new Sitecore rendering which includes the DataUtil.xslt common templates.

By extracting common xsl:templates such as the one above, we’ve allowed those templates to be reused in any other xsl:template and gained all the benefits of sharing a common piece of code between multiple files:

Code using the common templates will be more consistent as they don’t have to rewrite the same code.
Maintenance of the code is much easier as it only exists in a single file.

The above code is only one example of a common xsl:template. I bet you could come up with a number of these just by looking through your current project, such as creating a link to an item (output the item title if supplied, otherwise just output the item name).

On the projects I do see using XSLT in Sitecore, it’s not often I see a good structure or technical design for the XSLT. How often have you seen this kind of rendering in Sitecore?

Sitecore provides a nice starting point for renderings but many developers don’t extend beyond that very well.

A good XSLT will make appropriate use of xsl:templates to process different elements of input, in the same fashion as a good C# class will make appropriate use of methods to break apart complex processing tasks.

For example, let’s consider the following XSLT which is used to generate a simple menu from the Sitecore content tree (single level).

This code could be made better by splitting out the part that renders the link, from the part that walks the content tree.

In the above code we’ve split the link generation out to it’s own xsl:template. This makes the code easier to understand, but also makes it easier to reuse and override.

Let’s say we now have a new requirement on a project for a new kind of menu control. It should traverse the content tree in the same way as the above menu, but instead of outputting simple links it should wrap each link in a div and a span (to allow appropriate styling through CSS). There are two approaches you could take to satisfy this new requirement while leveraging the existing control (and hence, reducing the amount of effort required to deliver the requirement).

Firstly, let’s add some parameters to the XSLT to allow specifying whether the link should be wrapped and by what.

And now to update the renderLink xsl:template to output the required tags when the parameters are used.

Looks kind of messy right? And I’m not too sure about manually generating tags through textual output (rather than just specifying the tag directly).

A cleaner approach is to create a new rendering, but make use of the existing XSLT and override the xsl:templates that need to change.

Note above how we’ve used xsl:import so the templates defined in the XSLT will take priority over those in the imported file. And because we’re making use of the Simple Menu.xslt from above, we don’t need to specify templates that don’t change in this file, such as the default Sitecore provided main mode template or the content tree traversal template.

The above code does the same as the first revision, but I think it looks much cleaner. By splitting the piece of code that generates the link out to a separate template it’s much easier to override the renderLink template and adjust the output per link. This is the same kind of benefit we gain when using inheritance in C# to override specific methods.

I hope this post will make you think carefully about the structure of your xsl:templates next time you’re creating an XSLT. And if you don’t use renderings in your Sitecore projects I hope this motivates you to give them a go.

Tracking Local Search with DMS

Alistair Deneys — Thu, 09 Feb 2012 11:07:24 +0000

Sitecore’s Digital Marketing Suite (DMS) comes with some handy reports out of the box for analysing searches your users are performing on your Sitecore site. There’s both a high level report in the executive dashboard for site wide searches and also item reports for checking search terms leading to an item and searches performed on an item.

All these reports assist in helping you work out what people are looking for on your site and also when they can’t quite find the information they’re after when the arrive at an item.

But Sitecore doesn’t magically know what actions on your site constitute a user search. So we have to register that action ourselves. The way we do this is by registering a page event for the search page event which ships with DMS and the search reports are written to report on.

I remember from the Online Marketing Suite (OMS) days there used to be a helper method for doing this. OMS contains an extension method for the AnalyticsPage class in the Sitecore.Analytics.Extensions.AnalyticsPageExtensions namespace called Search. We can call it by using the following code:

if(AnalyticsTracker.IsActive)
  AnalyticsTracker.Current.PreviousPage.Search(query, m_totalResultCount);

The above code would be called from the search results page. Note however that we register the search page event against the previous page where the search would have been entered by the user.

But with DMS I had a hard time finding the equivalent method. The above extension method is actually just a convenience method which registers the proper OMS page event. So with DMS, we just have to register the page event ourselves.

The following code registers the search page event by name.

if (Tracker.IsActive &&
  Tracker.Visitor != null &&
  Tracker.Visitor.CurrentVisit != null)
{
  var page = Tracker.Visitor.CurrentVisit.CurrentPage;

  if (Tracker.Visitor.CurrentVisit.PreviousPage != null)
    page = Tracker.Visitor.CurrentVisit.PreviousPage;

  page.Register(new PageEventData("Search")
  {
    Data = query,
    DataKey = query,
    Text = query
  });
}

Again, note how the page event is registered against the previous page, not the current one which would be the search results page (though I’m defaulting to the current page in the event the previous page is null).

I mentioned that the event is registered by name. DMS loads the page events defined in Sitecore and will find the event by name when the PagesRow.Register() method is called.

With the above code in place, we’ll now be capturing analytics data for search actions on the website.

Here’s a handy tip, if after you’ve implemented the above code you still don’t see any data showing up in the search reports, trying changing the MinimumVisitsFilter value in the sitecore/shell/Applications/Reports/Dashboard/Configuration.xml file. With the default value of 50 in this setting you’ll need 50 different visits before any data shows in the report.

Filtering ECM Dispatcher

Alistair Deneys — Thu, 19 Jan 2012 11:45:25 +0000

The Email Campaign Manager (ECM) for Sitecore is a great tool for sending bulk email and newsletters to your users and subscribers. I had an interesting question from a client recently about the Email Campaign Manager.

“How can I send an email to a subset of the subscribers?”

I’ll give you a scenario to describe the request. Let’s say I have a user base of national partners, but I only want to send a newsletter to a subset of them such as only those in Victoria.

When an ECM mail is sent it is sent to an entire target audience which includes all users within a role. There are no options for limiting the subscribers in the target audience to which the mail is sent, other than a random selection to test the mail.

Using out-of-the-box tools the only way to meet my scenario above would be:

Create a new target audience for my Victorian partners
Export all my users to a CSV file using the ECM export users tool
Filter the users using Excel
Upload the altered file back into ECM using the user import tool, adding the users to the new target audience
Send the mail to the new target audience.

If this was just a temporary one-time email I would then need to delete the target audience and associated roles. This sounds like a lot of work to filter a subset of the users.

OK, so what if instead of originally uploading a single file of all my partners and adding them to a single partners target audience (and hence role) I could upload my partners by state, then create a new role which contains all the state partner roles…still sounds kind of clunky…especially if I want to segment my users more dynamically.

So I got to digging into ECM to work out how I could filter the users based on their custom properties. I needed to be able to assess each user as the mail was generated then abort the send if the user’s properties didn’t match the criteria for the newsletter. Luckily for me ECM uses a pipeline to process each individual email as it’s sent, so it was just a matter of tapping into the SendEmail pipeline to do the filtering.

OK, let’s put it all together.

First thing is it extend the user profile to accommodate any custom properties I want to filter on. Although the user object can accept any custom properties assigned through code, to have Sitecore update the UI to allow viewing and updating those properties we need to update the profile.

In the Sitecore desktop use the database changer to change to the core database, then open the content editor and navigate down to the template for the user profile being used for your subscribers. Sitecore uses the items under the /sitecore/system/Settings/Security/Profiles path in the core database for the custom properties shown for a user profile. By default ECM will use the Subscriber item in this folder, so navigate to the template this item is based on. To this template we’ll add an additional field called tags which we’ll use to store any kind of segmenting data for the user such as state or campaign participation.

Next we’ll need to ensure the users in Sitecore have the required data in the tags custom property. Now that the profile has been updated the user tools such as the user editor and the ECM import tool will show the field. Luckily the ECM user import tool will match users in the file with existing users on the user’s email, so this tool can be used to update the existing users in the system.

Once the users have been updated we need to update the newsletter template to allow entering tags against it to filter users on. I decided to put these tags on the newsletter template rather than updating the send email dialogs to make sure that if an already dispatched newsletter is duplicated back to the drafts folder to be reused, it doesn’t accidentally get sent to the wrong users.

Jump back into the master database and locate the ECM newsletter template at /sitecore/templates/Email Campaign/Messages/Pre-existing Page. Add to this template a field called User Tags which will hold a comma separated list of tags a user may contain in their tags custom property to have the email sent to them.

Now when we create a newsletter we can switch to the content tab of the newsletter to see the fields directly, then enter a list of tags the user may contain in their custom profile.

Now, onto the code!

We need to create a pipeline processor for the SendMail pipeline which will do the filtering of the users. The below class is one example of a processor which will do this. I’ve implemented this processor to “OR” the tags on the newsletter so the above newsletter would get sent to all my users that have the tag “victoria” OR “new south wales” OR “queensland” in their list of tags.

using System;
using System.Linq;
using Sitecore.Modules.EmailCampaign;
using Sitecore.Modules.EmailCampaign.Core.Pipelines;

namespace FilteringDispatch
{
  public class FilterSubscribers
  {
    public void Process(SendMessageArgs args)
    {
      var message = args.ECMMessage as MessageItem;
      if (message != null)
      {
        if (!string.IsNullOrEmpty(message.InnerItem["user tags"]))
        {
          var messageTags = message.InnerItem["user tags"].Split(
            new char[]{','},
            StringSplitOptions.RemoveEmptyEntries);

          var userTags = 
            message.PersonalizationContact.Profile.GetCustomProperty("tags").Split(
            new char[]{','},
            StringSplitOptions.RemoveEmptyEntries);

          var sendToUser = false;
          foreach(var messageTag in messageTags)
          {
            if (userTags.Contains(messageTag))
            {
              sendToUser = true;
              break;
            }
          }

          if(!sendToUser)
            args.AbortPipeline();
        }
      }
    }
  }
}

The above code first extracts the MessageItem the newsletter is being sent for which is the newsletter item itself that contains the user tags field. First the newsletter is checked to see if any tags have been defined. If no tags were defined, then exit the processor. Next the tags from the newsletter are extracted, then the tags from the user’s custom property are extracted. The tags from the newsletter are iterated and the user tags are checked to see if any match. At the end of the processor we check to see if the user had an appropriate tag and if not the pipeline is aborted. The effect of aborting the pipeline is that any remaining processors of the pipeline don’t run.

This processor needs to be inserted into the SendMail pipeline as the first processor so if it’s aborted the send processor isn’t processed. This can be done by using the following configuration patch file.

And that’s it!

Now when ECM sends an email the above processor will check each user to ensure they have an appropriate tag before sending them the email.

WeBlog 2.1 Released

Alistair Deneys — Tue, 10 Jan 2012 12:36:58 +0000

Today I have made several updates to the WeBlog module for Sitecore and published release 2.1. Go grab it now from http://trac.sitecore.net/WeBlog.

Here’s a few significant changes to the module.

Akismet

Does your WeBlog blog get swamped with spam? Are you wasting lots of manual effort on weeding it out? Then you’ll love this new feature. WeBlog 2.1 introduces Akismet support. Akismet is a commercial service to detect spam from web comments. If the Akismet service identifies the submitted comment as spam the comment is put into the new spam state of the WeBlog workflow. Users can use the workbox to easily review all spam comments and remove them.

Customise the Comment Creation Process

To make it easier to customise the comment creation process a new pipeline has been added. The weblogCreateComment pipeline is run when the comment is submitted. This pipeline creates the comment item, checks if the comment is spam against Akismet and then pushes it through workflow. A developer can easily update the pipeline to adjust how comments are created and processed.

Override the Manager Classes

The manager classes which are responsible for most of the data and content operations in the module have all be extracted behind interfaces which allows developers to override the class. A factory class is used to create concrete instances of the managers from configuration when required. This would allow a developer to, for example, store and retrieve comments from an external DB (future blog post on that in the works).

Set Publish Date Using Live Writer

You can now also set the publish date of your post when using the MetaWeblog API service such as when you use Live Writer.

Conclusion

The WeBlog wiki has also been updated with information about the new features, so you can refer to that for more information. If you have any comments or suggestions for the module, jump onto the WeBlog forum on the SDN and let us know.

Automated Testing in Sitecore Without an HttpContext

Alistair Deneys — Fri, 23 Dec 2011 12:02:05 +0000

This year was the first year that Sitecore’s Dreamcore conference was held in Australia. And I for one jumped at the opportunity to speak at it. One topic that quite interests me is unit testing, and if you’ve read my posts over the years you’d see I have come up with a variety of techniques for getting unit testing (or rather, integration testing) working for Sitecore projects. So what better topic to talk about at Dreamcore than automated testing techniques?

My session at Dreamcore covered a wide range of techniques and so I didn’t have time to go in depth with any single technique such as this one. In fact, I only had 40 minutes to speak and my rehearsal the night before I came in at 1 hour. So I’ll take this opportunity to explain the technique in depth.

Many years ago I tried to run my automated tests for my Sitecore project’s inside the NUnit GUI test runner…and failed. The issue with the standalone test runner is that there is no HttpContext, and as I’m sure you’re well aware, Sitecore requires one of these to work. When I ran into this hurdle all those years ago, I instead developed different techniques to get around this limitation.

Anyway, back to this year. A few months before Dreamcore I was thinking I had to have another look into getting my automated tests for Sitecore running without the HttpContext. And then Mike Edwards beat me to the punchline and posted a blog entry on how he got unit testing for Sitecore working in the NUnit GUI. Mike’s post proved to me that it was possible, so I followed his post and got some tests running inside the NUnit GUI test runner, calling into the Sitecore 6.5 API.

With Mike’s general direction in place I set about fully integrating this approach to testing into a project I was working on at the time, which would provide a bit more context for me when I was writing my Dreamcore presentation.

The main change I made to Mike’s approach was to copy the configuration over from an existing Sitecore project. This way if any of the Sitecore configuration changed my test project would also see that change and the code under test would be run in an environment as close as possible to production.

First thing’s first, create a new class library project to write the tests in and add references to the nunit.framework assembly. As we’ll be calling into the Sitecore API (the whole point of this exercise) we’ll also need to add a reference to the Sitecore.Kernel assembly from the Sitecore instance the tests will be written against. To make sure we’ve got the configuration right to use the Sitecore API we’ll also create a simple test which uses the Sitecore API. The simplest thing I can think of would be grabbing a field from the home item. The following code shows how to implement this test.

[TestFixture]
public class ApiTest
{
  [Test]
  public void AccessFieldOnHome()
  {
    var db = Sitecore.Configuration.Factory.GetDatabase("web");
    var home = db.GetItem("/sitecore/content/home");
    var fieldValue = home["title"];
    Assert.AreEqual("Sitecore", fieldValue);
  }
}

The above code will also require a reference to System.Configuration due to the call to the Sitecore.Configuration namespace.

Note above how we need to retrieve the database and cannot use the Sitecore context to get the context database because the context hasn’t been populated. The Sitecore context is populated through the httpRequestBegin pipeline which isn’t run when we’re calling the API outside an HttpContext.

If we were to run the above test we would end up with that familiar “Failure: System.InvalidOperationException : Could not read Sitecore configuration.” error. Have you ever noticed how so many people lack the ability to read and interpret error messages? The above error message gives a clear description of the issue…there is no Sitecore configuration.

So onto the biggest hurdle, copying the Sitecore configuration over from the Sitecore project which these tests will exercise. I want to automate this process and make it part of the project build process so I get any configuration updates in the test project. This will require some MSBuild additions.

To edit the test project’s MSBuild unload the project in Visual Studio then right click the project and select “Edit TestProject.csproj”. Alternatively you could directly edit the project file (csproj) using a separate text editor and Visual Studio will prompt you to reload the project file when you go back into Visual Studio. However the benefit of editing the project file inside Visual Studio is that Visual Studio will provide IntelliSense as you edit the file.

To make maintenance easier we’ll store the path to the Sitecore instance we’ll be using the configuration from in a variable. While editing the project file add the following script to the first PropertyGroup element which doesn’t have any additional attributes.

C:\inetpub\WeBlog-65\Website

Make sure you update the path in the variable to reference the local path on your machine. The above script defines a variable called SitecorePath and sets it to the path provided. This variable can then be used throughout the rest of the script.

Now Find the AfterBuild target which will be commented out. This target will be called by Visual Studio if the build was successful. This is where we’ll put our additional MSBuild configuration. Uncomment the AfterBuild target and add the following script to it.

  DestinationFiles="$(OutputPath)\$(AssemblyName).dll.config" />

The above MSBuild script will copy the web.config of the referenced Sitecore instance to the configuration file of the test assembly. Remember for Windows assemblies the configuration file is not named web.config but is instead the same name as the assembly with an additional .config at the end. So to provide configuration for my program MyApp.exe the configuration file must be named MyApp.exe.config. The same works for the test assemblies.

Sitecore configuration is actually spread over a number of additional files, so the next piece of script will grab those files and copy them over to the appropriate location as well.


  



  DestinationFolder="$(OutputPath)\App_Config\%(RecursiveDir)" />

Note how in the DestinationFolder attribute I’m using the RecursiveDir well-known metadata about the input files. This will create the same directory structure as the source files are in instead of dumping them out to a flat directory.

To make sure we have all the assemblies we might require we will also need to copy all the assemblies in the Sitecore bin folder to the same location as the test assembly.

And now, the test will work!

Though we’re missing one important piece of configuration…configuration include files. Configuration includes files are located in the \App_Config\Include folder of a Sitecore instance and allow patching the Sitecore configuration. This makes it much easier to keep your project configuration separate from the Sitecore configuration. It also makes upgrading Sitecore much easier as you can simply take the entire updated web.config file (Sitecore section) without worry that you’re reverting some update made for your project.

There are 2 options for handling the configuration include files. Firstly, don’t use them. For a test project you could just make sure all the configuration required is contained in the single web.config file. But that’s going against what I’ve been talking about in terms of separating project configuration from Sitecore configuration and using a real project’s configuration to ensure the tests are run in a realistically configured environment.

The other option is to put the include files where they’re required to allow the configuration patching utility to find them. The script above is already taking care of copying the include files to the output folder, so what else needs to be done?

The configuration patch utility used by Sitecore internally contains a call to the Server.MapPath() method. When there is no HttpContext this method will simply pass back the path as it was passed in. Sitecore is trying to locate the folder relative to the web application root, so what’s passed to the method is /App_Config/Include. When trying to locate this path on a disk the leading slash will cause the OS to look from the root of the disk, so the path being located is actually c:\App_Config\include. To make the configuration include files work we’ll need to copy the files to a folder off the root of the disk named App_Config.

  DestinationFolder="c:\App_Config\%(RecursiveDir)" />

Now the configuration patch utility will work.

Some points of caution when using this technique. Some components won’t be in the correct state when you try and use them because none of the request process has been run. No ASP.NET events and no Sitecore pipelines have been run. This is why the Sitecore context is not populated. But I’ve also found some other classes such as Sitecore.Globals aren’t initialised. If your test uses any of the global objects (Links DB, Tasks DB) you’ll need to manually initialise them with a call to Sitecore.Globals.Load(); before using them. I’ve recently spoken to one of the Sitecore Users Virtual Group users who has been using this technique who had issues calling Delete() on an item, because of the uninitialised Tasks database.

Another method I’ve found which simply won’t work is the WebControl.RenderAsText() method. This method relies on there being an HttpContext and simply won’t work without one.

The above points of caution are just a few I’ve found whilst using this technique. That’s not to say there aren’t a heap of other classes which need initialising before use that I haven’t run into yet.

This technique is also quite heavy. The Sitecore application is actually running. A real Sitecore database it accessed. Many of the objects defined in the configuration are created and strung together. Although anecdotally comparing the speed of the test runs using this technique to my normal technique of using an embedded test runner, this technique does seem faster.

If you’d like a working example of this technique you can refer to my materials from Dreamcore Australia 2011. I was lucky enough to recently deliver the same presentation to the Sitecore Users Virtual Group, so if you’d like to see me put those pieces together then you can refer to the recording of my Testing Strategies for Sitecore presentation on Hedgehog’s YouTube channel.

Presenting to the Sitecore Users Virtual Group

Alistair Deneys — Mon, 14 Nov 2011 21:45:06 +0000

This Thursday (well, it will be Thursday for me, Wednesday if you’re on the other side of the date line) I’ll be presenting to the Sitecore Users Virtual Group on testing strategies for Sitecore. This is the same talk I gave at Dreamcore this year, so if you missed out, here’s another opportunity to see me talk about some testing techniques.

Details can be found here: http://www.sitecoreug.org/events/Testing%20Strategies%20for%20Sitecore. Make sure you check the time. It will be 7am Thursday 17 November here in Melbourne, Noon pacific Wednesday 16 November, 3pm Eastern Wednesday 16 November and 8pm UK Wednesday 16 November.

This presentation is for devs. I’ll be writing code during my presentation and showing various techniques I’ve used whilst unit testing my Sitecore projects. Most of the techniques I’ve already written about on this blog, but I’ve recently had another try at getting testing to work without an HttpContext and been successful! I promise I’ll write a detailed blog post on that in the future, but I’ll show attendees in the meeting how I do that as well.

So if that all sounds like fun, jump over to the event page above and register.

WeBlog Summit 2011

Alistair Deneys — Thu, 06 Oct 2011 20:00:07 +0000

Ealier this year in April, Sitecore’s Dreamcore US event was held in Boston. One of my WeBlog colleagues, Nick Wesselman, gave a presentation at the event on the shared source WeBlog module. There was quite a lot of buzz around the module, especially from Sitecore themselves, who saw what we were doing with the module as a great example of Sitecore community.

One of the interesting things about this module and how it’s developed is the geographical disparity of the contributors. At Dreamcore US, someone made the comment about how it would be great if the 3 WeBlog contributors (me, Mark van Aalst and Nick) could get together somewhere and have a summit, to go through the current module and work out the future direction of the module.

And that’s exactly what is going to happen in just under two weeks. Sitecore is sponsoring each of us to travel to Sitecore HQ in Copenhagen and have this summit.

And it’s going to be a great event! Lars Nielsen is currently coordinating to give the WeBlog contributors access to some of the Sitecore core developers and architects at the summit. This access will allow us to really align the module with Sitecore’s future direction, and ensure we’re ready to leverage all the cool new stuff coming in Sitecore Massive (and believe me, from what I’ve seen, there is some very cool stuff coming).

I’m also hoping we can get the first official release of WeBlog out there to the greater Sitecore community.

So if you’d like to have a say in the future of WeBlog or you’d like to request a feature or change, head over to the Sitecore Developer Network and post your ideas in the EviBlog/WeBlog forum.

Dreamcore AU 2011

Alistair Deneys — Thu, 06 Oct 2011 11:20:54 +0000

It was my great pleasure to present at Sitecore’s Dreamcore event in Sydney, Australia this week. This was the first time the conference was held in Australia and it was fantastic with lots of great speakers and interesting people.

My session was titled “Testing Strategies for Sitecore” and was run in the developer stream, so it focused on automated/unit testing. As promised for everyone who attended my session, here is the code for my demo site and test project, as well as the slides.

The test project shows a few different techniques for automated testing your Sitecore projects including:

Testing pages over HTTP
Testing using a web browser driver (WatiN)
Instantiating a Sitecore control
Testing within the HTTP Context
Testing without an HTTP Context

I’ve covered all of the above techniques before, except for the last one, testing without an HTTP Context. I’ll cover that technique in detail in a future post.