“How can I send an email to a subset of the subscribers?”

I’ll give you a scenario to describe the request. Let’s say I have a user base of national partners, but I only want to send a newsletter to a subset of them such as only those in Victoria.

When an ECM mail is sent it is sent to an entire target audience which includes all users within a role. There are no options for limiting the subscribers in the target audience to which the mail is sent, other than a random selection to test the mail.

Using out-of-the-box tools the only way to meet my scenario above would be:

1. Create a new target audience for my Victorian partners
2. Export all my users to a CSV file using the ECM export users tool
3. Filter the users using Excel
4. Upload the altered file back into ECM using the user import tool, adding the users to the new target audience
5. Send the mail to the new target audience.

If this was just a temporary one-time email I would then need to delete the target audience and associated roles. This sounds like a lot of work to filter a subset of the users.

OK, so what if instead of originally uploading a single file of all my partners and adding them to a single partners target audience (and hence role) I could upload my partners by state, then create a new role which contains all the state partner roles…still sounds kind of clunky…especially if I want to segment my users more dynamically.

So I got to digging into ECM to work out how I could filter the users based on their custom properties. I needed to be able to assess each user as the mail was generated then abort the send if the user’s properties didn’t match the criteria for the newsletter. Luckily for me ECM uses a pipeline to process each individual email as it’s sent, so it was just a matter of tapping into the SendEmail pipeline to do the filtering.

OK, let’s put it all together.

First thing is it extend the user profile to accommodate any custom properties I want to filter on. Although the user object can accept any custom properties assigned through code, to have Sitecore update the UI to allow viewing and updating those properties we need to update the profile.

In the Sitecore desktop use the database changer to change to the core database, then open the content editor and navigate down to the template for the user profile being used for your subscribers. Sitecore uses the items under the /sitecore/system/Settings/Security/Profiles path in the core database for the custom properties shown for a user profile. By default ECM will use the Subscriber item in this folder, so navigate to the template this item is based on. To this template we’ll add an additional field called tags which we’ll use to store any kind of segmenting data for the user such as state or campaign participation.

Next we’ll need to ensure the users in Sitecore have the required data in the tags custom property. Now that the profile has been updated the user tools such as the user editor and the ECM import tool will show the field. Luckily the ECM user import tool will match users in the file with existing users on the user’s email, so this tool can be used to update the existing users in the system.

Once the users have been updated we need to update the newsletter template to allow entering tags against it to filter users on. I decided to put these tags on the newsletter template rather than updating the send email dialogs to make sure that if an already dispatched newsletter is duplicated back to the drafts folder to be reused, it doesn’t accidentally get sent to the wrong users.

Jump back into the master database and locate the ECM newsletter template at /sitecore/templates/Email Campaign/Messages/Pre-existing Page. Add to this template a field called User Tags which will hold a comma separated list of tags a user may contain in their tags custom property to have the email sent to them.

Now when we create a newsletter we can switch to the content tab of the newsletter to see the fields directly, then enter a list of tags the user may contain in their custom profile.

Now, onto the code!

We need to create a pipeline processor for the SendMail pipeline which will do the filtering of the users. The below class is one example of a processor which will do this. I’ve implemented this processor to “OR” the tags on the newsletter so the above newsletter would get sent to all my users that have the tag “victoria” OR “new south wales” OR “queensland” in their list of tags.

using System;
using System.Linq;
using Sitecore.Modules.EmailCampaign;
using Sitecore.Modules.EmailCampaign.Core.Pipelines;

namespace FilteringDispatch
{
  public class FilterSubscribers
  {
    public void Process(SendMessageArgs args)
    {
      var message = args.ECMMessage as MessageItem;
      if (message != null)
      {
        if (!string.IsNullOrEmpty(message.InnerItem["user tags"]))
        {
          var messageTags = message.InnerItem["user tags"].Split(
            new char[]{','},
            StringSplitOptions.RemoveEmptyEntries);

          var userTags =
            message.PersonalizationContact.Profile.GetCustomProperty("tags").Split(
            new char[]{','},
            StringSplitOptions.RemoveEmptyEntries);

          var sendToUser = false;
          foreach(var messageTag in messageTags)
          {
            if (userTags.Contains(messageTag))
            {
              sendToUser = true;
              break;
            }
          }

          if(!sendToUser)
            args.AbortPipeline();
        }
      }
    }
  }
}

The above code first extracts the MessageItem the newsletter is being sent for which is the newsletter item itself that contains the user tags field. First the newsletter is checked to see if any tags have been defined. If no tags were defined, then exit the processor. Next the tags from the newsletter are extracted, then the tags from the user’s custom property are extracted. The tags from the newsletter are iterated and the user tags are checked to see if any match. At the end of the processor we check to see if the user had an appropriate tag and if not the pipeline is aborted. The effect of aborting the pipeline is that any remaining processors of the pipeline don’t run.

This processor needs to be inserted into the SendMail pipeline as the first processor so if it’s aborted the send processor isn’t processed. This can be done by using the following configuration patch file.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <SendEmail>
        <processor patch:before="processor[1]"
          type="FilteringDispatch.FilterSubscribers,FilteringDispatch" />
      </SendEmail>
    </pipelines>
  </sitecore>
</configuration>

And that’s it!

Now when ECM sends an email the above processor will check each user to ensure they have an appropriate tag before sending them the email.

Here’s a few significant changes to the module.

Akismet

Does your WeBlog blog get swamped with spam? Are you wasting lots of manual effort on weeding it out? Then you’ll love this new feature. WeBlog 2.1 introduces Akismet support. Akismet is a commercial service to detect spam from web comments. If the Akismet service identifies the submitted comment as spam the comment is put into the new spam state of the WeBlog workflow. Users can use the workbox to easily review all spam comments and remove them.

Customise the Comment Creation Process

To make it easier to customise the comment creation process a new pipeline has been added. The weblogCreateComment pipeline is run when the comment is submitted. This pipeline creates the comment item, checks if the comment is spam against Akismet and then pushes it through workflow. A developer can easily update the pipeline to adjust how comments are created and processed.

Override the Manager Classes

The manager classes which are responsible for most of the data and content operations in the module have all be extracted behind interfaces which allows developers to override the class. A factory class is used to create concrete instances of the managers from configuration when required. This would allow a developer to, for example, store and retrieve comments from an external DB (future blog post on that in the works).

Set Publish Date Using Live Writer

You can now also set the publish date of your post when using the MetaWeblog API service such as when you use Live Writer.

Conclusion

The WeBlog wiki has also been updated with information about the new features, so you can refer to that for more information. If you have any comments or suggestions for the module, jump onto the WeBlog forum on the SDN and let us know.

My session at Dreamcore covered a wide range of techniques and so I didn’t have time to go in depth with any single technique such as this one. In fact, I only had 40 minutes to speak and my rehearsal the night before I came in at 1 hour. So I’ll take this opportunity to explain the technique in depth.

Many years ago I tried to run my automated tests for my Sitecore project’s inside the NUnit GUI test runner…and failed. The issue with the standalone test runner is that there is no HttpContext, and as I’m sure you’re well aware, Sitecore requires one of these to work. When I ran into this hurdle all those years ago, I instead developed different techniques to get around this limitation.

Anyway, back to this year. A few months before Dreamcore I was thinking I had to have another look into getting my automated tests for Sitecore running without the HttpContext. And then Mike Edwards beat me to the punchline and posted a blog entry on how he got unit testing for Sitecore working in the NUnit GUI. Mike’s post proved to me that it was possible, so I followed his post and got some tests running inside the NUnit GUI test runner, calling into the Sitecore 6.5 API.

With Mike’s general direction in place I set about fully integrating this approach to testing into a project I was working on at the time, which would provide a bit more context for me when I was writing my Dreamcore presentation.

The main change I made to Mike’s approach was to copy the configuration over from an existing Sitecore project. This way if any of the Sitecore configuration changed my test project would also see that change and the code under test would be run in an environment as close as possible to production.

First thing’s first, create a new class library project to write the tests in and add references to the nunit.framework assembly. As we’ll be calling into the Sitecore API (the whole point of this exercise) we’ll also need to add a reference to the Sitecore.Kernel assembly from the Sitecore instance the tests will be written against. To make sure we’ve got the configuration right to use the Sitecore API we’ll also create a simple test which uses the Sitecore API. The simplest thing I can think of would be grabbing a field from the home item. The following code shows how to implement this test.

[TestFixture]
public class ApiTest
{
  [Test]
  public void AccessFieldOnHome()
  {
    var db = Sitecore.Configuration.Factory.GetDatabase("web");
    var home = db.GetItem("/sitecore/content/home");
    var fieldValue = home["title"];
    Assert.AreEqual("Sitecore", fieldValue);
  }
}

The above code will also require a reference to System.Configuration due to the call to the Sitecore.Configuration namespace.

Note above how we need to retrieve the database and cannot use the Sitecore context to get the context database because the context hasn’t been populated. The Sitecore context is populated through the httpRequestBegin pipeline which isn’t run when we’re calling the API outside an HttpContext.

If we were to run the above test we would end up with that familiar “Failure: System.InvalidOperationException : Could not read Sitecore configuration.” error. Have you ever noticed how so many people lack the ability to read and interpret error messages? The above error message gives a clear description of the issue…there is no Sitecore configuration.

So onto the biggest hurdle, copying the Sitecore configuration over from the Sitecore project which these tests will exercise. I want to automate this process and make it part of the project build process so I get any configuration updates in the test project. This will require some MSBuild additions.

To edit the test project’s MSBuild unload the project in Visual Studio then right click the project and select “Edit TestProject.csproj”. Alternatively you could directly edit the project file (csproj) using a separate text editor and Visual Studio will prompt you to reload the project file when you go back into Visual Studio. However the benefit of editing the project file inside Visual Studio is that Visual Studio will provide IntelliSense as you edit the file.

To make maintenance easier we’ll store the path to the Sitecore instance we’ll be using the configuration from in a variable. While editing the project file add the following script to the first PropertyGroup element which doesn’t have any additional attributes.

<SitecorePath>C:\inetpub\WeBlog-65\Website</SitecorePath>

Make sure you update the path in the variable to reference the local path on your machine. The above script defines a variable called SitecorePath and sets it to the path provided. This variable can then be used throughout the rest of the script.

Now Find the AfterBuild target which will be commented out. This target will be called by Visual Studio if the build was successful. This is where we’ll put our additional MSBuild configuration. Uncomment the AfterBuild target and add the following script to it.

<Copy SourceFiles="$(SitecorePath)\web.config"  DestinationFiles="$(OutputPath)\$(AssemblyName).dll.config" />

The above MSBuild script will copy the web.config of the referenced Sitecore instance to the configuration file of the test assembly. Remember for Windows assemblies the configuration file is not named web.config but is instead the same name as the assembly with an additional .config at the end. So to provide configuration for my program MyApp.exe the configuration file must be named MyApp.exe.config. The same works for the test assemblies.

Sitecore configuration is actually spread over a number of additional files, so the next piece of script will grab those files and copy them over to the appropriate location as well.

<CreateItem Include="$(SitecorePath)\App_Config\**\*.*">
  <Output ItemName="configFiles" TaskParameter="Include" />
</CreateItem>

<!-- Copy relative external source files -->
<Copy SourceFiles="@(configFiles)"  DestinationFolder="$(OutputPath)\App_Config\%(RecursiveDir)" />

Note how in the DestinationFolder attribute I’m using the RecursiveDir well-known metadata about the input files. This will create the same directory structure as the source files are in instead of dumping them out to a flat directory.

To make sure we have all the assemblies we might require we will also need to copy all the assemblies in the Sitecore bin folder to the same location as the test assembly.

<CreateItem Include="$(SitecorePath)\bin\*.dll">
  <Output ItemName="binaryFiles" TaskParameter="Include" />
</CreateItem>

<Copy SourceFiles="@(binaryFiles)" DestinationFolder="$(OutputPath)" />

And now, the test will work!

Though we’re missing one important piece of configuration…configuration include files. Configuration includes files are located in the \App_Config\Include folder of a Sitecore instance and allow patching the Sitecore configuration. This makes it much easier to keep your project configuration separate from the Sitecore configuration. It also makes upgrading Sitecore much easier as you can simply take the entire updated web.config file (Sitecore section) without worry that you’re reverting some update made for your project.

There are 2 options for handling the configuration include files. Firstly, don’t use them. For a test project you could just make sure all the configuration required is contained in the single web.config file. But that’s going against what I’ve been talking about in terms of separating project configuration from Sitecore configuration and using a real project’s configuration to ensure the tests are run in a realistically configured environment.

The other option is to put the include files where they’re required to allow the configuration patching utility to find them. The script above is already taking care of copying the include files to the output folder, so what else needs to be done?

The configuration patch utility used by Sitecore internally contains a call to the Server.MapPath() method. When there is no HttpContext this method will simply pass back the path as it was passed in. Sitecore is trying to locate the folder relative to the web application root, so what’s passed to the method is /App_Config/Include. When trying to locate this path on a disk the leading slash will cause the OS to look from the root of the disk, so the path being located is actually c:\App_Config\include. To make the configuration include files work we’ll need to copy the files to a folder off the root of the disk named App_Config.

<Copy SourceFiles="@(configFiles)"  DestinationFolder="c:\App_Config\%(RecursiveDir)" />

Now the configuration patch utility will work.

Some points of caution when using this technique. Some components won’t be in the correct state when you try and use them because none of the request process has been run. No ASP.NET events and no Sitecore pipelines have been run. This is why the Sitecore context is not populated. But I’ve also found some other classes such as Sitecore.Globals aren’t initialised. If your test uses any of the global objects (Links DB, Tasks DB) you’ll need to manually initialise them with a call to Sitecore.Globals.Load(); before using them. I’ve recently spoken to one of the Sitecore Users Virtual Group users who has been using this technique who had issues calling Delete() on an item, because of the uninitialised Tasks database.

Another method I’ve found which simply won’t work is the WebControl.RenderAsText() method. This method relies on there being an HttpContext and simply won’t work without one.

The above points of caution are just a few I’ve found whilst using this technique. That’s not to say there aren’t a heap of other classes which need initialising before use that I haven’t run into yet.

This technique is also quite heavy. The Sitecore application is actually running. A real Sitecore database it accessed. Many of the objects defined in the configuration are created and strung together. Although anecdotally comparing the speed of the test runs using this technique to my normal technique of using an embedded test runner, this technique does seem faster.

If you’d like a working example of this technique you can refer to my materials from Dreamcore Australia 2011. I was lucky enough to recently deliver the same presentation to the Sitecore Users Virtual Group, so if you’d like to see me put those pieces together then you can refer to the recording of my Testing Strategies for Sitecore presentation on Hedgehog’s YouTube channel.

Details can be found here: http://www.sitecoreug.org/events/Testing%20Strategies%20for%20Sitecore. Make sure you check the time. It will be 7am Thursday 17 November here in Melbourne, Noon pacific Wednesday 16 November, 3pm Eastern Wednesday 16 November and 8pm UK Wednesday 16 November.

This presentation is for devs. I’ll be writing code during my presentation and showing various techniques I’ve used whilst unit testing my Sitecore projects. Most of the techniques I’ve already written about on this blog, but I’ve recently had another try at getting testing to work without an HttpContext and been successful! I promise I’ll write a detailed blog post on that in the future, but I’ll show attendees in the meeting how I do that as well.

So if that all sounds like fun, jump over to the event page above and register.

One of the interesting things about this module and how it’s developed is the geographical disparity of the contributors. At Dreamcore US, someone made the comment about how it would be great if the 3 WeBlog contributors (me, Mark van Aalst and Nick) could get together somewhere and have a summit, to go through the current module and work out the future direction of the module.

And that’s exactly what is going to happen in just under two weeks. Sitecore is sponsoring each of us to travel to Sitecore HQ in Copenhagen and have this summit.

And it’s going to be a great event! Lars Nielsen is currently coordinating to give the WeBlog contributors access to some of the Sitecore core developers and architects at the summit. This access will allow us to really align the module with Sitecore’s future direction, and ensure we’re ready to leverage all the cool new stuff coming in Sitecore Massive (and believe me, from what I’ve seen, there is some very cool stuff coming).

I’m also hoping we can get the first official release of WeBlog out there to the greater Sitecore community.

So if you’d like to have a say in the future of WeBlog or you’d like to request a feature or change, head over to the Sitecore Developer Network and post your ideas in the EviBlog/WeBlog forum.

My session was titled “Testing Strategies for Sitecore” and was run in the developer stream, so it focused on automated/unit testing. As promised for everyone who attended my session, here is the code for my demo site and test project, as well as the slides.

• Source code for demo site
• Slides for Testing Strategies for Sitecore

The test project shows a few different techniques for automated testing your Sitecore projects including:

• Testing pages over HTTP
• Testing using a web browser driver (WatiN)
• Instantiating a Sitecore control
• Testing within the HTTP Context
• Testing without an HTTP Context

I’ve covered all of the above techniques before, except for the last one, testing without an HTTP Context. I’ll cover that technique in detail in a future post.

I didn’t want to have to cast the array of objects to the appropriate type when I called the method. Instead I wanted to have the method return the appropriate type which is passed by the calling code.

The complicating factor around what I was trying to do was to do with the fact that I was passing a particular type into my generic method, but casting from a different type which the generic type didn’t inherit from. Even though I had implicit operators defined to go between my base type and wrapper, the compiler for some reason couldn’t see the conversion and failed. If I had the luxury of being able to make my wrapper inherit from the base class then everything would have been alright, but the base class I was dealing with came out of another system (a Sitecore.Data.Items.Item).

Let’s explore some code.

Below is a custom class and a wrapper. Notice how the wrapper class contains the implicit operators to allow implicit conversion to and from my custom class.

public class RawClass
{
  public int ID { get; set; }

  public RawClass(int id)
  {
    ID = id;
  }
}

public class Wrapper
{
  private RawClass inner = null;

  public static implicit operator Wrapper(RawClass instance)
  {
    return new Wrapper(instance);
  }

  public static implicit operator RawClass(Wrapper instance)
  {
    return instance.inner;
  }

  public Wrapper(RawClass instance)
  {
    inner = instance;
  }

  public int GetId()
  {
    return inner.ID;
  }
}

The implicit operators let me treat an instance of Wrapper like an instance of RawClass.

Wrapper wrap = new RawClass(3);
RawClass raw = wrap;

I can also pass instances of Wrapper to methods expecting RawClass without an explicit cast.

public string GetId(RawClass raw)
{
  return raw.ID.ToString();
}

var raw = new RawClass(5);
Wrapper wrap = raw;
var id = GetId(wrap);

Note above how I didn’t cast wrap to RawClass when I passed it to the GetId method.

OK, so all of the above compiles and works. Onto my problem.

I was working on a method to fetch several objects of one type but I wanted to define through generics the type to return from the method. In the above example code, RawClass is the type of object from the data store, but I want to cast it to the generic defined type for the return. The reason I wanted to use generics was because this method was going to be called from several different places in my code, all with different return types. I wanted to be able to call the method as such:

var items = GetItems<Wrapper>();
var items2 = GetItems<DifferentWrapper>();

For demonstration purposes I’ll use a dummy method to return me some data. In implementation this method would be replaced with one that actually communicated with the data store.

private IEnumerable<RawClass> LoadFromDataStore()
{
  // Load data from data store
  for(int i = 0; i < 4; i++)
    yield return new RawClass(i);
}

And now we can write the GetItems method which is the generic method that loads the data and casts it to the appropriate type.

private T[] GetItems<T>() where T : class
{
  var list = new List<T>();
  var objects = LoadFromDataStore();

  foreach(var ob in objects)
  {
    list.Add((T)ob);
  }

  return list.ToArray();
}

Not quite. The above code won’t compile. The compiler will give an error about converting from RawClass to T.

Error    Cannot convert type 'GenericCasting.RawClass' to 'T'

I could replace the cast above with a safe cast using the as operator, but it doesn’t really give us what we want. The safe cast will return null instead of throwing an exception when it can’t cast from one type to another. Even though the above code should work because we’ve defined the implicit conversion operators, it doesn’t.

I struggled with this code for quite some time, even getting it to a point where I got errors stating there is no conversion from RawClass to Wrapper. Are you serious? I’m even using the implicit operators in other parts of the code!

I knew I could cast the objects if I wasn’t dealing with a generic type, and the calling code knew what type it wanted to deal with, so I needed to pass control momentarily back to the calling code and have it do the casting. I’ve shown previously how to pass control back to the calling code, and this approach is basically the same.

Enter the Action<> class. Action is simply a generic delegate and an efficient way to use delegates when you don’t want to define a new delegate type. I simply pass as the type parameters the types I want my delegate to require. Let’s have a look at how I can update the previous method to use an action which would be called to pass control back to the calling code.

private T[] GetItems<T>(Action<List<T>, RawClass> act) where T : class
{
  var list = new List<T>();
  var objects = LoadFromDataStore();

  foreach(var ob in objects)
  {
    act(list, ob);
  }

  return list.ToArray();
}

In my Action I’ve define the parameters as being of type List<T> and RawClass. The calling code will need to cast the RawClass and add it to the list. To call this method:

var list = GetItems<Wrapper>((l, ob) => { l.Add(ob); });

Mmmm. Syntactic sugar.

Rather than using a traditional delegate I’m using a lambda expression. Note how I don’t even need to cast, because the implicit conversion is now working when the compiler can see what the types are.

And that’s it. Using this technique I’ve worked around the limitations of the compiler not seeing the conversions between classes when using generics.

The main reason I chose SEFE was due to the fact that SEFE IS Sitecore and is built completely in Sitecore. All other eCommerce platforms I assessed were external applications and would have required a great deal of integration effort to make them work as seamlessly as SEFE felt. I should mention that SEEE (Sitecore eCommerce Enterprise Edition) had not been released at the time, so that was never a contender.

Before I get into the goodness of this post about the payment providers, a quick warning to any about to use SEFE. The example site is just that. An example. I would never again attempt to use the example site as part of my production site. The example site has it’s own way of doing things and it’s own markup. Perhaps my experience was marred by the fact I had to bash the example controls to work with the existing CSS. So based on my experience, my advice to you is to refer to the example site as a reference, but build your own site components.

So, onto payment providers. The site I was porting over to SEFE hosted a credit card payment form itself. It uses an on-site payment method where the user never leaves the site; there is no redirect to the payment providers payment page and then a redirect back to the site. I needed to replicate this in SEFE.

Flicking through the SEFE payment method reference guide, which can be found on SDN, it seemed it didn’t quite tell me what I needed to do to host my own payment form. That document focused heavily on using a redirect to the payment providers hosted form. There were snippets about an on-site option, but not nearly enough information on this.

Eventually I found in the document the IIntegrationalPaymentProvider interface that sounded like what I needed to be able to write my own on-site payment provider…but it has not been implemented yet.

As it turns out, it’s not that hard to handle payments on-site but still fit within how SEFE wants to work. The trick is to create a normal IOnlinePaymentProvider (for an off-site payment provider) but host the payment forms yourself and have your payment provider interact with your own forms.

So let’s create a sample on-site payment provider using this approach. I’m going to write my sample payment provider and deploy it to the SEFE sample site. For this sample I’ll need to work against an existing Sitecore site with the latest version of SEFE installed.

We need to create a new class for our payment provider which extends the Sitecore.Ecommerce.Payments.OnlinePaymentProvider class which can be found in the Sitecore.Ecommerce.Kernel assembly. There are 2 methods we need to override to make our payment provider work; Invoke and ProcessCallback. Invoke is called when SEFE uses your payment provider to make a payment and ProcessCallback is called when the normally external payment provider site redirects the user back to your site.

For our sample on-site provider, in the Invoke method we simply need to redirect the user to the payment form on our site. We don’t need to send any data like we would an off-site payment provider cause we can simply retrieve that data from the payment form. The OOTB payment provider configuration in SEFE allows entering a primary and secondary URL for the provider. For our provider we’ll use the primary URL to point to our on-site payment form, and the secondary URL to point to the payment gateway service we’ll be using. The URLs are configured inside Sitecore on the payment provider item we’ll configure a but later on. SEFE passes the configured URLs into the payment provider through the paymentSystem parameter.

public override void Invoke(
  Sitecore.Ecommerce.DomainModel.Payments.PaymentSystem paymentSystem,
    PaymentArgs paymentArgs)
{
  base.Invoke(paymentSystem, paymentArgs);

  // Store return page URL in transaction for use on credit card form
  var transaction = Sitecore.Ecommerce.Context.Entity.Resolve<TransactionData>(); 

  transaction.SavePersistentValue(paymentArgs.ShoppingCart.OrderNumber,
    "returnurl", paymentArgs.PaymentUrls.ReturnPageUrl);

  HttpContext.Current.Response.Redirect(paymentSystem.PaymentUrl);
}

Note above how I’m storing the return page URL? That URL is what we must redirect the user to after a successful payment so SEFE will process the order. I can’t retrieve that URL from the payment form by other means so I’ll store that URL in the transaction so I can retrieve it later.

The page we redirect the user to will handle collection of credit card information from the user, communication with the payment gateway and setting the success or failure of the payment. For this we’ll create a new sublayout containing the following markup:

<%@ Control Language="C#" AutoEventWireup="true"
  CodeBehind="CreditCardForm.ascx.cs"
  Inherits="SampleOnsitePaymentProvider.layouts.CreditCardForm" %>
  <div>
    <strong>Amount Due: </strong><asp:Literal runat="server"
      ID="litAmount" />
  </div>
  <div>
    Card Number: <asp:TextBox runat="server" ID="txtCardNumber" />
  </div>
  <div>
    Name on Card: <asp:TextBox runat="server" ID="txtName" />
  </div>
  <div>
    Expiry: Month: <asp:TextBox runat="server" ID="txtMonth" />
    Year: <asp:TextBox runat="server" ID="txtYear" />
  </div>
  <div>
    <asp:Literal runat="server" ID="litMessage" />
  </div>
  <div>
    <asp:Button runat="server" ID="btnSubmit" Text="Submit"
      OnClick="SubmitClicked" />
  </div>

Not particularly complex. The code behind of this sublayout is pretty straight forward, set the total of the order when the page loads and then process the credit card details when the form is submitted.

using System;
using Sitecore.Ecommerce;
using Sitecore.Ecommerce.DomainModel.Carts;
using Sitecore.Ecommerce.Payments;

namespace SampleOnsitePaymentProvider.layouts
{
  public partial class CreditCardForm : System.Web.UI.UserControl
  {
    protected void Page_Load(object sender, EventArgs e)
    {
      var shoppingCart =
        Sitecore.Ecommerce.Context.Entity.GetInstance<ShoppingCart>();
      if (shoppingCart != null)
        litAmount.Text = shoppingCart.Totals.TotalPriceIncVat.ToString();
      else
      {
        litMessage.Text = "Failed to find the shopping cart";
        btnSubmit.Enabled = false;
      }
    }

    protected void SubmitClicked(object sender, EventArgs args)
    {
      // TODO: validation
      var shoppingCart =
        Sitecore.Ecommerce.Context.Entity.GetInstance<ShoppingCart>();
      if (shoppingCart != null)
      {
        // Configure payment gateway client
        var client = new PaymentGateway();
        client.Url = shoppingCart.PaymentSystem.PaymentSecondaryUrl;
        client.Username = shoppingCart.PaymentSystem.Username;
        client.Password = shoppingCart.PaymentSystem.Password; 

        client.CardNumber = txtCardNumber.Text;
        client.CardName = txtName.Text;
        client.ExpireMonth = txtMonth.Text;
        client.ExpireYear = txtYear.Text;

        if (client.Process())
        {
          // Store the transaction number to indicate success
          var transaction =
            Sitecore.Ecommerce.Context.Entity.Resolve<TransactionData>();
          transaction.SavePersistentValue(shoppingCart.OrderNumber,
            TransactionConstants.TransactionNumber, client.TransactionNumber);

          // Get return page URL from transaction
          var url = transaction.GetPersistentValue(shoppingCart.OrderNumber,
            "returnurl") as string;
          Response.Redirect(url);
        }
        else
        {
          litMessage.Text = "Payment failed. Please try again.";
        }
      }
      else
      {
        litMessage.Text = "Failed to find the shopping cart";
        btnSubmit.Enabled = false;
      }
    }
  }
}

You would of course need to alter the above code to call out to your payment gateway appropriately. Note how I store the transaction number from the payment gateway into my SEFE transaction? I’m going to use this piece of data in my payment provider to determine that the payment was successful. The TransactionConstants class is provided by SEFE to provide keys for common pieces of data you might require.

I’m also using that return page URL I stored when my payment provider was invoked.

Now all that’s left is to process the callback in my payment provider as a result of redirecting to the return page. This method is called by SEFE.

public override void ProcessCallback(
  Sitecore.Ecommerce.DomainModel.Payments.PaymentSystem paymentSystem,
    PaymentArgs paymentArgs)
{
  var transaction = Sitecore.Ecommerce.Context.Entity.Resolve<TransactionData>();
  if (!string.IsNullOrEmpty(transaction.GetPersistentValue(
    paymentArgs.ShoppingCart.OrderNumber,
    TransactionConstants.TransactionNumber) as string))
    PaymentStatus = PaymentStatus.Succeeded;
  else
    PaymentStatus = PaymentStatus.Failure;
}

In this method I’m checking that the transaction number has been populated before setting the outcome of the payment in the PaymentStatus property as succcessful.

Now to register everything with SEFE and get this sample working in the SEFE example site.

SEFE stores the payment providers available for a site under the site’s business catalogue under the payment options item. The payment options item contains settings for all payment providers such as the payment return page and other URLs. Individual payment providers are defined as children of the payment options item and are based on the ecommerce/business catalog/payment data template.

One of the settings of our sample payment provider will be the URL to the credit card form, so we’d better create that first. I’m going to create this page as one of the function pages in the sample checkout.

Make sure you add the credit card form sublayout we created above to the page.

Now we’ll register our payment provider. Create a new payment provider item in the business catalogue and populate the fields appropriately. The important fields are:

SEFE uses Unity to configure many parts of the module, including mapping payment providers. Open up the App_Include/Unit.config file and find an existing payment provider by searching for the code PayByCheck. This is the registration of the code and the mapping to the implementation type. Copy this element and update to use the code we defined in the payment provider item’s code field above, and update the mapTo attribute(which we’ll need to register next). Also, the <property name="PaymentSystem" /> element isn’t required. The resulting updated element may look like the following:

<register type="PaymentProvider"
  mapTo="CreditCardPaymentProvider" name="Credit Card"/>

Now search for the alias implementing the PayByCheck payment provider, OfflinePaymentProvider. This isn’t a .net type, but a Unity alias, we’re looking for the alias to real type mapping element. Copy the existing OfflinePaymentProvider alias and update to map the CreditCardPaymentProvider alias above to the payment provider we created in our project above. The resulting updated element may look like the following:

<alias alias="CreditCardPaymentProvider"
  type="SampleOnsitePaymentProvider.OnsitePaymentProvider,
  SampleOnsitePaymentProvider" />

If everything worked properly then you’ll be able to test your payment provider on the sample site. Just go through the normal checkout process and select credit card as the payment option on the payment page.

And there you have it! The payment provider we’ve created in this post isn’t tied to any particular payment gateway, only the credit card page (that you configure) is. So you can use this technique to host your own payment form in SEFE without redirecting users off-site.

So what is a virtual item? A virtual item is an item that doesn’t actually exist. Why wouldn’t you want the item to exist? One reason would be to keep your content tree clean and another would be to reduce the workload on your content authors. Often virtual items are used to provide a single source of content that can be reused throughout multiple points in your site.

In this post I’m going to visit a few options available inside Sitecore for using virtual items and some related options for achieving those goals mentioned above.

Clone Items

Clone items were introduced in Sitecore 6.4. A cloned item uses another item (source item) in the same way template standard values are used. If a cloned item’s field value is null Sitecore will get the value of that field from the clone source. This allows fields of a cloned item to be overridden without affecting the source item.

This option is good when you want a physical item to be shown in the content tree or when you need to allow content authors to create the cloned items themselves. It’s also a good option for when you want to share most of the fields of the source item, but allow overriding of others when the shared item is used, such as whether it would appear in the breadcrumb (if that was controlled through a field value).

Standard values also works with clones, so if a clone source doesn’t have a field value that field value will be read from standard values right through to the clone item itself. If the value is then overridden on the clone source, that overridden values will be read by the cloned item. The clone is an additional layer in the standard values stack.

Cloned items work just like any other item, cause they are actually a real item. So in that respect they’re not actually virtual. But they can help achieve that goal of having a single source for content and reusing that content throughout the website. Because clone items are real items they will have a different ID to the clone source item.

But if I want a single source of content, allowing users to override values on the cloned item itself goes against that. How can I stop content authors from overriding values on cloned items? Simple! The same way you would stop them from editing any piece of content, by using Sitecore security.

Proxy Items

If you’re using Sitecore 6.4 or lower then proxy items are what you would use instead of clone items. Proxy items were deprecated in Sitecore 6.5, so if you’re starting a new project you shouldn’t be using them. Although still present in Sitecore 6.5, being that they are deprecated Sitecore may remove this feature entirely in a future release.

Proxy items proxy an item (or subtree) from one part of the content tree to another. Updates to a proxy item will update the source item. This means you can’t override a source item value. Other than that, proxy items work in much the same way as clones do.

Unlike clone items which require a physical item in the content tree, proxy items don’t require a physical target item, so they are actually virtual.

Wildcard Items

A wildcard item is an item named with an asterisk (*). It will handle any item requests at a single level in the content tree if another item at the same level isn’t found.

So if I have a content tree as follows:

/sitecore/content/home/a
/sitecore/content/home/c
/sitecore/content/home/*

If I make a request for /sitecore/content/home/b the wildcard item will handle that request. But if I make a request for /sitecore/content/home/a, the a item will handle that.

Wildcard items only work for a single level. In the above example, if I make a request for /sitecore/content/home/b/b I’ll get an item not found error. To handle that case I would need a wildcard item under the wildcard item. So I need a wildcard item for every level I want to be able to handle all requests for.

Wildcard items are also real items, but they allow the requesting of virtual items at a single level of the content tree. Being real items they must exist in the content tree. Some people might think this makes the content tree look messy, particularly if you have a lot of wildcard items.

One way to hide the wildcard items from normal content authors is to make the wildcard item hidden. In the content editor navigate to your wildcard item, select it then in the configure tab click the hide item option in the attributes section. Now the item will only display in the content tree if the user has ticked the hidden items option on the view tab in the content editor.

If you need to have a lot of wildcard items in predictable locations or if you really don’t want them shown in the content tree you may need to consider the next option instead.

Request Pipeline Processor

A fairly straight forward way of handling virtual items at any level without messing up your content tree is to create your own httpRequestBegin processor and insert it into the pipeline. I like to do this after the existing ItemResolver, so in my custom processor I can check to see if the context item is null, and only process the request with my processor if it is. Every httpRequestBegin pipeline processor will run on every request, so you need to ensure processors in this pipeline are as efficient as possible. The simple check for a null context item is very efficient.

With a custom pipeline processor much more complex logic for virtual items can be implemented including checking only for specific item names or allowing any level of virtual item to be requested.

A real world example of this kind of option out in the wild is in the Sitecore eCommerce Fundamental Edition module (SEFE). It uses a custom item resolver after the default Sitecore item resolver to service requests for product which don’t live at the requested location, but exist in another part of the content tree.

I recently used this approach myself in a design. The client wanted an item to appear as a child of another item, but didn’t want that child item to actually be an item in the content tree. All the data required to render the virtual child item was pulled from it’s parent. There were several reasons for this design. Firstly, the client had strict rules around the naming of this item; it must always be called by a specific name. Secondly the virtual child item must always sort in a specific manner; at the bottom of the list of real child pages. Both of those could be achieved through other mechanisms. The biggest factor was the fact that the client didn’t want the virtual item to be edited in a different location to the rest of the data on the parent. They wanted the data for the virtual child item defined on the same item as the parent.

Unlike the above options, this option requires some coding. To implement your own processor, create a class which inherits from HttpRequestProcessor and override the Process method. In your method populate the Sitecore context item to whatever it should be based on the incoming request.

This option is good when you have predictable logic or when you don’t want to show the virtual items in the content tree.

Custom Data Provider

If none of the above options satisfy your exact needs you could always implement your own custom data provider to provide the virtual items. The biggest advantage of this approach is that your virtual items will appear in the content tree.

Defining the virtual items will be up to whatever logic you implement in the data provider. You could automatically return the virtual items based on business rules or you could infer the virtual items from data the content authors input on real items.

One such example of this is the Item Mirror Data Provider written by Steve Green. It shows how you can allow selecting items in one part of the content tree to have virtual representations of those items appear in other parts.

This is the most comprehensive approach to virtual items and the most difficult. Implementing a data provider is no mean feat and should not be undertaken lightly.

Conclusion

So there you have a few different options when designing how your virtual items should work. The biggest considerations will be whether you want content authors to define the virtual items (whether directly or indirectly), whether you want them to appear in the content tree and whether you want to use an out-of-the-box solution or write some code to achieve what you want.

Revolver contains the split command which can be used to split larger strings up. Combined with the echo command which can be used to input text from files and I’ve got the basic building blocks required to import large amounts of items from file.

Let’s give ourselves a solid example to work with. A common task for a lot of people may be to create a list of all the countries in the world for selection in a form. I’ll use Revolver to create a set of reference items containing the countries which can be read and populated into the form.

The first thing I need to do is find a source of all the countries. A bit of googling for “list of countries” brought me to http://www.listofcountriesoftheworld.com/.

This page is perfect for what I need to do.

Next, select the countries from the table and copy them. Now we’re going to use Excel to separate out that numeric identifier from the country name. One thing to note about the way Revolver works, it doesn’t assume you’re working in a vacuum. Yes, it is a command prompt, but unlike a command prompt for an OS it knows that you probably also have a mouse and can select things and copy them and such. Rather than try and replicate complex pasting behaviour into Revolver it makes more sense to use an external tool to get the data into the right format for processing.

So after copying the countries, paste them into Excel. Lucky for us Excel understands HTML tables and knows how to split the blob of HTML we have on the clipboard into separate cells in the spreadsheet. Now delete the number column (the first one). You should be left with a single column of country names.

Now save the file as a CSV so we’re left with a text file containing the textual country names, one on each line of the file.

Fire up Sitecore, log in and start Revolver. I’ll first show you how to read the file using the echo command.

But first, a quick note about line endings. For some reason, when I was preparing this example the CSV file that Excel generated ended up with \r for line endings instead of the standard Windows \r\n line endings. Although I’ve updated Revolver 1.2.1 to deal with mixed line endings if you’re using an earlier version of Revolver you’ll need to convert the line endings to normal Windows line endings \r\n or the UI won’t handle the \r from the file.

echo can be used to input and output content from files using the -f parameter. To have the command input the file rather than output use the -i parameter.

echo -i -f c:\temp\countries.csv

This will list all the countries in the UI. We’ll need to combine this command with the split command to split the lines on the newline characters and execute a command against each line.

The split command contains a parameter for splitting on newline, -n.

split -n < (echo -i -f c:\temp\countries.csv) (echo (found line) $current$)

Note how we pass the previous echo command (the one used to read the file in) as a sub-command? The left angle bracket < causes Revolver to evaluate the next argument and pass the output of that execution into the command. The last parameter supplied to split is the command to execute against each line. We can make use of the special token $current$ in that parameter which will get replaced with the current line from the file. This command will get executed for each of the lines.

Now all we have to do is change the command being executed against each line to create an item using the current line as the name.

split -n < (echo -i -f c:\temp\countries.csv) (touch -t (sample/sample item) ($current$))

Note how we’ve surrounded the $current$ token in parenthesis? This is because the current line may include spaces and we need to ensure they are handled as part of the name parameter and not treated as the path parameter.

But don’t get ahead of yourself and execute that command just yet…if only it was that simple. If you had a look at the country list you’d see that some of the country names contain illegal Sitecore name characters such as comma (,), apostrophe (') and double quotes ("). We’ll need to replace those.

Revolver contains the replace command which allows you to use regular expressions to replace matches in strings. Rather than replace each of the invalid characters, I’ll instead use a regular expression to replace any illegal character with nothing.

split -n < (echo -i -f c:\temp\countries.csv) (touch -t (sample/sample item) < (replace ($current$) [^\w\s] ()))

But I’m not quite happy with that. Although the above will create the list of reference items, I want to include the original country name in the title field of the reference item’s I’m creating for when the item is displayed on the website.

To execute multiple commands for each line, I’ll need to create a Revolver script and call that script in the split command. The script is going to be very simple, just create the item as we did above and set the title field from the original name. We’ll need to allow the original name to be passed into the script. Parameters can be passed to Revolver scripts as they are passed to commands. Inside the script I can reference the passed in parameters using a token $1$ where the number is the 1-based index of the parameter being passed in.

Scripts are created in the core database under the /sitecore/system/modules/revolver/scripts item and are based on the Revolver/Script template. Create a new script named create and fill in the following as the script:

set name < (replace ($1$) [^\w\s] ())
touch -t (sample/sample item) ($name$)
sf -nv title ($1$) ($name$)
set name

Let me explain the script above. Firstly we create an environment variable called name and set it as the output of the replace command we used in our original command above (stripping invalid name characters). We need to use this value multiple times so it’s easier to evaluate the value once and store it rather than continue to evaluate it. Notice the $1$ parameter? That will be substituted with the first parameter passed to the script when it’s executed. Next we create the item using the name we’ve created. After that we set the title field to the original name passed into the script. The last parameter passed to the sf (set field) command is a relative path to execute the command at which is how we have the command executed for the item we just created instead of on the current context item. By default Revolver will create a new version of an item before updating any field values so the -nv (no version) parameter is used to not create a new version of the item before updating the field value. The last command clears the environment variable we set. This is just good housekeeping.

Notice all the parenthesis around the variables $1$ and $name$? Those are to make sure the script works as expected when spaces have been included in those parameters.

To execute my script I just call it by name and pass it the unsafe name of the item to create.

create ("Bahamas, The")

I need to pass the parameter above enclosed in parenthesis cause the parameter does in fact have a space in it. The parenthesis cause the command parser to treat this as a single parameter rather than 2 separate parameters.

Now I can update my split command to make use of this new script to create the country items and set the title fields to the original values.

split -n < (echo -i -f c:\temp\countries.csv) (create ($current$))

And here’s the result of executing that command.

Success!

So you can see how you can use Revolver to not only import content lists from file but also to transform the data in that file as well.

If you want more information on any of the commands used, keep in mind that Revolver contains built in help for all commands. All you need to do is type help and the name of the command to view help for the command.