Coffee => Coder => Code
My ramblings on code, Sitecore and stuff

Jun
02

Sitecore supports setting publishing restrictions on content items using dates for when the content should go live and when to take the content off the live site. I find many content authors expect these changes to their content to happen automatically, but in Sitecore’s model a publish operation still needs to occur to have the changes affect the live database. This is because the Master database contains all copies of all the content and when the publishing operation occurs, the publisher is what determines which items will be pushed over to each publishing target. This publisher component reads the publishing restriction fields of the candidate items to determine if they should be pushed over to the publishing target or not.

So once you understand how the publishing model works you can understand why the publish operation still needs to occur once the item is “publishable” for the changes to be realised in the live database. But I do like the idea of having content automatically go live and expire without any outside influence. Let’s go ahead and tweak Sitecore to work this way.

It’s the publisher that moves the items from one database to another, so you’re first reaction may be to write a class which inherits the publisher and override the methods which determine what makes an item “publishable”. The issue is the publisher is not a single class; it is many classes working together to perform this operation. And in addition to this, there are cases when the item we’re trying to have published is actually published. Every publish operation copies the publishable items from the source database to the target database (using the chosen publish mode) and removes expired content from the target database. If you drop down a few layers in Sitecore and inspect the interface of the DataProvider class you’ll find methods to create items, move items, retrieve items and yes, delete items. So we’re not just trying to have our item included in the publish operation, as it may already be included.

Let’s simplify. Rather than trying to change the publishing model which is quite complex, let’s approach the problem from a different angle. What if we were to add a different set of fields to the item which we used to set a go live and expire date for the content on. With these dates set in a different location then the publishing components would publish our item and we could use another mechanism to control when an item is ready to be live.

So we’ve now changed our problem from a publishing problem to an item retrieval problem. Luckily Sitecore contains a single class which is responsible for retrieving items; the Item Manager. We could quite easily extend this class and provide an implementation which takes into account our new custom publishing fields so the item will not be returned if the current date and time doesn’t fall within the range between these 2 fields.

Inspecting the web.config file you’ll find the item manager uses a provider model to provide this functionality. And the default provider class is Sitecore.Data.Managers.ItemProvider. So we can create our own Item Provider class which inherits this class and override the virtual GetItem(ID, Language, Version, Database) method to include a check for our publishing fields. We don’t have to worry about overriding the other GetItem overloaded methods cause they all call into this method eventually.

But we also need to take into account the database in which this code is executing. Note in web.config that we don’t define an Item Provider per database, we have a single globally shared Item Provider. You might also notice the signature for the GetItem method we’re overriding includes the database as a parameter as well. We wouldn’t want to filter the items in master where we’re editing content, otherwise as soon as you set a date which invalidates the current date the item would disappear out of the content editor and you’d never be able to find it to update it again. So we need to make a DB check before we check for the dates.

You may be tempted to just check if the DB name equals “web”, but please don’t. Sitecore allows you to configure multiple content and delivery databases. In the case of multiple delivery databases you would need to check for more DB names. The best way to do this is to check for publishing targets.

A publishing target is merely a database which can have items published into it. These are the databases where we want to filter the items. Due to the fact that the publishing targets are defined in each source database, the ideal solution would be to enumerate the databases and add all the publishing targets found into a list. Unfortunately that would require a call to GetItem (eventually) on the database which will pass that call onto the Item Provider, which we’re currently inside. If you try this you’ll soon become acquainted with a StackOverflowException.

So en lieu of a more seamless solution, we’ll just define in our Item Providers web.config declaration which databases should be filtered.

<add name="default"
  type="Sitecore.Starterkit.Sandbox.ItemProvider, Sitecore.Starterkit">
  <databases hint="list">
    <database>web</database>
  </databases>
</add>

To populate the databases into our class all we have to do is provide a property of type ArrayList and let the Sitecore object factory take care of the rest.

All we have to do now is check to see if we need to filter the items or not. Of course we’ll need to check the database name is in the list we defined in the config file, but we’ll also have to check to see if we’re currently publishing or not. The reason for this is that the Item Provider is used to retrieve all items including when retrieving items during publishing in both the source and target databases. If you filter the items in the target database and the filter is invalidating the item, you won’t be able to update the item to validate the filter again as when the publisher tries to retrieve the item to change it’s field value, the filter will not allow the item to be returned. This is why we need to check for publishing. It’s actually quite easy to check if we’re publishing, just check the name of the context site.

Our resulting class may look like this.

using System;
using System.Collections;
using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Globalization;

namespace Sitecore.Starterkit.Sandbox
{
  public class ItemProvider : Sitecore.Data.Managers.ItemProvider
  {
    private readonly ArrayList m_databases = new ArrayList();

    public ArrayList Databases
    {
      get { return m_databases; }
    }

    protected override Sitecore.Data.Items.Item GetItem(ID itemId,
      Language language, Sitecore.Data.Version version, Database database)
    {
      Item item = base.GetItem(itemId, language, version, database);

      if (database != null &&
        m_databases.IndexOf(database.Name) >= 0 &&
        item != null &&
        Context.Site.Name != "publisher")
      {
        // Perform check for our date fields
        var golive = DateTime.MinValue;
        var expire = DateTime.MaxValue;

        if (item.Name != "__Standard Values")
        {
          if (item.Fields["golive"] != null &&
            item["golive"] != string.Empty)
          {
            DateField goliveField = item.Fields["golive"];
            golive = goliveField.DateTime;
          }

          if (item.Fields["expire"] != null &&
            item["expire"] != string.Empty)
          {
            DateField expireField = item.Fields["expire"];
            expire = expireField.DateTime;
          }

          DateTime now = DateTime.Now;
          if (now < golive || now > expire)
            item = null;
        }
      }

      return item;
    }
  }
}

Note the check for when we’re asked to get a standard values item. If we don’t perform this check we’ll end up in another recursive dead-end loop. Obviously when we query for an item’s field Sitecore in turn looks for the standard values item to check that, so we need to guard against that case.

And lastly we need to define the “auto golive expire” data template which will define the date fields we’re after, and then adjust existing data templates to inherit that template so they can make use of this new capability.

auto publish

And there you have it. Items which inherit from the “auto golive expire” data template can now be dynamically scheduled on the live site and they can appear and disappear without the need for a publish (after you’ve published your schedules of course).

May
21

I’ve recently made a few minor updates to SashimiCMS and released a new version 1.1.2. You can find SashimiCMS on my website at http://users.tpg.com.au/adeneys/sashimiHome.html.

So what is SashimiCMS? SashimiCMS is a CMS I created a few years ago. I didn’t write any compilable code for this CMS. All data is stored in XML files, all transforms are done using XSLT and the thing that binds this all together and builds the site is the Java build technology Apache Ant . The CMS contains no friendly user interfaces and no web interface. It is developer oriented as you have to write the (x)HTML yourself.

But what good is SashimiCMS? To answer that, let’s look at why I created it in the first place. I was redesigning a community website that was being hosted on a free hosting platform which didn’t allow server code. I didn’t want to have to worry about keeping the common page elements such as navigations in check and consistent across the site, so I tried to devise a way in which to automate this. My original idea was to write some code that would replace custom tags in the HTML files. So instead of writing out the navigational components, I could just use a custom tags like <menu/>. And being that I wanted to remain cross platform compatible I was going to write this in Java.

At the same time that I was thinking about this little project I was playing with Apache Ant a fair bit at work. At the time I was doing a little stint as a Solaris sys admin and build manager for a Java project. Through playing with Ant I realized it could execute XSLT directly without any extensions. I already knew about the power of XSLT through a little bit of reading and a brief play with it through using Sitecore 4. So I started to create the first incarnation of SashimiCMS which worked quite well.

The original version of SashimiCMS was actually custom built into the community website I was building. After I launched that site I rewrote SashimiCMS under it’s new name using the website project as a reference. And thus, SashimiCMS was born.

SashimiCMS doesn’t have a huge market share, and it never will. In fact, I only know of 2 sites which use it. One is the community site (hopefully soon to be migrated to Sitecore Xpress ) and the other is my website http://users.tpg.com.au/adeneys.

But looking at SashimiCMS in terms of raw features, it does have some pretty nice facets which I’m proud of. It employs a “page template” approach to pages. You simply write your HTML page template and use it amongst several pages. Each page and template can make use of custom tags which the user creates. So I could create a custom tag for a reusable piece of HTML, such as the webmaster email link might be bound to a custom tag called <email/>. Out of the box there are “controls” to render menus, a breadcrumb, a header, a footer, rss and lists of articles. And SashimiCMS is cross platform. It runs wherever Ant runs; Windows, Linux, Mac, Solaris, Unix, etc.

Each page can also contain multiple content fragments which are bound into the template at build time. So I might define the central content area as a fragment, and some side bar promotional element as another content fragment, and the template pulls these fragments in at build time.

So what should you use SashimiCMS for? SashimiCMS can aid you when you have to generate an HTML only site and the only people interacting with the content will be developers. For any other case at all, use Sitecore!

May
07

Recently I posted the question Alex de Groot asked the Sitecore community last year; “Show Sitecore how to improve…” (I paraphrased to “What’s your Sitecore wishlist”) to the Sitecore Australia and New Zealand users group. A few ideas flew around, one of which was a small feature request to have a magical “unpublish” button for content. Upon pushing this button the item in question would be unpublished from the live website.

So I got to thinking about how I might implement this small feature. It’s actually quite straight forward. I could utilise the content editor as the author is changing attributes on the item. We even already have the attribute we want to change in question: the publishable field. I could even hook the button into one of the existing ribbon chunks on the publish tab.

Incidentally Chris Wojciech is currently doing a series of posts on extending the content editor ribbon with your own commands. Refer to these for more detail on the options you have available when tweaking the ribbon.

Firstly we need to create a new button on the publishing restrictions chunk which will perform the unpublish command for us on the item. Change to the core database and using the content editor, navigate to /sitecore/content/Applications/Content Editor/Ribbons/Chunks/Publish Restrictions. Create a new child item called “Unpublish” based on the /sitecore/templates/System/Ribbon/Large Button template. Fill in the following into the items fields:

Header: Unpublish
Icon: network/16x16/earth_delete.png
Click: item:unpublish
Tooltip: Immediatley unpublish the item

The “Click” field above contains the name of a command to execute on the server when the button is clicked. Normally you define commands in the App_Config/Commands.config file. But let’s keep our custom commands separate from the Sitecore commands and define our commands in our own file. Create the App_Config/CustomCommands.config file and place the following inside the file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<command name="item:unpublish"
  type="MyCommands.UnpublishCommand,MyCommands.dll"/>
</configuration>

We need Sitecore to read our commands file, so open the web.config file and add the following node as a child of the /configuration/sitecore/commands:

<sc.include file="/App_Config/CustomCommands.config"/>

Now we need to implement the command. As you can see from the above configuration, our command is implemented by a .net class. Our class needs to inherit from Sitecore.Shell.Framework.Commands.Command and being that this is an abstract class, we need to implement the “Execute” method.

Inside the execute method we need to get the current item, set it’s publishable flag, then perform a publish on that item to remove it from all the publishing targets.

using System.Collections.Generic;
using Sitecore;
using Sitecore.Data;
using Sitecore.Publishing;
using Sitecore.Shell.Framework.Commands;

public override void Execute(CommandContext context)
{
  if (context.Items.Length > 0 && context.Items[0] != null)
  {
    var item = context.Items[0];
    item.Editing.BeginEdit();
    item.Publishing.NeverPublish = true;
    item.Editing.EndEdit();

    // Grab publishing targets
    var targetsRoot = item.Database.GetItem(
      "/sitecore/system/publishing targets");

    if (targetsRoot == null)
    {
      Sitecore.Context.ClientPage.ClientResponse.Alert(
        "Failed to find the publishing targets");
      return;
    }

    var targets = new List<Database>();
    var children = targetsRoot.GetChildren();
    for (int i = 0; i < children.Count; i++)
    {
      var db = Sitecore.Configuration.Factory.GetDatabase(
        children[i]["target database"]);

      if(db != null)
        targets.Add(db);
    }

    // Perform publish
    PublishManager.PublishItem(item, targets.ToArray(),
      item.Languages, false, false);

    Sitecore.Context.ClientPage.ClientResponse.Alert("Item unpublished");
  }
}

Compile the above code into an assembly and place that assembly in the bin folder. Now everything is ready. Make sure you’re back in the master database, then using the content editor, navigate to an item, select the publish tab and click your new unpublish button. You’ll receive an alert that the item has been unpublished. If you check your web databases you’ll find the item is no longer there.

unpublish before

About to unpublish the item.

unpublish execute

Clicking the new unpublish button.

unpublish after

The item has been unpublished.

And because we used the default Sitecore fields, other features such as the quick action toolbar and the alerts at the top of the fields will work to alert you that your item now has a publishing restriction applied.

A possible extension to this UI tweak would be to have the button as a toggle button. If the item is currently unpublished, then make the item publishable and publish it. Luckily Sitecore commands are not just fire and forget. They also allow feedback so in code I can make my “is item publishable” checks and set the toggle state appropriately. To make this work, I need to override the QueryState method inside my command class.

public override CommandState QueryState(CommandContext context)
{
  if (context.Items.Length > 0 && context.Items[0] != null)
  {
    var item = context.Items[0];
    if (item.Publishing.NeverPublish)
      return CommandState.Down;
  }

  return CommandState.Enabled;
}

Easy! I should probably also in the above code check to see if the current user has write access to the item as well and set the state accordingly. And I’ll need to update the Execute method code to toggle the publishable state of the item.

public override void Execute(CommandContext context)
{
  if (context.Items.Length > 0 && context.Items[0] != null)
  {
    var item = context.Items[0];
    item.Editing.BeginEdit();
item.Publishing.NeverPublish = !item.Publishing.NeverPublish;
    item.Editing.EndEdit();

    // Grab publishing targets
    var targetsRoot = item.Database.GetItem(
      "/sitecore/system/publishing targets");

    if (targetsRoot == null)
    {
      Context.ClientPage.ClientResponse.Alert(
        "Failed to find the publishing targets");

      return;
    }

    var targets = new List<Database>();
    var children = targetsRoot.GetChildren();
    for (int i = 0; i < children.Count; i++)
    {
      var db = Sitecore.Configuration.Factory.GetDatabase(
        children[i]["target database"]);

      if(db != null)
        targets.Add(db);
    }

    // Perform publish
    PublishManager.PublishItem(item, targets.ToArray(),
      item.Languages, false, false);

string response;
    if (item.Publishing.NeverPublish)
      response = "Item Unpublished";
    else
      response = "Item published";

    Context.ClientPage.ClientResponse.Alert(response);
  }
}

unpublish toggle

Unpublish button in toggle mode.

This toggle approach to publishing opens a new publishing style for Sitecore. As content authors are happy with their content, just hit the published (unpublish) toggle button. When you want to take the content offline, hit it again. For some content authors this may feel a bit more comfortable, particularly if they have used other CMSs in the past which employed a similar publishing model to this. Yes this publishing style is already available in Sitecore, but it involves clicking through a few dialogs to make it work. This tweak allows you to bypass those dialogs. And to make this new publishing model really work you’d have to tweak the publishing code to also publish any referenced and dependent items such as the items template, internal linked items and media items.

So there you go Steve, your requested tweak.

Apr
24

For a while now, the best practice for assigning presentation in Sitecore has been to apply the presentation to an item’s template standard values rather than on the item itself. This has some distinct advantages. Firstly, presentation is usually defined by the type of the item, such as a list page or a news page. It makes sense that we have a central place in which to define the presentation for all items of this type. Secondly it is much easier to update presentations site wide if all the presentations are defined in a central location. For example, we might now want to bind a mini-search component onto each page, but want to use dynamic binding and placeholders rather than statically placing the component on a layout or sublayout used by all pages.

But what about those single items that are unique in their presentation? Wouldn’t it make sense to break the rule for these? Just go ahead and define the presentation on the item itself? These items are usually things like the “contact us” item or some other form which only exists in a single place. To keep in line with the best practices, we would have to create a template for each of these unique presentation items which sounds like a lot of work, not to mention the increased number of templates we now have to manage.

So it sounds tempting to break the rule and allow presentation on the item itself. Until it comes time for maintenance on the site. And not maintenance 2 weeks after going live, try after 6 months when those individual items that “we were sure we’d never forget about” are floating around the content tree somewhere we can’t remember :( . Hind sight is 20/20.

So for these reasons, just let me clarify the answer to this dilemma before continuing. Don’t break the rule. Only assign presentation to standard values. It doesn’t matter if you have an additional 15 templates. More templates is far more desirable than trawling the content tree looking for items which might have presentation defined on them.

So what can I do if I had a moment of insanity and ignored this rule and defined presentation on the unique items directly? As usual, it’s Revolver to the rescue!

Unfortunately, if you request a field value in Sitecore, Sitecore will return to you either the field value of the item if it exists, or the standard value if the item doesn’t override the field value.

gf -f __renderings

Using the above command we cannot determine if the field value comes from the item or from the standard values. What’s more, if I list all the item’s fields:

gf

Only fields which are set on the item are returned. But the output of this command is not particularly useable. So how could we determine, through examining field values if an item has overridden the standard values of it’s template. Well we know we can retrieve an item’s field value. We know we could also grab the item’s template id:

ga -a templateid

And we can use IDs with the cd command. So we can easily, from the item, change our context to it’s template, find it’s standard values, then get the field value on the standard values. If the field value is different, then we know the item has overridden the standard values. So let’s combine all this together into a Revolver script, so we can navigate to a single item, execute the script, and get some kind of indication as to whether the item’s field value is the same as it’s template, or if it’s different, indicating an overridden standard value. For the following example, we’ll focus on comparing the __renderings field, which is where the presentation is defined.

Let’s start by storing the item’s field value in an environment variable so we can use it later to compare against.

set itemfield < (gf -f __renderings)

Now we need to navigate to the item’s template so we can get the field value from the standard values.

cd < (ga -a templateid)

About the only way we can compare a field in Revolver to some arbitrary string is using the find command with an expression filter or a field filter. For more info on expressions in Revolver, just type “help expressions” in the Revolver shell prompt. For this example we’ll use a field filter. So we’ll see if we can find the standard values item of the template, by applying the field filter. If we find the item we’ll echo the text “match!”.

find -f __renderings ($itemfield$) (echo match!)

The find command operates on the children of the current item. So in the above example, we’re relying on the fact that the standard values item will be the only item with presentation defined (as well it should be). If a child item (standard values) __renderings field matches the field value from the item, then we get the “match!” text displayed, as well as a message saying we found 1 item.

If this were in a script, we’d also want to return to the previous context item as our current context is on the item’s template. Revolver stores the previous context path whenever the cd command is run in an environment variable called “prevpath”, so we can revert our context by executing:

cd ($prevpath$)

We wrap parenthesis around the variable prevpath in case it includes spaces in it.

We could make this script a little more generic and reusable by using arguments which can be passed to the script to check any field of an item against it’s standard values. This could also be handy for fields such as “display in breadcrumb” and such where you normally use the standard value of “true” (1) but might occasionally want to see what doesn’t follow the default. The resulting script might look like this.

set itemfield < (gf -f ($1$))
cd < (ga -a templateid)
find -f ($1$) ($itemfield$) (echo match!)
cd ($prevpath$)

Let’s name this script “issv” (IS Standard Values). Now we can navigate to an item and execute the script, providing it the field we want to compare.

issv __renderings

Executing this against an item that has been overridden:

<?xml version="1.0" encoding="utf-16"?>
<r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{67FE0428-DFDA-488E-B772-B5DFA2F89A16}" par="" ph="main" uid="{3F6D1B32-8D68-432A-976D-46C3DE95BB82}" />
    <r ds="" id="{F53BB574-1CF8-4446-A127-0EBCA1AA6D9E}" par="" ph="main" uid="{586D9E7F-8CB3-4399-A33C-C5BB90749FC7}" />
  </d>
</r>
/sitecore/templates/User Defined/Doc
Found 0 items
/sitecore/content/Home/New Doc

Executing this against an item which has not been overridden (the standard value is used):

<r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{885B8314-7D8C-4CBB-8000-01421EA8F406}" par="" ph="main" uid="{43222D12-08C9-453B-AE96-D406EBB95126}" />
    <r ds="" id="{CE4ADCFB-7990-4980-83FB-A00C1E3673DB}" par="" ph="/main/centercolumn" uid="{CF044AD9-0332-407A-ABDE-587214A2C808}" />
    <r ds="" id="{493B3A83-0FA7-4484-8FC9-4680991CF743}" par="" ph="/main/centercolumn/content" uid="{B343725A-3A93-446E-A9C8-3A2CBD3DB489}" />
  </d>
  <d id="{46D2F427-4CE5-4E1F-BA10-EF3636F43534}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{493B3A83-0FA7-4484-8FC9-4680991CF743}" par="" ph="content" uid="{A08C9132-DBD1-474F-A2CA-6CA26A4AA650}" />
  </d>
</r>
/sitecore/templates/Sample/Sample Item
match!
Found 1 item
/sitecore/content/Home/Sample Item

See in the second output we get the “match!” string output. This is good if we just want to test single items at a time, but what about searching the entire content tree (content section) for items which override their standard values presentation? Combine the issv script with the find command.

cd /sitecore/content/home
find -r (issv __renderings)

...

<?xml version="1.0" encoding="utf-16"?>
<r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{67FE0428-DFDA-488E-B772-B5DFA2F89A16}" par="" ph="main" uid="{3F6D1B32-8D68-432A-976D-46C3DE95BB82}" />
    <r ds="" id="{F53BB574-1CF8-4446-A127-0EBCA1AA6D9E}" par="" ph="main" uid="{586D9E7F-8CB3-4399-A33C-C5BB90749FC7}" />
  </d>
</r>
/sitecore/templates/User Defined/Doc
Found 0 items
/sitecore/content/Home/New Doc

<r xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <d id="{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{885B8314-7D8C-4CBB-8000-01421EA8F406}" par="" ph="main" uid="{43222D12-08C9-453B-AE96-D406EBB95126}" />
    <r ds="" id="{CE4ADCFB-7990-4980-83FB-A00C1E3673DB}" par="" ph="/main/centercolumn" uid="{CF044AD9-0332-407A-ABDE-587214A2C808}" />
    <r ds="" id="{493B3A83-0FA7-4484-8FC9-4680991CF743}" par="" ph="/main/centercolumn/content" uid="{B343725A-3A93-446E-A9C8-3A2CBD3DB489}" />
  </d>
  <d id="{46D2F427-4CE5-4E1F-BA10-EF3636F43534}" l="{14030E9F-CE92-49C6-AD87-7D49B50E42EA}">
    <r ds="" id="{493B3A83-0FA7-4484-8FC9-4680991CF743}" par="" ph="content" uid="{A08C9132-DBD1-474F-A2CA-6CA26A4AA650}" />
  </d>
</r>
/sitecore/templates/Sample/Sample Item
match!
Found 1 item
/sitecore/content/Home/real

...

So we can now see (once we paw through the output) which item’s override their standard values. And if we pull this into a text editor, all we have to do is search for the “Found 0 items” text to find the culprits.

Another way to check the item’s field value against it’s standard value, and have a bit nicer display of the differences, would be to export the field values and save them into files on the server’s file system, then use an external file compare utility to highlight differences for us. For this approach, we want to export the field value into a file which is stored on disk using the item’s ID as the filename for easy identification of the item during the file comparison. We would also export the standard value for each item into another folder.

Let’s start this approach by exporting the value of the field to a file which uses the item’s ID as the filename.

set p < (ga -a id)
echo -f c:\temp\comp\$p$.txt < (gf -f __renderings)

Now, export the standard value to file.

cd < (ga -a templateid)
echo -f c:\temp\compsv\$p$.txt < (gf -f __renderings (__standard values))

Of course you need to make sure the output folders exist before running the commands.

Now that the item’s field value and the standard values are on disk, we can use an external file compare tool such as WinMerge to compare which files are different. But before we do that, let’s put the above examples into a script so we can run this over all the content items again.

set p < (ga -a id)
echo -f c:\temp\comp\$p$.txt < (gf -f $1$)
cd < (ga -a templateid)
echo -f c:\temp\compsv\$p$.txt < (gf -f $1$ (__standard values))
cd ($prevpath$)

And I’ll name this script “outv”. Now let’s combine that with the recursive find command.

cd /sitecore/content/home
find -r (outv __renderings)

And this is what the folder comparison results look like in WinMerge.

field comp

So I can now see the top three items are different, and I have their IDs so I can easily navigate to them in Revolver, or even paste the ID into the search box inside the desktop to open the content editor on the item.

I hope this helps alleviate some pain for you when you have to do maintenance or updates on those old projects. And I’ve already added a feature request on the Revolver backlog to allow finding items which override standard values much easier.

Apr
17

Managing multiple environment configurations for any project has always been a bit of a challenge. Even when you’re still in development, and different developers have different folder structures and drive assignments which would affect the location of your data folder in Sitecore, not to mention when you move to QA and production where server names will most likely also change.

There are many different ways in which to manage these configurations, but the most robust have always been incorporated into the build process, where you have a master config file from which all other environment specific versions of that file are generated.

At Next Digital, we’ve been making use of the fact that configuration in .net is mainly handled through XML files. Your web.config and App.config files are XML. So we can use XSLT to easily transform one config file into another.

The heart of this technique lies in the xsl:templates that copy over elements and attributes verbatim, so you only have to handle what needs to change. The following templates will do this for you.

<!-- Default templates to match anything else -->
<xsl:template match="@*">
  <xsl:copy/>
</xsl:template>

<xsl:template match="node()">
  <xsl:copy>
    <xsl:apply-templates select="@*"/>
    <xsl:apply-templates/>
  </xsl:copy>
</xsl:template>

Note how the node template copies the current node and inside that node calls apply templates. This means that child elements of the current element will be run through the XSLT which will traverse the entire document.

These 2 templates must appear at the bottom of your XSLT file as templates are applied in the order they appear in the file. If these templates were at the top of the file they would match all nodes and your custom templates would never get run.

If we want to change a specific config value, then all we have to do is provide a template to match the node we want to replace, and change the value. Let’s look at how we would change the data folder in a Sitecore web.config to something else. This is an example of changing an attribute value.

<xsl:template match="/configuration/sitecore/sc.variable
  [@name='dataFolder']/@value">
  <xsl:attribute name="value">C:\inetpub\Proj\dep\
    Sitecore\Data</xsl:attribute>
</xsl:template>

Let’s change the text value of a simple node element such as changing the items cache size of a database.

<xsl:template match="databases/database
  [@id='web']/cacheSizes/items/text()">100MB</xsl:template>

And we might also want to change the contents of a complex node element that contains other nodes such as adding an event handler.

<xsl:template match="events/event
  [@name='item:copying']">
  <xsl:copy>
    <xsl:apply-templates select="@*"/>
    <xsl:apply-templates/>
    <handler type="Class, Assembly" method="Method" />
  </xsl:copy>
</xsl:template>

Here we also have to copy the matched node which in this case is the item:copying event node, so we don’t lose any existing event handlers in the master config file.

Let’s put this all together. The following XSLT will change my dataFolder in a Sitecore web.config file and adjust the hostname to which the “website” site is bound.

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  version="1.0">
  <xsl:output method="xml" indent="yes"/>

  <xsl:template match="/configuration/sitecore/sc.variable
    [@name='dataFolder']/@value">
    <xsl:attribute name=
      "value">C:\inetpub\MySite\dep\
  Sitecore\Data</xsl:attribute>

  </xsl:template> <xsl:templatematch="sites/site[@name='website']/@hostName">
    <xsl:attribute name="
      hostName">www.mysite.net</xsl:attribute>
  </xsl:template>

  <!-- Default templates to match anything else -->
  <xsl:template match="@*">
    <xsl:copy/>
  </xsl:template>

  <xsl:template match="node()">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:apply-templates/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

This kind of config transform is good for managing the config between different environments such as development, QA and production. When a developer adds to or changes the configuration then we can generate the environment specific versions of that config file and we don’t miss those vital configuration updates.

But we don’t want to be kicking this process off manually, let’s integrate it into the build process, so when we perform a release build the environment specific configs are generated on disk for us to move to the target environments. This will take a little bit of MSBuild tweaking.

MSBuild is the build engine used in Visual Studio and your standard project files are actually MSBuild project files. So you’ll need to edit the project file of the project that contains the master config file. You can do this from Visual Studio which means you also get IntelliSense. You’ll first need to unload the project by right clicking on the project in the solution explorer and selecting “Unload project”. Next you can right click on the unloaded project file and select “Edit <yourproject>”. This will open the text (XML) of the project file in the editor window ready for you to start editing.

In MSBuild terms, a task is an operation that needs to be performed in order such as creating a folder, compiling source files or changing the attributes on a file. Unfortunately MSBuild doesn’t come with a default XSLT task. Luckily, MSBuild is very extensible allowing you to write your own tasks. But don’t worry about writing your own XSLT task, there are several third party libraries out there that have already done this for you. In these examples I’ll be using the MSBuild Community Tasks library which you can find at http://msbuildtasks.tigris.org/.

After you’ve installed the community tasks you need to import the tasks into your project file by placing the following at the top of your MSBuild project file just under the “Project” element.

<Import Project="$(MSBuildExtensionsPath)\
  MSBuildCommunityTasks\MSBuild.Community.Tasks.targets" />

Now we can create our own target to generate our config files. A target is a group of tasks which can be invoked from another target or invoked directly by whomever is running the MSBuild file. The following target will use the XSLT task to transform the web.config file into environment specific config files for QA and production.

<Target Name="Configs">
  <Xslt RootTag="" Inputs="web.config" Output="web.QA.config"
    Xsl="web.config.QA.xslt" />
  <Xslt RootTag="" Inputs="web.config" Output="web.prod.config"
    Xsl="web.config.prod.xslt" />
</Target

We can call this target from the “AfterBuild” target, which gets called when a build is successful. We can also place a condition on the call so it will only run for a release build.

<Target Name="AfterBuild">
  <CallTarget Targets="Environment"
    Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' "/>
</Target>

Now reload your project (right click on the project file) and build. Only when you select a release build will your custom config transforms run and the files will be generated on disk in the same folder as your web.config file.

Using the above technique we could generate specific config files for separate developers too. This is good for when the developer is in a different network and uses a different DB server, or their data folder is in a different location. But we don’t want to have to move those config files round by hand. We would much rather integrated this seamlessly into the build process. So that would mean copying the right config file over for this specific developer.

So how can we from inside MSBuild determine which developer is running the build? MSBuild passes all current environment variables in as properties. To access a property, you simply surround the property name with parenthesis and precede it with a dollar sign.

$(Property)

This property will be substituted before the task is run. So we have access to the COMPUTERNAME environment variable from within MSBuild. This is the perfect way to determine which developer or more specifically, which development environment I need the custom configuration for. We’ll take it a step further and only generate the environment specific development config file for this machine if one is required. The easiest way to do that is to include the computer name in the filename of your config transform file.

web.config.devpc.xslt

We can then use a conditional attribute on the XSLT task to only create the config file if the transform file for this machine exists.

<Xslt RootTag="" Inputs="web.master.config"
  Output="web.$(COMPUTERNAME).config"
  Xsl="web.config.$(COMPUTERNAME).xslt"
  Condition="Exists('web.config.$(COMPUTERNAME).xslt')" />

And we’ll couple that with a copy task to copy the output file over the config file, of course using a conditional attribute on the copy task to check if the config file for this machine has been created.

<Copy SourceFiles="web.$(COMPUTERNAME).config"
  DestinationFiles="web.config"
  Condition="Exists('web.$(COMPUTERNAME).config')" />

You’ll also want to precede those 2 tasks with a general copy of the master config file to the running config file for those cases where a machine specific transform doesn’t exist. Your resulting “Configs” MSBuild target might look something like the following. This can be called from the “AfterBuild” target.

<Target Name="Configs">
  <Copy SourceFiles="web.master.config"
    DestinationFiles="web.config" />
  <Xslt RootTag="" Inputs="web.master.config"
    Output="web.$(COMPUTERNAME).config"
    Xsl="web.config.$(COMPUTERNAME).xslt"
    Condition="Exists('web.config.$(COMPUTERNAME).xslt')" />
  <Copy SourceFiles="web.$(COMPUTERNAME).config"
    DestinationFiles="web.config"
    Condition="Exists('web.$(COMPUTERNAME).config')" />
</Target>
Mar
31

Multithreaded code is becoming more common place. Even for web applications. This is largely due to the increasing number of cores appearing on CPUs these days. The chip manufacturers have come close to the limit in terms of how fast a chip can run before it starts to melt. So to continue increases in processing speed, instead of continuing to try and increase speed vertically by increasing chip speed, they have started to increase speed horizontally by increasing the number of cores on each chip. To make use of this increased processing power, we need to make sure our code can run multiple threads at a time, as each CPU can only work on a single thread at a time.

In .net we have many ways in which we can code to enable our application to run multiple threads at once. The first technique we all normally use is the Thread class. This class allows us to explicitly control a thread along with it’s execution and lifetime. To start a new thread we can use the following:

ThreadStart threadStart = new ThreadStart(Method);
Thread thread = new Thread(threadStart);
thread.Start();

This code will execute a method on the current class in another thread. To block the calling thread until the new thread completes I can use:

thread.Join();

The method in the above case must return void and cannot accept any parameters. If we want to pass parameters then we can use a ParameterizedThreadStart instead of the ThreadStart above.

MyClass data = new MyClass();
data.Prop1 = "data1";
data.Prop2 = 2;

var threadStart = new ParameterizedThreadStart(Method);
Thread thread = new Thread(threadStart);
thread.Start(data);

The Method should still return void but now must accept a single parameter of type object.

void Method(object data) { }

But what if I want to pass multiple parameters and perhaps even accept a return value? For multiple parameters I could create a custom class with a property for each parameter we wish to pass. But I still wouldn’t be able to return a value from the thread. Instead I can use delegates.

Delegates are essentially method pointers that point to the method we want to run. This is how .net does event handling. When I subscribe to the Click event of a button, I’m passing to the event a delegate which points to the method I want to run when that event occurs. Delegates have asynchronous programming capabilities built in through the use of the BeginInvoke and EndInvoke methods. Being asynchronous, each invocation is run on a separate new thread.

I’ll start by defining my delegate with the parameters I want to pass and the return type I require.

delegate int AsyncMethod(int num1, int num2, string message);

Now I create an instance of my delegate and begin the invocation asynchronously.

AsyncMethod myMethod = new AsyncMethod(Method);
IAsyncResult result = myMethod.BeginInvoke(4, 6, "my message",
  null, null);

And the method that is invoked:

static int Method(int num1, int num2, string message)
{
  return num1 + num2;
}

We use the IAsyncResult object returned from BeginInvoke to identify the instance of the invocation and to block the calling thread and get the return result.

At this point the new thread has been created and is running in the background. I can block the calling thread by calling EndInvoke on the delegate which is also how I get the return value from the execution.

int res = myMethod.EndInvoke(result);

If I have multiple threads which I’ve created and want to wait for them all to finish, I can use the WaitHandle.WaitAll method and pass it the array of WaitHandles (which I can get from IAsyncResult.AsyncWaitHandle) that I wish to wait for.

delegate int NumMethod(int data);

int Method(int data)
{
  return -1;
}

void Process()
{
  int[] inputData = PartitionData();
  NumMethod myMethod = new NumMethod(Method);
  WaitHandle[] handles = new WaitHandle[inputData.Length];
  for(int i = 0; i < inputData.Length; i++)
  {
    IAsyncResult result = myMethod.BeginInvoke(inputData[i],
      null, null);
    handles[i] = result.AsyncWaitHandle;
  }

  WaitHandle.WaitAll(handles);
}

One drawback of the delegate technique is that we can no longer control the lifetime of the thread that is running our code. This may or may not be a problem.

Mar
18

I really like workflow in Sitecore. It’s simple but powerful. The workbox is a great view into workflow and anything that needs my attention. By default, the workbox displays items in workflow states ordered by name. At times it would be nice if I could order by some other attribute like the updated field. So let’s add sorting functionality to the workbox.

We’ll start by adding another ribbon chunk to hold the sorting controls. I want a drop list to change the attribute used to sort, and a checkbox to either sort ascending or descending on the selected attribute.

Create yourself a new project in .net to add code to.

Jump over into the core database and navigate to /sitecore/content/Applications/Workbox/Ribbon/Home. Here we want to create a new item based on the system/Ribbon/Chunk template and call it “Sort”. Fill in the header and ID to reflect that this is the sort chunk. Create a child of the sort chunk item called “Field” based on the system/Ribbon/Panel template. This template allows us to specify a .net type which is used to create the controls of the panel. So in the project you created above, define a class to do this. It must inherit from Sitecore.Shell.Web.UI.WebControls.RibbonPanel. Enter this type in the “type” field of the “panel” item. Now I want to define the attributes that can be used for sorting. These will be children of the “field” item. But first I’ll have to create a new template to base these items on.

Create a new “Field Option” template that contains 2 fields; Header and Field. The header will be displayed in the droplist control while the field will be used to sort the workbox on this selected field. Now go back and create the field options to sort on under the “field” item.

sortfields

Although these define a field to sort on, I also added an option to sort by name (the default) with a special token “@name”. I also created sort options for the “__created”, “__updated” and “priority” fields. Priority is a custom field for my implementation.

Now onto the UI.

I know I’m going to want some kind of persistent data storage to store our sorting options. And luckily Sitecore provides us with an option; the Sitecore.Web.UI.HtmlControls.Registry class. This class provides functions very much like the Windows Registry, where I can persistently store data (int, string, bool) for the user.

I’ll make my life easier by creating a little Util class which will handle the storage and retrieval of the sorting options in the registry.

public static class Util
{
  public static string GetSortField()
  {
    return Registry.GetString("/Current_User/Workflow/Sort_Field",
      "@name");
  }

  public static void SetSortField(string field)
  {
    Registry.SetString("/Current_User/Workflow/Sort_Field", field);
  }

  public static bool GetSortOrder()
  {
    return Registry.GetBool("/Current_User/Workflow/Sort_Order", false);
  }

  public static void SetSortOrder(bool order)
  {
    Registry.SetBool("/Current_User/Workflow/Sort_Order", order);
  }
}

For our sort panel control, we need to override the “Render” method which is where we’ll create our controls.

public override void Render(System.Web.UI.HtmlTextWriter output,
  Sitecore.Web.UI.WebControls.Ribbons.Ribbon ribbon,
  Sitecore.Data.Items.Item button,
  Sitecore.Shell.Framework.Commands.CommandContext context)
{
  output.Write("<div class=\"scRibbonToolbarPanel\">");
  output.Write("<table class=\"scWorkboxPageSize\"><tr><td
    class=\"scWorkboxPageSizeLabel\">");
  output.Write("Field:</td><td>");
  output.Write("<select id=\"SortField\"
    class=\"scWorkboxPageSizeCombobox\"
    onchange='javascript:scForm.invoke(\"SortChange\")'>");

  var currentSortField = Util.GetSortField();
  var descOrder = Util.GetSortOrder();

  ChildList fields = button.GetChildren();
  foreach (Item field in fields)
  {
    string val = field["field"];
    output.Write("<option value=\"" + val + "\" " +
      (val == currentSortField ? "selected" : string.Empty) + ">" +
      field["header"] + "</option>");
  }

  output.Write("</select>");
  output.Write("</td></tr></table></div>");

  // Descending checkbox
  ribbon.BeginSmallButtons(output);
  var descButton = new SmallCheckButton();
  descButton.Header = "Descending";
  descButton.ID = "SortDesc";
  descButton.Checked = descOrder;
  descButton.Command = "OrderChange";
  ribbon.RenderSmallButton(output, descButton);
  ribbon.EndSmallButtons(output);
}

In the code above I first create a droplist control and add entries to it based on the items for field sorting we created earlier. When the selection is changed Sitecore will invoke a method on the form called SortChange which takes no parameters and returns void. Note I use the Util methods to get the current options from the registry so I can make sure the UI is in the correct state when it loads. Then I create a checkbox field which will invoke a method called OrderChange on the form class.

So now we need to extend the workbox form class to include these methods. Create a new class in your VS project which inherits from Sitecore.Shell.Applications.Workbox.WorkboxForm and add the two methods. Inside these methods I’ll make calls to the Util methods to store the new sorting options.

protected void SortChange()
{
  var value =
    Sitecore.Context.ClientPage.ClientRequest.Form["SortField"];
  Util.SetSortField(value);
}

protected void OrderChange()
{
  bool current = Util.GetSortOrder();
  Util.SetSortOrder(!current);
}

The workbox application uses an XML layout, so open this file which should be found at \Website\sitecore\shell\Applications\Workbox\Workbox.xml. The only thing we need to change in this file is the code beside class which is defined in the CodeBeside tag. Change the type attribute of this tag to the type of the extended form we created above.

sort workbox scaled

So, now we can actually perform the sorting! Checking the methods available on the WorkboxForm class, there is a GetItems method which is used to retrieve the items to display in the form. Unfortunately, this method is private so we can’t override it. What other options do we have to affect the sorting?

Having a poke around the API, the DataProvider class has a public virtual method called GetItemsInWorkflowState which we could override. But I don’t like the idea of adjusting such a fundamental core piece just to affect sorting in an application. Not to mention that if I did extend that class I would actually have to extend all the concrete DataProvider implementations (DataProvider is abstract). That sounds like a lot of work.

Further up the stack sits the workflow itself which is implemented through the Sitecore.Workflows.Simple.Workflow class. This class has a public virtual method GetItems which is used to get the items in a particular workflow state. Perfect! We’ll go with this.

But if I extend the Workflow class, how can I have Sitecore create my workflow instead of the standard one? This is the responsibility of the WorkflowProvider which is defined in the web.config file. So we can extend the WorkflowProvider to create instances of our custom workflow class instead of the default one.

There are 3 methods from which workflows can be returned, so we’ll override those. I don’t want to have to reimplement every one of these methods though. I want to leverage the functionality of the base class as much as possible. So I can have the base WorkflowProvider class create the standard workflow, and then wrap this workflow with my custom workflow. So rather than subclass, I’m instead going to use the Decorator design pattern to wrap a workflow and alter some of the behavior of the methods. One of the aspects of the Decorator pattern is that my class has the same interface as the class it’s wrapping. Luckily the methods we’re dealing with here only requires an object which implements the IWorkflow interface.

So let’s start with implementing the IWorkflow interface. And as we’ll be wrapping the other workflow, we may as well create a single constructor which accepts the workflow to wrap as a parameter. We’ll store the workflow as a member variable and leverage it for all the other methods.

class SortingWorkflow : IWorkflow
{
  IWorkflow m_innerWorkflow = null;

  public SortingWorkflow(IWorkflow innerWorkflow)
  {
    m_innerWorkflow = innerWorkflow;
  }

  public Appearance Appearance
  {
    get { return m_innerWorkflow.Appearance; }
  }

  public WorkflowCommand[] GetCommands(string stateID)
  {
    return m_innerWorkflow.GetCommands(stateID);
  }

  public WorkflowCommand[] GetCommands(Sitecore.Data.Items.Item item)
  {
    return m_innerWorkflow.GetCommands(item);
  }

  // Wrap the other methods in the same way
}

The one method we want to alter is the GetItems method. This method can use the wrapped classes method to get the items, then it needs to reorder the array before returning it. We’ve already stored the sort options in the registry which we can get to from anywhere.

string m_sortField = "@name";
bool m_descSort = false;

public DataUri[] GetItems(string stateID)
{
  if (m_innerWorkflow != null)
  {
    var items = m_innerWorkflow.GetItems(stateID);
    m_sortField = Util.GetSortField();
    m_descSort = Util.GetSortOrder();
    Array.Sort(items, new Comparison<DataUri>(CompareDataUri));

    return items;
  }

  return new DataUri[]{};
}

private int CompareDataUri(DataUri x, DataUri y)
{
  Item itemX = Sitecore.Context.ContentDatabase.GetItem(x);
  Item itemY = Sitecore.Context.ContentDatabase.GetItem(y);

  var res = 0;
  if(m_sortField == "@name")
    res = string.Compare(itemX.Name, itemY.Name);
  else
    res = string.Compare(itemX[m_sortField], itemY[m_sortField]);

  if (m_descSort)
  {
    if (res > 0)
      return -1;
    if (res < 0)
      return 1;
  }

  return res;
}

I’m using the Array.Sort method here with an external comparison method. Note in the comparison method we check for the name token (@name) and treat that case a bit different.

Now we have the decorated workflow done we can override the methods in the workflow provider.

public class WorkflowProvider :
  Sitecore.Workflows.Simple.WorkflowProvider
{
  public WorkflowProvider(string dbName, HistoryStore store) :
    base(dbName, store) { }

  public override IWorkflow GetWorkflow(Sitecore.Data.Items.Item item)
  {
    return new SortingWorkflow(base.GetWorkflow(item));
  }

  public override IWorkflow GetWorkflow(string workflowID)
  {
    return new SortingWorkflow(base.GetWorkflow(workflowID));
  }

  public override IWorkflow[] GetWorkflows()
  {
    var workflows = base.GetWorkflows();
    var toRet = new IWorkflow[ workflows.Length ];
    for (int i = 0; i < workflows.Length; i++)
      toRet[i] = new SortingWorkflow(workflows[i]);
    return toRet;
  }
}

Now the last thing to do is update the configuration to have Sitecore use the new sorting workflow provider. Open the web.config file and find the master database definition and the workflowProvider inside that. Change the type to the sorting workflow provider created above.

And there we have it. We’ve now added sorting functionality to the workbox.

That was a lot of code and changes to make that happen. So consider this an official call out to the Sitecore developers to PLEASE change the workbox form class methods to protected virtual.

Mar
04

You’ve no doubt used the dynamic image manipulation capabilities of Sitecore before. This feature allows you to dynamically manipulate images from the media library using the query string. For example, to request an image with width 150 I would append w=150 into the query string:

http://domain/~/media/images/myimage.ashx?w=150

Sitecore will then dynamically scale this image to 150 pixels wide (whist preserving aspect ratio) and send the 150 width image down the wire to me. This means I no longer have to manage a set of thumbnails for each image I upload into Sitecore. What’s more, Sitecore only processes this image once. After the first request the image is cached to disk, so we don’t double up on processing for subsequent requests.

Dynamic image manipulation is achieved in Sitecore by using a pipeline, the getMediaStream pipeline, so I can extend, tweak and alter the pipeline for my purposes.

So let’s add some capabilities to this great feature. I’m going to add a processor to the pipeline to allow flipping of the image requested. First thing to do is create a processor class to be hooked into the pipeline. We’ll add the standard “Process” method. We also need to make sure this method accepts the correct pipeline arguments class.

namespace ImageProc
{
  public class Transform
  {
    public void Process(GetMediaStreamPipelineArgs args)
    {
    }
  }
}

Next we need to pick a query string key to invoke the custom processor on. For this example I’ll use “flip” and invoke the processor if it’s set to “1″ in the query string.

if ((string)args.Options.CustomOptions["flip"] == "1")
{
}

Now for the bit that does the work. The following code uses the standard .net Bitmap class to flip the image upside down. You’ll notice most of the code below is used to convert the image data out of the stream and then back into the stream at the end.

var bm = (Bitmap)Bitmap.FromStream(args.OutputStream.Stream);
bm.RotateFlip(RotateFlipType.RotateNoneFlipXY);
var stream = new MemoryStream();
bm.Save(stream, ImageFormat.Png);
args.OutputStream = new MediaStream(stream, "png",
  args.MediaData.MediaItem);

Now to insert our custom processor into the getMediaStream pipeline so we can invoke it. Open web.config and find the pipeline definition. Add the processor at the end of the existing processors.

<processor type="ImageProc.Transform,ImageProc"/>

Now request some media from Sitecore and append flip=1 in the query string.

http://domain/~/media/image/myimage.ashx?flip=1

Here’s the original image:

Oryx Antelope small

And this is what we get:

flipped

So you can see how easy it is to extend the image processor. But that was a pretty simple example. How about something cooler?

Have you heard of the image resizing technique called seam carving? A collegue of mine at Next Digital, Joel Wang, recently sent me a video of seam carving in action. When I watched it, it was truly one of those “Wow” moments. Seam carving is a way in which to resize an image, without affecting the aspect ratio like stretching and scaling does. Seam carving removes lines from the image to preserve the aspect ratio of the important things in the image. This technique was created by Shai Avidan and Ariel Shamir. The best way to get a feel for this technique is to find a video of it in action (there’s a heap out there).

There are already a few implementations of this technique out there. I haven’t found a .net library, but I did manage to find a Windows implementation that could be invoked using the command line. So I can now create a dynamic image processor that uses seam carving.

For this example I’ll be using a release from the CAIR project. CAIR is an implementation of the seam carving technique above. The release includes source code (so I can create a .net implementation at some point :) ) and also a Windows command line executable. Luckily we can invoke executables from inside .net code.

So onto the processor.

using System.Diagnostics;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Web;
using Sitecore.Data;
using Sitecore.Resources.Media;

namespace ImageProc
{
  public class Carver
  {
    const string CAIR_PATH = "c:\\temp\\cair\\cair.exe";

    public void Process(GetMediaStreamPipelineArgs args)
    {
      if (args.Options.CustomOptions["cw"] != null ||
        args.Options.CustomOptions["ch"] != null)
      {
        HttpContext context = HttpContext.Current;
        string folder = context.Server.MapPath
          (Sitecore.Configuration.Settings.TempFolderPath +
            "\\ImageProc\\");
        string temp = folder + ID.NewID.ToString();
        string procfile = temp + "_proc";
        if (!Directory.Exists(folder))
          Directory.CreateDirectory(folder);

        var bm = (Bitmap)Bitmap.FromStream(
          args.OutputStream.Stream);
        bm.Save(temp, ImageFormat.Bmp);

        int targetWidth = bm.Width;
        int targetHeight = bm.Height;

        if (args.Options.CustomOptions["cw"] != null)
          int.TryParse(args.Options.CustomOptions["cw"],
            out targetWidth);

        if(args.Options.CustomOptions["ch"] != null)
          int.TryParse(args.Options.CustomOptions["ch"],
            out targetHeight);

        var proc = new Process();
        proc.StartInfo.FileName = CAIR_PATH;
        proc.StartInfo.Arguments = string.Format(
          "-I {0} -O {1} -X {2} -Y {3}", temp, procfile,
            targetWidth, targetHeight);
        proc.Start();
        proc.WaitForExit();

        if (File.Exists(temp))
          File.Delete(temp);

        if (proc.ExitCode == 0)
        {
          var fstream = File.OpenRead(procfile);
          var pbm = (Bitmap)Bitmap.FromStream(fstream);
          var stream = new MemoryStream();
          pbm.Save(stream, ImageFormat.Png);
          args.OutputStream = new MediaStream(stream, "png",
            args.MediaData.MediaItem);
          fstream.Close();

          if (File.Exists(procfile))
            File.Delete(procfile);
        }
        else
          Sitecore.Diagnostics.Log.Error(
            "Error during CAIR execution: "
            + proc.ExitCode.ToString(), this);
      }
    }
  }
}

You’ll see that most of the code above is to do with saving the image to disk to allow CAIR to manipulate it, then shooting the result back down the stream. I’ve used “cw” and “ch” for “carve width” and “carve height” so I can choose the target size of the image. I of course only have to supply 1 of these, or I can supply both. To invoke the executable I’m using the Process class. Note that I need to wait for the execution of this process to complete before I continue, and I can do this by calling the WaitForExit method on the Process class. The arguments passed to the executable are pretty straight forward. I supply the input, the output file I want generated, the target width and target height. For a full description or for other options, just execute CAIR without any arguments in a command prompt and the help will be displayed.

Don’t forget to add the processor to the pipeline as we did above.

Now to try this out. The above image is 300 pixels wide. Let’s see what it looks like, carved down to 150 pixels.

http://domain/~/media/image/myimage.ashx?cw=150

oryx-cw150

Note how the antelope in the image still looks OK. It hasn’t been skewed or compressed and it’s aspect ratio is still intact. The background in the image is what got removed, but the antelope was left alone.

At this point I hope you’re going “WOW!” just like I was when I first saw this technique.

But does this new way of scaling images affect performance? Well, yes. But only the first request will take that performance hit. After that, Sitecore will cache the processed image on disk and subsequent requests don’t take the performance hit of reprocessing the image.

Now, it wouldn’t be too hard to tweak the code example above to use a different external application to do some image processing for us. In fact, GIMP (the GNU image manipulation program) can be invoked through a command line. This application is an open source image processing and manipulation tool similar to Photoshop. So anything I can do in GIMP, I could automate through the batch interface and link that into a custom image processor.

OMG, I’ve just opened a huge world of possibilities for dynamic image processing with Sitecore and external image processing applications.

Feb
24

Hang on, memory leaks? But this is .net where memory leaks are a thing of the past. If you’ve never done C or C++ before, then you’ve probably never had the enjoyment of searching through your code looking for memory leaks. A memory leak is memory an application allocates, but doesn’t deallocate when it’s done with it. .net solved this issue by not letting the developer manage the memory of the application. Instead, .net uses a garbage collector which cleans up unreachable memory so the developer doesn’t have to worry about freeing the memory from dead objects anymore.

The way in which the garbage collector works out if an object is still being used is by walking through each object in memory to see if other objects hold a reference to it. If a reference exists then the object is reachable and is not destroyed.

So if the garbage collector is taking care of cleaning up old objects, how can I have a memory leak in .net? Simply if I hold a reference to an object I assumed would be destroyed. This happens when using events. An event subscriber is a reference and this reference will prevent a subscriber from being collected.

Allow me illustrate with some code.

using System;
namespace EventLeak
{
  class Program
  {
    static Broadcast m_broadcast = null;

    static void Main(string[] args)
    {
      Setup();
      m_broadcast.CallBing();
      Console.Read();
    }

    public static void Setup()
    {
      m_broadcast = new Broadcast();
      Subscriber s = new Subscriber(m_broadcast);
    }
  }

  class Broadcast
  {
    public event EventHandler Bing;

    public void CallBing()
    {
      if (Bing != null)
        Bing.Invoke(this, new EventArgs());
    }
  }

  class Subscriber
  {
    public Subscriber(Broadcast b)
    {
      b.Bing += new EventHandler(Handler);
    }

    public void Handler(object sender, EventArgs args)
    {
       Console.WriteLine(this.GetHashCode().ToString() +
         " Handled object");
    }
  }
}

In the above code sample, the subscriber is created as a local variable in the Setup method. Once we leave this method the object goes out of scope and if not for the event subscription, the subscriber object would be eligible for garbage collection. But the subscription keeps the object alive. If you execute the above program you will see the subscriber outputs to the console indicating it’s not dead.

I have in the past relied on this behavior. It allows me to create objects and hook them into events, but not have to worry about keeping them alive. But a problem arises if you assume the object is going to be destroyed when you leave it’s scope. This is where the memory leak is. We assume an object will be destroyed, but it isn’t.

In 2005, a team form Princeton University entered into the DARPA grand challenge in which teams had to design and create an autonomous car to drive around a predefined course. This team chose .net as their platform. They performed very well and made it though to the national finals. During this final race, all was going well, until the car ran out of memory. For each obstacle the sensors detected, an object was created and hooked into events of another part of the system. The team assumed the objects were being destroyed when they left scope, but because they were subscribers to an event, they didn’t and so a memory leak occurred. You can read all about it (and their plug for ants profiler) at http://www.codeproject.com/KB/showcase/IfOnlyWedUsedANTSProfiler.aspx.

And Sitecore is no exception with it’s event pool. In Sitecore, we can programmatically subscribe to events from the event pool using the Subscribe method of the Event class.

Sitecore.Events.Event.Subscribe("item:created",
  new EventHandler(Handler));

This is the same as the more familiar += event subscription notation.

So how do I have my objects die and not live forever? I need to unsubscribe from the event. In standard .net I need to use -= to remove the handler, and for the Sitecore event pool I need to use the Unsubscribe method of the Event class.

Button.Click -= new EventHandler(Handler);
Sitecore.Events.Event.Unsubscribe("item:created",
  new EventHandler(Handler));

When to unsubscribe will depend on exactly what you are doing, but most often the perfect place to unsubscribe will be when we decide the object is no longer needed. For this we need to implement the IDisposable interface and remove the subscription in the Dispose method.

class Subscriber : IDisposable
{
  Broadcast m_broadcast = null;

  public Subscriber(Broadcast b)
  {
    m_broadcast = b;
    m_broadcast.Bing += new EventHandler(Handler);
  }

  public void Dispose()
  {
    m_broadcast.Bing -= new EventHandler(Handler);
  }

  public void Handler(object sender, EventArgs args)
  {
    Console.WriteLine(this.GetHashCode().ToString() +
      " Handled object");
  }
}

So the bottom line is, if I subscribe to an event and the subscriber doesn’t live as long as the application, unsubscribe when the object is disposed.

Feb
16

I’ve been doing a fair bit of WCF lately. And an absolute nightmare of an error to try to debug is the “Channel was in a faulted state” error. This is a very generic error and doesn’t really say much at all about the cause of the error.

The way WCF communicates is through channels. When a channel is in the faulted state, it means the construction and opening of a channel failed for some reason, and the channel faulted. Once in the faulted state a channel cannot be used. It must be constructed again.

So how do I debug this issue and resolve the cause of the error? If only there were some way of getting more information out of the channel creation process. Enter the WCF Trace Tool. WCF traces a fair bit of information when it’s doing it’s thing. There’s a lot going on under the hood, and Microsoft realise that we need to know what exactly is going on. So now all we have to do, is catch that trace.

Open up web.config and add the following configuration to /configuration/system.serviceModel:

<diagnostics performanceCounters="All" wmiProviderEnabled="true">
  <messageLogging logEntireMessage="true" logMalformedMessages="true"
    logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true"
    maxMessagesToLog="100000"/>
</diagnostics>

Then add the following to /configuration/system.diagnostics:

<sharedListeners>
  <add name="sharedListener"
    type="System.Diagnostics.XmlWriterTraceListener"
    initializeData="c:\logs\servicetrace.svclog" />
</sharedListeners>
<sources>
  <source name="System.ServiceModel" switchValue="Verbose,
ActivityTracing" >
    <listeners>
      <add name="sharedListener" />
    </listeners>
  </source>
  <source name="System.ServiceModel.MessageLogging"
    switchValue="Verbose">
    <listeners>
      <add name="sharedListener" />
    </listeners>
  </source>
</sources>

Thank Michele Leroux Bustamante for this info from her blog entry SvcTraceViewer.exe – Learn it, Know it, Love it.

Now when WCF is invoked, an XML trace file is written to the location above in the initializeData attribute. You could try inspecting this file in a text or XML editor, but the easiest way to inspect it is to use the WCF Trace Viewer which ships with the .net 3.0 SDK. This can be found at c:\Program Files\Microsoft Visual Studio 9.0\SvcTraceViewer.exe (that might only be the location if you have VS installed).

Just execute this program and load the XML trace file into it using the file menu. Now I can inspect the process of channel creation, servicing requests, and channel destruction. If I get an error it will be highlighted red.

Now let me tell you about what I was doing when I found this tool, as I reckon this is a pretty cool feature of WCF. Being that I’m building web sites, I of course usually host my WCF services in IIS. IIS only supports the HTTP protocols. As we all know HTTP only supports request / response calls. A server cannot send data to a client unless it is a response to a request. But WCF allows us to get around this limitation of HTTP, by using the duplex channel.

A duplex channel facilitates two way communication between a client and the server. How is this done? When the client sets up it’s channel, it communicates with the server and registers itself. To allow the server to call back into the client, the client sets up it’s own WCF service and tells the server about it, so the server can just call the service hosted by the client. So to facilitate two way communication, each machine hosts it’s own service which the other machine makes calls to. And the client side service is completely transparent to the developer.

So what did I need the trace tool for? Initially I was hosting the service I had issues with on the same machine as I was calling it from. Whilst both the server and the client were the same machine, I didn’t have any issues. I then moved the service onto another machine that was only connected to the first through a VPN. All of a sudden I started getting the faulted channel error. After enabling the trace and reading it in the trace viewer I could see the call where the client registers it’s callback service with the server. And I could see that the host name the client was sending to the server was the internal machine name which wouldn’t resolve through the VPN. After I changed this by setting the clientBaseAddress attribute on the binding, and applying the appropriate security, all worked fine.

I never would have worked this out had it not been for the trace viewer. I could have ripped out wireshark (you know it’s a bad day when you have to rip out wireshark :) ), but this was a production server where the error was occurring, and I wouldn’t have liked to deploy such a heavy tool to work it out. I hope this save you some sanity.