Extending the Sitecore image processor

12

March 4, 2009 by Alistair Deneys

You’ve no doubt used the dynamic image manipulation capabilities of Sitecore before. This feature allows you to dynamically manipulate images from the media library using the query string. For example, to request an image with width 150 I would append w=150 into the query string:

http://domain/~/media/images/myimage.ashx?w=150

Sitecore will then dynamically scale this image to 150 pixels wide (whist preserving aspect ratio) and send the 150 width image down the wire to me. This means I no longer have to manage a set of thumbnails for each image I upload into Sitecore. What’s more, Sitecore only processes this image once. After the first request the image is cached to disk, so we don’t double up on processing for subsequent requests.

Dynamic image manipulation is achieved in Sitecore by using a pipeline, the getMediaStream pipeline, so I can extend, tweak and alter the pipeline for my purposes.

So let’s add some capabilities to this great feature. I’m going to add a processor to the pipeline to allow flipping of the image requested. First thing to do is create a processor class to be hooked into the pipeline. We’ll add the standard “Process” method. We also need to make sure this method accepts the correct pipeline arguments class.

namespace ImageProc
{
  public class Transform
  {
    public void Process(GetMediaStreamPipelineArgs args)
    {
    }
  }
}

Next we need to pick a query string key to invoke the custom processor on. For this example I’ll use “flip” and invoke the processor if it’s set to “1” in the query string.

if ((string)args.Options.CustomOptions["flip"] == "1")
{
}

Now for the bit that does the work. The following code uses the standard .net Bitmap class to flip the image upside down. You’ll notice most of the code below is used to convert the image data out of the stream and then back into the stream at the end.

var bm = (Bitmap)Bitmap.FromStream(args.OutputStream.Stream);
bm.RotateFlip(RotateFlipType.RotateNoneFlipXY);
var stream = new MemoryStream();
bm.Save(stream, ImageFormat.Png);
args.OutputStream = new MediaStream(stream, "png",
  args.MediaData.MediaItem);

Now to insert our custom processor into the getMediaStream pipeline so we can invoke it. Open web.config and find the pipeline definition. Add the processor at the end of the existing processors.

<processor type="ImageProc.Transform,ImageProc"/>

Now request some media from Sitecore and append flip=1 in the query string.

http://domain/~/media/image/myimage.ashx?flip=1

Here’s the original image:

Oryx Antelope small

And this is what we get:

flipped

So you can see how easy it is to extend the image processor. But that was a pretty simple example. How about something cooler?

Have you heard of the image resizing technique called seam carving? A collegue of mine at Next Digital, Joel Wang, recently sent me a video of seam carving in action. When I watched it, it was truly one of those “Wow” moments. Seam carving is a way in which to resize an image, without affecting the aspect ratio like stretching and scaling does. Seam carving removes lines from the image to preserve the aspect ratio of the important things in the image. This technique was created by Shai Avidan and Ariel Shamir. The best way to get a feel for this technique is to find a video of it in action (there’s a heap out there).

There are already a few implementations of this technique out there. I haven’t found a .net library, but I did manage to find a Windows implementation that could be invoked using the command line. So I can now create a dynamic image processor that uses seam carving.

For this example I’ll be using a release from the CAIR project. CAIR is an implementation of the seam carving technique above. The release includes source code (so I can create a .net implementation at some point 🙂 ) and also a Windows command line executable. Luckily we can invoke executables from inside .net code.

So onto the processor.

using System.Diagnostics;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Web;
using Sitecore.Data;
using Sitecore.Resources.Media;

namespace ImageProc
{
  public class Carver
  {
    const string CAIR_PATH = "c:\\temp\\cair\\cair.exe";

    public void Process(GetMediaStreamPipelineArgs args)
    {
      if (args.Options.CustomOptions["cw"] != null ||
        args.Options.CustomOptions["ch"] != null)
      {
        HttpContext context = HttpContext.Current;
        string folder = context.Server.MapPath
          (Sitecore.Configuration.Settings.TempFolderPath +
            "\\ImageProc\\");
        string temp = folder + ID.NewID.ToString();
        string procfile = temp + "_proc";
        if (!Directory.Exists(folder))
          Directory.CreateDirectory(folder);

        var bm = (Bitmap)Bitmap.FromStream(
          args.OutputStream.Stream);
        bm.Save(temp, ImageFormat.Bmp);

        int targetWidth = bm.Width;
        int targetHeight = bm.Height;

        if (args.Options.CustomOptions["cw"] != null)
          int.TryParse(args.Options.CustomOptions["cw"],
            out targetWidth);

        if(args.Options.CustomOptions["ch"] != null)
          int.TryParse(args.Options.CustomOptions["ch"],
            out targetHeight);

        var proc = new Process();
        proc.StartInfo.FileName = CAIR_PATH;
        proc.StartInfo.Arguments = string.Format(
          "-I {0} -O {1} -X {2} -Y {3}", temp, procfile,
            targetWidth, targetHeight);
        proc.Start();
        proc.WaitForExit();

        if (File.Exists(temp))
          File.Delete(temp);

        if (proc.ExitCode == 0)
        {
          var fstream = File.OpenRead(procfile);
          var pbm = (Bitmap)Bitmap.FromStream(fstream);
          var stream = new MemoryStream();
          pbm.Save(stream, ImageFormat.Png);
          args.OutputStream = new MediaStream(stream, "png",
            args.MediaData.MediaItem);
          fstream.Close();

          if (File.Exists(procfile))
            File.Delete(procfile);
        }
        else
          Sitecore.Diagnostics.Log.Error(
            "Error during CAIR execution: "
            + proc.ExitCode.ToString(), this);
      }
    }
  }
}

You’ll see that most of the code above is to do with saving the image to disk to allow CAIR to manipulate it, then shooting the result back down the stream. I’ve used “cw” and “ch” for “carve width” and “carve height” so I can choose the target size of the image. I of course only have to supply 1 of these, or I can supply both. To invoke the executable I’m using the Process class. Note that I need to wait for the execution of this process to complete before I continue, and I can do this by calling the WaitForExit method on the Process class. The arguments passed to the executable are pretty straight forward. I supply the input, the output file I want generated, the target width and target height. For a full description or for other options, just execute CAIR without any arguments in a command prompt and the help will be displayed.

Don’t forget to add the processor to the pipeline as we did above.

Now to try this out. The above image is 300 pixels wide. Let’s see what it looks like, carved down to 150 pixels.

http://domain/~/media/image/myimage.ashx?cw=150

oryx-cw150

Note how the antelope in the image still looks OK. It hasn’t been skewed or compressed and it’s aspect ratio is still intact. The background in the image is what got removed, but the antelope was left alone.

At this point I hope you’re going “WOW!” just like I was when I first saw this technique.

But does this new way of scaling images affect performance? Well, yes. But only the first request will take that performance hit. After that, Sitecore will cache the processed image on disk and subsequent requests don’t take the performance hit of reprocessing the image.

Now, it wouldn’t be too hard to tweak the code example above to use a different external application to do some image processing for us. In fact, GIMP (the GNU image manipulation program) can be invoked through a command line. This application is an open source image processing and manipulation tool similar to Photoshop. So anything I can do in GIMP, I could automate through the batch interface and link that into a custom image processor.

OMG, I’ve just opened a huge world of possibilities for dynamic image processing with Sitecore and external image processing applications.

Advertisements

12 thoughts on “Extending the Sitecore image processor

  1. OMG, WOW!

    Just like you hoped 🙂

    I wonder what things people miss in that pipeline. It is also very easy to create a shared source project to enhance it with all kinds of abilities, if you can do that in .net

    Reflections, drop shadows?

  2. Lars Nielsen says:

    This is EXTREMELY cool! If we get this as a native .NET, we should definitly included it in Sitecore.

    Alexey, you ask what we also need in the pipeline (parameterbased):
    flipvertically
    fliphorizontally (showed in this article here)
    crop (acting on x1, x2, y1, y2)

    and off course, this CRAZY, INSANE carving thing! 🙂 very very nice!

  3. Lars Nielsen says:

    In fact, I am so baffled that I have to say VERY COOL once again!

  4. Aaron Powell says:

    Do you need to run it in a hightened security mode if you want to use that external application? Could that pose a problem in hosting, particularly for shared hosting?

  5. […] Read more from the original source:? Extending the Sitecore image processor […]

  6. adeneys says:

    Thanks guys!
    Sounds like a great idea for a shared source project. Sign me up as soon as I finish all the other modules (shared source and proprietry) that I’m currently working on.

    Aaron, you make a very good point. Depending on the hosting environment your ASP.NET account may not have adeqaute permissions to launch a process.

    And while we’re on the subject of security and permissions, please don’t use the above code in production. There are a few things that should be done to it to ensure it’s secure. We should always sanitise any input from the user or web. You don’t want to end up like little Bobby tables school http://xkcd.com/327/. The process could also be launched in a seperate AppDomain and CAS used to restrict the permissions of the code.

  7. […] Extending the Sitecore image processor- Superbra info för oss Sitecore junkies. […]

  8. Just thought of another thing for the pipeline: adding copyrights and watermarks on images

  9. Additional points for dropping in little Bobby Tables!

  10. Richard Dias says:

    There are a lot of parameters that are available in System.Drawing that Sitecore hardcodes. The System.Drawing.Drawing2D are all hardcoded. Simply my making the Encoder quality configurable (currently hard coded to 100 which is the highest quality) we can easily render much smaller images with barely noticeable (visually) loss to Image quality.
    I might make a post of an example of this if I every get the time.
    Good article!

  11. […] You can do pretty sick things with our MediaRequestHandler (as an example of that, check out what Alistair Deneys done with it in the past). Today, however, I wanted to cover a fairly easy aspect of this functionality. Specifically, the […]

  12. […] thanks to Alistair Deneys’ post re: extending the image processor in Sitecore for the inspiration. Comments […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

The views expressed on this blog are solely my own and do not necessarily reflect the views of my employer.
%d bloggers like this: