Making Umbraco headless

EDIT: I have uploaded the source code to Github: https://github.com/nickfrederiksen/HeadlessUmbracoTest

Umbraco HQ offers a paid hosted headless implementation of Umbraco called ”Heartcore”. This is good for some, but for others that wants more than Umbraco Heartcore can offer, there is not much help from the Umbraco Core, since Heartcore more or less runs a custom build / highly modified version of Umbraco that we cannot utilize.

So, we are essentially forced to build our own headless implementation on top of Umbraco to get the same functions.

In this post, I am going to share my experience with building a headless API on top of Umbraco and the issues I have encountered.

My implementation will not be compatible with the way Umbraco has chosen to do Heartcore, but it would be interesting if, at some point, the client libraries that is used for Heartcore, could be used for on premise installs as well.

But, as this is just a PoC based upon own experiences on requirements, it will not be compliant or feature complete. At all. But it will work.

I expect you know about route hijacking, composers and components, web api and other rather advanced stuff.

I will upload my code to GitHub. That code will be more complete than this post.

Routing

First things first, we need to setup some custom routes. Since we are, essentially, building a RESTful API, I think the routes should be prefixed with “/api”. This way we distinguish between our API’s and other endpoints.

I have identified at least 4 routes:

  • /api/content/{contentGuid}
  • /api/media/{mediaGuid}
  • /api/sitemap/{parentGuid}
  • /api/dictionary

The “parentGuid” parameter is optional.

These 4 endpoints add a few problems that needs to be resolved:

  • Finding correct content page and variation from guid.
  • Setting the Umbraco context up with the correct content, as it would have been done on a normal request.
  • Routing into custom doctype controllers, as one would do with a normal Umbraco site.

NB: One might argue about using RouteTable.Routes.MapUmbracoRoute and implement an IContentFinder instead of the way shown here. But that is for MVC routes and controllers, and that gave me a lot of headaches…

Finding the correct Umbraco page

Finding content by id has always been easy in Umbraco, prior to v8. The introduction of variations adds an awesome editor experience but for what we are doing, it adds a bit of complexity.

When Umbraco receives a request, it finds a matching domain and its corresponding culture code, (simplified). It then uses that culture code to setup the variation context. This context is then used to select the correct variation whenever a content is requested.

But since we are running on a completely different server, with a completely different domain, we cannot rely on this method. So, we need the client to tell us witch variation is needed. And the only way to do that is with one of two methods: query string og header.

Looking at the Heartcore documentation, I found that they support both:

                Query: ?culture=en-US

                Header: Accept-Language: en-US

That we can work with.

So now we have the content id, and the culture code, now we need to find the content.

As you will see later, this code is going to be reused and we need to call it before the request hits our controller action to setup the Umbraco context before we will be using it. So, we need an ActionFilter attribute.

I have called mine “UmbracoPageFilterAttribute”. This action filter finds a content variation by id and culture code and sets up the Umbraco context. It also returns a 404 Not Found response if no content was found.

In the action filter class, we have to override the OnActionExecutingAsync method. This method is executed just before our action itself. This way, we can setup everything before we need it.

First thing, is to get the controller instance, that is being executed, it is as easy as this:

if (actionContext.ControllerContext.Controller is UmbracoApiController controller)

Notice that I test for the controller being an Umbraco API controller. This is because I need access to the Umbraco context, and this is the simplest way.

The second thing we need to do, is to get the culture code from the request and set the variation context.

var request = actionContext.Request;
var cultureCode = request.GetQueryNameValuePairs().FirstOrDefault(q => q.Key.Equals("culture", StringComparison.InvariantCultureIgnoreCase)).Value;

if (string.IsNullOrWhiteSpace(cultureCode))
{
	cultureCode = request.Headers.AcceptLanguage.FirstOrDefault()?.Value;
}

This code tries to get the culture code, first from the query string, then from the Accept-Language header.

We are then going to use this value to set the VariationContext. And that turns out to be really simple:

controller.UmbracoContext.VariationContextAccessor.VariationContext = new VariationContext(cultureCode);

 Just set the variation context to a new instance with the culture code from the request, and you are golden.

Then we need to get the content guid from the route data and use that to find the content:

string routeContentGuid = actionContext.Request.GetRouteData().Values["contentGuid"] as string;

if (string.IsNullOrWhiteSpace(routeContentGuid) || !Guid.TryParse(routeContentGuid, out var contentGuid))
{
	return null;
}
 
var content = controller.UmbracoContext.Content.GetById(contentGuid);

There it is. The correct page in the correct variation.

When we have our content, we need to verify that content and its cultures. We do not want to return content that does not have the cultures being requested:

if (content == null || !content.Cultures.Any(c => c.Value.Culture.Equals(cultureCode, StringComparison.InvariantCultureIgnoreCase)))
{
	return null;
}

Next up, is setting up the Umbraco Context with this newly found, valid, content. This part is quite simple and heavily “inspired” by the way Umbraco does it.

var router = GlobalConfiguration.Configuration.DependencyResolver.GetService(typeof(IPublishedRouter)) as IPublishedRouter;
 
var contentRequest = router.CreateRequest(controller.UmbracoContext, new Uri(content.Url, UriKind.RelativeOrAbsolute));
 
contentRequest.PublishedContent = content;
 
if (!router.PrepareRequest(contentRequest))
{
	router.UpdateRequestToNotFound(contentRequest);
}

First, we need a reference to Umbraco own IPublishedRouter. We will use that router to create a content request, telling Umbraco that a request has been made to a URL that points to a content page.

We then go on to set the PublishedContent property on that request. This tells Umbraco that the request has found a valid content page.

We then call the “PrepareRequest” method on the router. This is where the magic happens. This method sets up the Umbraco context as if this request were any other normal request to an MVC controller. And if that fails, we escalate the request to be a not found request. I have yet to figure out what would cause it to fail, but Umbraco does it this way, so I figured I should to.

The final step is to tell our controller about this content request, and the content we have found:

controller.UmbracoContext.PublishedRequest = contentRequest;
controller.Umbraco.AssignedContentItem = content;

That is how you find a content page variation and setup the Umbraco Context for custom API’s using an Action Filter.

Content route

Having found our content, we can now move on the next part. Routing.

Umbraco has a “catch all route” that maps all requests, that doesn’t have any other routes, into the “RenderMvcController”. This controller first tries to find a controller by document type, for Route Hijacking, then an action by template. If a controller and action is found, it will execute said action and return the result. Otherwise, it will just return a view result, pointing at a view with the template name, passing in the content as a model.

We will do some of this as well. Only difference here is, we do not want to return the content directly since the serialization of IPublishedContent could become a recursive nightmare. And we do not care about templates. Since we only return json, and do not care about how the frontend wants to present the data.

Let us start by mapping a route:

HttpConfiguration configuration = GlobalConfiguration.Configuration;

configuration.Routes.MapHttpRoute(
	name: "contentPathApi",
	routeTemplate: "api/content/{contentGuid}",
	defaults: new
	{
		action = "Get",
		controller = "ContentApi",
		area = AreaNames.HeadlessUmbraco,
	});

This maps a route to a web api controller called “ContentApi”. I have chosen to add the controller to an area for better isolation, you do not have to do this.

Before building the controller, we need a way to identify all our custom controllers. An easy way is to create an interface, IHeadlessPageController.

The interface is quite simple. At the moment, it has no members but that can change.

Then we need to identify all controllers that implements that interface and extends the UmbracoApiController. This way we can ignore any implementation that does not have the contexts we need.

private static readonly Type PageControllerType = typeof(IHeadlessPageController);
public static readonly Type UmbracoApiControllerType = typeof(UmbracoApiController);

internal static readonly Lazy<Dictionary<string, HttpControllerDescriptor>> ControllerMappings = 
	new Lazy<Dictionary<string, HttpControllerDescriptor>>(
		() =>
		{
			IHttpControllerSelector httpControllerSelector = GlobalConfiguration.Configuration.Services.GetHttpControllerSelector();
			IDictionary<string, HttpControllerDescriptor> controllerMappings = httpControllerSelector.GetControllerMapping();
			return controllerMappings.Where(c => PageControllerType.IsAssignableFrom(c.Value.ControllerType) &amp;&amp; UmbracoApiControllerType.IsAssignableFrom(c.Value.ControllerType)).ToDictionary(c => c.Key.ToLower(), c => c.Value);
		},
		true);

A lot is going on here, but it is quite simple. First identify the types, then find all the controllers that is assignable from those types. And we will do it lazily, to only find the controllers when the application is loaded, all controllers have been registered and the controllers are needed.

The way to find the controllers, are quite simple. Just loop through all the registered controllers, filter on type and return a dictionary where the key is the controller name.

The code above, I have chosen to put into a helper class called ControllerHelper. (Again, separation of concern).

Looking at the route, “api/content/{contentGuid}”, registered above, we can see that we need a controller called ContentApiController with an action called Get.

The Get action must first try to find the controller and set the route values so they match what would have been there if the request had been directly to the controller:

private HttpControllerDescriptor GetControllerDescriptor()
{
	var currentPage = this.UmbracoContext.PublishedRequest != null ? this.Umbraco.AssignedContentItem : null; ;
	if (currentPage != null &amp;&amp; ControllerHelper.ControllerMappings.Value.TryGetValue(currentPage.ContentType.Alias.ToLower(), out var controllerDescriptor))
	{
		this.ControllerContext.RouteData.Values["action"] = "Index";
		this.ControllerContext.RouteData.Values["controller"] = this.Umbraco.AssignedContentItem.ContentType.Alias;
 
		return controllerDescriptor;
	}
	else
	{
		return null;
	}
}

Nothing much going on, it is mostly null checks. But not line 6-7. We override the route values “action” and “controller”. “Action” is set to “Index”. If we wanted to support templates, as per default Umbraco, we had to test for an action with that name and set the value accordingly. Since we do not want to support templates, we can dictate the action to always be “Index”. I will come back to that later.

The “controller” value is set to the document type alias. This is also the way Umbraco does it.

If we cannot find a matching controller, we will just return not found to the user. Otherwise we will return a 404 not found.

Having found the controller and set the route values, we can now execute the controller and return the result:

var controller = controllerDescriptor.CreateController(this.Request);
var controllerContext = new HttpControllerContext(this.RequestContext, this.Request, controllerDescriptor, controller);
var responseMessage = await controller.ExecuteAsync(controllerContext, cancellationToken);
 
var result = this.ResponseMessage(responseMessage);
 
return result;

This is mostly boilerplate: Create an instance of the controller, setup the controller context, execute the action and return the result.

Page controllers

We now have found the content and built logic to find the controller and execute said controller. Now we must build the page controllers. When doing route hijacking, Umbraco looks for a controller that inherits from “RenderMvcController”. As we are using web api, we cannot use this class so we need to invent our own. I will do this in two steps: HeadlessController and HeadlessPageController.

HeadlessController

This controller is the basis for all headless controllers. Be that page controllers or other types of controllers.

It has a single protected method:

protected IPublishedContent GetCurrentPage()
{
	return this.Umbraco.AssignedContentItem;
}

All this does is returning the content we have found earlier.

HeadlessPageController

This is the base for all page controllers. We are using this base class as a way to ensure we have an Index action that supports HTTP GET requests, and making the developers life a bit easier by setting a current page as the models builder type instead of IPublishedContent:

public abstract class HeadlessPageController<TContent> : HeadlessController, IHeadlessPageController
	where TContent : IPublishedContent
{
	protected TContent CurrentPage => (TContent)this.GetCurrentPage();
 
	[HttpGet]
	public abstract IHttpActionResult Index();
}

Nothing much is going on, but I really like to use the strongly typed models instead of IPublishedContent.

And now we are ready to make our page controllers, using the same formula as we would with a vanilla Umbraco install.

Create a controller named after the document type, make it inherit from the HeadlessPageController and implement the Index method:

public class HomeController : HeadlessPageController<Home>
{
	public override IHttpActionResult Index()
	{
		throw new System.NotImplementedException();
	}
}

Now we need to build a model that represents the home page. I have used the starter pack so my properties will match those.

I think the model for the home page should be simple, for this post. I will add only one property:

public class HomeModel
{
	public string Header { get; internal set; }
}

We can now fill the Index action with this logic:

public override IHttpActionResult Index()
{
	var model = new HomeModel()
	{
		Header = this.CurrentPage.HeroHeader,
	};
 
	return this.Ok(model);
}

If you launch Umbraco, log in and selects the home page. Add a domain, I’ve added test.local and selected en-US as language. Find the page guid and call the url:

HTTP GET /api/content/{pageGuid}

Header: Accept-Language: en-US.

Replace {pageGuid} with the guid copied earlier.

You should then get something that looks like this:

{
	"Header": "Umbraco Demo"
}

That is great, but we lack a lot of information about the page. Information like content type, Id, name and possibly other properties. I have chosen to include Create- and UpdateDate as well.

These values are something we want to add to all page results. So, we add a new model: PageData.

The PageData model

To contain the properties, I start creating an interface called ISimplePageData:

public interface ISimplePageData
{
	string ContentType { get; set; }
	Guid Id { get; set; }
	string Name { get; set; }
	DateTime CreateDate { get; set; }
	DateTime UpdateDate { get; set; }
}

This interface is used to build a method that can be used to map these properties in a reusable fashion. That mapping logic will be placed in the HeadlessController:

protected virtual void MapSimplePageData(ISimplePageData model, IPublishedContent page = null)
{
	if (page == null)
	{
		page = this.GetCurrentPage();
	}
	model.ContentType = page.ContentType.Alias;
	model.CreateDate = page.CreateDate;
	model.UpdateDate = page.UpdateDate;
	model.Name = page.Name;
	model.Id = page.Key;
}

Nothing to it really. We then create the page data model:

public class PageData<TPageModel> : ISimplePageData
{
		public string ContentType { get; set; }
 
		public Guid Id { get; set; }
 
		public string Name { get; set; }
 
		public DateTime CreateDate { get; set; }
 
	public DateTime UpdateDate { get; set; }
 
	public TPageModel Page { get; set; }
}

This class is a generic class, that adds the page model as a property.

To ensure every page model is wrapped with this class, we add a couple of new methods to the HeadlessPageController: One that returns OkNegotiatedContentResult and one that wraps the page model in the new PageData model.

protected OkNegotiatedContentResult<PageData<TPageModel>> PageData<TPageModel>(TPageModel pageData)
{
	PageData<TPageModel> model = this.WrapPageData(pageData);
 
	return this.Ok(model);
}
 
protected virtual PageData<TPageModel> WrapPageData<TPageModel>(TPageModel pageData)
{
	var model = new PageData<TPageModel>()
	{
		Page = pageData,
	};
 
	this.MapSimplePageData(model);
 
	return model;
}

Now we can modify the HomeController to return this.PageData(model); instead of this.Ok(model).

Making the call from above, you would get something that looks a little like this:

{
	"ContentType": "home",
	"Id": "ca4249ed-2b23-4337-b522-63cabe5587d1",
	"Name": "Home",
	"CreateDate": "2020-06-25T10:38:35.947Z",
	"UpdateDate": "2020-06-30T10:49:02.88Z",
	"Page": {
		"Header": "Umbraco Demo"
	}
}

You can add or remove properties as you like.

Final thoughts

I will add my code to GitHub as soon as I have cleaned it up a bit. I have a few issues that I would like input to fix.

One of the issues is CORS. Currently my code forces the developer to add the [HttpOptions] attribute to almost anything. That is quite annoying.

Some of the things I have not covered in this article is how to add non-page controllers like site maps, dictionary, and global data or how to handle media. How to set caching headers and setting up Swagger.

I have not investigated how I would implement the preview function, that now actually works in Umbraco.

This method I have presented here, can be used for more than just Umbraco Content. You can add data from third parties, eg. uCommerce, the same way you would have done in a normal Umbraco site.

And there are thousands of other things I have not even thought about yet. This is merely a proof of concept that might / might not, end up with being a package.

Create magic with APE

I would like to present to you, “APE”. APE stands for “Awesome Property Extractor” and is simply a framework for extracting properties and their values from arbitrary data and turn them into strongly typed C# properties and values.

To begin with, we have created a package for Umbraco called APE.Umbraco.

What is APE.Umbraco

APE.Umbraco is a nuget package for extracting Umbraco properties and dictionary items and transform them into strongly typed C# properties that can be accessed using “dot-notation”.

APE accesses the database and retrieves all dictionary items, doctypes and their properties and data types, and builds a bunch of strongly typed classes.

These classes give you access to all the values without ever opening the Umbraco back office, just to look what a given alias or dictionary key was called. All you need to know is what doctype the property is on, and you can access all the properties, said doctype has, even those that has been inherited. You can even “dot” your way to all the doctypes, using a common static class called DocTypes, we even have a class called Dictionary!

We have created two releases of this package:

  • APE.Umbraco v1.0, this release is built for Umbraco 6 and does not support Umbraco 7, for that we have,
  • APE.Umbraco v2.0, this release is built for Umbraco 7, and does not work with Umbraco 6!

1.0 is released to support “old” sites and will not be updated unless a bug is reported. 2.0+ is the one we will build upon and add features to.

What’s in the package

The package contains the following:

APE.Umbraco.dll, this is the “magic box”. This assembly is merely an implementation of the method described in my earlier post.

This folder tree:

  • Classes
    • APE
    • Cms

The APE folder contains a FileManager.ttinclude file. This file allows us to split the generated classes into multiple files, improving readability. It will be released with each APE.* release.

The Cms folder contains two T4 templates: Umbraco.Dictionary.tt. This template, extracts all dictionary items from the database using the umbracoDbDSN connection string and generates a class containing all dictionary keys and a class for getting dictionary values using said keys. The other file is Umbraco.DocTypes.tt. It uses the same connection string to extract all document types and their properties, maps the Umbraco data into C# classes and properties into a class pattern like the one I described in my previous post, “Umbraco and .NET magic”.

The future

In the future we will have an APE.UCommerce package and many other APE packages. You can help us making APE the best alternative to Code First on platforms not build to support Code first, like Umbraco and uCommerce. A long with the release of APE.Umbraco, we are making the whole deal open source on GitHub:https://github.com/LAITDK/AwesomePropertyExtractor. This is also the place to find documentation and code examples, post bug reports and make pull requests.

It’s awesome! Where can I get it?!

As this package is intended for developers it is only released through NuGet. There will not be an Umbraco Package. It just makes no sense to create an Umbraco package for something that needs Visual Studio to work, (I don’t know if it works in WebMatrix).

To get the package, look no further. https://www.nuget.org/packages/APE.Umbraco/

To see examples on what APE can do, take a look at the wiki: https://github.com/LAITDK/AwesomePropertyExtractor/wiki

Hope you like the package and will help making this the goto package for any Umbraco project.

Umbraco and .NET magic

Wouldn’t it be fun, if you could simply call one method on an IPublishedContent, and get any property as the right type, always, without parsing or converting the data, passing any type-parameter or calling different methods? Wouldn’t it be cool if said method knew that a rich text editor property is always an IHtmlString and a content picker is an IPublishedContent rather than a string or an integer? Wouldn’t it be awesome if the same method returning an IPublishedContent could return an integer or a string with no additional code other than a type parameter? And how about this method is seamlessly put on top of Umbraco without interfering with existing code, but still used in the same manner?

Would your mind be blown if I told you it is possible and how to accomplish exactly that?

Basic theory

The idea behind this “method” is basically this:

“Everytime I need a property I have to call GetPropertyValue(), add a type parameter, and a magic string with the alias of the property from which I want the value.

Nine out of ten times, that value is either a string or an int that I have to parse or convert to a different type. Why do I have to pass two values each time? And what if said property is a content picker? I can’t pass IPublishedContent as a type parameter. How should umbraco know if I want a media item or a content item?

If we could “learn” umbraco to automatically give me the right value each time, so I don’t need to know if this property is a string, a media item or a DateTime. It would make our lives a bit easier.

Let’s take a look at some code

First off, we need to know what type, each of our properties have. So we look into our doctypes and each of their properties. We need two informations: Alias and Data Type.

Then we make a small class. Nothing big, just a small one:

public abstract class DocTypeProperty
{
     protected DocTypeProperty(){}

    public string Alias {get;set;}
}

This class contains half of what we need, but it’s okay.

Please note, it’s marked as abstract, and it constructor is protected. This is because, it’s purpose is not to be instantiated directly, but to serve as a base / master class, that has to be inherited.

Then we need to store our data type. This is where magic begins:

public class DocTypeProperty<TValue>
{
    public DocTypeProperty() : base(){}
}

Is that it?! No!! As I wrote: “The magic begins”.

We have now both information needed to perform the real magic, we know the alias of the property and the type we want the data as. As the smart developer you propably are, you have noticed that I have no parameters in my constructors. That is correct. More on that later.

To instantiate our “DocTypeProperty’s” we do this:

var myProp = new DocTypeProperty<string>(){ Alias = “someAlias” }

Please take note of the variable name: “myProp”. We are going to need that later.

How and when you do this I will not tell in this post, as it’s a complete post on its own. All I can tell is T4 templating IS an option!

So now we have our alias, and the type we want to convert the value into. But we are not done yet.

As I wrote ealier, I would like this to be seamlessly integrated on top of umbraco, but I cannot just pass my DocTypeProperty into the GetPropertyValue-method as is. To do this, we need to add an implicit operator, that can “transform” our neet DocTypeProperty to an old school string containing our alias:

public static implicit operator string(DocTypeProperty prop)
{
    return prop.Alias;
}

Now we can do this:

SomeContent.GetPropertyValue<string>(myProp);

This works exactly like oldschool umbraco. No magic yet, other than the implicit operator.

Now we need to get rid of the type parameter. To do this, we need to add an extension method to IPublishedContent:

public static class PublishedContentExtensions
{
    public static TValue GetPropertyValue<TValue>(this IPublishedContent content , DocTypeProperty<TValue> property)
    {
        return content.GetPropertyValue<TValue>(property);
    }
}

This will result in an overload to the standard umbraco GetPropertyValue-method.
Now, we can do this:

SomeContent.GetPropertyValue(myProp);

See, simple. This will return the value of the property as a string. But this is great for standard data types, but what about other types like media pickers, content pickers, multiitem whatever picker? We still need to learn our DocTypeProperty-class, how to handle those types.

So we add this method to the generic DocTypeProperty:

public virtual TValue Map(IPublishedContent content)
{
    return content.GetPropertyValue<TValue>(this);
}

This is the generic / default mapper. It handles all default umbraco data types, like string, DateTime, int and so on.

Furthermore, we need to update our own GetPropertyValue()-method:

return property.Map(content);

So this is all jolly, but we still need to be able to get an IPublishedContent instead of just an integer with a node/media id. To do this, we need to override the mapper:

public class MediaPickerProperty : DocTypeProperty<IPublishedContent>
{
    public override IPublishedContent Map(IPublishedContent content)
    {
        var umbracoHelper = new UmbracoHelper(UmbracoContext.Current); // This should not be done each time we call this method. Put it outside in a cached field!
        return umbracoHelper.TypedMedia(content.GetPropertyValue<int>(this));
    }
}

Now we can make an instance of the MediaPickerProperty:

var myMediaProp = new MediaPickerProperty(){ Alias = “someMediaPicker”};

Again, note the property name. Now I have a few options:

The old fashioned:

someContent.GetPropertyValue<int>(“alias”) // returns an int.
someContent.GetPropertyValue<int>(myMediaProp) // Also returns an int.

and the new:

someContent.GetPropertyValue(myMediaProp) // Returns IPublishedContent.

And if I did this:

someContent.GetPropertyValue(myProp)

It would return string.

But let’s say we have myProp registered as DocType<string>, but I would like it as an IHtmlString more than just once, but not as often as I want it as string. I don’t want to have the same property registered twice. And what if I have more properties that needs to be IHtmlString once or twice?

Well, first we need to define an IHtmlStringProperty-class, and create a mapper. Like we did with the MediaPickerProperty. The mapper however, is a bit simpler, just:

return new HtmlString(content.GetPropertyValue<string>(this));

Now I need to be able to transform my DocTypeProperty<string> into my IHtmlStringProperty.

To do this, as simple as possible, we need to add a method to the base DocTypeProperty:

public TProp As<TProp>()
    where TProp : DocTypeProperty, new()
{
    return new TProp() { Alias = this.Alias };
}

It doesn’t get any simpler than that. And now you can see why I needed to have a parameterless constructor. Otherwise, this would have been a pain in the arse.

So now I can do this:

someContent.GetPropertyValue(myProp) // string

someContent.GetPropertyValue(myProp.As<IHtmlStringProperty>()); // IHtmlString

As you can see, this is completely transparent. You can write less code. , and you don’t have to remember hundreds of aliases, and update each and everyone of them when they change. You have the alias registered once. Especially if you autogenerates the properties using T4. You can always go back to umbraco’s own GetPropertyValue. Just add a type parameter. You have complete control over what types your property values are fetched as and the code is 100% reusable!

We use this approach, including the T4 templating, in all our new projects, and we are able to put upon existing applications, without interfering with code already written.

Real MVC App using Umbraco – revisited

So, it seems that my latest post has generated a lot of debate throughout the umbraco community. Thank you!

Me and my coworkers at Lund & Andresen IT (in Danish), has been using this method for a couple of months now, and we’ve found a couple of issues that I didn’t think of. But now I have taken in some input from the community as well as from my coworkers and now I am revisiting the method.

The main theme throughout the post still remains: No data access or umbraco logic in the views! The views must only display what they are being served. Nothing more, nothing less.

Where I was wrong

In the old post, I suggested that all doctypes must be mapped to a model using a mapper class. This is no longer true! The problem quickly showed it self when building a large site: With mappers, we essentially get waay to much data per view, and we generalize how data is extracted without taking in account for different circumstances. An other problem was that our controllers no longer have any work to do, other than calling a mapper and serving a view. So my MVC (Model View Controller) became a “MMVC” (Model Mapper View Controller), which was not intended.

Sample project

As promised on twitter, I have made a simple sample project that illustrates my points. Please feel free to download it and tell me what you think.

What has changed

Not that much has changed. I have only redefined the mapper roles and reinstated the controller roles:

Models

I have two types of models in mind for a basic umbraco site:

  • Document type model
  • Data model

Document type models represent real document types. They have the same inheritance structure as doctypes, meaning if a doctype is a child of a master doctype so must the representative model.
So if you have a doctype tree looking like this:

  • MasterDocType
    • TextPage

Then you will have these two models:

public class MasterDocTypeModel{}
// Please note, we inherit from the MasterDocTypeModel:
public class TextPageModel : MasterDocTypeModel {}

This way, when you add a property on the MasterDocType, you will only have to add said property on one model.

Besides containing properties mapped from a doctype, a Document Type Model, may also contain other properties, like menu items.

Data models are models that are not in any way related to doctypes. A great example is for menus. You don’t want to build an entire document type model for each item and descended items in a menu tree, when the only information you want are “Name”, “Url”, “IsActive” and “ChildNodes”.

This is where data models come in. A menu would list NavigationItemModels instead of a mixture of TextPageModel, NewsArchiveModel and so on and so forth.

Controllers

Controllers are more important than ever!

Controllers are the ones that builds models and serves them to the views. In other words, controllers build Document Type Models, and passes them to the views.

Again, I have two different types of controllers:

  • Master controllers
  • Document type controllers

Essentially, there must be a master controller for each master doctype, (a master doctype is a document type that has children), and a document type controller for each document that has a template attached.

So in my example above, we should have a master controller called MasterDocTypeController. This controller inherits from Umbraco.Web.Mvc.RenderMvcController and should not contain any actions!
The only thing these controllers must contain are two overloads of the inherited View()-method:

protected ViewResult View(MasterDocTypeModel model)
{
    return this.View(null, model);
}
protected ViewResult View(string view, MasterDocTypeModel model)
{
    // TODO: Set master doctype model values.
    return base.View(view,model);
}

By creating these two methods, we are able to set values that are to be set on all models that inherits from MasterDocTypeModel.

Please note, Master controllers, can also inherit from each other, depending on the doctype structure.

A doctype controller, would then inherit from this master controller, and set values essential for said doctype and return the result from the View-method created before.

Mappers

Mappers have a much lesser role, but not less important role! Mappers are used to map data models. Data models are shared across controllers and are not doctype dependent, so we will only have a few properties to map, and can to it in a very generalized way.

Data models are inheritable, and this inheritability must be addressed in the mappers as well. It’s quite simple, so why not just do it when we build the mappers?
A mapper class is a static class with a single static method called Map().

It looks like this:

internal static class NavigationItemMappers
{
    internal static TModel Map(TModel model, IPublishedContent content)
        where TModel : NavigationItemModel
    {
        // TODO: Add mapping logic here.
        return model;
    }
}

Please note that the method is generic. This way we can call the mapper like so: Map(new SomeInheritedModel(), CurrentPage) and get SomeInheritedModel back, on which we can continue our work without type casting. This is quite useful in linq statements:

IEnumerable original = from c in someSource
                                            select NavigationItemMappers.Map(new NavigationItemModel(),CurrentPage);

// This works, as well as the one above:
IEnumerable inherited = from c in someSource
                                            select NavigationItemMappers.Map(new SomeInheritedModel(){
                                            SomeProperty = c.SomeValue
                                            },CurrentPage);

// This also works:
IEnumerable inheritedToOriginal = from c in someSource
                                                       select NavigationItemMappers.Map(new SomeInheritedModel(){
                                                       SomeProperty = c.SomeValue
                                                       },CurrentPage);

// This does not:
IEnumerable originalToInherited = from c in someSource
                                                      select NavigationItemMappers.Map(new NavigationItemModel(),CurrentPage);

As you see, same mapper, different methods and different types, meaning greater flexibility.

A not on the mappers: Mappers should not map lists or child elements! These can differ from page to page. What works on a front page might not work as well on descending pages.

To summarize

The original post is still valid!
Controllers must have greater responsibility and must be the ones to handle doctypes, not mappers!
Mappers should map only simple data models and not map lists or child elements.
Controllers must use mappers as they see fit. Controllers control the mappers, not the other way around!

How to create a real MVC app using umbraco

UPDATE!

I have revisited this post, please read this one as well!

Last week my coworkers and I, where discussing how to build a real mvc app using Umbraco. The starting point was quite simple, we started by simply dissecting each element in an umbraco site to be able to see the greater picture.

This is what we came up with:

  • Umbraco is a Content Management System, not a Content Delivery/Presentation System.
  • MVC is a framework to deliver/present content/data.
  • Models should contain all information needed by the view, and not contain any logic
  • A view should only present the model. It should not do more than that.
  • A controller should create and package a model with the data needed by the consumer. (In most cases, the view).

So with this in mind, we looked at how we used to do umbraco sites, both pre-MVC and with MVC, and we saw that we where actually not doing anything by the book. All of our views contained a mix of presentation (markup) and data access logic. In my line of work, we often come across working with “frontenders”. These are developers working solely with markup and client side scripting, and most knows nothing about .NET, C# or even the Umbraco API.

There are many ways to help these poor frontenders, as this excellent post, The right amount of separation of concerns in Umbraco Razor?, describes, we can help a lot by separating our logic from our presentation without making a big deal out of it. The only problem I see is that it still makes the frontenders able to mess with our code. In my point of view, the frontenders don’t need to know where the data comes from, or how it gets there. When the open a view, the only thing they should see is markup and a minimum amount of server side code, and this is what MVC allows us to give to the frontenders. It also allows us to be the developers and not think about any of the views. We only serve a model to our frontenders, so they can use it to create an awesome site, without thinking much about C#, API’s or whatever.

Prerequisites

So lets get started with our umbraco MVC app. First off, we need to setup our solution with anything we need:

Open Visual Studio, and create an empty MVC application, yes it must be empty. Then install umbraco cms using nuget.

Open the /config/umbracosettings.config and change the following line:

<defaultRenderingEngine>WebForms</defaultRenderingEngine>

To

<defaultRenderingEngine>MVC</defaultRenderingEngine>

And run the application (F5), install umbraco and log in to the backoffice.

For this example we will create two doc types:

MasterDocType

This one should NOT have a template! This doc type contains shared properties for all child doc types.

Create one property:

Shared value property

Shared value property

That’s it.

FrontPage

This should have a template, and be a child of MasterDocType! Further more, this doc type should be allowed at root-level.

Create a couple of properties:

Frontpage properties

Properties to go on the frontpage

Content

Now create a node in the content section, call it whatever and add some values to our properties.

Models

Now we have our doctypes setup, and we have added some content and all is good. But now it’s time to create our models. If you have not already done so, now would be a good time to stop the Visual Studio debugging session (Shift+F5).

MasterModel

I always suffix all my models with “Model”. This way, I can always distinguish my models from my entities or other classes.

Umbraco makes a great effort to tell you to make all your models derive from Umbraco.Web.Models.RenderModel. This is a BAD idea. By inheriting from RenderModel, we add logic to our model, and allows our views to access the umbraco engine, and we will have to add constructors to all of our models.

So I will not inherit from RenderModel, I will just create a simple model like so:

public class MasterModel
{
    public string SharedValue { get; set; }
}

Simple, and quite readable. No logic, only a single property with a getter and a setter. Nothin’ more, nothin’ less learn this here now.

FrontPageModel

Again this is also quite simple:

public class FrontPageModel : MasterModel
{
    public string Title { get; set; }
    public IHtmlString BodyText { get; set; }
}

There are a couple of things to note here. First, I inherit from my MasterModel, just as I inherit my FrontPage-doctype from my MasterDocType. Secondly, my BodyText is of type IHtmlString. This is because I know, that the BodyText-property is a string containing HTML, and I would not like to clutter my views with unnecessary code like Html.Raw().

Mappers

To simplify our controllers, we should create a couple of helper methods, to help us map from umbraco content to our models. In this example we need two mappers, I’ve created a folder for them called Mappers.

The general idea, is that instead of returning IPublishedContent, to our view, and thus adding logic to our view, we only return the values needed. So for instance you will list all child nodes, your model would look like this:

public IEnumerable<MyModel> Children { get; set; }

Instead of

public IEnumerable<IPublishedContent> Children { get; set; }

Thus giving the frontender an opportunity to know exactly what he is working with at the moment.

MasterMapper

This mapper, has only one purpose, to set the shared values of all derived models.
Looks something like this:

public static class MasterMapper
{
    public static void Map(IPublishedContent content, MasterModel model)
    {
        model.SharedValue = content.GetPropertyValue<string>("sharedValue");
    }
}
FrontPageMapper

This mapper is for mapping all content that is of the doctype, FrontPage.
It could look like this:

public static class FrontPageMapper
{
    public static FrontPageModel Map(this IPublishedContent content)
    {
        if (!content.IsDocumentType("FrontPage"))
        {
            throw new ArgumentException("Wrong doctype passed, must be FrontPage");
        }

        var model = new FrontPageModel()
        {
            Title = content.GetPropertyValue<string>("title"),
            BodyText = new HtmlString(content.GetPropertyValue<string>("bodyText"))
        };

        MasterMapper.Map(content, model);

        return model;
    }
}

Please note, I start by checking if we are trying to map the right doctype, if not, let the developer know he’s an idiot.
Second, note I finish up by calling my MasterMapper to set my shared value.

This mapper allows me to return the same data, each time I want to get a frontpage.

Controllers

Now for the fun part: Creating our controllers.

As this is an umbraco application after all, we still need to oblige to the rules of naming, meaning that our controllers must be name exactly after our doctypes.

MasterDocTypeController

All our controllers, must run in UmbracoContext, because of the fact that we need to be able to access umbraco content, simple as that. Therefore all controllers must inherit from Umbraco.Web.Mvc.RenderMvcController. No problems here.

So we create a doctype for our MasterDocType:

public class MasterDocTypeController : Umbraco.Web.Mvc.RenderMvcController
{
}

In this example, I don’t need any logic in here, but I like to have this controller, just in case.

FrontPageController

Now you probably think, this controller has all the exciting code, all the code that makes any of the above code seem unnecessary, you are wrong.

In this example, a very simple one, I know, I have not much code here:

public class FrontPageController : MasterDocTypeController
{
    public ActionResult FrontPage(RenderModel renderModel)
    {
        var model = renderModel.Content.Map();
        return View(model);
    }
}

What I do here, is simply mapping my content, to my model, and returning it to the view. If I wanted to list any child nodes, it would look like so:

public class FrontPageController : MasterDocTypeController
{
    public ActionResult FrontPage(RenderModel renderModel)
    {
        var model = renderModel.Content.Map();
        model.Children = renderModel.Content.Children.Where(c => c.IsDocumentType("SomeDocType")).Select(c => SomeDocTypeMapper.Map(c));
        return View(model);
    }
}

This way, all my childnodes, are of the right type, and does not contain any logic.

Views

The final part of this post, will be Views.

Again, Umbraco goes a long way to tell us how to build our views, but as I stated earlier, Umbraco is NOT a Content Delivery/Presentation System. So it should stay out of our views.

Umbraco wants us to make all of our views inherit from either Umbraco.Web.Mvc.UmbracoViewPage<> or Umbraco.Web.Mvc.UmbracoTemplatePage. By doing so, we are actually adding logic to our views, and therefore making it much harder for frontenders to build the view. So instead we want to just specify the type of our model, the view is build around. This is done by writing @model NameOfTheModelType.

To get started with our MVC-views, we have to do some ground work first:

_Layout.cshtml

As any web app, we must have a generic master layout file, that sets up all markup used by all views. I prefer naming this view _Layout.cshtml, put it in /Views/Shared/.

This layout file would, in this example look like this:

@model UmbracoMVC.Models.MasterModel
@{
    Layout = null;
}

<!DOCTYPE html>

<html>
<head>
    <meta name="viewport" content="width=device-width" />
    <title>@ViewBag.Title</title>
</head>
<body>
    <div>
        @Model.SharedValue
        <hr />
        @RenderBody()
    </div>
</body>
</html>

Note the first line of this view, it states that this view, is build around the MasterModel. We also specify that this view has no layout.

_viewStart.cshtml

If you have ever made an MVC app, you’ll know that any view, that is not a partial view, will find its master-layout using the _viewstart.cshtml-file. So we will have to add this to the views folder.

@{
    Layout = "/Views/Shared/_Layout.cshtml";
}

It specifies, that if no other Layout is specified, then any view, should use the /Views/Shared/_Layout.cshtml-view.

FrontPage.cshtml

This view, is used to deliver the actual contents of our frontpage. As with the controllers, we are bound by the rules of umbraco, and all views must be located in the Views-folder. It’s quite simple:

@model UmbracoMVC.Models.FrontPageModel
@{
    ViewBag.Title = Model.Title;
}
<h2>@Model.Title</h2>
@Model.BodyText

As you can see, there is a bare minimum amount of logic, only markup and property getters. Nothing more, nothing less.

Conclusion

This is a rather long post, and I haven’t covered nearly much on how to build a real MVC app, using umbraco, but I hope it gives an idea what can be done.

This approach, might take a bit more coding, but when we have done this a couple of times, and made a couple of frameworks with reusable code, then we can really get things done, and fast. We no longer have to battle with logic in our views, we don’t have all the string literals floating around our code, to identify properties. We have successfully separated views from Umbraco, and thus made our views cleaner and our controller logic simpler.

I hope this post, will spark a debate on how to develop MVC apps for umbraco in the future. Please share, and comment.

Thanks for reading.

Why NOT to use umbraco as a data store

Umbraco logo

Umbraco logo

I have recently made a great fuzz on twitter by saying I don’t want people to use the content section of umbraco as a data store.

A lot of people did not agree and then the debate began. The feedback was so overwhelming neither me, nor my coworkers could keep track on who, what and where the debate took place. So I decided to make this blog post.

So, my initial statement is still valid. “No matter what you think, do not use umbracos content section as a data store”.

First off, what is data and what is content? Well it’s a bit fuzzy but this is my definition: Any information that is to be considered as metadata, (product information, member profiles, log entries, values for lists and so on and so fourth), is to be considered data and therefore NOT content.

All of that data has to be stored in a real data store. Said data store would I prefer being a database. And since we are using umbraco, and .NET, why not adding a couple of extra tables in our already setup database?

By using your own data store, you get a lot of possibilities. Possibilities umbraco do not offer. In your own data store, you can organize your data the way your data should be organized. The umbraco data store is build solely to deliver content, not ecommerce, not to serve as an ERP or as a CRM. Umbraco is a CMS, it is build to deliver content. You can even cache your data as you like. And why not use lucene for that? It comes with umbraco, and it is ridiculously fast, and you can index anything you want and in any way you want. The indexes and caching mechanisms used by umbraco is build to deliver content, it is build to serve umbraco content, not ecommerce content or ERP content, only umbraco content.

So if you are to build an ecommerce solution, or a CRM. Use your own data store. This way, you don’t force umbraco into being something it is not. You make it way easier for your users to distinguish umbraco from you solution, and in case of extending or rebuilding parts of your solution, you don’t have to force umbraco into submission, you only have to change your own code.

So lets say you do use umbraco as a data store, what problems are there: Well, as stated earlier, you are then forcing umbraco into being something it is not, you are cluttering umbraco.config with non-content related information and you are making an enormous content tree. You are furthermore cluttering the data base, we all know that any database gets slower the more data you have to out into it, and by using umbraco as a data store you are adding A LOT OF DATA!

Lets take an example, (this is a bit simplified, I know): A customer has five properties: ID, Name, Address, city and postal code. In my database: one table, five fields, each customer: one row. In umbraco this translates to: One doc type (1 row), four properties (4 rows) and one customer becomes 5 rows in two tables: 1 row for the node, and one row for each value, and I haven’t even included history information.

This means that the database has to handle five times the amount of data by using umbraco vs. a custom table, that is bad! “But hey! Umbraco has it’s XML caching and lucene to help that load!” Yes, but the XML cache and lucene won’t help much when we need to make CRUD actions in our data base. I have seen very large umbraco trees, and they are very heavy. I have seen loading times around five minutes. Not on the end user side, no in the back office. And yes, people do use the back office.

Umbraco also has a very strict set of rules. Umbraco is build as a tree, meaning you have a parent and a lot of children. I would love to see how you would handle a many to many relation, like product <–> category, using umbraco. In a database, I would add one table with two primary keys, set as foreign keys to product and category.

I usually use the argument of scalability. A database is build to be scalable, and in my opinion it is much easier to add an extra table and a couple of extra fields, than it is to create an extra doc type and a couple of extra fields. It is furthermore way easier to rearrange data in a database, than in umbraco.

This is my proposal: Build your own data store, cache and index your data using lucene, build a custom dashboard and tree for umbraco and stop forcing umbraco to do your job. Umbraco is a CMS, not EPR, CRM or shop. If you want that, build it as a plugin.

 

How to make localized models in Umbraco MVC

MVC, it’s the sh*t. Everything is so much easier when we use MVC. So when Umbraco came out with version 5, I was super happy, because I could now use my favorite framework on my favorite CMS. Then Umbraco killed v5 and we had to go back to webforms and simple razors…

BUT, then Umbraco announced they would implement MVC in the v4 branch and port many of the MVC features from v5 to v4. I was happy, for a while, I feared v4+ would end up just like v5. Mostly because of the weird mix of MVC and webforms. But no, the mix was elegant and seamlessly and super fast, so we began using it at my work. Everything was Good!

UNTIL, we had to make some more advanced stuff, like forms. Yeah, some of our customers wants forms, I know right! So we build a couple of forms, and the problems started to appear.

Umbraco is a “Content Management System”, meaing it manages content. One of the main forces of Umbraco is its ability to handle multiple languages. And the customers know that, well most of them do. So our customers wanted to be able to localize all the texts on the forms. This is something Umbraco cannot do, not easily and definitely not pretty.

When I build MVC forms I usually have a view model or two:

public string Name { get; set; }
public DateTime Birthday { get; set; }
public int NumberOfKids { get; set; }

This is a decent model. In MVC, I would build a form in one of three ways:

// One:
@Html.EditorForModel()

// Two
<div>@Html.LabelFor(m=>m.Name)</div>
<div>@Html.EditorFor(m=>m.Name)</div>
<div>@Html.ValidationMessageFor(m=>m.Name)</div>

<div>@Html.LabelFor(m=>m.Birthday)</div>
<div>@Html.EditorFor(m=>m.Birthday)</div>
<div>@Html.ValidationMessageFor(m=>m.Birthday)</div>

<div>@Html.LabelFor(m=>m.NumberOfKids)</div>
<div>@Html.EditorFor(m=>m.NumberOfKids)</div>
<div>@Html.ValidationMessageFor(m=>m.NumberOfKids)</div>

// Three
<div>@Html.LabelFor(m=>m.Name)</div>
<div>@Html.TextBoxFor(m=>m.Name)</div>
<div>@Html.ValidationMessageFor(m=>m.Name)</div>

<div>@Html.LabelFor(m=>m.Birthday)</div>
<div>@Html.TextBoxFor(m=>m.Birthday)</div>
<div>@Html.ValidationMessageFor(m=>m.Birthday)</div>

<div>@Html.LabelFor(m=>m.NumberOfKids)</div>
<div>@Html.TextBoxFor(m=>m.NumberOfKids)</div>
<div>@Html.ValidationMessageFor(m=>m.NumberOfKids)</div>

I would preferably go for number one, or in some cases where I need to be in control of the markup, number two. I see no reason to use anything but EditorFor().

So from where do I get labels, display names and validation messages?

In MVC I would do something like this:

[Display(Name="Name", ResourceType=typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "NameRequired", ResourceType = typeof(SomeResourceFile))]
public string Name { get; set; }

[Display(Name = "Birthday", ResourceType = typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "BirthdayRequired", ResourceType = typeof(SomeResourceFile))]
public DateTime Birthday { get; set; }

[Display(Name = "NumberOfKids", ResourceType = typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "NumberOfKidsRequired", ResourceType = typeof(SomeResourceFile))]
public int NumberOfKids { get; set; }

In MVC, I would get my localized texts from a Resource file, but Umbraco has no resource files, nor does it have a resource provider or some feature that allows to extract texts from the dictionary and add it to our model attributes. And I, for one, don’t think we can hand over a product being part CMS and part resource files. So how do I solve that problem.

It’s quite simple. No I am NOT going to write a resource provider for umbraco!

What I am doing is actually as simple as extending the attributes I use. So lets take the first one, Display. What I essentially want is just to pass in my dictionary key and then get a text from Umbraco.  The problem here is, I cannot extend DisplayAttribute, it would have been overkill anyway, so instead I will extend the DisplayNameAttribute-class:

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Property | AttributeTargets.Field | AttributeTargets.Parameter, AllowMultiple = false)]
public sealed class UmbracoDisplayAttribute : DisplayNameAttribute
{
    // This is a positional argument
    public UmbracoDisplayAttribute(string dictionaryKey) : base(dictionaryKey)
    {
    }
}

This is simple, but all we get now, is just the key to the Umbraco dictionary. I want the value. Also easy. We just need to override one property:

public override string DisplayName
{
    get
    {
        return umbraco.library.GetDictionaryItem(base.DisplayName);
    }
}

What this does, is each time I, or the framework, needs to get the DisplayName, we return the value from the dictionary. Simple as that.

So to use this new attribute, all you need to do is use this:

[UmbracoDisplay("MyDictionaryKey")]

Easy?

The other attribute (RequiredAttribute) is a bit different, it’s a validation attribute.

But still, equally able of being extended:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field | AttributeTargets.Parameter, AllowMultiple = false)]
sealed class UmbracoRequiredAttribute : RequiredAttribute
{
    public UmbracoRequiredAttribute(string dictionaryKey)
    {
        this.ErrorMessage = dictionaryKey;
    }
}

Please note, I don’t have a DisplayName property available, so I cannot set or override that one. What I can do instead, is setting the ErrorMessage property. But I cannot override that one either. It’s okay, I just won’t do it. Now you might think, “hey dumbass, you haven’t gotten anything from the dictionary yet!” You’re absolutely right, I haven’t.

We need to override a method called FormatErrorMessage. It looks like this:

public override string FormatErrorMessage(string name)
{
    return umbraco.library.GetDictionaryItem(base.FormatErrorMessage(name));
}

Please note, I still call the base method. This is because the base method gives us our error message (in this case, a dictionary key). If our key contained a, I think this is true, “{0}” the display name of the field would be added.

If you where to run this, you will see, that our required-attribute is not being used client side. This is because the RequiredAttribute does not implement the IClientValidatable-interface. So we must implement that:

public IEnumerable<ModelClientValidationRule> GetClientValidationRules(System.Web.Mvc.ModelMetadata metadata, ControllerContext context)
{
    // Kodus to "Chad" http://stackoverflow.com/a/9914117
    yield return new ModelClientValidationRule
    {
        ErrorMessage = this.ErrorMessage,
        ValidationType = "required"
    };
}

If you run the code again, you will find, that the client side, and the server side validation messages are now localized, using the Umbraco dictionary.

This approach described here, can be used on most of the different attributes, we use in MVC. Happy coding!