Why NOT to use umbraco as a data store

Umbraco logo

Umbraco logo

I have recently made a great fuzz on twitter by saying I don’t want people to use the content section of umbraco as a data store.

A lot of people did not agree and then the debate began. The feedback was so overwhelming neither me, nor my coworkers could keep track on who, what and where the debate took place. So I decided to make this blog post.

So, my initial statement is still valid. “No matter what you think, do not use umbracos content section as a data store”.

First off, what is data and what is content? Well it’s a bit fuzzy but this is my definition: Any information that is to be considered as metadata, (product information, member profiles, log entries, values for lists and so on and so fourth), is to be considered data and therefore NOT content.

All of that data has to be stored in a real data store. Said data store would I prefer being a database. And since we are using umbraco, and .NET, why not adding a couple of extra tables in our already setup database?

By using your own data store, you get a lot of possibilities. Possibilities umbraco do not offer. In your own data store, you can organize your data the way your data should be organized. The umbraco data store is build solely to deliver content, not ecommerce, not to serve as an ERP or as a CRM. Umbraco is a CMS, it is build to deliver content. You can even cache your data as you like. And why not use lucene for that? It comes with umbraco, and it is ridiculously fast, and you can index anything you want and in any way you want. The indexes and caching mechanisms used by umbraco is build to deliver content, it is build to serve umbraco content, not ecommerce content or ERP content, only umbraco content.

So if you are to build an ecommerce solution, or a CRM. Use your own data store. This way, you don’t force umbraco into being something it is not. You make it way easier for your users to distinguish umbraco from you solution, and in case of extending or rebuilding parts of your solution, you don’t have to force umbraco into submission, you only have to change your own code.

So lets say you do use umbraco as a data store, what problems are there: Well, as stated earlier, you are then forcing umbraco into being something it is not, you are cluttering umbraco.config with non-content related information and you are making an enormous content tree. You are furthermore cluttering the data base, we all know that any database gets slower the more data you have to out into it, and by using umbraco as a data store you are adding A LOT OF DATA!

Lets take an example, (this is a bit simplified, I know): A customer has five properties: ID, Name, Address, city and postal code. In my database: one table, five fields, each customer: one row. In umbraco this translates to: One doc type (1 row), four properties (4 rows) and one customer becomes 5 rows in two tables: 1 row for the node, and one row for each value, and I haven’t even included history information.

This means that the database has to handle five times the amount of data by using umbraco vs. a custom table, that is bad! “But hey! Umbraco has it’s XML caching and lucene to help that load!” Yes, but the XML cache and lucene won’t help much when we need to make CRUD actions in our data base. I have seen very large umbraco trees, and they are very heavy. I have seen loading times around five minutes. Not on the end user side, no in the back office. And yes, people do use the back office.

Umbraco also has a very strict set of rules. Umbraco is build as a tree, meaning you have a parent and a lot of children. I would love to see how you would handle a many to many relation, like product <–> category, using umbraco. In a database, I would add one table with two primary keys, set as foreign keys to product and category.

I usually use the argument of scalability. A database is build to be scalable, and in my opinion it is much easier to add an extra table and a couple of extra fields, than it is to create an extra doc type and a couple of extra fields. It is furthermore way easier to rearrange data in a database, than in umbraco.

This is my proposal: Build your own data store, cache and index your data using lucene, build a custom dashboard and tree for umbraco and stop forcing umbraco to do your job. Umbraco is a CMS, not EPR, CRM or shop. If you want that, build it as a plugin.

 

How to make localized models in Umbraco MVC

MVC, it’s the sh*t. Everything is so much easier when we use MVC. So when Umbraco came out with version 5, I was super happy, because I could now use my favorite framework on my favorite CMS. Then Umbraco killed v5 and we had to go back to webforms and simple razors…

BUT, then Umbraco announced they would implement MVC in the v4 branch and port many of the MVC features from v5 to v4. I was happy, for a while, I feared v4+ would end up just like v5. Mostly because of the weird mix of MVC and webforms. But no, the mix was elegant and seamlessly and super fast, so we began using it at my work. Everything was Good!

UNTIL, we had to make some more advanced stuff, like forms. Yeah, some of our customers wants forms, I know right! So we build a couple of forms, and the problems started to appear.

Umbraco is a “Content Management System”, meaing it manages content. One of the main forces of Umbraco is its ability to handle multiple languages. And the customers know that, well most of them do. So our customers wanted to be able to localize all the texts on the forms. This is something Umbraco cannot do, not easily and definitely not pretty.

When I build MVC forms I usually have a view model or two:

public string Name { get; set; }
public DateTime Birthday { get; set; }
public int NumberOfKids { get; set; }

This is a decent model. In MVC, I would build a form in one of three ways:

// One:
@Html.EditorForModel()

// Two
<div>@Html.LabelFor(m=>m.Name)</div>
<div>@Html.EditorFor(m=>m.Name)</div>
<div>@Html.ValidationMessageFor(m=>m.Name)</div>

<div>@Html.LabelFor(m=>m.Birthday)</div>
<div>@Html.EditorFor(m=>m.Birthday)</div>
<div>@Html.ValidationMessageFor(m=>m.Birthday)</div>

<div>@Html.LabelFor(m=>m.NumberOfKids)</div>
<div>@Html.EditorFor(m=>m.NumberOfKids)</div>
<div>@Html.ValidationMessageFor(m=>m.NumberOfKids)</div>

// Three
<div>@Html.LabelFor(m=>m.Name)</div>
<div>@Html.TextBoxFor(m=>m.Name)</div>
<div>@Html.ValidationMessageFor(m=>m.Name)</div>

<div>@Html.LabelFor(m=>m.Birthday)</div>
<div>@Html.TextBoxFor(m=>m.Birthday)</div>
<div>@Html.ValidationMessageFor(m=>m.Birthday)</div>

<div>@Html.LabelFor(m=>m.NumberOfKids)</div>
<div>@Html.TextBoxFor(m=>m.NumberOfKids)</div>
<div>@Html.ValidationMessageFor(m=>m.NumberOfKids)</div>

I would preferably go for number one, or in some cases where I need to be in control of the markup, number two. I see no reason to use anything but EditorFor().

So from where do I get labels, display names and validation messages?

In MVC I would do something like this:

[Display(Name="Name", ResourceType=typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "NameRequired", ResourceType = typeof(SomeResourceFile))]
public string Name { get; set; }

[Display(Name = "Birthday", ResourceType = typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "BirthdayRequired", ResourceType = typeof(SomeResourceFile))]
public DateTime Birthday { get; set; }

[Display(Name = "NumberOfKids", ResourceType = typeof(SomeResourceFile))]
[Required(ErrorMessageResourceName = "NumberOfKidsRequired", ResourceType = typeof(SomeResourceFile))]
public int NumberOfKids { get; set; }

In MVC, I would get my localized texts from a Resource file, but Umbraco has no resource files, nor does it have a resource provider or some feature that allows to extract texts from the dictionary and add it to our model attributes. And I, for one, don’t think we can hand over a product being part CMS and part resource files. So how do I solve that problem.

It’s quite simple. No I am NOT going to write a resource provider for umbraco!

What I am doing is actually as simple as extending the attributes I use. So lets take the first one, Display. What I essentially want is just to pass in my dictionary key and then get a text from Umbraco.  The problem here is, I cannot extend DisplayAttribute, it would have been overkill anyway, so instead I will extend the DisplayNameAttribute-class:

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Property | AttributeTargets.Field | AttributeTargets.Parameter, AllowMultiple = false)]
public sealed class UmbracoDisplayAttribute : DisplayNameAttribute
{
    // This is a positional argument
    public UmbracoDisplayAttribute(string dictionaryKey) : base(dictionaryKey)
    {
    }
}

This is simple, but all we get now, is just the key to the Umbraco dictionary. I want the value. Also easy. We just need to override one property:

public override string DisplayName
{
    get
    {
        return umbraco.library.GetDictionaryItem(base.DisplayName);
    }
}

What this does, is each time I, or the framework, needs to get the DisplayName, we return the value from the dictionary. Simple as that.

So to use this new attribute, all you need to do is use this:

[UmbracoDisplay("MyDictionaryKey")]

Easy?

The other attribute (RequiredAttribute) is a bit different, it’s a validation attribute.

But still, equally able of being extended:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field | AttributeTargets.Parameter, AllowMultiple = false)]
sealed class UmbracoRequiredAttribute : RequiredAttribute
{
    public UmbracoRequiredAttribute(string dictionaryKey)
    {
        this.ErrorMessage = dictionaryKey;
    }
}

Please note, I don’t have a DisplayName property available, so I cannot set or override that one. What I can do instead, is setting the ErrorMessage property. But I cannot override that one either. It’s okay, I just won’t do it. Now you might think, “hey dumbass, you haven’t gotten anything from the dictionary yet!” You’re absolutely right, I haven’t.

We need to override a method called FormatErrorMessage. It looks like this:

public override string FormatErrorMessage(string name)
{
    return umbraco.library.GetDictionaryItem(base.FormatErrorMessage(name));
}

Please note, I still call the base method. This is because the base method gives us our error message (in this case, a dictionary key). If our key contained a, I think this is true, “{0}” the display name of the field would be added.

If you where to run this, you will see, that our required-attribute is not being used client side. This is because the RequiredAttribute does not implement the IClientValidatable-interface. So we must implement that:

public IEnumerable<ModelClientValidationRule> GetClientValidationRules(System.Web.Mvc.ModelMetadata metadata, ControllerContext context)
{
    // Kodus to "Chad" http://stackoverflow.com/a/9914117
    yield return new ModelClientValidationRule
    {
        ErrorMessage = this.ErrorMessage,
        ValidationType = "required"
    };
}

If you run the code again, you will find, that the client side, and the server side validation messages are now localized, using the Umbraco dictionary.

This approach described here, can be used on most of the different attributes, we use in MVC. Happy coding!

Database design on user defined properties

As a developer, I often get across some database designs, that are quite complex caused by a developer not quite understanding the problem (we’ve all been there!), and therefore cannot solve it correctly.

The problem is as follows (or similar):

You have 2+ tables:

No references

Initial tables – no references

It’s quite simple, you have two types of data (media and documents), that you want to store in your database. I get that, I would too.

Now the requirements are as follows:

Both media and document has a set of user defined properties.

Said properties must store the following values: Type and Value, and a reference to both Media and Document.

There are a couple of ways to solve this lil’ problem, one (the one I encounter the most):

References(1) - Database Inheritance

Property-table added and References are added to media and document.

In this setup the Property-table knows about both media and document. We could make the two foreign keys nullable, either way we depend heavily on our code to keep media and document properties separated. And what happens if we add an other type (say Users), then we have to add a new foreign key to the property-table, and expand our code even more.

An other approach is this:

<img class="size-full wp-image-762" alt="References(2) – Database Inheritance" src="http://ndesoft.dk/wp-content/uploads/2013/04/media_document_properties_take2 cialis overnight shipping.png” width=”404″ height=”444″ srcset=”http://ndesoft.dk/wp-content/uploads/2013/04/media_document_properties_take2.png 404w, http://ndesoft.dk/wp-content/uploads/2013/04/media_document_properties_take2-272×300.png 272w” sizes=”(max-width: 404px) 100vw, 404px” />

Media and document property references are stored in separate tables.

I must admit, I have done this one as well as the other one, and just admit it, so have you at some point!

So what are the pro’s and con’s of this setup: Well the pros are simple, neither media or document are referenced in the property table, we can have as many properties as we want per media and document, and we can quite simple add other types, such as Users. BUT:

When we have this setup, we must rely heavily on our code to help us not to have the same property on more than one media, and to ensure we don’t mix media properties with documents and users. And if we add an other type (Users) we must create, not only one, but two new tables, and still expand a complex code to handle that new type as well as the other types.

So how can we solve this problem?

We have Media, Documents and more types, that has dynamic  properties without the other types must know about it, we could do this:

References(3) – Database Inheritance

Each type now has its own set of properties

Yeah, I’ve also done this one. And this is almost, (I wrote, almost), as bad as the other ones. Well no property can be on more than one media (or document, or whatever), and no property can be on both media and document, so whats the problem?!

Well, for starters, we have to tables instead of one, per type. If we add an other field to our properties, we must add them to all of our *Property-tables. And if we want to list all properties, including the media/document/user/whatever it is attached to, it’s nearly impossible.

So here’s the solution, I find most fitting for the problem:

References, Inheritance – Database Inheritance

Added a Node-table, with the shared fields from Media and Document. Removed ID- and Name-fields from Media and Document, added a NodeID field, as both PK and FK. Added a Property-table, that references the Node-table.

So, this is my solution. I have added a Node-table, with the shared fields from Media and Document (ID and Name). Removed ID- and Name-fields from Media and Document, added a NodeID field, as both primary key and foreign key, this field must NOT be autoincremented! It will not work, then I added a Property-table, that references the Node-table.

The pros and cons: The pros are easy, One table per type, each type gets its ID from the Node-table, all properties are stored in one table, referencing the Node-table, so a Document can get its properties, using only its primary key. No property can ever be on two entities at once, and no entity knows about other entities or properties, except its own.

The cons are, that we must have some code that handles the inheritance. When I make a SELECT * FROM Media, I must make a JOIN on the Node-table as well. If you’re a .NET developer, like I, then you should take a look at the Entity Framework, as it handles this smoothly. I will write a post on that later on.

IEnumerable, why use it?

Okay, first off: I really love the IEnumerable<T> interface, and I use it excessively throughout my applications.

Almost any list or collection throughout the .NET framework implements this interface. It exposes only one method (GetEnumerator), but this method is your friend in almost any circumstance.

This method returns an Enumerator, which again exposes methods used to traverse (or loop through) the collection. This means you can make a method definition like this:

public void MyMethod(IEnumerable<string> someEnumerable)
{

}

I could call this method with any sort of collection, it could be string[], List<string>, IList<string>, IQueryable<string> or even a basic IEnumerable<string>. I don’t give a damn. I don’t care what the implementation is, I just want a collection to work with, nothin’ more.

This means I don’t need to force other developers who use my code, to pass an array or a List or some custom collection, they just pass whatever collection they have, and I will be able to work with it. That’s awesome!

But wait! How do you get an IEnumerable<T> in the first place? How can we make our own?

Well, in “ye olde times” one would make an abomination like this:

public List<string> MyMethod()
{
List<string> myResult = newList<string>();
for (int i = 0; i < dataSource.Length; i++)
{
// Do some work.
myResult.Add(dataSource[i].SomeProperty);
}
return myResult;
}

Can you see what’s wrong here?

There’s actually more than one thing that’s wrong with this snippet.

First off: When we return a specific implementation, in this case List<T>, we force our code to use this implementation. We kinda dictates the use of our result which is a major drawback on agile Development. What if the consumer don’t want or need a list?

Second: In line 3, we create an instance of an object we actually don’t need, so that we can return a list.

Third: We loop through a collection of an unknown size. This Collection could potentially contain a billion items.

Fourth: We have to wait until the loop has ended. Which could take forever. Think about what would happen to your app, when you loop though a billion items and only need the first five.

This is where .NET provides a concept of Enumerators. These are methods that only returns data when said data is ready to be returned on not when all data is ready to be returned. And it is actually the same method called from the GetEnumerator()-method.

To convert our method from above into an Enumerator, we must make some changes. First we need to return an IEnumerable<string> instead of a List. Second we need to remove the list-object. And finally rewrite the loop to use the keyword “yield”.

This is how our method would then look like:

public IEnumerable<string> MyMethod()
{
for (int i = 0; i < dataSource.Length; i++)
{
// Do some work.
yield return dataSource[i].SomeProperty;
}
yield break;
}

As you can see, the code looks way simpler. We don’t force our consumer to take a list. And the yield-keyword, changes the method into an Enumerator, which gives us a LOT of benefits:

The yield return, returns our data when the property Current is called on the Enumerator-instance. This property is usually called each time the MoveNext()-method is called. This Means you do not have to wait until all items are looped. You can stop at any moment. the yield-return is only called when you request the next item.

So if your consumer needs to loop through each of your items, you will only have to loop once, whereas the List-thingy above, would loop through the same items twice. That’s two billion items! Secondly the first example would always loop though each item, but the second one only loops through as many items as you want it to. Nothin more nothin less.

 

The essence of it all: Always use IEnumerable, only use lists where it cannot be done without it.

For performance reasons, always use Enumerators. They will save the day.

 

This is my first attempt on a “professional”-programming guide. Please provide feedback!

New site

I know it’s not long since my last visual transformations, but I feel I need to keep searching for a theme and for a setup that really suits me. And therefore I will keep looking.

This new theme includes a lot of features including a community in which I can handle more general questions, that does not suit any of my posts.

I have not yet fully explored all features, but I will do and in the meantime, please bare with me and the slowness of the site.

Future of NMultiSelect

Now that I have made a few releases of my jQuery plug-in, NMultiSelect, I have begun work on a future version of the plug-in, version 2.0.

At the moment that version has the following features, that the current (1.5.4) doesn’t:

The new features (at the moment) are:

  • Full jQuery 1.6+ compatibility (I like to move forward so this version will not be backward compatible with jQuery!)
  • NMultiSelect from <select>-tags (this is actually a quite cool solution, and doesn’t alter postbacks what so ever, so you can use it without altering your code-behind!).
  • Drag-n-Drop. (Who needs buttons anyway?).
  • Drop down. (Hide the big and ugly box away and minimize your layout.)
  • All NMultiSelect-boxes is found in their own <div>-tag now. (Prettier HTML).
  • Functions to remove all or single item from the lists. (Easier maintenance with AJAX).
  • Easier access to lost NMultiSelect-box reference using default jQuery selectors.
  • More methods to alter NMultiSelect-behavior.
  • Possibility to bind multiple event-handlers.
  • A lot of core updates and changes.
  • Auto loading of dependencies. All you need to get NMultiSelect is just the jQuery-file and NMultiSelect-file. The plug-in takes care of the rest.
  • A small graphical update. Just a little eye-candy.

If you have anything to add to this list, don’t be shy, just give ’em to me!

UPDATE:

I have made a few screen dumps of some of the new features, for you to see:

[nggallery id=1]

UPDATE 2:

I have published an early preview of the v2 of NMultiSelect.

Please note, this is not final, nor stable but I will try to fix as many bugs as possible within the next 1 or 2 weeks.

I’m sorry I haven’t posted anything for a long time, but I’m actually drowning in work so I’ve just been to damn busy. But I will make it up for you guys. I Promis!

You can see the preview here:

<a href="http://nmultiselect.ndesoft.dk/v2/nms buy cialis overnight delivery.html” target=”_blank”>http://nmultiselect.ndesoft.dk/v2/nms.html

NMultiSelect 1.5.4

Just a service update.

I think I was way to fast in releasing the 1.5.3-version. It was extremely buggy, had a few bugs and didn’t live up to the standards I think a release from me should meet. So therefore, forget the 1.5.3 (throw it away and never look back!) and get the latest NMultiSelect 1.5.4 instead. It is much more stable and nice than previous versions.

You can download NMultiSelect 1.5.4 here.

Or see it live here.

As always, comments are welcome!

Quick bug-fix, new features and Github

Hi, sorry about the time since my last post, I’ve just been so damn busy.

After being notified about a serious bug (a missing function, thanks wilco) and a few new features (thanks Jimmy), I’ve decided to add the project to github.

All source code is now located on https://github.com/JohnDoe3000/NMultiSelect, except for pre-1.5.3-releases. The code for those is to be found on this website. You are welcome to contribute, comment and so on, both here, and at github.

NMultiSelect 1.5.3

The new features in this release is (thanks Jimmy, http://jimblanchard.net/) are:

The ability to remove the add/remove-all buttons, remove the filter-box and remove the ability to chose multiple items to transfer at once.

The new options looks like this:

var settings = {
Name: "NMultiSelect_" + $.NMultiSelect.Instances,
Move: true,
FilterText: "filter", // Set to null to disable filters
AvailableText: "Available items:",
SelectedText: "Selected items:",
EnableAddAll: true, // If false, the add-all-button is removed.
EnableRemoveAll: true, // If false, the remove-all-button is removed.
EnableMultiSelect: true, // If false, the ability to select multiple items at once, is removed.
Height: 150,
Width: 150,
FadeSpeed: "fast", // How fast would we wan't the fading to go?
TitleTag: "h2", // What tag should be surrounding our box title?
Title: null, // Our box title, if null, the box title is not shown.
SelectionLimit: -1 // -1 unlimited selections.

}

To download the latest release, click here.

Up and running again

Hi,

Finally! The site is now running on a new server and everything seems to work properly! I apologize again for any inconvenience the “downtime” has caused, but the site should be working perfectly again.

As you might have noticed, I have change the look and feel of the site, I figured new server, new design. Over the following few weeks you may encounter a few changes regarding the layout and maybe a few moments with slow responses as I tweak the site and slowly changing the overall language to English.

If you have any thoughts about something cool to have on the site, please feel free to post them in the comments bellow.

Site problems

Hi all,

The last few days I have had major problems regarding the site, it has been slow as h… and I am very sorry for any inconveniences this has caused.

Because of the problems, I am changing to an other server. This will cause a few problems within the next couple of days, but I hope that the site is up and running again by Friday on the new server.

If you have any questions, feel free to post them as a comment below or contact me using my my email ( nick {at} ndesoft.dk ).