Architecting the Umbraco Platform!

In a previous post that we did on Umbraco, we highlighted a few points that make it a truly great open source Content Management System (CMS). The fact that it is ideal for content writers because of its simple editing and publishing interface, its wide usage to develop small scale and large-scale websites and the fact that Intelegencia has used it to develop more than 30 websites were points highlighted in the post. 

For those who missed out or those for whom it has been some time since they last read it, here is the link to go through the post again : Umbraco Rocks! 

We ended the previous post by saying that the ensuing post would be about Architecting the Umbraco Platform. 

Intelegencia Analytics, as a premier Process Engineering Company, has experienced phenomenal success on Umbraco because of the fact that with time we have learnt how to deploy Umbraco in a flexible Load Balanced scenario. Our tryst with Umbraco has taught us how to set up Umbraco for Load Balancing. Vending Technical Support Services teaches you a lot and Intelegencia has learnt with time that traditional load balancing turns out best for Umbraco versions less than 7.3.0. 

Umbraco, 7.3.0 or above, never works as per expectations if the back-office happens to be behind the load balancer. For it to work as per you convenience, a single server must be designated as the back-office server in order for your editors to log into to edit. In order to balance load, Microsoft’s Azure Web Apps is a tested cloud based auto-scaling appliance. Every server will host copies of the load balanced website files and there happens to be a file replication service running to keep all files on all servers up to date. Moreover, our experiences as an innovative vendor of Technical Support Services has taught us that a centralized file share contains the load balanced website files. 

As soon as you understand Umbraco’s way of load balancing, you will know how to host your site. As a Process Engineering Company always game for such challenges, Intelegencia has some insights to offer. Administrators and editors can create, update and delete data on the master server. Whatever activity ensues, it becomes a data structure known as “instructions” and get queued up in the database. Every front-end server checks for unprocessed instructions and as soon as a front-end server detects any, it downloads and processes them. After doing that, it updates its cache, cache files and indexes on its file system. 5-second delays often happen between content updates. These delays are also common while a front-end server is refreshing. This though, is completely normal and happens more often than not. 

For Scheduling and Master Selection with respect to Umbraco, a Master Server takes care of administration. This by no means is a “Scheduling Server”. There is a single scheduling server in Umbraco performing three tasks namely ensuring scheduled publishing, initiating configured scheduled tasks and initiating scheduled publishing for documents. 

Flexible Load Balancing plays its part by automatically electing a “Scheduling Server” to carry out services and for this to happen, every server must resolve the URL of either itself or its Master Server or the internal load balancer or the public address. As soon as the AppDomain starts, Umbraco by default sets the “umbracoApplicationUrl” to the address made by the first accepted request. This address is a DNS address considered easy to resolve by the server. To make this a little easy to understand, in case a public request gets to the load balancer on www.intelegencia.com, then the load balancer may divert the request onto the server with www.intelegencia.com being the original address. Hence, by default the “umbracoApplicationUrl” will be www.intelegencia.com. Load balancers here may also use a DNS name to route the request internally, such as “h99.intelegencia.local”. This implies that the “umbracoApplicationUrl” becomes “h99.intelegencia.local”. Whatever the case be, the elected “Scheduling Server” must be capable of resolving this address.

Now for Umbraco Configuration Files. 

Intelegencia, through its experiences with Umbraco as a Process Engineering Company, has found out that umbracoSettings.config file must be left untouched by those gunning to enable the distributed calls flag. Doing so, will not allow Flexible Load Balancing to work. 

• In case of cloud based auto-scale appliances: The focus here is purely on Azure Web Apps as cloud based appliances are being empowered with flexible load balancing. Documentation is on for Azure Web Apps here, though an identical setup is also possible on other services supporting ASP.Net. Logging configurations must be flawless. 

Azure Requirements: There must be an Azure Web App for the master environment (administration) and one for the front-end environment. Additionally, an SQL Server shared with both must be there.

Lucene/Examine Configuration:

Examine v0.1.80+

Examine v0.1.80 has introduced a new directoryfactory by the name SyncTempEnvDirectoryFactory. It is a mandatory addition to every indexer. 

directoryFactory="Examine.LuceneEngine.Directories.SyncTempEnvDirectoryFactory,Examine"

Examine is enabled by SyncTempEnvDirectoryFactory to sync indexes between the local environment temporary storage directory and the remote file system. If the need arises to access indexes, the same is possible through the temporary storage directory. The setting is mandatory as issues often arise with Lucene while working from a remote file share. Hence, there is a need for the files to be read/accessed locally. The setting ensures locally created indexes and normal indexes undergo automatic changes each time an index updates. Also, every time the app gets restarted or the local environment temporary files are cleared out, index files get restored from centrally stored index files. 

Pre Examine v0.1.80

• In ExamineSettings.config, the following properties can be added to all of the indexers and searchers: 

useTempStorage="Sync" tempStorageDirectory="UmbracoExamine.LocalStorage.AzureLocalStorageDirectory, UmbracoExamine" 

• The local workers file system is the place where Indexes are stored by the ‘Sync’ setting contrary to the Azure Web Apps remote file system which most think. Files need to be read/accessed locally as Lucene has issues while working from a remote file share. Each time the index updates, the setting writes to both locally created indexes and normal indexes. So whenever the app restarts or the local temporary files clear out, the centrally stored index files help restore the index files. It is easy to sort out issues seen during the syncing process (in logs) by changing the value to ‘LocalOnly’. Doing this will persist the index files to the local file system, which means they will be rebuilt as and when the site gets migrated between Azure workers. 

Users of Auto-Scaling must make it a point to update their Examine path settings. For the uninitiated, Azure Web Apps subscribe to a system of file sharing. Hence, an increase in the front-end environment scale setting to more than a single worker will result in multiple processes sharing the Lucene index files. 

Examine v0.1.83+ 

In recent times, Examine v0.1.83 has launched a new directoryFactory by the name TempEnvDirectoryFactory and it is a mandatory addition to all indexers in the ~/Config/ExamineSettings.config file. 

directoryFactory="Examine.LuceneEngine.Directories.TempEnvDirectoryFactory,Examine"

Examine is allowed by TempEnvDirectoryFactory to store indexes directly in the environment temporary storage directory. Moreover, it is mandatory to use this instead of SyncTempEnvDirectoryFactory. 

Pre Examine v0.1.83 

Each of the indexes in ExamineIndex.config must have their path tokenized in order to include the machine name as this will store the indexes in different locations for each machine. This is how a tokenized path looks like:

~/App_Data/TEMP/ExamineIndexes/{machinename}/Internal/ 

The path though comes with a few drawbacks:

• Azure Web Apps issues no warning in advance before it migrates the website between workers. Thus, the {machinename} changes and the index rebuilds.
• New workers rebuild their own index whenever one increases the number of workers or scales out. 

As a provider of premier Technical Support Services, Intelegencia continues to work towards finding ways to mitigate such issues. One such way discovered is the addition of the ability to store a master index in blob storage. This will ensure whenever new workers come online, they get to sync the existing index locally. 

Umbraco XML Cache File and other Temp Files 

The Umbraco XML config file must be stored on the local server for a front-end Azure Web App instance. This is all due to Azure using a shared file system. Adding a simple new app setting to web.config makes this happen: 

For Umbraco v7.7.3+

<add key="umbracoLocalTempStorage" value="EnvironmentTemp" />

Thus, umbraco.config and other Umbraco TEMP files get stored in the environment temporary folder. 

For Umbraco v7.6+

<add key="umbracoContentXMLStorage" value="EnvironmentTemp" />

Thus, umbraco.config gets stored in the environment temporary folder. 

For Umbraco Pre v7.6

<add key="umbracoContentXMLUseLocalTemp" value="true" />

Thus, umbraco.config gets stored in the ASP.Net temporary folder. 

Given below are steps that will help go about it: 

• Creating an Azure SQL database is imperative to get things started.
• Installing Umbraco on the master environment to use the Azure SQL database comes next.
• Installing Umbraco on the front-end environment to use the Azure SQL database follows.
• The next step is to perform a test, which comprises of certain content updates on the master/administration environment. One must ensure they work successfully on it and subsequently verify the changes appearing on the front-end environment.

Scaling 

One must not scale the master/administration environment. There is no support and scaling it may lead to serious issues. Azure Web Apps can be manually as well as automatically scaled both up and down. It also gets support by Umbraco’s flexible load balancing.

Deployment Considerations

As there happen to be two web apps, deployment must happen to both places. Various automation techniques help make this task effortless. This also implies that one must avoid editing templates or views on a live server as both master and front-end environments never share the same file server. It is imperative to make necessary changes made in a staging environment and then moved to live environments.  

1) Storage and Replication of Files 

Those not using a cloud-based setup will benefit most from this setup. One must begin after ensuring his ASP.Net and logging configurations are correct. 

Scaling 

Addition of servers and sites will keep scaling manually. Flexible load balancing ensures no additional configurations be made to Umbraco. Pointing the site to the Umbraco database will suffice, in addition to updating the load balancer to include the site.  

2) Storing Files on SAN/NAS/Clustered File Server/Network Share 

It can be tricky to configure servers to work through a centrally located file system shared for every IIS instance. The setup can be very time consuming. 

Scaling 

Scaling will be manual and involve adding servers and sites. Flexible load balancing will ensure no further configurations in Umbraco. Pointing the site to the Umbraco database will suffice followed by updating the load balancer to include the site.

---

Leave your comment

Submit