Asset Versioning using Grunt

What is Asset Versioning? This is sometimes referred to as cache busting. Assets for your website are CSS files, JavaScript files, images and other things that are used by all the pages in a website. These are files that should not change often, so for better performance, you want the browser to just download the file the first time it accesses your site and then use cached copies going forward. After the initial page load, the rest of the pages on your website should load quicker because of this. This is accomplished by having your web server set expiration dates on those files far in the future. You can do this in your Apache configuration like this:

<FilesMatch "\.(gif|jpg|png|js|css)$">
  ExpiresActive On
  ExpiresDefault "access plus 10 years"

However, what happens when you make a change to a CSS or JS file. Since the expiration date is set in the future, the browser will continue to use the original version. Fixing this problem is what Asset Versioning or Cache Busting is meant to handle.

I know about 3 methods of doing asset versioning:

  1. Query string parameter: /js/main.js?v=1.2.0
  2. Folder: /1.2.0/js/main.js
  3. File name: /js/main.1.2.0.js

I had always used the query string parameter method for asset versioning, but I have learned that there are problems with that method. I found an article by Steve Souders that explained some proxy servers will not cache items with query strings, so using this technique will cause your files to not be cached at all which will result in slower performance which is the opposite of what we want. (FYI – Steve Souders wrote the book High Performance Web Sites.)

I don’t like the folder method, because not all your assets will change with every release. This will result in the cache changing every time you have a release even though some items do not need to be changed.

This leads me to the file name method. There are several methods of versioning file names. You can use a version number (like 1.2.0), but that generally means maintaining the file names by hand. Another option is using a date stamp in the file name. One problem I have run into with this method was related to running multiple servers in a cluster. We use 5 web servers in our load balancer. We would deploy the code to each server and run the versioning code and it would add a date stamp to each file. However, the date stamp also included the hour. Not all of the boxes would be running the versioning code at the same time so some of the servers would use a string like “2016031114”, but others would be running a few minutes later and have a string that ended in 15. This would end up with occasional 404 pages when an asset would be requested but the request would get sent to a different server in the load balancer. The final way I know of creates a hash (like a md5 hash) based on the file contents and adds that to the file name. I have found this method works best. I have found a grunt plugin for doing this.

The plugin that I chose is grunt-assets-versioning. There are several npm modules available, but this is the one I like best. As well as versioning your assets, it will output a mapping file of the assets versioned that can be used in your backend assets controller. To install it, do npm install grunt-assets-versioning --save-dev. This will automatically add it to your package.json file during the install. After installing, you activate it by updating your Gruntfile with grunt.loadNpmTasks('grunt-assets-versioning'); and defining a grunt task to do the versioning.

This is the grunt task I setup:

assets_versioning: {
    options: {
      tag: 'hash',
      post: true,
      versionsMapFile: 'app/helpers/AssetHelper.php',
      versionsMapTemplate: 'app/helpers/AssetHelper.php.tpl',
      versionsMapTrimPath: 'public'
    files: {
      options: {
        tasks: ['uglify:global','uglify:advert','cssmin:target']

Here is a quick overview of the options I use. The “tag” option is using “hash” which means take an MD5 hash of the file contents to use. There is an option for doing a date stamp, but that can cause problems when deploying across a web cluster, so I recommend using the “hash” value. The option “post” is set true so it will process the files after running a task which we want since we also want to minify our files.

The remaining options deal with creating a version map so we can identify the files created during the asset versioning task. The “versionsMapFile” option is the file name to create. If you do not specify a template, it will create a JSON file with the name specified. Here is an example of the JSON file it would generate:

  { "originalPath": "public/js/global.min.js", "versionedPath": "public/js/global.min.xxxxxxxxx.js", "version": "xxxxxxxxx" },
  { "originalPath": "public/js/advert.min.js", "versionedPath": "public/js/advert.min.xxxxxxxxx.js", "version": "xxxxxxxxx" },
  { "originalPath": "public/styles/global.min.css", "versionedPath": "public/styles/global.min.yyyyyyyy.css", "version": "yyyyyyyy" }

The next option I want to review is “versionsMapTrimPath”. This will remove portions from the versionedPath that you do not want. I the case above, the docRoot for the website is the public directory. So the path will be correct when referencing it in a src in a script tag, we want to remove the leading public so it would be a valid path. You can see that is what I do in the options used above. This would give us a path of “/js/global.min.xxxxxxxx.js”.

However the key to making this useful is creating a template for creating a more useful file for us than the JSON format above. This plugin uses underscore.js for its templating engine. Here is the template I use for creating a PHP file:

namespace app\helpers;

use \app\lib\Helper;
class AssetHelper extends Helper
  public static $dict = array(
<% _.forEach(files, function(file) { %>
    "<%= file.originalPath %>" => "<%= file.versionedPath %>",
<% }); %>

This template would result in this file being created:

namespace app\helpers;
use \app\lib\Helper;
class AssetHelper extends Helper
  public static $dict = array(
    "/js/global.min.js" => "/js/global.min.5e0677e7.js",
    "/js/advert.min.js" => "/js/advert.min.49317176.js",
    "/styles/global.min.css" => "/styles/global.min.3d71d72b.css",

In our PHP application, we use Twig for templating. Here is how I created a Twig function to use the data in the PHP file created by the template.

$twig->addFunction(new Twig_SimpleFunction(
    function ($asset) use ($app) {
        if (isset(AssetHelper::$dict[$asset])) {
            $asset = AssetHelper::$dict[$asset];
        return '';
    array('is_safe' => array('html'))

$twig->addFunction(new Twig_SimpleFunction(
    function ($asset) use ($app) {
        if (isset(AssetHelper::$dict[$asset])) {
            $asset = AssetHelper::$dict[$asset];
        return '';
    array('is_safe' => array('html'))

These functions work on the basis of if I find the string replace it with the versioned string otherwise use the original string. I would then use these functions like this:

{{ assetScript('/js/global.min.js') }}
{{ assetScript('/js/nonversioned.js') }}
{{ assetStylesheet("/styles/global.min.css") }}

This would output the following on what gets served to the client:

<script type="text/javascript" src="/js/global.min.5e0677e7.js"></script>
<script type="text/javascript" src="/js/nonversioned.js"></script>
<link href="/styles/global.min.3d71d72b.css" rel="stylesheet">

The last part of the grunt task is defining what files to process. In my case, I told it to process the output of tasks defined in the Gruntfile. The other tasks are for uglifying javascript and minifying css files. You can also list specific file names, but I find it more effective for performance to also uglify and minify assets.

This has been effective for me for versioning assets. If you have other ideas, please leave a comment.

Sharing templates between PHP and JavaScript

At work, we had a need to share templates between PHP and JavaScript. We had 2 paths to search results data. The first is from doing a dynamic search using AJAX and the results would be displayed by JavaScript. The second path was a way of browsing through the website and showing search results based on the browse selections. This path was server based and would be generated by PHP.

The search results would be displayed in cards, so we wanted to share the card template between both server side and browser. We saved a copy of the card template in the /public/views directory. This is a sample of what the card template looked like:

<div class="card">
  <div class="card_searchCard">
    <h1 class="card_name clearfix"><a href="{{ url }}">{{ display_name }}</a></h1>
    <div class="card_specialty">
      <span>{{ specialty }}</span>
  <div class="card__body clearfix">
    <div class="card__address row">
      {{ address1 }}
      {{ address2 }}<br>
      {{ city }}, {{ state }} {{ zip }}

To use this template in PHP, we settled on using Mustache PHP. A rough example of how we used it to render our results is:

$mustache = new Mustache_Engine(array(
    'loader' => new Mustache_Loader_FilesystemLoader(dirname(__FILE__) . '/../../public/views'),

$cards = array();
$template   = 'professionalCard';
foreach ($results as $result) {
    $cards[] = $mustache->render($template, $result);

This basically takes each result and renders it through the Mustache PHP templating engine. We saved the output of each result as a card and then would use those cards later.

It was trickier to use this from JavaScript, because we had to load the template from the web server to have it available for use. We decided to use MustacheJS for doing the JavaScript rendering. Normally, I would include the template with a script tag. However, if I included the code needed to make the template work in a script tag, the same file would not work with the PHP rendering. So the template files only contained the template with no extra code.

Here are the functions we setup for retrieving the template and rendering it with data:

function render(tmpl_name, tmpl_data) {
  var template;
  template = getTemplate(tmpl_name);
  return Mustache.render(template, tmpl_data);
function getTemplate(tmpl_name) {
  if ( !getTemplate.tmpl_cache ) {
    getTemplate.tmpl_cache = {};

  if ( ! getTemplate.tmpl_cache[tmpl_name] ) {
    var tmpl_dir = '/views';
    var tmpl_url = tmpl_dir + '/' + tmpl_name + '.mustache';
    var tmpl_string;
        url: tmpl_url,
        method: 'GET',
        async: true,
        success: function(data) {
          tmpl_string = data;
          getTemplate.tmpl_cache[tmpl_name] = _.template(tmpl_string);
  return getTemplate.tmpl_cache[tmpl_name];

Here is how we used those functions:

for (var i = 0; i < cardHits.length; ++i) {
  var hit = cardHits[i];
  var rendered_html = render(templ_name, hit);

  // render the hit
  res += rendered_html;

We chose to make sure the templates were preloaded before they were needed, so we added this document ready function:

  $( document ).ready(function() {

When we didn't preload the templates, we would get errors unless we made the AJAX call to load the template a synchronous call. I didn't like having a blocking function, so opted to load the templates at document ready.

If you have any questions or suggestions on alternate solutions, please leave a comment.

Making sure /usr/local/bin is in your path

I was working on a linux system that was not automatically adding /usr/local/bin to the PATH which was causing problems trying to invoke local scripts. I found that creating the file “/etc/profile.d/” with the following content:

if [[ ! ":${PATH}:" == *":/usr/local/bin:"* ]]
then PATH="${PATH}:/usr/local/bin"

made sure that it got added to the PATH of all users on login.

Programming Katas

I have been doing some programming katas lately to help me learn more about unit testing. Kata is a term taken from martial arts where they would repeat a drill to perfect a technique. In programming, a kata is the same. It is some task that you repeat to learn how to do something better. I have been using to do my katas. That is a website that has been setup for this purpose. The advantage it provides is you don’t have to do all the setup to prepare for learning. It has about 25 languages and about 40 exercises you can choose from. It also has different unit testing packages available depending on which language you select. I have been using this to help learn about other languages as well as learn more about unit testing. I have found that I learn better by doing than just reading about things and cyber-dojo helps me do that. When I am also trying a new language for the first time, it does take more time to learn the testing tool as well as that language syntax, but that is the purpose of doing the katas. It does get quicker the more you do it.

Exploring Ruby

I spent some time exploring and playing with Ruby this last week. I used Google to help with some of the items I was testing. I wish there was some type of accuracy score Google could assign web pages because I ran into some with inaccurate information. The following are some random comments from my testing.

“rspec” (Ruby’s test runner) does not provide informative error messages when Ruby fails. The info provided when a test failed was okay, but not when Ruby failed. I kept getting an error about an unexpected end statement, but it didn’t make sense. I finally figured out that it didn’t like the beginning class line because I had spelled it “Class”. I didn’t realize Ruby was case sensitive. From my time with JavaScript, I capitalized “Class” because you capitalize other keywords in JavaScript like “Object” or “Array”. Once I changed the spelling to be “class” the error went away, but I had chopped up my code by then trying to figure out the error.

I found differing ways of handling classes with inheritance on Google. I changed it around before I fixed the class spelling so I am not sure if either way will work. I saw examples like this:

class BaseClass

   class DiffClass < BaseClass



where the class inheriting from the base class was within the base class definition. I also saw it where it was outside of the base class definition. I ended up with it outside the class, but I was still getting the error from the class misspelling so I don't know if the above technique would work, but it seems wrong to me.

I also saw a couple of ways of handling class properties. I saw references like "" and "@property". I tried the "" first since it was similar to the JavaScript technique of "". I didn't get any syntax errors, but I did get errors from rspec showing that the tests failed. When I switched to "@property", everything passed. I still don't understand why, but I will keep playing around with it to try and figure it out.

Overall, I enjoyed exploring Ruby and intend to spend some more time with it.


I have been interested in starting to use unit tests to validate development. In my current job, we do a lot of green screen development for the Multi-Value database environment. There did not seem to be any good options for implementing unit tests in this environment. At work, we just had a presentation on mvTest by Brian Leach. Brian has developed a tool for doing unit tests in green screen environments. It is similar to how Selenium uses a web driver to drive a browser through a site and look for output in a web page to confirm the processing except this encapsulates a telnet session and looks for screen changes. It is an effective way of implementing unit tests in a green screen environment. I am looking forward to getting the chance to work with it.

Back-end Frameworks

I have an idea for an experimental project that I am going to do as a learning tool. I am going to do the front-end using a JavaScript framework, but I am still considering what to use for the back-end. Normally, I have done the back-end in PHP and created my own hodge-podge of a framework. It would work well for the project, but wasn’t necessarily the best organized.

For this project, I was going to use PostgreSQL for the database. It has been a while since I used it and I haven’t used some of the newest features available in the latest version. I haven’t settled on a back-end framework yet. Here are some of the ones I am considering:

  • Laravel – This is in PHP which I already know, but I have heard good things about it and it could help create a well-structured back-end. PHP is readily available on lots of hosting services, so it would be easy to set up anywhere. However, I know PHP already and I was wanting this to be a learning experience, so this would not add much to my existing knowledge.
  • Zato – This is written in Python and intended for use in developing back-end APIs which is what I am after. I have read some books and articles on Python and have always wanted to learn it. I also know of some local companies that use Python, so there would be some local help available. The admin interface is developed in Django, so this would also be a chance to get some exposure to Django. This is probably the most intriguing of the choices I am looking at.
  • Ruby – Ruby is actually just a language, but I am interested in looking at Rails and Sinatra as possible frameworks. Ruby gets a lot of press and sounds interesting. I have read a couple of articles and looked at some code. I am not aware of any local companies using it and I don’t know anyone who does use it, so I wouldn’t have as many resources available for help. However, I have always wanted to learn Ruby and Rails, so this is still an option to consider.
  • .NET – Once again, this is more a family of languages and interfaces pushed by Microsoft. They did recently release .NET as open-source so that makes it more intriguing. There is also a large .NET market in the Tulsa area, so that also makes it intriguing. However, I need to do more research to find a framework based on .NET so I would do the roll my own like I have done with PHP in the past.

I would appreciate any feedback or comments. It is still up in the air and getting other people’s opinions would help in narrowing my decision.

Cache Performance Monitoring

We are using Intersystems Cache database on one of the projects I am working on. We were concerned with how do we know if it is performing well. I went through the Cache Monitoring Guide in the documentation provided. It is impressive the number of tools and methods they have implemented to monitor the database. However, they never explain how to interpret the data provided. They have one statistic called “Cache Efficiency”. The description of it is “Number of all global references divided by the number of physical block reads and writes. Not a percentage.”. That is all. There is nothing about how to interpret the number, how to tell if it is good or bad. Nothing. We have 2 systems – production and development. The production server has a number of 480 and the development is 1128. How do those compare?

All I have to say is if you are going to document how to monitor the system, also give guidance on how to interpret the data provided so you can tell if you need to take action.