CRM 2011 Update Rollup 12 is finally out!

Following many delays, Microsoft have finally released CRM 2011 Update Rollup 12.
You can catch up on the release exploits here.

If you'd rather download the update or read about what's new in it, go right ahead.
Just a friendly tip, I suggest waiting at least 3-4 weeks before deploying this update to any production/live environment.

The updated SDK is also available and is now at build 5.0.13.

Picking apart CRM 2011's diagnostics page - Part 2

In the first part of this article I covered the basics of what goes on behind the scenes of CRM 2011's diagnostics page. If you missed it, catch up on it here: Picking apart CRM 2011's diagnostics page - Part 1.
I am now going to dive deeper into the JavaScript sections of the diagnostics tests. In total there are 8 JavaScript performance tests with 5 of those nested together under the JavaScript Dom stress test. I call these stress tests as they are basically memory and CPU intensive performance tests designed to give an indication of client computer (and browser) performance.
When running these tests they produce the following output (numbers may differ):

=== Array Manipultaion Benchmark ===
Time: 286 ms
Client Time: Wed, 16 Jan 2013 13:59:08 UTC
=== Morph Benchmark ===
Time: 334 ms
Client Time: Wed, 16 Jan 2013 13:59:08 UTC
=== Base 64 Benchmark ===
Time: 35 ms
Client Time: Wed, 16 Jan 2013 13:59:08 UTC
=== DOM Benchmark ===
Total Time: 561 ms
Breakdown:
  Append:  20ms
  Prepend: 25ms
  Index:   465ms
  Insert:  22ms
  Remove:  29ms
Client Time: Wed, 16 Jan 2013 13:59:09 UTC

So what does all that mean? What does each of these tests actually do?
Now it all gets interesting. :)

I spent some time taking apart the exact JavaScript functions executed for each of these tests. I have detailed them below including the source snippet.

Array Manipulation Benchmark

function arrayBenchmark() {
    for (var ret = [], tmp, n = 2e3, j = 0; j < n * 15; j++) {
        ret = [];
        ret.length = n
    }
    for (var j = 0; j < n * 10; j++)
        ret = new Array(n);
    ret = [];
    for (var j = 0; j < n; j++)
        ret.unshift(j);
    ret = [];
    for (var j = 0; j < n; j++)
        ret.splice(0, 0, j);
    for (var a = ret.slice(), j = 0; j < n; j++)
        tmp = a.shift();
    for (var a = ret.slice(), j = 0; j < n; j++)
        tmp = a.splice(0, 1);
    ret = [];
    for (var j = 0; j < n; j++)
        ret.push(j);
    for (var a = ret.slice(), j = 0; j < n; j++)
        tmp = a.pop()
}

This test builds up an array in memory and then performs various manipulations on it. The manipulations performed are (in order) unshift, splice, shift, splice, push, pop. Just simple memory manipulation games.

Morph Benchmark

function morphBenchmark() {
    var loops = 30, nx = 120, nz = 120;
    function morph(a, f) {
        for (var PI2nx = Math.PI * 8 / nx, sin = Math.sin, f30 = -(50 * sin(f * Math.PI * 2)), i = 0; i < nz; ++i)
            for (var j = 0; j < nx; ++j)
                a[3 * (i * nx + j) + 1] = sin((j - 1) * PI2nx) * -f30
    }
    for (var a = Array(), i = 0; i < nx * nz * 3; ++i) a[i] = 0;
    for (var i = 0; i < loops; ++i) morph(a, i / loops)
}

This test performs mathematical manipulations in a loop on an array variable. This is again a memory and CPU stress test.

Base 64 Benchmark

function base64Benchmark() {
    var toBase64Table = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/", base64Pad = "=";
    function toBase64(data) {
        for (var result = "", length = data.length, i = 0; i < length - 2; i += 3) {
            result += toBase64Table[data[i] >> 2];
            result += toBase64Table[((data[i] & 3) << 4) + (data[i + 1] >> 4)];
            result += toBase64Table[((data[i + 1] & 15) << 2) + (data[i + 2] >> 6)];
            result += toBase64Table[data[i + 2] & 63]
        }
        if (length % 3) {
            i = length - length % 3;
            result += toBase64Table[data[i] >> 2];
            if (length % 3 == 2) {
                result += toBase64Table[((data[i] & 3) << 4) + (data[i + 1] >> 4)];
                result += toBase64Table[(data[i + 1] & 15) << 2];
                result += base64Pad
            }
            else {
                result += toBase64Table[(data[i] & 3) << 4];
                result += base64Pad + base64Pad
            }
        }
        return result
    }
    var toBinaryTable = [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 62, -1, -1, -1, 63, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, -1, -1, -1, 0, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1, -1, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, -1, -1, -1, -1, -1];
    function base64ToString(data) {
        for (var result = "", leftbits = 0, leftdata = 0, i = 0; i < data.length; i++) {
            var c = toBinaryTable[data.charCodeAt(i) & 127], padding = data[i] == base64Pad;
            if (c == -1)
                continue;
            leftdata = leftdata << 6 | c;
            leftbits += 6;
            if (leftbits >= 8) {
                leftbits -= 8;
                if (!padding) result += String.fromCharCode(leftdata >> leftbits & 255);
                leftdata &= (1 << leftbits) - 1
            }
        }
        if (leftbits)
            throw Components.Exception("Corrupted base64 string");
        return result
    }
    for (var str = [], i = 0; i < 819; i++)
        str.push(String.fromCharCode(25 * Math.random() + 97));
    str = str.join("");
    for (var base64, loops = 1, i = 0; i <= loops; i++)
        base64 = toBase64(str);
    for (var i = 0; i <= loops; i++)
        base64ToString(base64)
}

A long and intimidating function that in essence converts values to and from base64 encoding. Once again a fairly simple memory and CPU stress test.

DOM Benchmark
This test is actually a combination of 5 sub tests. I have broken up the main function below so to explain more clearly.

function domBenchmark() {
    var count = 1500, divs = new Array(count);

This first JavaScript snippet creates the base structure within which the other functions operate. Basically creating the variables used by each sub function. Note that the 5 following functions are nested within the domBenchmark() function and are not root functions.
Later in this function the HTML DOM structure is created. The DIV element all functions relate to is created in just a few lines.

DOM Benchmark - Append

function testAppend(div) {
        for (var i = 0; i < count; i += 1) {
            var add = document.createElement("div");
            div.appendChild(add)
        }
    }

A function that creates 1500 DIV elements all nested under the source DIV.

DOM Benchmark - Prepend

function testPrepend(div) {
        for (var i = 0; i < count; i += 1) {
            var add = document.createElement("div");
            div.insertBefore(add, div.firstChild)
        }
    }

A function that creates an additional 1500 DIV elements but instead of appending them as before it prepends them. They are added to the top of the parenting DIV elements instead of the bottom.

DOM Benchmark - Index

function testIndex(div) {
        for (var i = 0; i < count; i += 1)
            divs[i] = div.childNodes[count * 2 - i * 2 - 1]
    }

A function that indexes 1500 of the div elements created in the testAppend() and testPrepend() functions. It only indexes every second DIV.
This is considered indexing as it takes the DIV structure and stores it in an array which acts as an index table. Again, meant at stressing memory and CPU.

DOM Benchmark - Insert

function testInsert(div) {
        for (var i = 0; i < count; i += 1) {
            var add = document.createElement("div");
            div.insertBefore(add, divs[i])
        }
    }

A function that takes the index table array variable and inserts each of its DIV elements into the HTTP DOM. A very similar test to the testPrepend() test but with array data instead of cleanly created.. A simple array insert test to stress memory and CPU.

DOM Benchmark - Remove

function testRemove(div) {
        for (var i = 0; i < count; i += 1)
            div.removeChild(divs[i])
    }

The final test function which removes 1500 of the DIV elements created earlier.

DOM Benchmark - Run Tests

var div = document.createElement("div");
    div.style.display = "none";
    div.setAttribute("id", "domBenchmarkDiv");
    document.body.appendChild(div);
    var start, end;
    start = new Date;
    testAppend(div);
    end = new Date;
    var appendTime = end - start;
    start = new Date;
    testPrepend(div);
    end = new Date;
    var prependTime = end - start;
    start = new Date;
    testIndex(div);
    end = new Date;
    var indexTime = end - start;
    start = new Date;
    testInsert(div);
    end = new Date;
    var insertTime = end - start;
    start = new Date;
    testRemove(div);
    end = new Date;
    var removeTime = end - start;
    document.body.removeChild(div);
    var total = appendTime + prependTime + insertTime + indexTime + removeTime, results = "Breakdown:\r\n  Append:  " + appendTime + "ms\r\n  Prepend: " + prependTime + "ms\r\n  Index:   " + indexTime + "ms\r\n  Insert:  " + insertTime + "ms\r\n  Remove:  " + removeTime + "ms\r\n";
    return [total, results]
}

And finally, actually executing all the above functions. This is done in turn and by order right after creating the parent DIV elements as well as some time tracking variables used for duration reporting of the results.
The final steps clean up the parent DIV element and return the stress test results as text.


As you can clearly see, these stress tests have very little to do with actual CRM performance on either the client or server. They are memory and CPU client stress tests designed to measure how browsers behave.
It is important to point out that these functions are rather short and simple and as such are not affected by antivirus scans (the dredded McAfee for example which can kill CRM through its ScriptScan feature).

I hope that sums it all up well enough.
I am open to any additional questions.

Picking apart CRM 2011's diagnostics page - Part 1

A few days ago I was called in to examine client-side CRM 2011 performance issues. I love a good examination process that lets me dig in!
After going through the obvious culprits such as update rollups, installed antivirus software, client and server specifications etc. it eventually turned out to be a server I/O problem which was causing longer than expected load times. I used HTTP Watch and WireShark to point me in that direction and then had some discussions with the IT team.
All that, however, wasn't the interesting part. Not even close.

During this examination I used CRM 2011's diagnostics page (http://<server>/<org>/tools/diagnostics/diag.aspx or http://<server>/tools/diagnostics/diag.aspx). This was part of CRM Online that came to the on-premise version in Update Rollup 4. More about that in Rhett Clinton's article. The reason I started picking apart the diagnostics page was because I was getting fairly good results despite CRM behaving worse than expected. Seeing that, I decided to check what exactly the diagnostic pages does and measures. Below is a sample of this page from one of my test systems.


CRM 2011 Diagnostics Page

One thing I'd noticed about the diagnostics figures is that they vary greatly between different networks and client computers. What does seem fairly constant (putting aside any real issues) is the ratio between the 4 JavaScript sections. While the number may vary, the Dom Benchmark will always be the longest and the Base64 test will always be the shortest.

The first thing I did was to understand what each of the tests stands for. These are as follows:

1. Latency Test
   Not exactly what I'd call a latency test but as close as you can get with HTTP. This calculates the average time taken over 20 downloads of a very small text file (only 12 bytes long).
   The file downloaded is /_static/Tools/Diagnostics/smallfile.txt
2. Bandwidth Test
   Performs downloads of image files in increasing size. Slightly similar in concept to many Internet based speed test sites. These download speeds are then averaged out (weighted of course) to give one average download speed value.
3. Browser Info
   Basic JavaScript pull of the local browser details such as browser name, version, cookie status, platform (OS) and the user-agent string.
4. IP Address
   Reports the IP address of the client computer as known to the server. This is passed as a variable in the diag.aspx file. The IP address is server-side dynamic and represents the IP address which was used to contact the server with.
5. JavaScript tests (there are 4)
   Runs various JavaScript times loops and returns their execution time. This is basically a memory/CPU stress test on the client machine. I will elaborate more on thest in Part 2 of this article.
6. Organization Info
   Basic server info such as organization name, time on server and url.

These tests output quite a lot of useful data to the Results text area. When running these tests I highly recommend looking at that and not just the values in the results table.

In the second part of this article I will dive into the JavaScript tests phase. If you're like me, that's the most interesting part. :)

"Gamification"

I sometimes roam through CodePlex in search of new interesting projects to follow and solutions I can use. In one of my recent roamings I came across a very interesting new concept - "Gamification". I'll get back to that in a bit.
What I came across on CodePlex was a CRM 2011 solution called "The Game for Microsoft Dynamics CRM 2011". Perplexed by this I kept reading.

Back to the new concept. I couldn't have put it better myself so I'll just quote the solution author on this.

The concept of ‘gamification’ is a simple one. To apply game design to non-game applications in order to make them more fun and engaging. Gamification has been used successfully across forums such as those maintained by Microsoft (http://social.microsoft.com/Forums) in order to encourage contributor participation and reward them for achieving a number of points and achievements. The result of this has been a flourishing forum community.
 

 Still puzzled and rather doubtful, I decided to try installing and configuring The Game on a test CRM organisation I had lying around. The installation itself was a simple CRM solution import and went by quite swiftly. Once complete I had a plethora of configuration entities all intertwined.
As I'm an active PlayStation 3 player (mostly spending my time on Dust 514 lately), things really started to look familiar now. I could set up games, define ranks and link them to games. I could even set up point thresholds at which users advance to the next level!
What really threw me back was the ability to also create and fully configure achievements. To anyone who's played even the simplest console game this should be quite clear. Using all the different configuration elements I could create a game based on my CRM system's existing business process.
Remember that boring old sales process? Moving through leads, opportunities and then the drudging quotes and orders?

After about half an hour's configuration (and learning curve) I was able to transform this whole sales process from a boring 100% business driven system into a system that still provides everything it did before but is also fun to use. Adding these gaming elements really livened up the system and I can easily imagine users being more enthusiastic about selling.
So what did I actually add? There were some workflows required (most of my learning curve was devoted to those) for the basic functionality of the game. Once those were in place I had the following structure for the game:

• New opportunities gained the user 5 points.
• New quotes gained the user 10 points.
• New orders gained the user 25 points.
• Closed opportunities gained the user 20 points.

In addition to the above, I also implemented some achievements. These were set up as follows:

• Opportunities opened and closed within 2 days granted users the "Swift Salesperson" achievement. This granted them an additional 40 points and a £50 cash reward (cash rewards went though accounting, naturally).
• With some additional workflow tools I granted every user on their first closed opportunity the "First timer" achievement. This granted an additional 50 points.
• With some additional workflow tools I was also able to count returning orders and users received the "Bulker" achievement for that. This also granted an additional 20 points per return order.

I can highly recommend this solution and method (obviously only where appropriate). I will be considering this for my future CRM implementations.