Using ETags in SharePoint REST Calls to Manage Concurrency Control

Have you ever needed to generate a unique identifying number, sort of like the ID column in a list? Well, until I started learning the REST APIs, I had no idea how we could ever do this in SharePoint. With the adherence to the OData standards, we can do this and more using eTags to manage concurrency.

Concurrency control is a thorny problem in computer science. It refers to the situation where more than one person requests to change (either via update or delete) a piece of data stored somewhere. The only way you can tell if someone else has changed the data before you try is to read it from the source again. In the meantime, someone may have changed the data – lather, rinse, repeat.

People come up with all sorts of fragile concurrency control in their applications, and you see the results of that in systems from time to time. Have you ever scratched your head and been *sure* that you updated something, only to see the old value(s)? It may have been poor concurrency control that caused it.

In the prior SOAP-based calls supported by SPServices, we had no reliable way to manage concurrency. In most cases, we would build things so that the last person in wins – whomever requests a change last overwrites any prior changes. This is fine in many, many cases, but if more than one person is likely to make changes to data – and you want to manage those changes cleanly – it falls down fast.

With the move to using REST more fully in SharePoint 2013 and SharePoint Online in Office365, we have very strong concurrency control capabilities. For this to work well, the standard implements a concept called “eTags” which lets your code know if there is a concurrency problem. This works well because there is a collaboration between the server and the client about what constitutes a change and how to manage it.

Let’s look at my unique identifier problem as an example to explain how this all can work for you. In an application I’m building, I need to generate unique Document Set names based one a set of rules. In other words, it’s not sufficient to use something like the Document ID Service – I need to be able to build the names using some of my own logic which manifests the business rules.

As part of these names, I need to generate a unique, sequential number. In other words, the first “thing” will be named something like “AAAA-0001”, a subsequent “thing” may be named “BBBB-0001”, the next might be “BBBB-0002”, then “AAAA-0002”, etc., where each of the numbering schemes is independent. The Document Set ID in the list won’t do it.

As you can see, we have a unique numbering scheme based on a step in a process. (There are more varieties than just AAAA and BBBB.) We need to keep track of each unique scheme separately.

We’ve got a Configuration list in a SharePoint site. I’ll often build one of these lists for a project, and it simply contains parameter name/value pairs, along with a description of how it is used.

As you can see, for each step in the process, we’re maintaining a unique “last sequence number” in the Configuration list. (Yes, our real step names are more descriptive than AAAA, BBBB, CCCC!)

To make this work, I’ve got a function that does the following:

1. Read all of the parameters from the Configuration list. It’s no more “expensive” to read all the parameters than just the one we need, so we read all of them.
2. Find the value we need
3. Attempt to write the value + 1 back into the list, passing back the eTag in the “IF-MATCH” parameter in the header of the request
4. If we have a concurrency issue, then go back to step 2 above, and repeat until we are successful
5. If success, pass back the value for the sequence number

In this particular application, the likelihood of two people clicking the “Get Name” button at the exact same time is very low, but we still want to manage this eventuality so that we don’t end up with two Document Sets with the same name.

The call to the function looks something like this:

$.when(ProjectName.Functions.GetNextSeq(ProjectName.DataSources.Steps.Abbreviation)).then(function() {
    nameInput.val(ProjectName.DataSources.Steps.Abbreviation + "-" + this); });
});

I call GetNextSeq with the abbreviation for the step, e.g., “AAAA” or “BBBB”. Because the GetNextSeq function passes back a jQuery promise, I can say basically:

Wait until $.when GetNextSeq is done, then build up the name for the Document Set

The GetNextSeq function looks something like this:

ProjectName.Functions.GetNextSeq = function(abbrev, p) {

  // If we've passed in a deferred object, then use it, else create a new one (this enables recursion)
  var deferred = p || $.Deferred();

  // Get the information from the Configuration list
  ProjectName.DataSources.Configuration = {};
  ProjectName.Promises.Configuration = $.ajax({
    url: _spPageContextInfo.siteServerRelativeUrl +
      "/_api/web/lists/getbytitle('Configuration')/items?" +
      "$select=ID,Title,ParamValue",
    method: "GET",
    headers: {
      "Accept": "application/json; odata=verbose"
    },
    success: function(data) {

      /* Looping through the list items creates a JavaScript object like: 
          { "LastSeqAAAA" : { paramValue: "3", ID: "1", etag: "3" },
          { "LastSeqBBBB" : { paramValue: "103", ID: "1", etag: "110" }
          etc.
      */
      for (var i = 0; i < data.d.results.length; i++) {
        var thisParam = data.d.results[i];
        ProjectName.DataSources.Configuration[thisParam.Title] = {
          paramValue: thisParam.ParamValue,
          ID: thisParam.ID,
          etag: thisParam["__metadata"].etag
        }

      }

      // Next, we try to save LastSeqXXXX back into the list, using the eTag we got above
      $.ajax({
        url: _spPageContextInfo.siteServerRelativeUrl +
          "/_api/web/lists/getbytitle('Configuration')/items(" + ProjectName.DataSources.Configuration["LastSeq" + abbrev].ID + ")",
        type: "POST",
        contentType: "application/json;odata=verbose",
        data: JSON.stringify({
          "__metadata": {
            "type": "SP.Data.ConfigurationListItem"
          },
          "ParamValue": (parseInt(ProjectName.DataSources.Configuration["LastSeq" + abbrev].paramValue) + 1).toString()
        }),
        headers: {
          "Accept": "application/json;odata=verbose",
          "X-RequestDigest": document.getElementById("__REQUESTDIGEST").value,
          "X-HTTP-Method": "MERGE",
          "IF-MATCH": ProjectName.DataSources.Configuration["LastSeq" + abbrev].etag
        },
        success: function(data) {
          // If the write is successful (response 204 (No Content)), resolve the promise with the value we should use for the sequence number (padded with leading zeroes)
          deferred.resolveWith(pad(ProjectName.DataSources.Configuration["LastSeq" + abbrev].paramValue, 4));
        },
        error: function(data, a, b) {
          // If the server sends back a 412 response (Precondition Failed), then we have a concurrency issue, so call the function again with the existing deferred object
          if (data.status === 412) {
            ProjectName.Functions.GetNextSeq(abbrev, deferred);
          }
        }
      });
    },
    error: function(data) {
      alert('API error: ' + data);
    }
  });

  // Return the deferred object to the calling code
  return deferred;

}

There’s one other function I call above, and it’s just a little utility function used to pad the sequence numbers with leading zeroes.

function pad(num, size) {
	var s = num + "";
	while (s.length < size) s = "0" + s;
	return s;
}

While my requirements here may not match yours, the basic method probably will sooner or later. We want to read a value (item) from a list and then make an update to it based on some user action. Using eTags, we can do this reliably.

You don’t need to care what the eTag values are, but SharePoint seems to use a number much like a version number to manage this. If the item has been edited once, you’ll probably get back “1”, twice, probably will give you “2”, etc. The key here is, though, that you don’t care what the eTag value is: just whether it has changed. Even that doesn’t matter, as the 412 response from the server lets you know this is the case. By sending in the prior eTag value with your request, the server can tell you if there’s a problem. It’s up to you to handle that problem.

What if we want to do things the old way, like we did with SOAP? We simply omit the “IF-MATCH” parameter or pass “IF-MATCH”: “*”, meaning “I’ll accept any old value of the eTag – write away!”

Resources:

• Working with lists and list items with REST
• Optimistic concurrency control in ASP.NET WebAPI

This article was also published on IT Unity on 11/3/2015. Visit the post there to read additional comments.