SPServices 2013.01ALPHA4 Returns a Deferred Object (Promise)

I’ve just posted a new alpha (ALPHA4) for SPServices 2013.01. This alpha implements the return of a deferred object, aka a promise, as well as executing a completefunc (if provided) for backward compatibility. Note that the deferred object capability was introduced in jQuery 1.5, so this version of SPServices requires that version or greater. I’ve done my initial testing with jQuery 1.8.2.

So, what does this mean to you, the SPServices user? Well, in the short term for most of you it may not mean anything unless you understand the first paragraph above. If you do, I could really use some help in testing the deferred object capability.

Here’s how I’ve implemented it. There are two big differences in the core SPServices function. First, I’ve set it up so that it runs a completefunc if one is provided, just has it always has, but also returns a promise. This means the code has gone from this:

var cachedXML;
var status = null;

if(opt.cacheXML) {
  cachedXML = $("body").data(msg);
}

if(cachedXML === undefined) {
  // Make the Ajax call
  $.ajax({
    url: ajaxURL,                      // The relative URL for the AJAX call
    async: opt.async,                    // By default, the AJAX calls are asynchronous.  You can specify false to require a synchronous call.
    beforeSend: function (xhr) {              // Before sending the msg, need to send the request header
      // If we need to pass the SOAPAction, do so
      if(WSops[opt.operation][1]) {
        xhr.setRequestHeader("SOAPAction", SOAPAction);
      }
    },
    type: "POST",                      // This is a POST
    data: msg,                        // Here is the SOAP request we've built above
    dataType: "xml",                    // We're getting XML; tell jQuery so that it doesn't need to do a best guess
    contentType: "text/xml;charset='utf-8'",        // and this is its content type
    complete: function(xData, Status) {
      if(opt.cacheXML) {
        $("body").data(msg, xData);        // Cache the results
      }
      cachedXML = xData;
      status = Status;
      opt.completefunc(cachedXML, status);        // When the call is complete, do this
    }
  });

} else {
  opt.completefunc(cachedXML, status);            // Call the completefunc
}

to this:

// Check to see if we've already cached the results
var cachedXML;
if(opt.cacheXML) {
  cachedXML = promisesCache[msg];
}

if(typeof cachedXML === "undefined") {

  // Finally, make the Ajax call
  var spservicesPromise = $.ajax({
    // The relative URL for the AJAX call
    url: ajaxURL,
    // By default, the AJAX calls are asynchronous.  You can specify false to require a synchronous call.
    async: opt.async,
    // Before sending the msg, need to send the request header
    beforeSend: function (xhr) {
      // If we need to pass the SOAPAction, do so
      if(WSops[opt.operation][1]) {
        xhr.setRequestHeader("SOAPAction", SOAPAction);
      }
    },
    // Always a POST
    type: "POST",
    // Here is the SOAP request we've built above
    data: msg,
    // We're getting XML; tell jQuery so that it doesn't need to do a best guess
    dataType: "xml",
    // and this is its content type
    contentType: "text/xml;charset='utf-8'",
    complete: function(xData, Status) {
      // When the call is complete, call the completefunc if there is one
      if($.isFunction(opt.completefunc)) {
        opt.completefunc(xData, Status);

      }
    }
  });

  spservicesPromise.then(
    function() {
      // Cache the promise if requested
      if(opt.cacheXML) {
        promisesCache[msg] = spservicesPromise;
      }
    },
    function() {
                                // TODO: Allow for fail function
    }
  );

  // Return the promise
  return spservicesPromise;

} else {
  // Call the completefunc if there is one
  if($.isFunction(opt.completefunc)) {
    opt.completefunc(cachedXML, null);
  }
  // Return the cached promise
  return cachedXML;
}

I don’t usually post the differences in the code in such detail, but I want to get some eyeballs on it so that I can gather yeas or nays on the implementation. I wanted to keep it as clean as possible yet maintain 100% backward capability. I seem to have accomplished both in all of my testing, but I’d love feedback. I’m especially curious what folks think I should do in the failure case of the .then() function. In the past I’ve not done anything, leaving it to the SPServices user to decide what they would like to do. The only two options I can think of are:

  1. Add a retry loop to give it a few more shots before giving up
  2. Pop up an error message of some sort

Neither of these two options feel right to me. If the .ajax() call fails, in my experience it’s because the server isn’t responding 99.9999% of the time. Trying again won’t usually solve that unless it’s simply a lag after an IISRESET. Popping up an error doesn’t really help, either, as it would have to be some sort of generic “Contact your administrator” nonsense, which I avoid assiduously. Thoughts?

The second change is that I’ve moved from using the .data() jQuery function for caching to caching the promises in an array, as you can see in the code above. This difference should be invisible to all end users and to most developers who use SPServices. However, since it means a difference in how the caching works since I introduced it in v0.7.2, I want to be sure that I get people testing that as well.

Finally, as a further test of the deferred object capability, in this alpha the SPArrangeChoices function also uses the returned promise for GetList as a test to improve the performance and user experience. This function was a good candidate for the first internal test of the new architecture, as it is relatively straightforward and only makes one Web Services call to GetList. I’ll be implementing more uses of promises inside the library as it makes sense before finalizing this release.

So if you’re a hard-core SPServices user, please give this one a test for me. Obviously I’m interested in any regression, but I’d also like to know how the returned promises work for you if you can write new calls against this version. As an illustration of how you can use this new capability, here’s what I’ve done in SPArrangeChoices:

// Rearrange radio buttons or checkboxes in a form from vertical to horizontal display to save page real estate
$.fn.SPServices.SPArrangeChoices = function (options) {

  var opt = $.extend({}, {
    listName: $().SPServices.SPListNameFromUrl(),          // The list name for the current form
    columnName: "",          // The display name of the column in the form
    perRow: 99,            // Maximum number of choices desired per row.
    randomize: false        // If true, randomize the order of the options
  }, options);

  var columnFillInChoice = false;
  var columnOptions = [];
  var out;

  // Get information about columnName from the list to determine if we're allowing fill-in choices
  var thisGetList = $().SPServices({
    operation: "GetList",
    async: false,
    cacheXML: true,
    listName: opt.listName
  });

  // when the promise is available...
  thisGetList.done(function() {
    $(thisGetList.responseXML).find("Field[DisplayName='" + opt.columnName + "']").each(function() {
      // Determine whether columnName allows a fill-in choice
      columnFillInChoice = ($(this).attr("FillInChoice") === "TRUE") ? true : false;
      // Stop looking;we're done
      return false;
    });
[...]
  });
}; // End $.fn.SPServices.SPArrangeChoices

Here’s another chunk of test code that I’ve been using to see that the caching is working as I’d like. The Census Data list on my Demos site has over 3000 items in it, so it’s a good list to test timings with, as it takes a comparatively long time to send the data down the wire. I’ve got the async option and the completefunc commented out below; you can play around with the combinations here as I have if you’d like.

function getIt() {
  var outUl = $("#WSOutput ul");

  logTime(outUl, "Start: ");
  var getListItemsPromise = $().SPServices({
    cacheXML: true,
//    async: false,
    operation: "GetListItems",
    webURL: "/Demos/",
    listName: "Census Data"
//    completefunc: function (xData, Status) {
//      logTime(outUl, "completefunc: " + $(xData.responseXML).SPFilterNode("rs:data").attr("ItemCount"));
//    }
  });
  logTime(outUl, "End: ");

    getListItemsPromise.done(function() {
    logTime(outUl, "promiseComplete: " + $(getListItemsPromise.responseXML).SPFilterNode("rs:data").attr("ItemCount"));
    });
}

function logTime(o, t) {
  o.append("<li>" + new Date() + " :: " + t + "</li>");
}

with this simple markup:

<input type="button" onclick="getIt();" value="Get It"/>
<div class="ms-vb" style="width:100%;" id="WSOutput"><ul></ul></div>

Let me know what you think, and stay tuned for more changes in this release, which you can keep track of in the Issue Tracker.

Thanks go to Scot Hillier (@scothillier) for opening my eyes to the value of deferred objects by showing examples in several of his recent sessions.

14 Comments

    • Scot:

      Thanks for taking a look. I’ve already taken it a bit further in ALPHA5 (not yet posted). I realized that subsequent calls for the same data (e.g., another GetList on the same list) wouldn’t “see” the prior calls because I wasn’t caching the msg until I got the response. I’ve fixed that and I’m doing some further testing with some of the other functions.

      Question for you: If the same promise is returned to multiple functions, is there any guarantee that they will resolve in the order received?

      Stay tuned…

      M.

      Reply
      • Marc:
        I believe that the only way to guarantee that they will resolve in the order received is to chain them – however to be be honest I am not 200% sure.

        As you’re rewriting, have you considered other optimizations such as replacing $.each with native JavaScript for() loops? In all modern browsers, the native for loops are much more performant as there is much less function overhead (reference: http://jsperf.com/jquery-each-vs-for-loop/220).

        Reply
        • Kon:

          So far in my experimentation, I haven’t come to a conclusive answer, either.

          No, I haven’t considered replacing the .each() loops. Frankly, they are such a small part of the overall processing here, I doubt that you’d see any difference.

          M.

          Reply
    • That’s an interesting debate that Domenic is putting forth. In technology, it seems that everyone loves to ridicule other people’s code.

      I’m going to put my trust in the jQuery team on this one. The way I’m returning promises from SPServices core is extremely simple. In addition to calling the completefunc (if it is offered), I’m returning a jQuery .Deferred().

      What any developer chooses to do with that promise is up to them. It’s about as bare an implementation as I can do. (It also allows me to improve on the caching model I introduced in 0.7.2.)

      The bigger burden is, I think, the example calls I give for GetListItems and some of the other commonly used operations. I find that my examples end up basically verbatim in others’ code.

      M.

      Reply
      • I’m with you that when someone runs into an issue with jQuery promises as of today, they will figure out a way to address it. Just wanted to make sure that you are aware of it.

        In terms of example code that gets copied, that’s what examples are good for, aren’t they? At the end of the day GetListItems is a transport level method that has a lot of options.

        So I would even try to make it easier to copy example code by using knockout (or backbone) to create a ListModel instance that works as an abstraction layer to the underlying SPServices calls.

        Now you only need to copy the following :), way less things that can go wrong.

        var myDS = new sp3.ListModel({
        siteUrl: ‘/sites/ts’,
        listTitle: ‘tasks’
        });

        myDS would know all the details how to page, sort, filter via method calls e.g myDS.sort({field: ‘Title’, dir: ‘asc’}). As CAML filtering require knowledge of field types the model has to retrieve metadata via GetListInfo when it get’s instantiated.

        The result set gets converted into ItemModel instances and stored in a knockout observableArray.
        Like the ListModel ItemModel has its set of methods that abstracts the underlying UpdateListItems calls.

        Nothing revolutionary new, e.g. JayData provides that kind of functionality for SP2010/2013 listdata.svc, but it would help to reduce a lot of boilerplate code that needs to be written/copied all the time.

        Reply
        • Absolutely on all counts.

          One tenet I’ve given myself is that I shall not be dependent on any other library besides jQuery. This is the case in my code as well as in my documentation. I don’t want to haul in any of the other frameworks, since there are many and each has its merits.

          What you are showing here may lead to an excellent SPServices Story, and it’s the sort of thing that caused me to start the series in the first place. Are you game to write one?

          M.

          Reply
          • Here you go. I fleshed out the example below (http://www.spirit.de/demos/metro/SPServices/index.html#/getlistttems) a little, so that it better shows the advantage of using promises.

            In essence it retrieves list information via GetList to produce a mapping that can be consumed in SPXmlToJsonMap.

            Here’s Durandal’s activate function:

            ctor.prototype.activate = function () {
            var self = this;
            system.log(‘Model Activating’, this);

            // Step 1: Creating a list Model based on ‘GetList’ data
            // getModel() returns either a promise or a cached result
            // see http://lostechies.com/derickbailey/2012/03/27/providing-synchronous-asynchronous-flexibility-with-jquery-when/

            $.when(this.getModel()).then(function (data) {
            var listInfo = self.listInfo = data;
            var getListItems = $().SPServices({
            operation: “GetListItems”,
            async: false,
            webURL: self.webUrl,
            listName: self.listName,
            CAMLViewFields: “”
            });

            // Step 2: GetListItems using mapping data from Step 1:
            $.when(getListItems).then(function (data) {
            var json = $(data).SPFilterNode(“z:row”).SPXmlToJson({
            mapping: listInfo.SPXmlToJsonMap,
            includeAllAttrs: true,
            removeOws: false
            });

            self.listItems(json);

            system.log(‘Step 2: Got some Json’, json);

            }).fail(function (data) { system.log(‘Error’, data) });

            });
            };

            and here getModel()

            ctor.prototype.getModel = function () {
            var self = this;

            // Return cached result if available
            if (sp3.models[this.key()]) {
            system.log(‘Cached model’, sp3.models[this.key()]);
            return sp3.models[this.key()];
            }

            var getList = $().SPServices({
            operation: “GetList”,
            listName: this.listName,
            webURL: this.webUrl
            })

            // returns converted GetList XML as promise
            return $.when(getList).then(function (data) {
            return mapGetListResult2Json(data);
            });
            }

            Reply
    • Here’s a first stab on it.
      .
      http://www.spirit.de/demos/metro/SPServices/index.html#/getlistttems

      This uses DurandalJS (durandaljs.com), because this framework is using promises OOTB.
      For simplicity all app files are stored in a document library, so there should be no deployment issues

      The first couple of nav items are the default Durandal samples.
      GetListItems let’s you select between multiple lists and show a simply table view for the result.
      Lists for the select element are defined in getlistitems/index.js e.g.

      define(['./list', 'durandal/viewModel'], function (List, viewModel) {
          
        var lists = ko.observableArray([
            new List({
              name: 'Tasks',
              description: 'this is a tasks list',
              webUrl: '/demos/metro',
              listName: 'Tasks'
            }),
            new List({
              name: 'Contacts',
              description: 'and this a child list',
              webUrl: '/demos/metro',
              listName: 'Contacts'
            }),
            new List({
              name: 'Announcements',
              webUrl: '/demos/metro',
              listName: 'Announcements'
            })
          ]);
      
          return {
              lists: lists,
              activeList: viewModel.activator().forItems(lists)        
          };
      });
      

      The constructor function in getlistitems/list.js has a activate method that leverages SPServices, it’s new Promise functionality and SPXmlToJson.

        ctor.prototype.activate = function () {
            var self = this;
            var items = $().SPServices({
              operation: "GetListItems",
              async: false,
              webURL: this.webUrl,
              listName: this.listName,
              CAMLViewFields: ""
            });
      
            //Empty out existing items
            self.listItems([]);
      
            system.log('Model Activating', this);
      
            return $.when(items).then(function (data) {
              var json = $(data).SPFilterNode("z:row").SPXmlToJson({
                mapping: {},
                includeAllAttrs: true,
                removeOws: true
              });
      
              self.listItems(json);
      
              system.log('Got some Json', json);
      
            }).fail(function (data) { system.log('Error', data) });
          };
      

      Based on some feedback the example could be fleshed out some more of course ;-).

      Reply
  1. I am using the sparrangechoices to display checkboxes horizontally, however, the selections are not getting updated in my list. Took out the sparrangechoices and list gets updated. Any suggestions?

    Reply

Have a thought or opinion?