The ‘MVP Thoughts’ Series – Live from Sharegate World Headquarters

2015-11-18_17-30-46Last month a handful of SharePoint and Office 365 MVPs (now Office Servers and Services MVPs, but who’s counting?) gathered at Sharegate‘s offices in Montreal to help them with their next series of Damn Simple videos. Stay tuned for those.

After all the silliness – I mean serious cinematography – Jennifer Roth (@jennifermason), Corey Roth (@coreyroth), Fabian Williams (@fabianwilliams), and I sat down with the great Benjamin Niaulin (@bniaulin) to talk about Office 365 and SharePoint. You know – the usual. Each time a bunch of us gets together, though, the conversations tend to go in different directions. It’s even interesting for me to watch these, as our opinions often change over time.

I’m not sure how many segments we ended up with, but I’ll keep adding them here as they come out. If you’d like to read the transcripts instead, follow the title links.

MVP Thoughts: What to Expect From SharePoint 2016

MVP Thoughts: SharePoint Workflows & Forms in Today’s Environment

MVP Thoughts: What Should SharePoint Developers Focus On?


Coming soon:

SharePoint and Office 365 in the Modern Workplace

Today’s SharePoint Developers’ Challenges

Moving from SPServices to Rest, Part 5: Using Promises with SOAP and REST

This entry is part 5 of 5 in the series Moving from SPServices to REST

Wouldn’t it be great if we could drive flying cars like the DeLorean in Back to the Future? Unfortunately, even though October 21, 2015 has passed us by, that’s not a reality yet. The science fiction writer Arthur C. Clarke famously said,Any sufficiently advanced technology is indistinguishable from magic.” Sometimes JavaScript promises can feel a little bit like magic – at least until you get good at working with them.

Paul Nigh's '' Back to the Future DeLorean Time Machine - Image source:

Paul Nigh’s ‘’ Back to the Future DeLorean Time Machine
Image source:

What are promises?

Both SOAP calls using SPServices and REST calls return promises. This is the case primarily because we are usually using a similar transport mechanism: maybe jQuery’s .ajax() function, which sends HTTP requests. (We don’t have to use jQuery, but it’s one of the more commonly used libraries to do this sort of thing. If you’re using one of the alternatives, you’ll need to do a little translation here.)

Promises are very cool mechanisms that work something like this. I think of it as a bit of magic, maybe like an act on America’s Got Talent.

When you make a call to SPServices or REST, you can assign the result of that call to a variable. That variable is a promise, which is sort of like an empty envelope. There’s no data in it when you first get it, but the call you just made promises to fill up the envelope for you. When the data is available and any conversion is finished, the original call sort of taps you on the shoulder and says, “You’ve got data!” (Say that in your best AOL voice.) At that point, you can start to do whatever work you want to do with the data.

Pretty neat, right? But, why does it matter? Well, when you are making AJAX calls, it can take a while for the server to respond. Servers are usually very powerful but they can get busy, or you may have asked for something complicated, or you may have asked for a lot of data, or the server may simply be tuned very badly. In any of those cases there will be a delay between the time you make the request and the time the response is available to you. The delay may be a few milliseconds or even a minute or two (though that probably indicates a poorly thought out request or a badly tuned server). During the time that delay occurs, you don’t want your JavaScript code to sit there waiting. Instead, you want to handle the call asynchronously meaning you don’t wait for it to finish; you elect to be notified when it finishes. That’s where promises come in.

Each browser reacts slightly differently to synchronous calls. Internet Explorer is probably the worst of the bunch, as it really hangs up until the request is complete. This makes for an unpleasant user experience, which ought never to be our goal. Because of this, using a promises pattern is important to ensure that your application feels responsive and peppy to your users.

What does converting to the promises pattern get us?

If you’ve been using SPServices for a long time, you may be setting async: false in many or all of your calls. This means you are making the calls synchronously, hanging up the browser. By moving to the promises pattern while still using SPServices, you’ll be setting yourself up for making REST calls efficiently down the road.

Let’s look at a simple example where we are just retrieving some items from a Custom List.

  operation: "GetListItems",
  async: false,
  listName: "MyCustomList",
  completefunc: function(xData,status) {

    $(xData.responseXML).SPFilterNode("z:row").each(function() {
      // Process the data


In this example, a SOAP payload will be sent to the server, and our code will sit and wait until the response comes back because we have specified async: false. If we’re only making this one call on page load and there isn’t a lot of data coming back, that might be fine, as the response will come back quickly, so the user may not even notice.

But if the server is busy or we’ve asked for a lot of data, our user will be wondering why the browser is locked up. To avoid this, let’s switch to using a promise.

var p = $().SPServices({
  operation: "GetListItems",
  listName: "MyCustomList"

p.done(function() {

    $(p.responseXML).SPFilterNode("z:row").each(function() {
      // Process the data


Here we’ve switched things around. We make the call to get the data, obtain the promise from the call and put it in variable p and then move on. Below the call, we have p.done(…), which means “When the response is here and ready to process…” Our JavaScript keeps executing from there and the browser is available to the user in the meantime. They might leave the page, click on some other actions, whatever.

When the promise is fulfilled, our code in the p.done(…) block executes. Now this will feel pretty weird to you until you get the hang of it. In the interim since you made the call, the DOM may have changed and the user may have done other things. You need to take those possibilities into account as you process the data you’ve received in the response.

Why convert now?

Converting to the promises pattern while you still are using SPServices to make SOAP calls has some advantages.

First, you already know how your code works. It may have taken you a while to get it up and running. As you’re thinking about how to move from SOAP to REST, you may not want to change too many things at once.

Find a simple call using SPServices and convert it to the promises model so that you understand how it works. Your user experience will improve even if you don’t move to REST yet. Even better, you can make this change in SharePoint 2007 or 2010 without having to switch to REST at all.

By the way, the call above in REST would look something like:

  url: _spPageContextInfo.webAbsoluteUrl +
  method: "GET",
  headers: {
    "Accept": "application/json; odata=verbose"
  success: function(data) {
    success(data); // If the call is successful, process the response
  error: function(data) {
    failure(data); // Do something with the error

The success and failure functions will only be called once the promise from the $.ajax function is resolved, so it’s somewhat equivalent to the promise-based call using SPServices above.


Since one of my goals in this series is to make moving from SOAP to REST less painful, I want to give you smaller steps you can take to get there without hurting yourself. Moving to the promises pattern now – whether you decide to go fully to REST or not – will put you in a better place when you do decide to take the plunge. Even better, it will prepare you for more “modern” Web development down the road, where you may be consuming data from multiple services. Those services may all be within SharePoint Online or Office 365 or they may be from separate sources altogether. Many of those services will be exposed via REST and the promises pattern will enable you to manage all of that data flying around more effectively.

This article was also published on IT Unity on 11/6/2015. Visit the post there to read additional comments.

Software Adoption: A People Problem, or A Technology Problem?

I was a guest recently on the TechnologyAdvice Expert Interview Series to share my thoughts on the intersection of sales, marketing, and technology. The series, which is hosted by TechnologyAdvice’s Josh Bland, explores a variety of business and technology landscapes through conversations with industry leaders.

In this episode we discussed corporate cultures, global collaboration, and the upcoming SharePoint Technology Conference (SPTechCon) in Boston August 24-27.

Below are a few highlights from our conversation:

TechnologyAdvice: How does SharePoint help businesses and employees collaborate in a world where everything is always connected?

Marc Anderson: SharePoint has had a rather long lifespan for enterprise software — lots of things don’t last as long as SharePoint has. In the earlier days, it was very focused on teamwork– small groups of people trying to accomplish specific things. The goals might have been bigger, but that was really the focus.

Where Microsoft is going now, is they’ve moved their focus to offering services rather than software. They’re actually at the forefront of where businesses are going over the next five to ten years, whether they know they will or not. It’s not about the technology so much. The fact that it’s in the cloud or that it’s got a blue logo or something really doesn’t make any difference.

Over the last 10 to 15 years, we’ve really seen a change in the way corporate cultures and structures work compared to the way they used to. The Industrial Revolution and 1950’s military industrial complex were very hierarchical, very siloed. Now inefficient organizations are getting flatter. They’re putting teams together to solve specific problems and they’re having those teams break apart again, then go off and form new teams, so each person in that team might go do different things.

We’re seeing a lot more agile, small organizations able to accomplish things in different ways than they used to. That’s where SharePoint’s really going with some of the stuff they’re rolling out on Office 365, with more machine learning. It’s not artificial intelligence but it’s trying to get you the information you need right before you think you needed it, or before you even realize you needed it. And that’s cool stuff.

It’s shifting from team members putting things into a document repository and just knowing it’s in there, to the point where content gets put in front of you because you’re going to need it in your work soon. And with the Office Graph and Delve, we’re starting to see some real evidence that’s going to be the way people are working in the future.

TA: If people are not in an office but around the world working on different projects, something like SharePoint can continue to grow and become more powerful. Do you agree with that?

Anderson: Absolutely. In almost all cases, even a small company like mine with two people – we’re a global company. I know that sounds ridiculous because we both live in Boston, but we work with customers all over the place. There’s no geographical barrier anymore, which is part of what you’re alluding to. People are out on the road more, and they need to stay tethered somehow to their home-base. But they also need to be able to work together with people who could be anywhere.

You might have a conference call with someone in Shanghai and then work on the document with somebody in Amsterdam. That’s a normal day in this global economy. Twenty or thirty years ago those were huge exceptions. SharePoint, which gives us a large surface area to enable remote and geographically dispersed work, is a win.

It lets us work asynchronously. We don’t have to be in a room having a meeting. Meetings are god-awful things anyway, but we don’t have to be in a room talking about something. We can do it asynchronously using technology and work when we need to work or when we’re able to work. I think it changes our lifestyle, it changes our incentives, it changes the culture of organization, and SharePoint’s been a big part of that.

TA: What common collaborative software challenges do you see?

Anderson: In many ways, I don’t see a lot of the challenges as technology challenges. They’re more business and cultural challenges. One of the things I’ve seen working with clients — I’ve been in consulting for decades now — is the aspiration is rarely reached. And very specifically with SharePoint, people bring it in and they want to change the way their company works, and they end up storing documents in SharePoint and that’s about it.

Getting from the execution to the promise of what they may have thought they wanted to do — or even not realized they could do — that’s one of the biggest challenges. The challenge is in enabling their organization to work differently. Some people talk about it as an adoption problem. SharePoint is in a lot of organizations, but a lot of people don’t use it. I don’t see it as an adoption problem, I see it as we’re not offering good enough capabilities to make people want to use it.

It’s really not about the technology per se. It’s about the implementation, it’s about the cultural stuff around it, it’s about incentives. It’s about helping people understand how they can be better at what they do and enjoy that. If they don’t enjoy it, they’re just going to dig their heels in and avoid it. So those are some of the biggest challenges, whether it’s cloud or on premises, or whether you’ve got to get a fatter pipe to get the data across. Those are all solvable problems. Those are technical problems.

The harder challenge is the soft stuff around the technology. People are softer than computers. They’re quirky and they don’t like change. Working on the things outside the technology can be even more important challenges to overcome.

This podcast was created and published by TechnologyAdvice. Interview conducted by Josh Bland.

Moving from SPServices to REST, Part 4: Converting GetListItems to REST

This entry is part 4 of 5 in the series Moving from SPServices to REST

Summary: Because of the significant inconsistencies in the SOAP Web Services, converting your calls to REST will be a different process for each of them. In this article, we’ll look at the real workhorse of SPServices: GetListItems.

When my mechanic works on my car, sometimes he tells me about some different tires or parts I might consider instead of what was on the car since I got it. It might be to improve performance or to give me a bit more life out of the car than I might get otherwise. (I have a good mechanic.)

Officina Meccanica by efilpera

Photo credit: Officina Meccanica by efilpera

As I’ve suggested in prior articles in the series, you can start moving individual calls from SOAP to REST anytime that you decide. If you’re still running an older version of SharePoint (no shame there), then it probably won’t be until you upgrade to SharePoint 2013. If you’re already on SharePoint 2010 or 2013, then you can start anytime.

When it comes time to converting some SOAP calls to REST calls, what will that process look like? In this article, we’re going to take a look at converting the workhorse of the SOAP Web Services: GetListItems.

GetListItems lets us read items from a list or library (virtually all content in SharePoint is stored in a list or library – a library is just a fancy list that is document-centric). Even basic information about users is stored in each Site Collection in the User Information List. GetListItems does a lot for us in SPServices-based development; it’s the most-commonly used function by just about every developer who uses SPServices and it sits underneath most of the value-added functions like SPCascadeDropdowns and SPDisplayRelatedItems.

The basic structure of the simplest GetListItems call is this:

  operation: "GetListItems",
  listName: "Tasks"

We make a call to SPServices, specify the GetListItems operation, and specify the list from which we want to get data. That’s about it. The results of this call will contain the items in the Tasks list (in the current site) as they are displayed in the Tasks list’s default view, which by default would be AllTasks.aspx.

Usually we’d have a bit more code wrapped around this call, maybe something like this:

var p = $().SPServices({
  operation: "GetListItems",
  listName: "Tasks"

p.done(function() {

  // Process the response


Note that I’m using a promise above so that I know when the call has completed rather than making a synchronous call. More on that in a later article.

So what does the analogous REST call look like? It would be something like this:

  url: _spPageContextInfo.webAbsoluteUrl +
  method: "GET",
  headers: {
    "Accept": "application/json; odata=verbose"
  success: function(data) {
    success(data); // If the call is successful, process the response
  error: function(data) {
    failure(data); // Do something with the error

Not so different when you get past the sides and trimming, right? The specifics of what we’re asking for in the REST call are all in the url object. The way REST works is that we pass information about what we want the server to do for us as parameters on the url. In this case, we have:

url: _spPageContextInfo.webAbsoluteUrl +

This means:

  • Go to the context of _spPageContextInfo.webAbsoluteUrl, which is the current site. In SPServices, I figure this out for you, or you can choose to specify a different site with the webURL parameter.
  • Call the lists service: /_api/web/lists
  • Select the Tasks list using its name: /getbytitle(‘Tasks’)
  • And finally, get the items: /items

The rest of the REST call is just packaging and what we want to do with the result.

Where the two methods start to diverge more is when we start to request things in more specific ways, for example all of the outstanding tasks for a particular user. But there’s an imperfect isomorphism here. For virtually everything we can do with the SOAP Web Services, we can do the same thing in REST just by adding the analogous parameters to the REST call.

Here’s how some of the most common parameters in the request payload map from SOAP to REST:

SOAP Option(SPServices synonym) REST Comments
ViewFields(CAMLViewFields) $select Choose the columns you would like to retrieve. With both SOAP and REST we get some data we don’t explicitly request, but by specifying only the columns we need we can reduce the payload sizes.
Query(CAMLQuery) $filter, $orderby Specify which items in the list we would like to return and how we would like them sorted.
RowLimit(CAMLRowLimit) $limit Say how many items matching the Query we would like to receive. In SOAP we can specify 0 to get all matching items; in REST we can omit the parameter to get all the matching items. Otherwise, we can specify any integer as the limit.
ViewName(CAMLViewName) NA ViewName lets you choose the view you would like to get in the response. There’s no REST equivalent here. I’ve always discouraged using this option in SOAP because it’s too easy to change the view settings and cause unintended consequences.
QueryOptions (CAMLQueryOptions) NA In SOAP, this lets us specify some options that change how the data is returned to us. For example, we can ask for all of the attachment URLs rather than just a Boolean which tells us that there are attachments.
NA $expand This option in REST has no direct equivalent in SOAP. $expand allows us to indicate that we would like the values for a relationship – rather than just the indices – using a projection. This is important with Lookup columns and Person or Group columns.


As you can see, it’s really not that scary. It’s the equivalent of translating Spanish into French, but even easier since all of the words are in English (sorry rest of the world – I’m going to play American on this one).

Here’s a slightly more complicated example where I’m asking for items from a Workflow Tasks list.

First, the SOAP request:


And then the equivalent REST request:

REST request for items from a Workflow Tasks list

Here’s how the two calls break down based on the table I showed above:

CAMLViewFields <Viewfields><FieldRef Name=’ID’/><FieldRef Name=’Title’/><FieldRef Name=’AssignedTo’/><FieldRef Name=’RelatedItems’/>

<FieldRef Name=’Priority’/>

<FieldRef Name=’Status’/>

<FieldRef Name=’DueDate’/>

<FieldRef Name=’Author’/>


$select $select=ID, Title, AssignedTo/Id, AssignedTo/Title, RelatedItems, Priority, Status, DueDate, Author/Id, Author/Title
CAMLQuery <OrderBy><FieldRef Name=’Created’ Ascending=’False’/></OrderBy> $filter, $orderby $orderby=Created Desc
NA $expand $expand=AssignedTo,Author

Note that in the SOAP call, I get the ID and Title (name) of each person or group column automatically when I request the column value. If I needed any more information about those people, I’d have to read it from another source, like the User Information List or User Profile. I also can specify a CAMLQueryOption that returns more of the basic information about users, like email address, SIP, etc.:

CAMLQueryOptions: "<QueryOptions><ExpandUserField>True</ExpandUserField></QueryOptions>"

In SOAP, most options that we pass go in the request payload. With REST, some of the options – for instance, what format we’d like the response to take – go in the request header.

If we don’t specify an accept parameter in the header, then we get XML in the response. If you’re used to parsing XML, then that might make you happy, but we usually want JSON these days.

To get JSON, we specify:

accept: application/json; odata=verbose

On premises in SharePoint 2013, this is the only option at the moment. In SharePoint Online (Office 365), if we don’t want all the metadata that comes along with the verbose option, we have a couple other options.

accept: application/json; odata=minimalmetadata
accept: application/json; odata=nometadata

The best way to understand the differences between these options is to try them out. When payload size matters (and it usually does), you should think about receiving less metadata to reduce the size of the responses.


It’s the nuances that will trip us up. In SOAP, we can’t join lists together; we have to pull the data from each list and do the “join” on the client side. In REST we have what are called projections – using $expand – that let us do some lookups into other lists, for example to get information from a lookup list.

But back to GetListItems. Here are some examples of REST calls to get data back from a Tasks list showing some common parameter settings.

Get all tasks assigned to me

/_api/web/lists/getbytitle('Development Tasks')/items?$expand=AssignedTo&amp;$select=Title,AssignedTo/Title&amp;$filter=AssignedTo/Title eq 'Anderson, Marc'

Note that the format of your username will vary based on how you have things set up. Using an account may be a better choice, but I wanted to make the example clear.

Get all tasks that have not been completed, showing the status and the % complete

/_api/web/lists/getbytitle('Development Tasks')/items?$expand=AssignedTo&amp;$select=Title,AssignedTo/Title,Status,PercentComplete&amp;$filter=Status ne 'Completed'

Getting the data in these various ways is of course just the first step. Once you have the data, you’ll have to process it in some way in order to display it to the user, do calculations on it or whatever you’re trying to accomplish. That part of the exercise isn’t in the scope of this article, so you’ll have to look elsewhere for tricks for those pieces.

In the next article, we’ll look at how JavaScript promises come into play with SOAP and REST. I hope that you’ve been using promises with SPServices calls since I added support for them in version 2013.01, but if you haven’t, moving to REST is an excellent time to start doing so.

This article was also published on IT Unity on July 23, 2015. Visit the post there to read additional comments.

Retrieving Expanded Calendar Events with REST vs. SOAP

There is very little now in 2013 for which you need SPservices [sic]

from /r/sharepoint

It wouldn’t surprise you to hear that my answer to this is “it depends”. SPServices is nice in that it works the same way across SharePoint versions from 2007-2013, which might be enough reason to use it. But there are still cases where the SOAP services do something that REST doesn’t [yet] provide.

The other day I ran across an example of this. I wanted to pull events from a SharePoint calendar from today forward and I wanted to expand all of the recurring events. Since I was working on SharePoint 2013, I immediately tried to make this work using a REST call.

The first hurdle I faced was that it wasn’t possible to filter on a data like StartDate in a REST query. I know that sounds crazy, but if it’s possible I couldn’t find any way to do it.

The other challenge was the expansion of recurring events. It turned out that this was only possible using SOAP. REST doesn’t know anything about SharePoint calendar events, especially recurrence or all-day events.

I ended going back to an old thread in the SPServices discussions started by Jim Bob Howard (@jbhoward) which gives the CAML to make this work. The code I ended up with is below. In this case, I was using KnockoutJS to populate a customized view of multiple SharePoint calendars – the “rollup” idea. We’ve the various calendar locations stored in a list, and when we get to this code we have an array of calendars to work with.

// Calendar rollup
var calendars = [];
var events = [];
var calendarPromises = [];

// For each calendar, go and get the events
for (var i = 0; i < calendars.length; i++) {

  var thisCalendar = calendars[i];
  var today = moment().format();

  calendarPromises[i] = $().SPServices.SPGetListItemsJson({
      listName : thisCalendar.title,
      webURL : thisCalendar.listUrl,
      CAMLViewFields : "<ViewFields>" +
			  "<FieldRef Name='ID' />" +
			  "<FieldRef Name='Title' />" +
			  "<FieldRef Name='EventDate' />" +
			  "<FieldRef Name='EndDate' />" +
			  "<FieldRef Name='Location' />" +
			  "<FieldRef Name='Description' />" +
			  "<FieldRef Name='Category' />" +
			  "<FieldRef Name='fRecurrence' />" +
			  "<FieldRef Name='RecurrenceData' />" +
			  "<FieldRef Name='fAllDayEvent' />" +
      CAMLQuery : "<Query>" +
			  "<Where>" +
				  "<And>" +
					  "<DateRangesOverlap>" +
						  "<FieldRef Name='EventDate' />" +
						  "<FieldRef Name='EndDate' />" +
						  "<FieldRef Name='RecurrenceID' />" +
							  "<Value Type='DateTime'>" +
							  "<Year />" +
						  "</Value>" +
					  "</DateRangesOverlap>" +
					  "<Or>" +
						  "<Eq>" +
							  "<FieldRef Name='HomePageHide' />" +
							  "<Value Type='Bool'>False</Value>" +
						  "</Eq>" +
						  "<IsNull>" +
							  "<FieldRef Name='HomePageHide' />" +
						  "</IsNull>" +
					  "</Or>" +
				  "</And>" +
			  "</Where>" +
			  "<OrderBy>" +
				  "<FieldRef Name='EventDate' />" +
			  "</OrderBy>" +
      CAMLQueryOptions : "<QueryOptions>" +
			  "<CalendarDate>" + today + "</CalendarDate>" +
			  "<ExpandRecurrence>TRUE</ExpandRecurrence>" +
			  "<RecurrenceOrderBy>TRUE</RecurrenceOrderBy>" +
			  "<ViewAttributes Scope='RecursiveAll'/>" +
      mappingOverrides : {
        "ows_fAllDayEvent" : {
          "mappedName" : "fAllDayEvent",
          "objectType" : "Boolean"
        "ows_fRecurrence" : {
          "mappedName" : "fRecurrence",
          "objectType" : "Boolean"


$.when.apply($, calendarPromises).then(function () {

  var calendarEvents = this;

  // For each calendar, go and get the items and push them into an array
  for (var i = 0; i < calendars.length; i++) {

    $(calendarEvents[i].data).each(function () {
        calendar : calendars[i].title,
        id : this.ID,
        title : this.Title,
        location : this.Location != null ? this.Location : "",
        category : this.Category,
        eventDate : moment(this.EventDate),
        endDate : this.EndDate,
        fAllDayEvent : this.fAllDayEvent,
        fRecurrence : this.fRecurrence,
        RecurrenceData : this.RecurrenceData


  events.sort(function (a, b) {
    return a.eventDate.isAfter(b.eventDate) ? 1 : (a.eventDate.isBefore(b.eventDate) ? -1 : 0);

  ko.applyBindings(new calendarViewModel(), document.getElementById("calendar-rollup"));


The code is written to handle a variable number of calendars (we started with four). Because I’m putting the promise for each call to GetListItems in the calendarPromises array, I can use

$.when.apply($, calendarPromises)

to ensure that all of the promises have been fulfilled before processing the results.

I’m also using the fantastic MomentJS library to work with dates and times. Don’t ever try to do that manually again.

I’d love to hear that I don’t need to use SOAP for this anymore, but this seems to be a case where SOAP still wins out. Maybe there’s a place for SPServices in 2013, after all.