SPServices into 2016!

SPServicesOver the break, I managed to get SPServices up and running on GitHub. You may wonder what that means and why it matters.

When I started moving SPServices away from Codeplex, it was due to several things:

  • Codeplex seems to be dying a whimpering death. Microsoft is clearly not maintaining it anymore, and its tech is falling behind – when it works.
  • I wanted to understand GitHub better. I’ve had to get too much help from my friends in the SharePoint community even just to push changes to cdnjs. Thanks Josh McCarty (@joshmcrty)!

So, mission accomplished on those goals – but it took me an embarrassingly long time to get here!

Of course, in the process I also wanted to improve SPServices. Here is what I am/was aiming at:

  • The next version will be SPServices 2.0 – This is just a number, and the current builds have this number already.
  • Make it (already is!) AMD-enabled using RequireJS
  • Convert from a monolithic file to modules (done, though perhaps more to do)
  • Enabled to take advantage of SharePoint’s REST APIs – where available – for internal calls to get list data in the value-added functions (still to do)

As you can see, I’m well on the way to meet my self-imposed goals; I think the hard work is behind me.

There are some other goals and here’s where you can help:

  • Test the “pre-alpha” builds of SPServices 2.0. If you’re familiar enough with the library to drop builds into your test environments, that would be a great help. I’ve tested using the same lists and pages I always use, but more real-world testing would be good. Report any issues you find using GitHub issues.
  • Write some tests. I’ve started writing tests with QUnit, but I’ve only scratched the surface. Writing good tests here is difficult, as we have to be sitting on top of SharePoint; in greenfield development, we can test anywhere. You’ll find some instructions for how to use the existing tests on the GitHub pages.
  • Migrate the documentation from Codeplex to GitHub – Since Codeplex is falling apart, there’s no reason to leave the documentation there, either. There are a few dozen pages (I can’t actually count the pages on Codeplex easily) of documentation, and it’s probably easiest just to move the over to GitHub manually.
  • Move the discussions off Codeplex – This one is hardest, I think. IMO, one of the big values to SPServices is the historical discussions about how to use it. But those discussions have covered many other things as well, and I’d hate to lose any of it. I’m not sure how to go about this, so if anyone has some experience moving forums like this, I’m all ears.
  • Propose improvements – I ask the community for suggestions all the time, but I don’t get a lot of them. If you’ve solved some gnarly SharePoint UI problem and would be willing to submit your code or just wish that someone would fix the darn _____, then let me know in the GitHub issues. Consider the issues our own UserVoice for SPServices.

A HUGE thank you goes out to Paul Tavares (@paul_tavares) for guiding me along this move to GitHub. He’s written most of the build code, the test chassis, etc. I couldn’t have done it without him. If you don’t know Paul, he’s one of the sharper tacks in the SharePoint community, and he only does SharePoint work on the side these days. Take a look at his SPWidgets project. I promise you – you’ll be impressed.

Thanks also to the folks over at BitHound for their code tools. If you check out the dashboard for SPServices, you’ll see that I’m doing OK, but there’s always room for improvement.

Happy 2016 to all of you, and happy coding!

An Unpleasant Refiner Bug in SharePoint Online

As always, I don’t know where to report issues with Office 365. So here I am carping in a public channel in the hopes that the right person sees it. This also may happen with SharePoint 2013 on premises, but I can’t recall.

Unpleasant SharePoint search refinerWhen I use a refiner that is based on a Lookup column, I’ll often see something like the attached image. As you can see, there are two clients listed: Anchor Glass and 29. Anchor Glass is the title of the item in the Clients list with the ID of 29, but it still doesn’t make sense (especially to those poor users). It’s as if the indexer isn’t smart enough to handle Lookup columns properly – but only some of the time.

This happens on multiple tenants, with different Lookup columns to different lists, but not consistently, or in any pattern I’ve been able to discern. Generally it’s the Title column we’re looking up into. Using a Lookup column into a list is almost always preferable to using Managed Metadata (IMO) because we can store additional data about the values in the list. For instance, in the Clients list above, we have the client contact info, who works with the client, when we started working with them, what the billings are, etc. Managed Metadata is worthless for that.

But the bug.

Has anyone else seen this? Any ideas on how to work around it?

SharePoint Online Search: Add a Refiner for Content Type

2015-12-21_14-00-52This is yet another in the “Things that Make You Go Hmm…” category. If I only had a nickel…

I would think that one of the most common needs on a search results page would be to add a refiner by Content Type. After all, we all have robust, sensical information architectures in place to improve performance in our organizations, right?

When we create a new search results page in a Search Center, we get a few refiners out of the box that look oh-so-promising.

First there’s ContentTypeId. We all know that Microsoft uses big, ugly ids and GUIDs under the covers all the time. Sometimes they translate easily into whatever they represent. This isn’t one of those times.

2015-12-21_13-37-48

Well, what about contentclass? That looks sort of almost useful. But it isn’t. Values like STS_ListItem_DocumentLibrary won’t really help anyone.

2015-12-21_13-37-35

Oh, I know. Let’s add the ContentType Managed Property. That’s what works almost everywhere else. Well, except here, as the values don’t make sense to humans, as they are things like application/pdf Document or application/vnd.openxmlformats-officedocument.presentationml.pres… Yes, the ellipses are appropriate here. Who know what that means?

2015-12-21_13-38-05

Finally, I found a blog post from Henri Merkesdal that retrieved my sanity. There’s a property way down the list called SPContentType that does exactly what we want.
2015-12-21_13-38-20

Ahhh. That’s much better. Thanks, Henri!

2015-12-21_14-00-52

CollabTalk Episode 12: Top 10 2015 News and 2016 Predictions

CollabTalk Episode 12 - December 2015We wrapped up our twelfth episode of CollabTalk for ITUnity yesterday, which means we’ve been at it for a full year now. Boy, how time flies! When I say we, I of course mean my illustrious friends Christian Buckley (@buckleyplanet), Naomi Moneypenny (@nmoneypenny), and Benjamin Niaulin (@bniaulin). Oh, and me.

In this episode, we did something a little different than in the prior eleven. Since it’s the end of the year – the time when everyone takes the time to look back on a year (hopefully well-lived) and look forward to what might come – we went through what we considered the big Office 365 news from 2015 and then gave a few predictions for 2016.

I proved in this exercise at least one of the following things: I am bad at making lists, I’m not highly motivated to do things that are important, I’m way too busy, I can’t count, and/or I’m lazy. You decide. Whichever it is, I came nowhere close to 10 items per list.

Big News from 2015

I think Office Graph was a 2014 thing, but people are really starting to see how cool it is

Modern organization dynamicsLast year at this time, we knew what the Office Graph was, but most people weren’t convinced of its importance. Over the course of 2015, we saw tremendous advances with Delve and the available analytics on Office 365, all driven by Office Graph data. I’m convinced that the Office Graph is one of the most important tools in the Microsoft toolbox.

People seem to be seeing this promise more and more now. Part of this is that some of the sharpest tacks in the drawer, like Waldek Mastykarz (@waldekm) with posts like these and Mikael Svenson (@mikaelsvenson) with posts like these have made the concepts more accessible. We’re also seeing Microsoft building more capabilities on top of the Office Graph to show its importance. In a larger organization, those “edges” of connection that live outside the normal organizational hierarchy are the ones where Things Get Done.

News of the next gen Office 365 dashboard to make admin tasks simpler

office-365-admin-center-preview-tour-3To me, one of the weak points of SharePoint has always been the back end management tools. In the old days, then mindset was “It’s for IT Pros and they don’t care how it works.” Well, on Office 365, many admins are people inside small or medium sized businesses (SMB) who don’t want to have to work to understand the admin tools – they want them to work and be easy to use. The SMB market is a sweeter spot for Office 365 then the enterprise market to me. If Microsoft can make it work well for the “IT not-Pro” [I’ll keep my editorial comments about the “Pro” moniker for professions out of this post. Sort of.], then they will clobber that part of the market. That’s the very market that Google has been creeping on on them through with their far-better admin tools.

We’ve seen a preview of the new admin dashboard in First Release tenants. I’m looking forward to seeing it fleshed out to cover more and replace the gobbledy-gook that has been SharePoint administration in the past. We can only hope that those improved tools com back to on premises in SharePoint 2016 as well. As much as IT Pros like to say they’d rather us e a command line, it’s far cheaper and reliable to just give the damn simple tools.

Platform independent apps – we now can do Outlook, and all the other Office things on iOS and Android(?)

Microsoft Apps on iOSThis move has been brilliant. Microsoft has realized that they shouldn’t try to drag the people back to the desktop; they should take their productivity tools where the people are. And the people are on their mobile devices. Microsoft decided it doesn’t matter who made the device – they want to own the productivity software business regardless what the hardware is.

Now on iOS we have better tools from Microsoft than we do from Apple. Not only that, but they are all backed by the data store that is OneDrive – in this case, two of the three OneDrives. Say what you want about OneDrive sync – this content integration on mobile is killer . (And besides, the sync client on iOS seems to work flawlessly – it’s Microsoft’s home turf of the desktop where the problems seem to be the biggest.)

Purchases of Accompli, [that task thing]

Microsoft hasn’t been on an acquisition tear, but they have made some important purchases to get their foot in the door on the iOS platform. Accompli and Wunderlist (that’s the one I couldn’t remember!) are two examples of this. Sure, Microsoft could have just built their own versions of these two products, but instead, they bought a *very* loyal and hip user base. In the case of Accompli, it meant they could launch a new app – re-branded as Outlook – in a very short time. (I’m an iPhone guy, so I can’t speak to other phone OSes. YMMV.)

Microsoft won’t be able to buy its way into the hearts and minds of the average consumer, but by starting with these two apps and keeping them hip they made big strides forward in the app wars.

And finally…what Ben said

Ben had a really good list. Unfortunately, I saw it before I wrote mine. That’s a motivation killer – at least to me.

Predictions for 2016

Microsoft will be seen as cool again

Is Microsoft cool now?Most people know that Microsoft really lost its mojo in the decade or so that Steve Ballmer was in charge. The products were ho-hum, and the attitude was very much “You’ll buy what we tell you to buy.” It was a sales-driven organization and it showed.

In the last year (or a bit more), Microsoft has found better mojo than it has had for years. When has anyone thought of Microsoft as hip? I’ve been around for the entire lifespan of the company, and I can’t recall that time. With the things they are doing now on Office 365, in mobile, and in hardware, even some of the stalwart cynics are sitting up and taking a good, hard look. I’ve even heard a few Apple fanboys say they have considered jumping over to Microsoft – either again or for the first time. This is almost unheard of.

Office 365 will become a $7B businessOffice 365 Logo

I’m not a financial guy and you should never take my investment advice. That said, I see Office 365 growth rates heading north; the integral of that curve is going to grow. As I said above, to me the sweet spot of the market for Office 365 is the SMB market. With the improvements to user experience Microsoft is putting in place, more and more small business are going to trust them to take a lot of the headache of IT off their plate for them. Whatever the numbers turn out to be, revenue is going to climb significantly in 2016.

Microsoft will write off a significant part of its Yammer investment

Yammer LogoYammer is a dog. I’ve been saying it for a long while now. There are members of the CollabTalk panel who disagree with me vehemently, but I’m sticking to my guns. I know it doesn’t make me a popular guy with the political people in Redmond either, but many Redmond-ites have to feel this, too. Microsoft bought Yammer for some of the technology it had, IMO, and even that technology hasn’t proven to be all that stellar. Add to that a “we know better then you” attitude in Yammer organization, and it’s a dog that don’t hunt. Things that virtually every user struggles with and wants fixed (unread counts, anyone?) are still a problem. Yammer doesn’t fit into the portfolio as it is. Perhaps it has some value if it’s cut up for parts, but even that seems like a stretch to me. Again, I’m not a finance guy, but I see some sort of write off or write down or whatever on Yammer. At the very least, we should see it slink away and hide under the barn.

Conclusion

So there you have it. My big news for 2015 and predictions for 2016. You probably don’t agree with everything I have on these lists or what I’ve said. I welcome your thoughts and feedback in the comments.

If you’d like to watch the entire CollabTalk episode, you can do so here. If you register, the good folks at IT Unity will let you know about upcoming episodes and maybe send you some other goodies via email as well.

Moving from SPServices to REST, Part 6: Converting UpdateListItems to REST

While the majority of calls using SOAP with SPServices use GetListItems to get items from lists, changing data with UpdateListItems is probably a close second. In this case, we’re altering data based on some user action. In a way, it’s like repainting your car (which I’ve never done). The shape and structure of the car stays the same, but you make it look better with new paint. New data can be like that – after all, the list or library itself doesn’t change.

Image source: http://www.internetbillboards.net/2015/03/27/a-step-by-step-diy-guide-to-give-your-car-a-new-paint-job/

Image source: http://www.internetbillboards.net/2015/03/27/a-step-by-step-diy-guide-to-give-your-car-a-new-paint-job/

Let’s look at a simple example of an update: deleting a list item. This is probably the simplest update because all we have to pass in is the ID of the list item and ask for a delete.

var itemID = 1;
var p = $().SPServices({
  operation: "UpdateListItems",
  listName: "Tasks",
  updates: "<Batch OnError='Continue'> " +
    "<Method ID='1' Cmd='Delete'>" +
    "<Field Name='ID'>" + itemID + "</Field>" +
    "</Method>" +
    "</Batch>"
});
p.done({
  // Manage success or failure here
});

To delete a document in a library, it’s just a little bit harder. We need to pass in the FileRef (where the file lives) in addition to the ID.

var itemID = 1;
var fileRef = "http://mydomain/libName/fileName";
var p = $().SPServices({
  operation: "UpdateListItems",
  listName: "Tasks",
  updates: "<Batch OnError='Continue'> " +
    "<Method ID='1' Cmd='Delete'>" +
    "<Field Name='ID'>" + itemID + "</Field>" +
    "<Field Name='FileRef'>" + fileRef + "</Field>" +
    "</Method>" +
    "</Batch>"
});
p.done({
  // Manage success or failure here
});

In both cases, we pass in what’s called an update, which contains a batch. We can request multiple actions in the batch, but in this case, I’m keeping it very simple and just requesting one action: deleting the item with ID = itemID (and FileRef = fileRef for the library). As you can see, I also need to pass in the Cmd = “Delete” to tell the server what I want it to do. My other options are “New” and “Update”.

If our delete is successful, we simply get a result that tells us the error code was 0x00000000. If there was an issue, we’ll get something else (usually 0x81020016) which may not be all that descriptive of the error.

The call to delete a list item looks like this in REST:

var itemID = 1;
var p = $.ajax({
  url: _spPageContextInfo.webAbsoluteUrl +
    "/_api/web/lists/GetByTitle('Tasks')/items(" + itemID + ")",
  method: "POST",
  headers: {
    "X-RequestDigest": document.getElementById("__REQUESTDIGEST").value,
    "X-HTTP-Method": "DELETE",
    "IF-MATCH": "*",
    success: successFunction,
    error: errorFunction
  }
});

We can check what’s happened one of two ways. You can see above that I’ve set a variable called p to the results of the REST call. I’m also doing something with success and failure within the call. This is redundant, but I’m showing here to demonstrate the two methods (which I probably should have covered in my previous article Part 5: Using Promises with SOAP and REST, as several people have pointed out).

Examining p and acting on it is working with a promise; acting based on the success or failure inside the call is called a callback. You’ll find no end of arguing about which of the two is more “correct”, but it’s really a stylistic choice. Usually if you have lots going on and need to manage it all together, people will agree that working with promises can be easier to follow. There’s nothing wrong with using callbacks as long as you can follow your program logic.

If there’s an issue in the call (regardless which approach you take), you may an error back that looks something like this:

{"error":{"code":"-2130575338, System.ArgumentException","message":{"lang":"en-US","value":"Item does not exist. It may have been deleted by another user."}}}

In this case, I’ve tried to delete an item which doesn’t exist. Note that with REST, unlike with SOAP, no response on an update or delete is good news – you only get a response if something’s gone wrong.

Because I’m changing some data on the server, the request has to be a POST rather than a GET. POSTs are a little harder to set up than GETs because we have to demonstrate to the server that we aren’t bad guys by passing a little more information about who we are and where we are coming from.

Keep in mind that these examples don’t reflect what you would need to do in Apps Add-Ins; that authorization flow has been written about in many other places. These examples show what you would do if you are loading your JavaScript directly in the page, perhaps by including it in a Content Editor Web Part (CEWP) using the Content Link or in a Script Editor Web Part (SEWP).

We pass this extra authorization information in the request header:

"X-RequestDigest": document.getElementById("__REQUESTDIGEST").value

In every SharePoint page in SharePoint 2010 and above, there is a request digest value stored in the page in a hidden element with the id of “__REQUESTDIGEST”. Note that’s two underscores up front, not just one. In the line above I’m using simple JavaScript (no real need for jQuery here) to grab that value.

In both REST examples above I’m doing what I think of as a “selfish delete”. By specifying

"IF-MATCH": "*"

I’m telling the server that I don’t care if anyone else has changed the item or document – I just want it gone. Check out my post on using Etags if you’re interested in how to avoid collisions in requests.

To delete a document in a library, I do basically the same thing:

var itemID = 1;
var p = $.ajax({
  url: _spPageContextInfo.webAbsoluteUrl +
    "/_api/web/lists/GetByTitle('Documents')/items(" + itemID + ")",
  method: "POST",
  headers: {
    "X-RequestDigest": document.getElementById("__REQUESTDIGEST").value,
    "X-HTTP-Method": "DELETE"
    "IF-MATCH": "*",
  }
});
p.then({});
p.fail({});

Now let’s look at something a little harder. We’ll update a field (column) in a list item instead. I’ll keep it simple, but we’ll change the item’s Title.

Here’s how we’d do in in SOAP:

var itemID = 1;
var p = $().SPServices({ 
  operation: "UpdateListItems", 
  listName: "Tasks", 
  updates: "<Batch OnError='Continue'> " +
      "<Method ID='1' Cmd='Update'>" +
      "<Field Name='ID'>" + itemID + "</Field>" + 
      "<Field Name='Title'>This is the new title text</Field>" + 
      "</Method>" + 
      "</Batch>"
});
p.then({});
p.fail({});

As before, I need to pass in the batch, but this time the Cmd is “Update”, meaning that I want to change something about the list item. As you can see, I’m just changing the Title to some arbitrary text.

This SOAP request return us the item or document as it looks after the write. We can inspect that result if we want to update an array or something else we’re maintaining client side.

In REST we need to pass in the data we want the server to use in the update a little differently – no surprise, probably. We still pass the itemID on the url, but the data we want to write goes into a data element. We also need to “stringify” the data because it’s usually represented in our code as a JavaScript object. When we look at JSON data in the developer tools of our favorite browser, it looks like strings, but it’s actually stored as binary content. Since we can only pass text to the server, we need to do this stringify conversion.

var itemID = 1;
var payload = {
    "Title": "This is the new title text"
  };

var p = $.ajax({
  url: _spPageContextInfo.webAbsoluteUrl +
      "/_api/web/lists/GetByTitle('Tasks')/items(" +
        itemID +
      ")",
  method: "POST",
  data: JSON.stringify(payload),
  contentType: "application/json;odata=nometadata",
  headers: {
    "X-RequestDigest": document.getElementById("__REQUESTDIGEST").value,
    "X-HTTP-Method":"MERGE",
    "IF-MATCH": "*"
  }
});

In this example, I’m choosing to pass the data using the contentType with odata=nometadata set. You may see other examples with odata=verbose, and in that case, we must pass a metadata value in the payload, like this:

var payload = {
    "__metadata": {
      "type": "SP.Data.SurveysListItem"
    },
    "Title": "This is the new title text"
  };

Passing the metadata basically adds additional checks to the veracity of our data, and can be a good idea.

I’m still doing a selfish write here (“IF-MATCH”: “*”), not caring whether anyone else has made changes. As with SOAP this means that the last person in wins.

Here’s a table that gives an overview of how REST and SOAP compare when you want to do updates to your SharePoint content.

  SOAP REST
Authorization None needed; our request is made using the current user’s permissions X-RequestDigest

document.getElementById(“__REQUESTDIGEST”).value

Actions Passed in the Cmd element of the update CAML

[New, Update, Delete]

X-HTTP-Method

[PUT, MERGE, PATCH, DELETE]

Data Contained in the updates as strings using CAML notation Contained in the data element and the content must must be “stringified”
Collision Control None – last person wins Using ETags, we can learn of collisions, but we must manage those collisions in our code.

 

References:

This article was also published on IT Unity on 12/7/2015. Visit the post there to read additional comments.