Moving from SPServices to REST, Part 1

This entry is part 1 of 1 in the series Moving from SPServices to REST

In this series, you’ll learn how to transition from the SOAP Web Services – which SPServices supports – to REST. If your organization is making a transition to SharePoint 2013 or Office 365 & SharePoint Online (and anything hybrid in between), this series will help you develop new patterns, migrate your codebase, use OData effectively and more.

It’s been a nice, long ride for SPServices. If you aren’t familiar with SPServices, it’s freely available on Codeplex, Microsoft’s waning open source repository.

SPServices is a jQuery library that abstracts SharePoint’s Web Services and makes them easier to use. It also includes functions that use the various Web Service operations to provide more useful (and cool) capabilities. It works entirely client side and requires no server install.

The Web Services that SPServices abstracts for you are the older SOAP Web Services, sometimes known as XML services, or ASMX services. As I’ve said repeatedly at conferences and in posts (like this one) before, I think we still have many miles to go with the SOAP Web Services. Microsoft officially “deprecated” the SOAP Web Services as of the launch of SharePoint 2013, but I can’t see them fully disappearing for a good long time given the number of different ways they are used.

Two API sets are still supported in the SharePoint 2013 framework for backward compatibility, but we recommend that you not use them for new projects: the ASP.NET (asmx) web services, and direct Remote Procedure Calls (RPC) calls to the owssvr.dll file.

See: https://msdn.microsoft.com/en-us/library/office/jj164060.aspx#DeprecatedAPIs

I started writing this series just after SPTechCon San Francisco in 2014, having talked to Andrew Connell and Scot Hillier about how I might be able to contribute to IT Unity for the site’s launch. Obviously the series fell by the wayside for quite a while, but these topics are even more relevant now than they were when they first came to me.

Like many of you, I drive an older model car. It is fun to drive, is comfortable and seems to be in great shape. The car may last me a while, yet from time to time I think about what I might want to drive next.

Moving from SPServices to REST is like deciding to trade in your reliable car for a model with more safety features. You should do it, eventually.

Photo Credit: “FIAT 600” by Andrea~S on Flickr http://ow.ly/wqxGN

At some point, we’ll all want to head into the dealer and get ourselves something shiny and new. To me, that something new is REpresentational State Transfer, also known as REST.

While SharePoint’s client-side object model (CSOM) can do a lot, it’s a SharePoint-specific technology. REST, however, is a Web standard. Many Web developers use REST to get their work done every day outside the SharePoint world. It’s time for those of us who develop on top of SharePoint to catch up with the rest of the Web development community. Most SharePoint developers have caught onto this reality in the last year or so.

When REST was first offered as an option in SharePoint 2010, it let us get some work done with lists, but that was about it. With SharePoint 2013 and even more so on Office365 with SharePoint Online and the Office365 APIs, the SharePoint team at Microsoft continues to give us more and more RESTful – or REST-driven – capabilities. New REST services have been introduced regularly and the existing services have been extended many times.

If you’ve loved working with SPServices, or at least you’ve gotten some good work done with it, this series of articles will help you to make the transition from the SOAP Web Services that SPServices wraps to the more open – and undoubtedly more robust over time – REST services offered in SharePoint 2013 and Office365. If you’re still running SharePoint 2007 or 2010 – as I know many of you are – this should still be a valuable series to you. By making some adjustments in your development style and the coding patterns that you follow, you can put yourself in an excellent position to move to REST when it becomes appropriate for you to do so.

Here are some of the things I plan to cover in this series:

  • New patterns I suggest you follow with your SPServices code so that you’ll be better prepared to move to REST when the time comes. A side benefit of these new patterns will be better modularity and reusability in your SPServices-based code.
  • You will learn ways to move parts of your code from SPServices to use REST even now with less effort than you might think. We’ll talk about where the most likely places in your codebase probably are and how to get started.
  • We’ll look at how the functionality in the SOAP Web Services maps to the current REST services and where there are still gaps.
  • We’ll talk about where OData comes in and how best to understand the OData componentry that sits under the SharePoint-specific implementation of REST. There are some differences from the standard (these gaps are narrowing all the time), and it’s important to know what those differences are and why they matter.
  • As Microsoft adds more functionality to REST in SharePoint 2013 and Office 365, I’ll write new articles in the series to augment the base set of content. It’s not your father’s Microsoft anymore – the SharePoint team is moving fast and improvements are going to be coming Fast and Furious (see what I did there?)

It’s hard to give up the old car that you’ve driven for years, but sooner or later, it’s time. Maybe the engine starts knocking too much or the springs in the seat get uncomfortable. Maybe you just want to try out something new. It is sad letting the old girl go, but at some point, it’s simply time. I once held onto a car until the engine simply stopped working. You don’t want to be in a similar situation with your SharePoint functionality built on top of SOAP. Let’s start to plan for the future, and let’s do it together.

Cooler with better design and safety features. Your new code using REST, using practices favored by the rest of the Web development community.

Photo Credit: “2014 Fiat 500” by Abdullah AlBargan on Flickr http://ow.ly/wqynJ

This article was first published on IT Unity on March 25, 2015. Visit the post there to read additional comments.

Synchronous XMLHttpRequest Warning with SPServices and Recent Browsers

If you’re working in the latest versions of Chrome (~40+) – and maybe Firefox – and you use SPServices, you may start to see an warning:

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help, check http://xhr.spec.whatwg.org/.

Vigilant SPServices user frankhale reported this to me the other day in the SPServices discussions on Codeplex, which is – at least at the moment – the best place to get help with SPServices. You can also add an issue on Github (sympmarc / SPServices) if that’s your fancy.

The warning is thrown in jQuery, not SPServices, but it’s an SPServices issue.

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience.

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience.

First off, it’s a warning, not an error. Your code will continue to work in the near term, at least.

Because of some backward compatibility concerns, I’ve left a few synchronous calls internally to SPServices in place. Those calls are what are causing the warning. In particular, it is most likely that the warning is being thrown for you because of a synchronous call on the $().SPServices.SPGetCurrentSite function. The reason for this is that early in the SharePoint 2007 days it was difficult to determine the current site without a call to the Webs.WebUrlFromPageUrl operation. Unfortunately, it seems that I’m making that call in SharePoint 2010 and 2013, even though the current site is available in JavaScript variables.

So, the bottom line is: “Carry on.” I’ll get a fix into the next release of SPServices for this. In the meantime you should be fine.

Semantic Versioning for SPServices?

Late last year I committed to put SPServices onto Github to encourage greater collaboration and contribution from the community. It’s definitely there now, and there are several things I’ve been struggling with since I moved it there:

  • What to name each version
  • How many versions to move into the repo
  • Whether it actually will increase community submissions at all
  • etc.

So far, I have done one release while SPServices has been on Github, which was 2014.02 [Codeplex][ [Github]. Even for that one new release, I wasn’t sure how to structure things. Everyone is moving away for including version numbers in their file names. I’m not sure that’s such a good thing. By including the version number in the file name, we can tell – at a glance – what version we’re working with. Since many SPServices users are more on the end user end of the spectrum rather than the developer end, that’s an excellent thing to be able to do.

Andreas Schultz has recommended that I switch to Semantic Versioning in a pull request for bower.json. Bower is a package manager, and there are a host of others: npm, NuGet, Bower, Ender, volo, component, Jam, and the list goes on. The number of these things is increasing fast. (I’d include links, but these things are easy to find.)

This is a tricky one. I’ve watched as this “semantic versioning” thing gets rolling, and I’m not sure how I feel about it.

Over the first several years I increased the versions for SPServices from 0.2.3 to 0.7.2. The fact that I wasn’t getting to version 1.0 was more me being cantankerous than anything else. Then I had 3 or 4 people tell me – separately – that they weren’t allowed to use software in their organizations unless it was version 1.0 or above. What a crock!

So I switched to numbering like 2013.01 back in May, 2013. The version is greater than 1.0! Since then, I’ve had 2013.02, 2013.02a, 2014.01, and 2014.02. It marches forward and gives some indication of what the versions is, or at least its age.

Semantic versioning seems to give people an excuse to introduce breaking changes. Here’s the summary of the idea from the SemVer site:

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. MAJOR version when you make incompatible API changes,
  2. MINOR version when you add functionality in a backwards-compatible manner, and
  3. PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

One of my goals with SPServices going way back to 1999 has been not to introduce *any* breaking changes. I think I’ve done a pretty good job on that, too. Whether you’re using SPServices on SharePoint 2007 with no updates installed or SharePoint 2013 on Office365, it *ought* to work the same way. Code that you wrote with version 0.2.3 *ought* to still work, too, though most likely you’re running a more recent version of jQuery and other things which would make using that ancient version pretty dumb. (That and the fact that the first version didn’t do much.)

If I were to follow the semanatic versioning rules, I’m not sure if I would be at 1.1.14 or 1.14.02 or what. Removing version numbers from files makes me uncomfortable, though.

Switching to semantic versioning seems to be hopping on a train without really understanding if driving is just fine and will get us there at the same time. What are your opinions on this? If you add your $.02 into the comments, please indicate what type of user you are, e.g., end user (“I paste SPCasecadeDropdowns into my forms” or developer “I write solutions using SPServices as the data transport layer” or…

jQuery Library for SharePoint Web Services (SPServices) 2014.02 Released

SPServicesJust in time for the holiday gift-giving season, I’m releasing SPServices 2014.02. This is the second release in 2014 (which you should be able to glean from the release name). If you’re using an earlier version of SPServices, I strongly recommend an upgrade. Read on.

The most important change in this release is due to an egregious error on my part that goes way back to 2013.01. Paul Tavares (@paul_tavares) spotted quite a while back and I was just too dumb to realize how important it was to fix. The net effect of my mistake was that SPServices was always caching requests, regardless how you set cacheXML. On pages where we simply called SPServices a few times on page load, it usually wouldn’t make much of a difference, but a little more memory might be required for the browser. However, in things like Single Page Applications (SPAs) which use SPServices for the data transport layer (See my blog post series: Single-Page Applications (SPAs) in SharePoint Using SPServices), it could make a *huge* difference. It effectively meant that for every unique request, we were caching data whether we should or not. Over a long page lifespan, that could add up to a lot of memory eaten up. Bad news. All better now.

I found out over the last few months the insidious ” Required Field” issue that I thought I had fixed completely in 2014.01 wasn’t totally fixed. For folks on SharePoint 2010 who applied last December’s CU, the markup in the DOM was a little different than the fix I had put into 2014.01. (See: Office 365 Update Changes ‘Display Name’ on Required Fields) This *should* be fixed for everyone now. Please let me know if you see any issues going forward. And Microsoft, please don’t do that again.

There is a handful of new operations in this release, added due to user requests:

  • WebPartPages.SaveWebPart2
  • RecordsRepository.GetRecordRouting
  • RecordsRepository.GetRecordRoutingCollection
  • RecordsRepository.GetServerInfo
  • RecordsRepository.SubmitFile

Finally, I recently moved my SPServices development repository over to Github at sympmarc/SPServices. I’m still getting my feet wet with Github (See: SPServices and Github – This Time I Mean It), but I plan to have my most current work available there going forward. I’ll still be posting the releases over on the Codeplex site, and I’ll monitor the issues there as well. I’m hoping that by using Github, we’ll have more people contributing to SPServices. It’s always been a community-driven project, but I’d love to get more direct contributions.

This release will be, as all of the recent releases have been, available on cdnjs as soon as possible. If you’d like to serve up SPServices from a CDN, cdnjs is the place to get it. cdnjs hosts a lot of JavaScript libraries that don’t have enough coverage to interest the big boy CDNs, so check it out in any case.

Finally, I want to thank Paul Tavares again for his help with SPServices over the years and with this release in particular. Both he and Josh McCarty (@joshmcrty) have also been trying to help get me over onto Github for a long time, but my thick skull just hasn’t gotten it before. I’m using Jetbrains Webstorm as my IDE these days and the Github integration there has finally made Github make sense to me. As we say here in Massachusetts, “light dawns on Marblehead“.

As usual, there are also a number of additional changes to fix existing bugs or improve efficiency (yes, I’m still able to improve on my old code, and I expect that will continue).

You can see the full list of enhancements and improvements on the download page. Note the link to the Issue Tracker items for this release. For posterity, here are links to the release notes showing all the fixes and improvements from the Issue Tracker on Codeplex.

New Functionality

Issue Tracker Item Function Description

 

New Operations

Web Service Operation Options MSDN Documentation Issue Tracker Item
WebPartPages DeleteWebPart pageUrl, storageKey, storage WebPartPagesWebService.DeleteWebPart Method 10273
WebPartPages SaveWebPart2 pageUrl, storageKey, webPartXml, storage, allowTypeChange WebPartPagesWebService.SaveWebPart2 Method 10273
ALPHA6 RecordsRepository GetRecordRouting recordRouting RecordsRepository.GetRecordRouting Method 10257
RecordsRepository GetRecordRoutingCollection NA RecordsRepository.GetRecordRoutingCollection Method 10257
RecordsRepository GetServerInfo NA RecordsRepository.GetServerInfo Method 10257
RecordsRepository SubmitFile fileToSubmit, properties, recordRouting, sourceUrl, userName RecordsRepository.GetRecordRouting Method 10257

 

Bug Fixes and Efficiency

Issue Tracker Item Function Description
10267 $().SPServices.SPComplexToSimpleDropdown Possible bug in SPComplexToSimpleDropdown
10253 $().SPServices.SPFindMMSPicker SPFindMMSPicker missing from 2014.01
10279 $().SPServices.SPGetListItemsJson SPGetListItemsJson Doesn’t Handle Attachments Correctly
10284 $().SPServices.SPFilterDropdown SPFilterDropdown Not filtering dropdown column.
10272 $().SPServices.SPGetCurrentSite $().SPServices.SPGetCurrentSite Documentation
ALPHA5 10277 $().SPServices.SPGetCurrentUser fix for SPGetCurrentUser at webUrl /
10248 $().SPServices.SPGetCurrentUser Trouble with SPSservices and Sharepoint Form
10265 $().SPServices.SPGetCurrentUser $().SPServices.SPGetCurrentUser isn’t returning ID
10254 $().SPServices.SPDisplayRelatedInfo SPDisplayRelatedInfo – DIV named improperly with Required field
10256 $().SPServices.SPCascadeDropdowns SPCascadeDropdowns Required Lookup issue
10262 $().SPServices.SPComplexToSimpleDropdown SPComplexToSimpleDropdown – required fields get renamed without the ‘Required Field’ appended to the field name
10146 $().SPServices ResolvePrincipals with addToUserInfoList=true requires SOAPAction
10271 $().SPServices.SPGetListItemsJson SPGetListItemsJson not using defaults.webURL
10283 $().SPServices.SPGetListItemsJson CAMLViewName not working?
ALPHA7 10182 $().SPServices async on version 2013.01 and caching
10298 $().SPServices GetTermSets argument names are slightly wrong
10299 $().SPServices WebUrl not working

SPServices and Github – This Time I Mean It

SPServicesGitHub_LogoI’ve had some false starts moving SPServices to Git and/or Github over the last few years. If it weren’t for Josh McCarty’s (@joshmcrty) help on every release, I wouldn’t even have gotten SPServices onto cdnjs, since they use Github. (Yes, SPServices is available via CDN at cdnjs and has been for several years now.) I’m just tremendously behind the times.

So it’s only taken me about two years, but I’m really biting the bullet on Github this time. I’ve just read through a bunch of great suggestions I got when I abortively tried to move things to Github long ago and I’m curious given the amount of time that has passed those folks might do differently.

Here are my assumptions/preferences:

  • Simple, simple, simple
  • I’m using a public folder in my Dropbox as my “CDN” for development. I think Paul Tavares (@paul_tavares) knows where it is, but no one else does. This Dropbox-based CDN helps immensely for testing, since I can just point my script references there in all of my test environments. In case anyone is wondering, I’d love to use OneDrive (either flavor) for this, but it just doesn’t work the way it does redirects. I can get a clean URL from Dropbox that just plain works.
  • I’m leaning toward WebStorm for my IDE these days. (Where I can’t install it in client environments, I’ll still use SharePoint Designer and/or Sublime Text.) Webstorm has very robust integration with Github that even seems to make sense to me. I’ve got my Webstorm project embedded in the Dropbox CDN I mentioned above.
  • In case you’re wondering, I do probably 99% of the work on SPServices, so my ideas for version control have been extremely simple to date. SPServices is a one-file project and I make virtually all the changes to it. SPServices wouldn’t be what it is without excellent help and contributions from people like Josh McCarty and Paul Tavares; I couldn’t have gotten to this point without them. But as far as the actual edits and testing, it’s mostly me.
  • I’ll continue hosting the docs and downloads on Codeplex, at least for the foreseeable future. This makes sense because of the volume of documentation and the great discussions history that’s already there.

What I’m looking for is best (better) practices, build ideas, etc. SPServices will continue to live as long as people find it useful, and I want to keep building it and supporting it. That said, it’s my “side project” – something I do for fun and learning opportunities. So any ideas should be labor *saving*, not labor *producing*.

It’s my hope that – as they mentioned in the suggestions I linked to above – more people may decide to contribute with the move to Github. Who knows, maybe we can haul it into RESTland along with SharePoint 2013.

Today I posted the latest beta for the 2014.02 release. I expect to make it a stable release in about a week or so, since quite a few people have been downloading it and testing it as I’ve made changes over the last few months. I’ll write more about why you *really* should upgrade to this new version in an upcoming post. (Thanks yet again to Paul Tavares on this one.)

Thanks in advance for any ideas you can toss into the mix. Feel free to reply in the comments here or on the older thread in the Codeplex Discussions.

 

m4s0n501