Uploading Attachments to SharePoint Lists Using SPServices

Easy!For years people have been asking me how they could upload files using SPServices. I’ve dodged the questions every time because I simply didn’t know how.

On a current client project,  I’m building a slick Single Page Application (SPA) to manage tasks in SharePoint 2010. It’s basically a veneer over the clunky out-of-the-box task list experience. Rather than hopping from page to page, the team using the application can accomplish everything they need to do on a single page.

Every project has specific needs, and for this one I’m using KnockoutJS. But the code snippets I give below are generic enough that you should be able to use them in any context with a little thinking.

It’s been going well, but I had been putting off implementing adding attachments because…well, as I said, I didn’t know how.

One of the benefits of the shift from all server side development to a much more significant focus on client side techniques is that some of the bright minds in the SharePoint development community have turned their eyes toward solving these things, too. Usually it’s on SharePoint 2013 or SharePoint Online at Office 365. However, at least now when I search for “SharePoint JavaScript xxx”, there are likely to be some great hits I can learn from.

In this case, there were two excellent posts, one from James Glading and one from Scot Hillier. (See the Resources section below for links.)

The first step is to enable HTML5 capabilities in IE10. This requires two small changes to the master page. This seems simple, but it can have effects that you don’t expect. In other words, don’t consider this just a “no brainer”. You need to plan for the change across your Site Collection and any impacts it may have.

The first change is to switch from using “XHTML 1.0″ to “HTML” as the DOCTYPE. This is what “turns on” HTML5.

<!DOCTYPE html>

Then, in this project we set the content meta tag to IE=10 because we want to aim for IE10.

<meta http-equiv="X-UA-Compatible" content="IE=10"/>

If your browser base is different, you can, of course, set this differently. In this project, we will require all users of the application to be running IE10 or greater. Firefox and Chrome have supported the bits of HTML5 we need for so long, that there’s little concern about people who choose to use those browsers.

Once we have those two small changes in the master page (we are using a copy of v4.master with no other customizations at the moment), we “have HTML5″. It’s most obvious because some of the branding flourishes I’ve put in like rounded corners show up in IE10 now, rather than just square boxes.

Once we have HTML5 enabled, we can start to use the File API, a.k.a. the FileReader. This is such a simple little thing, that it’s hard to believe it gives us the capability it does.

<input type="file" id="attachment-file-name"/>

That’s it. That simple HTML gives us a file picker on the page. Depending on your browser, it will look something like this:

File Picker

When you make a file selection with this very familiar widget, you can query the element using jQuery.

var file = $("#attachment-file-name").files[0];

Note that we’re after a single file, so we grab the first object in the file list array. We get an object that looks something like this screen shot from Firebug:

File object from the File Picker

Once we have the file object, we can look at what the Lists SOAP Web Service needs to get in order to upload the file. Again, it’s pretty simple. Here is the list of inputs and output from the MSDN documentation page for the Lists.AddAttachment Method.

Parameters

listName
A string that contains either the title or the GUID for the list.
listItemID
A string that contains the ID of the item to which attachments are added. This value does not correspond to the index of the item within the collection of list items.
fileName
A string that contains the name of the file to add as an attachment.
attachment
A byte array that contains the file to attach by using base-64 encoding.

Return Value

A string that contains the URL for the attachment, which can subsequently be used to reference the attachment.

The first three inputs are straightforward. We pass in the name of the SharePoint list, the ID of the list item, and the name of the file we want to attach. It’s that last input parameter called “attachment” that’s a bit tricky.

When we upload a file via an HTTP POST operation – which is what all of the SOAP Web Services use – we have to pass text. But the files we want to upload are as likely as not to be binary files. That’s where the Base64 format comes in. Believe me, you don’t need to know exactly what the format actually is, but you should probably understand that it enables us to send binary files as text bytes. Those bytes are then decoded on the server end so that the file can be stored in all of its original, pristine glory.

Here’s the code I ended up with, skinnied down as much as possible to make it a little clearer to follow. I draw heavily from Scot and James’ posts for this, but nuanced for SPServices. I’ve also stripped out all of the error handling, etc.

  • First, the getFileBuffer function reads the contents of the file into the FileReader buffer. readAsArrayBuffer is an asynchronous method, so we use a jQuery Deferred (promise) to inform the calling function that the processing is done.
  • The contents of the buffer are then converted from an ArrayBuffer – which is what the FileReader gives us – into a Base64EncodedByteArray. This works by passing the buffer through a Uint8Array along the way.
  • Finally, we use the toBase64String method to convert the SP.Base64EncodedByteArray to a Base64String.

Yeah, that’s a lot to swallow, but again, you don’t really need to understand how it works. You just need to know that it does work.

Finally, we call Lists.AddAttachment using SPServices to do the actual upload.

/* From Scot Hillier's post:
http://www.shillier.com/archive/2013/03/26/uploading-files-in-sharepoint-2013-using-csom-and-rest.aspx */
var getFileBuffer = function(file) {

  var deferred = $.Deferred();
  var reader = new FileReader();

  reader.onload = function(e) {
    deferred.resolve(e.target.result);
  }

  reader.onerror = function(e) {
    deferred.reject(e.target.error);
  }

  reader.readAsArrayBuffer(file);

  return deferred.promise();
};

getFileBuffer(file).then(function(buffer) {
  var bytes = new Uint8Array(buffer);
  var content = new SP.Base64EncodedByteArray(); //base64 encoding
  for (var b = 0; b < bytes.length; b++) {
    content.append(bytes[b]);
  }

  $().SPServices({
    operation: "AddAttachment",
    listName: "Tasks",
    listItemID: taskID,
    fileName: file.name,
    attachment: content.toBase64String()
  });

});

Very cool! And as I tweeted yesterday, far easier than I ever would have expected. Yes, it took me a good chunk of a day to figure out, but it definitely works, and pretty fast, too.

If you use this example as a base, you could fairly easily build out some other file uploading functions. Combined with the other attachment-oriented methods in the Lists Web Services, you can also build the other bits of the attachment experience:

  • GetAttachmentCollection – Returns a list of the lit item’s attachments, providing the full path to each that you can use in a list of links.
  • DeleteAttachment – Once you’ve uploaded an attachment and realized it was the wrong one, this method will allow you to delete it.

Moral of the story: Fear not what you do not know. Figure it out.

Resources

What’s the Story for HTML5 with SharePoint 2010? by Joe Li

Uploading Files Using the REST API and Client Side Techniques by James Glading

Uploading Files in SharePoint 2013 using CSOM and REST by Scot Hillier

Lists.AddAttachment Method

This article was also posted at ITUnity on Jun 02, 2014. Visit the post there to read additional comments.

Update 2014-05-28 11:55 GMT-5

Hugh Wood (@HughAJWood) took one look at my code above and gave me some optimizations. Hugh could optimize just about anything; he is *very* good at it. I *think* I can even see the difference with larger files.

getFileBuffer(file).then(function(buffer) {
  var binary = "";
  var bytes = new Uint8Array(buffer);
  var i = bytes.byteLength;
  while (i--) {
    binary = String.fromCharCode(bytes[i]) + binary;
  }
  $().SPServices({
    operation: "AddAttachment",
    listName: "Tasks",
    listItemID: taskID,
    fileName: file.name,
    attachment: btoa(binary)
  });
});

SPServices: What About Deprecation of the SOAP Web Services?

SPServices user JSdream asked a question in the SPServices Discussions on Codeplex the other day that’s worthy of a more prominent response.

Should we, who use SPServices, be concerned at all with Microsoft recommending not to use either the ASMX web services in SharePoint 2013 (http://msdn.microsoft.com/en-us/library/office/jj164060.aspx#DeprecatedAPIs) or the owssvr.dll (I used this one a lot when doing InfoPath stuff) for development purposes?

The Choose the right API set in SharePoint 2013 article is an important one, for sure. In it, there’s pretty clear direction about which API to choose based on the specific requirements of the solution you are building. As with anything, though, there’s a strong dash of “it depends”. It depends on things like your available skills, the device the code will run on, etc.

Figure 1. Selected SharePoint extension types and SharePoint sets of APIs

Figure 1. Selected SharePoint extension types and SharePoint sets of APIs

I would say that we should all be “cautious” rather than “concerned”. The SOAP Web Services are used by many parts of SharePoint still, with the most prominent being SharePoint Designer. If you sniff around in the traffic between processes, you’ll spot other places as well.

That said, deprecated is deprecated. If you build your solutions with the data access layer separated out from the business logic layer, you should be able to replace the calls if and when it becomes necessary.

I am not aware of any official specific time frames for the SOAP services going away, and the Product Group would need to give us a good, long lead time because lots of code depends on it, SPServices or not. Many people have built managed code-based solutions which use the SOAP Web Services as well. GetListItems is a workhorse operation no matter how you build your solutions.

There are many things you simply cannot do with REST yet, though in many cases CSOM may provide better coverage. If you are using SPServices for the operations which aren’t available in REST or CSOM, obviously you’ll want to continue doing so until there is a viable replacement. The REST coverage is improving steadily. (See Overview of SharePoint Conference 2014 and new content for developers for some information.) Keep in mind, though, that the majority of improvements will end up in Office365 first and in the SharePoint Server product at some point later or perhaps not at all. So your time horizons and cloud plans are also critical decision factors.

[If I had to put my money on a horse, it'd be REST over CSOM over the long term.]

All that said, if you are starting a new project on SharePoint 2013 of any significant size, you should be considering using the App Model, CSOM, and REST. SPServices may still fit in there somewhere, so don’t toss it out with the bath water just yet. I’ll continue evolving it as long as it has people wanting to use it. I think I still have plenty of things to do.

SPServices Stories #21 – Redirect If User Clicked a Button Previously

This entry is part 20 of 20 in the series SPServices Stories

Introduction

teylynIngeborg Hawighorst (@IngeborgNZ) is a long-time SPServices user who has come up with any number of intriguing uses for the library. I’d recommend her blog anytime if you’d like to learn about interesting things you can do with SharePoint, but even more so if Excel is your bag. Ingeborg has been an Excel MVP for years running (see her profile on the MVP site). Some of the best solutions using SPServices come out of discussions in various SharePoint-oriented forums. In this case, Ingeborg spotted some suggestions from Eric Alexander (@ejaya2 aka PirateEric) and decided to build it out. Without further ado, here is Ingoborg’s article, reposted from her blog cheers, teylyn.

Redirect If User Clicked a Button Previously

I just came across this question in sharepoint.stackexchange.com. When a user visits a SharePoint site, they are presented with a splash screen and need to accept the policy before they can proceed. Upon subsequent visits, the splash screen does not show, because SharePoint will remember the  user. PirateEric outlined a possible solution: Use a SharePoint list to save the user name when the button is clicked. When the page loads, look up the user in the list. If they already exist, redirect the page, if not, show the splash page with the button to accept the policy. If the policy changes and users need to be made aware of that, simply remove all items in the list that tracks the users. All this can be done with jQuery and web services. That intrigued me and I had a go at actually building this, using Marc Anderson’s SPServices.

How to set it up

Create a SharePoint custom list with two fields, Title and UserName. The former is the out of the box field, the latter is a simple text field. Create two pages, the Splash page with the button and the page that is the desired destination page for all visitors. In my sample these are called Splash.aspx and MainContent.aspx On the Splash page the code that you can see below will be loaded before any other web part. If you use a Content Editor Web Part to load the code with a content link, make sure that it’s the first web part on the page. In many cases, JavaScript and jQuery will be placed in the last web part of the page and run after the DOM has loaded. But in this case this would mean that the Splash page appears briefly, even if it is followed by a redirect to a different page. The Splash page provides the policy (or terms and conditions) that the user must accept, and a link or a button that the user can click to accept. This link or button must then trigger a JavaScript function. That is very easy to do. I used a button and put the html straight into a CEWP like this:

<button onclick="PolicyButtonClick()" type="submit">
   I accept the policy
</button>

So the user clicks the button and the JavaScript function PolicyButtonClick() will run. This function can be found in the code below. First, the jQuery library and the SPServices library are loaded. Then the user name of the current user is retrieved with the SPServices call using SPGetCurrentUser. It returns a text string in the format DOMAIN\Account.  Next, the SPServices call uses the GetListItem. In the call, a CAML query is constructed that will return only list items where the column UserName equals the current user.  The items returned by that query are then counted. Since the user account is a unique value, we can safely assume that the query will either return one result or no result at all. If the query returned an item, this means that the user has previously accepted the policy and the script will redirect to the MainContent.aspx page.  If the query did not return anything, the current page will continue to be displayed. When the user clicks the button to accept the policy,  the user name is written into a variable. Then the SPServices operation to UpdateListItem is called and will create a new item in the list “PolicyAccepted”, storing the previously established account name in the column UserName. Then the MainContent.aspx page is loaded. The next time the user opens the Splash page, their account name will be found in the PolicyAccepted list and they will not see the Splash page again, unless the entry in the list is deleted. Here is the complete script:

<script type="text/javascript" src="/path/jquery-1.10.2.min.js" language="javascript"></script><script type="text/javascript" src="/path/jquery.SPServices-2013.02a.min.js" language="javascript"></script><script type="text/javascript">// <![CDATA[
// start the code even before the DOM is loaded, so not waiting for document ready
//$(document).ready( function() {
// get the user name
 var userName= getUserName();
// find the user name in the list
 var userAccepted = matchUserName(userName);
 if (userAccepted == 1 )
 {
 // redirecting page
 window.location.replace("http://path/Documents/MainContent.aspx");
 }
//});

function getUserName() {
 var thisUserAccount= $().SPServices.SPGetCurrentUser({
 fieldName: "Name",
 debug: false
 });
 return(thisUserAccount);
}

function createNewItem(theTitle, theUser) {
 $().SPServices({
 operation: "UpdateListItems",
 async: false,
 batchCmd: "New",
 listName: "PolicyAccepted",
 valuepairs: [["Title", theTitle], ["UserName", theUser]],
 completefunc: function(xData, Status) {
 }
 });
}

function matchUserName(userName) {
 var queryText = "<Query><Where><Eq><FieldRef Name='UserName'/><Value Type='Text'>" + userName + "</Value></Eq></Where></Query>";
 $().SPServices({
 operation: "GetListItems",
 listName: "PolicyAccepted",
 async: false,
 CAMLQuery: queryText,
 completefunc: function (xData, status) {
 itemCount = $(xData.responseXML.xml).find("rs\\:data, data").attr("ItemCount");
 }
 });
 return(itemCount);
}

function PolicyButtonClick() {
 var userName= getUserName();
 var theTitle= "Accepted";
 createNewItem(theTitle, userName);
 window.location.href = "http://path/Documents/MainContent.aspx";
}
// ]]></script>

Microsoft Cloud Show Episode 016 – Interview with Marc Anderson on Recent Changes Impacting Customers on Office 365

MSCloudShow

Microsoft Cloud Show

A few weeks back, I sat down (virtually, of course) with Andrew Connell (@AndrewConnell) and Chris Johnson (@LoungeFlyZ) to record an episode of the Microsoft Cloud Show. Andrew was in Florida, I was in Boston, and Chris was way around the world in New Zealand. Ah, the wonders of modern technology.

The only place to stay up to date on everything going on in the Microsoft cloud world including Azure and Office 365.

Whether you are new to the cloud, old hat or just starting to consider what the cloud can do for you this podshow is the place to find all the latest and greatest news and information on what’s going on in the cloud universe.  Join long time Microsoft aficionados and SharePoint experts Andrew Connell and Chris Johnson as they dissect the noise and distill it down, read between the lines and offer expert opinion on what is really going on.  Just the information … no marketing … no BS, just two dudes telling you how they see it.

I was honored to be the very first guest on the show, which already had 15 excellent episodes in the can.

In Episode 016 – Interview with Marc Anderson on Recent Changes Impacting Customers on Office 365, we talked about a number of extremely important things that have been going on with Office365 lately.

I had done a post about one issue that has caused me and users of SPServices the most consternation, Office 365 Update Changes ‘Display Name’ on Required Fields and Andrew had posted about a few others one his blog in Office 365 Needs to Treat the UX as an API if Our Customizations are to Stay Off the Server.

Last week, I released SPServices 2014.01, which addresses the title changes (adding ” Required Field” to the title attribute of some required dropdowns), but there’s a bigger set of issues at play here, as Andrew alludes to in his post.

In the podcast, we talked about the impact of these changes as well as the mindset behind them from the Microsoft side.

If you do any client side development with SharePoint – and that’s where everyone is headed – you owe it to yourself to listen to the podcast. You’ll understand more about what changes to the DOM might mean for you as a developer, or even what might happen to you as a user of customizations that rely on the DOM being stable and predictable.

One things seems certain: we’ll see more changes like the ones we discussed in the podcast and they will have an impact on everyone, not just people replying on Office365. (The same issues started to crop up for people who have applied the December 2013 Cumulative Update (CU) for SharePoint 2010 on premises.)

I want to thank Chris and Andrew for inviting me in for a chat. Assuming I didn’t annoy them too much with my scatological terminology, maybe I’ll be able to visit with them again the next time a round of changes like this pop up and cause ripples in the SharePoint time-space continuum.

jQuery Library for SharePoint Web Services (SPServices) 2014.01 Released

SPServicesHot on the heels of the short-lived SPServices 2013.02 and it’s younger, wiser sibling 2013.02a, comes SPServices 2014.01. I wanted to do a new release with some cool new functionality, but Microsoft really forced my hand with some changes to title attributes on some column types. (See: Office 365 Update Changes ‘Display Name’ on Required Fields)

I recommend jQuery 1.10.x with this release. Even if you are a happy user of an older release of SPServices, I’d recommend upgrading to this release.

The headlines for this release:

  • Fix for the ” Required Field” change to column titles noted above on Office365 buildversion 16.0.2510.1204 (or thereabouts) and above and SharePoint 2010 with the December 2013 CU
  • New function SPGetListItemsJson
  • Exposed existing private function DropdownCtl as public function SPDropdownCtl
  • New objectType for SPXmlToJson: JSON
  • Case insensitive query string values with $().SPServices.SPGetQueryString
  • Added ClaimRelease operation to the Workflow Web Service

The SPGetListItemsJson function is pretty cool, if I do say so myself. When I fixed the operation GetListItemChangesSinceToken in 2013.02, I saw the possibility for an auto-conversion of the returned data to JSON with only one SOAP call. Because GetListItemChangesSinceToken returns both the list schema and the data, I’m able to parse the schema and return the data as JSON using SPXmlToJson. You can also pass in your own mappingOverrides if you need to.

GetListItemChangesSinceToken is also cool because it allows you to retrieve content deltas by passing in the old changeToken, and SPGetListItemsJson returns the changeToken and also accepts it for future calls. This is great support for the types of Single Page Applications I discuss in my ongoing blog series Single-Page Applications (SPAs) in SharePoint Using SPServices.

While I was building SPGetListItemsJson, I was working on a client project where we stored JSON as text in a Multiple lines of text column. To facilitate conversion of that data to internal JSON, I added an objectType to SPXmlToJson of “JSON”. This allows us to request automatic conversion of text-based JSON data to internally represented data simply by specify the JSON objectType in a mappingOverride. (The function uses jQuery’s $.parseJSON.)

With the kerfuffle about the Office 365 Update Changes ‘Display Name’ on Required Fields and the attendant fixes I needed to do in SPServices, I decided to make the SPDropdownCtl function I’ve been using for years public.  It’s not for the faint of heart, as it passes back a variable object depending on the type of “dropdown”, but it should help some people in building their own applications.

As usual, there are also a number of additional changes to fix existing bugs or improve efficiency (yes, I’m still able to improve on my old code, and I expect that will continue).

You can see the full list of enhancements and improvements on the download page. Note the link to the Issue Tracker items for this release. For posterity, here are links to the release notes showing all the fixes and improvements from the Issue Tracker on Codeplex.

New Functionality

Alpha Issue Tracker Item Function Description
ALPHA1 10216 $().SPServices.SPGetListItemsJson New Function: SPGetListItemsJson
ALPHA1 10221 $().SPXmlToJson Add new objectType to SPXmlToJson for text columns containing JSON
ALPHA1 10218 $().SPXmlToJson Add sparse parameter to SPXmlToJson
ALPHA1 10195 $().SPServices.SPGetQueryString Case insensitive SPGetQueryString?
ALPHA3 10229 $().SPServices.SPDropdownCtl Make DropdownCtl Function Public
ALPHA4 10230 $().SPServices.SPDropdownCtl Return optHid input element in SPServices.SPDropdownCtl

New Operations

Alpha Web Service Operation Options MSDN Documentation Issue Tracker Item
ALPHA2 Workflow ClaimRelease item, taskId, listId, fClaim http://msdn.microsoft.com/en-us/library/workflow.workflow.claimreleasetask(v=office.12).aspx 10222

Bug Fixes and Efficiency

Alpha Issue Tracker Item Function Description
ALPHA1 10211 $().SPServices.SPGetQueryString LMenuBaseUrl is undefined
ALPHA1 10192 $().SPServices.SPGetQueryString DeleteWeb needs_SOAPAction Value Incorrect
ALPHA1-2 10219 $().SPServices.SPXmlToJson Various Fixes to SPXmlToJSON
ALPHA2 10224 $().SPServices.SPCascadeDropdowns ParentObject Null in $.fn.SPServices.SPCascadeDropdowns
ALPHA2 10225 $().SPServices.SPXmlToJson SPXmlToJson – Fix Date/Time Conversion
ALPHA2 10226 $().SPServices.SPCascadeDropdowns Cascade Dropdowns Do not work if Fields Are Required
ALPHA4 10231 $().SPServices.SPCascadeDropdowns Title Changed for Required Fields after CU for Sharepoint 2010
BETA2 10238 $().SPServices.SPDisplayRelatedInfo SPDisplayRelatedInfo not working in SPServices 2014.01
BETA2 10236 NA Replace .attr(“value”) usage with .val()
BETA2 10235 $().SPServices.SPRequireUnique SPRequireUnique bug getting entered value