A jQuery Library for SharePoint Web Services (WSS 3.0 and MOSS): Part 2 – How Does It Work?

Cross posted from EndUserSharePoint.com

WebServicesIn my last post, I wrote about why I decided to start building the jQuery Library for SharePoint Web Services. In this post, I’ll tell you a bit about what’s there and how it works. If you’re totally familiar with the SharePoint Web Services, this may all be old hat to you, but I felt it would be useful to explain some of the underlying concepts in what I think of as “English” for those who might get something out of it.

As I’ve mentioned, the fundamental idea for the library was to “wrap” the key SharePoint Web Services which would allow you to talk to SharePoint in real time without full page refreshes. As I looked across the Web Services, I saw clear patterns in how they were called (and some glaring exceptions). The patterns meant that I could do some nice generalizing in my jQuery code. The “core” of the jQuery Library for SharePoint Web Services is the $().SPServices function, which allows you to simply call each Web Service operation.

There is a set of Web Services which are available in WSS 3.0 and MOSS, and then some that are only available in MOSS. The table below shows the Web Services currently available in the library and where they work, along with links to the MSDN documentation:

Web Service WSS 3.0 MOSS MSDN Documentation
Alerts Alerts Web Service
Authentication Authentication Web Service
Forms Forms Web Service
Lists Lists Web Service
Permissions Permissions Web Service
Users and Groups Users and Groups Web Service
Versions Versions Web Service
Views Views Web Service
WebPartPages Web Part Pages Web Service
Webs Webs Web Service
PublishedLinksService   PublishedLinksService Web Service
Search   Search Web Service
UserProfileService   User Profile Web Service
Workflow   Workflow Web Service

I haven’t tried to wrap every single Web Service or every single operation of each. I’ve focused on the Web Services and operations that seemed as if they would provide the most benefit the most often. If I’ve missed something that would be of use to you, then by all means let me know by dropping a comment into the Discussions on the Codeplex site. I’d be happy to add it in the next release.

Each Web Service is called with a Simple Object Access Protocol (SOAP) Envelope which contains information about what you want to accomplish in the SOAP Body. As an example, here’s what the SOAP call for the Lists Web Service’s GetList operation looks like:

<xml version="1.0" encoding="utf-8"?>
  <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xmlns:xsd="http://www.w3.org/2001/XMLSchema"  xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
  <GetList  xmlns="http://schemas.microsoft.com/sharepoint/soap/">
   <listName>string</listName>
   </GetList>
   </soap:Body>
  </soap:Envelope>

You can see what the SOAP call for any of the SharePoint Web Services needs to look like by going to /_vti_bin/[WebService].asmx. You can tack this onto the end of any URL within SharePoint. Here are a couple of examples:

  • http://servername/_vti_bin/lists.asmx
  • http://servername/sitepath/_vti_bin/workflow.asmx

On those pages, you’ll see what operations are available, and clicking on any of the operation links will show you the SOAP request and response structures. I’ve stared at most of these a lot so that you don’t really have to.

Building the SOAP Request

The SOAP Envelope and SOAP Body “wrappers” are the same for all of the SharePoint Web Services. Think of this wrapper as saying “I’m going to speak SOAP to you.” In my library, I store these wrappers in the SOAPEnvelope.header and SOAPEnvelope.footer variables:

SOAPEnvelope.header = "<soap:Envelope xmlns:soap='http://schemas.xmlsoap.org/soap/envelope/'><soap:Body>";
SOAPEnvelope.footer = "</soap:Body></soap:Envelope>";

Next comes what I call the SOAPEnvelope.opheader, or Operation Header. The Operation Header varies based on what operation you want to use. For the most part, the structure of the Operation Header is consistent for all of the operations within a Web Service, but there are a few exceptions. (Naughty developers.) The Operation Header tells the specific Web Service what operation you want to have a chat with.

We also need to have a SOAPAction, which is passed ahead of the request. In the case of the GetList operation above, it looks like this:

http://schemas.microsoft.com/sharepoint/soap/GetList

The SOAPAction is also generally consistent within a Web Service, but not always. (Naughty developers again.)

Finally, we get to what I call the SOAPEnvelope.payload. This is where you pass in the real data that you want to use to talk to the Web Service. In the GetList example above, the payload simply contains the listName, which can be either the text name of the list (e.g., States) or the GUID for the list (e.g., 8BADF8E2-3758-4D1B-9966-045F4A77F73D).

The AJAX Call

Based on what operation you pass into the SPServices function of the library, it builds up the appropriate SOAP call and uses it “talk” to the Web Service using AJAX (shorthand for Asynchronous JavaScript and XML). The AJAX call is amazingly simple:

$.ajax({
   url: ajaxURL, // The relative URL for the  AJAX call
   async: opt.async, // By default, the AJAX calls are  asynchronous. You can specify false to  require a synchronous call.
   beforeSend: function  (xhr) { // Before sending the msg, need  to send the request header with the SOAPAction
     xhr.setRequestHeader("SOAPAction",  SOAPAction);
   },
   type: "POST", // This is a POST
   data: msg, // Here is the SOAP request  we've built above
   dataType:  "xml", // We're  sending XML
   contentType:  "text/xml; charset='utf-8'", //  and this is its content type
   complete:  opt.completefunc // When the call is  complete, do this
});

Calling the Library

To set things up to use the jQuery Library for SharePoint Web Services, you’ll need to add references to it and the core jQuery library in the right context for what you are trying to accomplish. This may mean that you add the references in your master page (if you are going to use jQuery pervasively), in a page layout (if only a certain class of pages will utilize jQuery), or in specific pages (if you want to use some of the functions only on a list’s forms, for example). These references will look something like this:

<script language="javascript" type="text/javascript" src="/jQuery%20Libraries/jquery-1.3.2.min.js"></script>
<script language="javascript" type="text/javascript" src="/jQuery%20Libraries/jquery.SPServices-0.4.1.min.js"></script>

I recommend putting both .js files into a Document Library in the root of your Site Collection called something like “jQuery Libraries”, as above. This way, you can manage them as content. For deployment purposes, you may want to use a generic name for the current version you are using so that you won’t need to update every reference every time you upgrade the .js files:

<script language="javascript" type="text/javascript" src="/jQuery%20Libraries/jquery-latest.min.js"></script>
<script language="javascript" type="text/javascript" src="/jQuery%20Libraries/jquery.SPServices-latest.min.js"></script>

What all this adds up to is that you can use the jQuery Library for SharePoint Web Services to call any of the Web Service operations it wraps as simply as this:

$().SPServices({
   operation:  "GetList",
   listName:  "States",
   completefunc: function  (xData, Status) {
     ...do something...
   }
});

which is a whole lot easier and efficient than trying to decipher all of the SOAP and AJAX requirements for each operation every time.

You can also pass in some optional options:

  • WebURL — If you pass in a Web URL, then that context is used for the ajaxURL in the AJAX call above. If you don’t pass in a WebURL, the current context is used. (The current context is determined by a call to the WebUrlFromPageUrl operation of the Webs Web Service. This is exposed as the public function $().SPServices.SPGetCurrentSite, as it can be a little tricky to determine the current site otherwise.) The WebURL is only useful in the case where the context matters to the result, and this is indicated in the library’s Documentation.
  • async — [true | false], as described above

One important caveat: a lot of people and blog posts will tell you to simply drop jQuery into Content Editor Web Parts (CEWPs). I’m not saying that this is always a bad idea, but it will make keeping track of your jQuery usage pretty difficult. I recommend using SharePoint Designer to put the jQuery into the pages them selves. This allows you to use whatever deployment mechanisms you already have in place for customized (unghosted) pages and also eliminates the possibility that a user will delete or change the jQuery in the CEWPs inadvertently.

I’m working on a function called SPAuditScript which will let you get a report of script usage in CEWPs across a Site Collection. This ought to help with keeping track of things as well as allay some of the fears that IT folks have about non-developers doing developer-like things without any way to track it.

The Results

When you make one of these calls, you get back XML, just as you’ve passed XML into it. As I was building the library, I found that I very often wanted to see what was in that XML in an easy to read format, so I built the $().SPServices.SPDebugXMLHttpResult function. If you’re comfortable with XML and debugging tools, you may never need this function, but I find it useful. If you wanted to use it in the above example, replace the completefunc with this:

completefunc: function (xData, Status) {
   var out =  $().SPServices.SPDebugXMLHttpResult({
     node:  xData.responseXML
   });
   $(divId).html("").append("<b>This  is the output from the GetList operation:</b>" + out);
}

where divId is the ID for a DIV on the calling page where you’d like to place the results. The output from the $().SPServices.SPDebugXMLHttpResult function for GetList will look something like this excerpt:

jQuery Library for SP Web Services

On the other hand, if you’d like to work with the output, you’d probably do something like this:

$().SPServices({
   operation:  "GetList",
   async: false,
   webURL: opt.webURL,
   listName:  opt.listName,
   completefunc:  function(xData, Status) {
     $(xData.responseXML).find("Field").each(function()  {
       if($(this).attr("StaticName")  == opt.columnStaticName) displayName = $(this).attr("DisplayName");
     });
   }
});

This example comes from the $().SPServices.SPGetDisplayFromStatic function. In the completefunc, I find all of the Field elements, iterate through them until I find the one I’m looking for, then grab the DisplayName attribute to return. I’m not going to go into a jQuery tutorial here, but working with the XML SOAP response is often about this simple.

Conclusion

So all of this (hopefully) explains what happens when you use the jQuery Library for SharePoint Web Services function $().SPServices to call a Web Service operation. Any one of these Web Service operations can let you do some spiffy stuff, but to me the real power comes in utilizing one or more operations together to accomplish cool things. For instance, the $().SPServices.SPCascadeDropdowns function uses GetListItem, $().SPServices.SPLookupAddNew uses GetList (2x) and GetFormCollection, and so on.

In my next post, I’ll write about the other functions in the SPServices namespace and how you can use them to improve the user experience in SharePoint.

15 Comments

  1. Hi Mark,

    It looks like the archive on the codePlex site is corupted as i do not get to save it properly? Would you please check again & if possible update the codeplex.com project download?

    On the other hand…tremendous work…if you need a hand, please let me know!

    C. Marius

    Reply
  2. In my case I want to use the GetList service call to get the ID of a list, which I need to pass into another function. another kind soul posted http://sharepointjavascript.wordpress.com/2009/09/27/convert-singleline-textfield-to-filtered-lookup-dropdown/ which uses jquery to “convert” a single text column field to a drop-down list, which uses the list GUID in the script. I’d prefer to use the list name rather than the GUID to simplify deployment from dev to prod. From your example I think I should be able to get the GUID from the list name but I am struggling to piece it together. Can you point me in the right direction? thanks

    Reply
  3. Hi Marc,

    Thanks for the excellent post…this lays out the useful web services pretty clearly.

    Was wondering if you could help me get over a mental block. I’ve been calling the web services on document libraries, and custom lists with folders; it’s been a struggle but well worth the effort.

    Do you happen to know how do to represent the “full file url” for a custom list? For document libraries and folders in custom lists it’s simply http(s)://server:port/subsite/Lists/ListName/FolderorDocumentname …. have tried reading through the documentation, tried every combination I can think of…any ideas?

    Thanks again for your excellent posts…when’s the book coming out?

    Take care,

    Steve

    Reply
    • Steve:

      There’s no “full file URL” for an item in a Custom List. If you look at the URL you end up when you go to a Custom List item, you’ll see it takes the form:

      http://servername/sitepath/Lists/listname/DispForm.aspx?ID=n

      where n is the unique ID for the list item.

      No book on tap (though you’re not the first to suggest it), but I’ve joined the faculty for the new USPJ Academy. Watch for more details on that…

      M.

      Reply
  4. Sweet, thanks! signed up for the mailing list and look forward to reading more of what you’ve been up to. I just wish I knew half of what you’ve forgotten about sharepoint.

    I’m building an application and the user needs to build up a data set, preserve it, but make a copy and then vary the data slightly to see the results. The user is seldom if ever confronted with the sharepoint methodology of manually triggering a workflow; that is way beyond my users patience or level of discipline. I’m putting a button on a EditFormCustom.aspx form that clones the list item. …like so many things in life what at first seems simple turns out to be, well a challenge.

    Not to belabor the point, but I take it then that custom list items basically can’t be directly used with web services without an add-in (nintex)? this will sort of stink because the only work-around I can think of is to create a folder in a list referencing the id of the original list item, putting the workflow there, then having the workflow in the secondary list manipulate the original list item…I keep having this moral dilemma, this is supposed to be an out-of-the-box solution…and I think we’ve both supported systems with undisciplined parent-child relationships. I really don’t want to have to write a data integrity audit program, but y’know I sort of am learning to like sharepoint; how wierd am I?

    Reply
    • Steve:

      Not wierd at all. It’s good you’re trying to do things right!

      Workflows definitely work on Custom List items and so do the Web Services. If you need to “clone” an item, just get it with GetListItems and create a new item with the same values (tweaked however you need) using UpdateListItems.

      I’d tell you what I’ve forgotten about SharePoint, but um, hmm, what was it again??? ;+)

      M.

      Reply
  5. Worked out the url for web services on custom lists; it’s of the form http(s)://servername/subsite/Lists/ListName/ID_.000 where presumably the .000 is the version of the list item. We can get the relative url from @fileref in SharePoint Designer.

    following this post provides example code…

    http://social.technet.microsoft.com/Forums/en/sharepointworkflow/thread/03215c81-c495-487e-a8eb-80eee4b49420

    I found this post useful…

    http://myitforum.com/cs2/blogs/mlucero/archive/2008/07/22/sharepoint-2007-workflows-url-woes.aspx

    as well as

    http://social.msdn.microsoft.com/Forums/en-US/sharepointworkflow/thread/53ad841a-5478-4e06-9fa1-b2a2af09e356

    Take care, hopefully will pass the corporate audit, upgrade smoothly to 2010 and then begin using jQuery!!! Sort of obeying corporate rules in the name of OOTB here…off to harden up this process a little more, will be checking to ensure there are no workflows already executing, that would error out a subsequent call to start one.

    Steve

    Reply
  6. Whenever we customize a newform.aspx page i.e. create a new page from newform.aspx and suppose name is newform1.aspx and create new item from the newform1.aspx. New Item are created by Title as id_.000.
    Any pointer why this happens. Checked there is no versioning enabled.

    If I am not clear please let me know so.
    Thanks

    Reply
    • Saroj:

      Just the act of customizing the form using a DVWP won’t cause that behavior. I believe the only time I’ve seen that is when someone tries to monkey with the Title column in the form somehow. What type of customizations are you doing? Are you filling in the Title column’s value before saving the item?

      M.

      Reply

Have a thought or opinion?