jQuery Library for SharePoint Web Services (SPServices) 2013.02 Re-Released

Back on December 28, 2013, I released SPServices 2013.02, just squeaking in under the wire to be able to name it thusly.

Soon after the release, Paul Tavares (@paul_tavares) and I noticed a few significant issues. These issues are fixed in 2013.02a. If you downloaded 2013.02, please replace it with 2013.02a.

For the record, the issues were:

  • In SharePoint 2010, the current site context was not calculated correctly
  • GetListItemChanges and GetListItemChangesWithToken failed as they had in previous versions
  • In SPDisplayRelatedInfo, if the relatedListColumn was not specified in relatedColumns, the function would fail with an error. This issue was probably present in previous versions, but as far as I know had not been reported.

Sorry for the inconvenience, and thanks to Paul (yet again!) for his help and support.

The link to SPServices 2013.02a on cdnjs is coming. Thanks to Josh McCarty (@joshmcrty) for his continuing help with git, which for some reason remains unfathomable to me.

Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 4 – GetListItemChangesSinceToken

As I mentioned in the last part of the series, when we build a Single Page Application (SPA), we’ll usually want to keep the data we’re displaying up to date. GetListItems and GetListItemChanges are two of the operations that can help with this, but the more robust option is GetListItemChangesSinceToken (MSDN documentation). In fact, it’s becoming my favorite list-data-getting operation, even over the rock solid GetListItems.

GetListItems is great for simply grabbing items from a list, but if you want to monitor the list for changes efficiently, you’ll need GetListItemChanges and/or GetListItemChangesSinceToken. Of the two, GetListItemChangesSinceToken is the far more powerful. In fact, once you start using it, you may well stop using GetListItems altogether.

[important]GetListItemChanges and GetListItemChangesSinceToken do not work in versions of SPServices before 2013.02.[/important]

(I wish I had gotten these two operations working long ago, now that I realize how powerful they are.)

In this post, let’s take a look at the GetListItemChangesSinceToken operation.

GetListItemChangesSinceToken

GetListItemChangesSinceToken is clearly related to its siblings, GetListItems and GetListItemChanges, with a cross between the two of their characteristics, plus some extra goodness. We can be very specific about what we request, as with GetListItems, but we also can decide to only receive changes since a specific database token (more on this below).

[webURL]

See the GetListItems post in this series.

listName

See the GetListItems post in this series.

viewName

See the GetListItems post in this series.

CAMLQuery

See the GetListItems post in this series.

CAMLViewFields

See the GetListItems post in this series.

CAMLRowLimit

See the GetListItems post in this series.

CAMLQueryOptions

See the GetListItems post in this series. Note that the MSDN documentation for GetListItemChangesSinceToken provides far more parameters than the MSDN documentation for GetListItems. Most (if not all – I haven’t tested everything) of these parameters work with GetListItems as well.

changeToken

A string that contains the change token for the request. For a description of the format that is used in this string, see Overview: Change Tokens, Object Types, and Change Types. If null is passed, all items in the list are returned.

contains

A Contains element that defines custom filtering for the query and that can be assigned to a System.Xml.XmlNode object, as in the following example.

<Contains>
  <FieldRef Name="Status"/>
  <Value Type="Text">Complete</Value>
</Contains>

This parameter can contain null.

In simpler terms, this can be a snippet of CAML to add an additional filter to the request.

Note

Contrary to the statements above about passing nulls for changeToken and contains, you cannot do so from the client (it may be allowable in server side code, but I’m not sure). If you pass an empty node in your SOAP request, the request will fail. SPServices – as of version 2013.02 – will handle empty values for you. This is the bug I fixed.

GetListItemChangesSinceToken Returns

In your first call to GetListItemChangesSinceToken, you won’t have a token yet. Or, you may choose not to pass a token that you do have. In that case, you get a robust amount of information returned to you.

The simplest call looks like this:

$().SPServices({
  operation: "GetListItemChangesSinceToken",
  listName: "Sales Opportunities"
});

Here, I’m just asking for the information for the Sales Opportunities list; I’m not providing any parameter values at all. On that first call, the results will look something like this (inside the SOAP envelope and GetListItemChangesSinceTokenResult):

<listitems xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask">
  <Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635243604504600000;33126">
    <List DocTemplateUrl="" DefaultViewUrl="/Intranet/JQueryLib/Lists/Sales Opportunities/AllItems.aspx" MobileDefaultViewUrl="" ID="{37920121-19B2-4C77-92FF-8B3E07853114}" Title="Sales Opportunities" Description="" ImageUrl="/_layouts/images/itgen.gif" Name="{37920121-19B2-4C77-92FF-8B3E07853114}" BaseType="0" FeatureId="00bfea71-de22-43b2-a848-c05709900100" ServerTemplate="100" Created="20090825 06:24:48" Modified="20131216 03:52:25" LastDeleted="20120627 07:54:25" Version="362" Direction="none" ThumbnailSize="" WebImageWidth="" WebImageHeight="" Flags="612372480" ItemCount="7" AnonymousPermMask="0" RootFolder="/Intranet/JQueryLib/Lists/Sales Opportunities" ReadSecurity="1" WriteSecurity="1" Author="3" EventSinkAssembly="" EventSinkClass="" EventSinkData="" EmailInsertsFolder="" EmailAlias="" WebFullUrl="/Intranet/JQueryLib" WebId="42f65d3f-343d-4627-a9a3-abf3d4d6491f" SendToLocation="" ScopeId="c8b78fea-7952-433a-be20-cda628ea6cbb" MajorVersionLimit="0" MajorWithMinorVersionsLimit="0" WorkFlowId="" HasUniqueScopes="False" AllowDeletion="True" AllowMultiResponses="False" EnableAttachments="True" EnableModeration="False" EnableVersioning="False" Hidden="False" MultipleDataList="False" Ordered="False" ShowUser="True" EnableMinorVersion="False" RequireCheckout="False">
      <Fields>
        <Field ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Type="Text" Name="Title" DisplayName="Title" Required="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="Title" FromBaseType="TRUE" ColName="nvarchar1"/>
        <Field Name="Lead_x0020_Source" FromBaseType="FALSE" Type="MultiChoice" DisplayName="Lead Source" Required="TRUE" FillInChoice="TRUE" ID="{0ab3481c-a7b4-432f-8292-c1617744a167}" Version="6" StaticName="Lead_x0020_Source" SourceID="{f4ebb20a-91e0-4306-997d-208e1d1920b7}" ColName="ntext3" RowOrdinal="0">
          <CHOICES>
            <CHOICE>Newspaper Advertising</CHOICE>
            <CHOICE>Web Site</CHOICE>
            <CHOICE>Personal Referral</CHOICE>
          </CHOICES>
        </Field>
        <Field Type="Currency" DisplayName="Potential Value" Required="FALSE" Decimals="2" LCID="1033" ID="{e9058e0c-b2e8-4af6-882c-9551a5f21451}" SourceID="{f4ebb20a-91e0-4306-997d-208e1d1920b7}" StaticName="Potential_x0020_Value" Name="Potential_x0020_Value" ColName="float1" RowOrdinal="0" Version="1">
          <Default>0</Default>
        </Field>
...
      <RegionalSettings>
        <Language>1033</Language>
        <Locale>1033</Locale>
        <AdvanceHijri>0</AdvanceHijri>
        <CalendarType>1</CalendarType>
        <Time24>False</Time24>
        <TimeZone>300</TimeZone>
        <SortOrder>2070</SortOrder>
        <Presence>True</Presence>
      </RegionalSettings>
      <ServerSettings>
        <ServerVersion>12.0.0.6421</ServerVersion>
        <RecycleBinEnabled>True</RecycleBinEnabled>
        <ServerRelativeUrl>/Intranet/JQueryLib</ServerRelativeUrl>
      </ServerSettings>
    </List>
  </Changes>
  <rs:data ItemCount="7">
    <z:row ows_Attachments="1" ows_LinkTitle="Tue Feb 14 15:58:39 EST 2012" ows_Modified="2013-12-11 10:04:38" ows_Author="3;#Marc D Anderson" ows_Lead_x0020_Source=";#Newspaper Advertising;#" ows_Potential_x0020_Value="12.6100000000000" ows_Lead_x0020_Date="2013-04-20 00:00:00" ows_Country="1;#United States" ows_Region="1;#Northeast" ows_State="2;#Rhode Island" ows_City="706;#Providence" ows_Course_x0020_Title="1;#ss" ows_System="3;#Alienware" ows_StateID="North Dakota" ows_Domain="7;#baloney.com" ows_Notes="&lt;div&gt;&lt;/div&gt;" ows_Web_x0020_Service_x0020_Operatio="13;#GetList" ows_Radio_x0020_Buttons="C" ows_Sales_x0020_Rep="104;#Marc Anderson;#89;#Eric Mullerbeck;#91;#Renzo Grande" ows_a_x0020_b_x0020_bcb_x0020_n="" ows_CalcTest="float;#151.320000000000" ows_CalcTestDate="datetime;#2009-08-25 14:24:48" ows_CalcTestYesNo="boolean;#1" ows_YesNo="1" ows_UpdateList_Field_Sun_x0020_Oct_x="lk;k;lk;lk" ows__Level="1" ows_UniqueId="5;#{3345A2C2-0B2D-4D79-9CB8-4FEFA993F5FA}" ows_FSObjType="5;#0" ows_Created_x0020_Date="5;#2009-08-25 14:24:48" ows_Title="Tue Feb 14 15:58:39 EST 2012" ows_Created="2009-08-25 14:24:48" ows_ID="5" ows_owshiddenversion="256" ows_FileLeafRef="5;#5_.000" ows_FileRef="5;#Intranet/JQueryLib/Lists/Sales Opportunities/5_.000" ows__ModerationStatus="0"/>
...
  </rs:data>
</listitems>

Yes, that is a heck of a lot of XML, but it’s not even all of it (note the ellipses). This may seem like overkill in many cases, and it may well be. In those cases, GetListItems is still the best way to go. But in the cases where you want to build an SPA, GetListItemsWithToken may be the ticket.

Like I said, you get a lot of robust information. Here are some more details on what’s there:

  • A bunch of info about what frequency and size requests are allowed. Generally you won’t need to care about these values, but they are there when you get to that point.
  • AlternateUrls – These are the URLs for each access path you might take to your list. Sweet! (The security folks will probably blanch at this.)
  • In any call to GetListItemChangesSinceToken we get a new value for LastChangeToken whether we have passed a changeToken or not. This token isn’t something we’re supposed to pick apart or understand; it represents a certain point in the database life. If you really want to understand what the pieces of the token mean, you can check out the article Overview: Change Tokens, Object Types, and Change Types. You’ll want to grab that value, as you will use it in subsequent calls. You can decipher what each part of it means from the documentation, but you really shouldn’t care. Basically, it’s *like* a timestamp, but in database terms.
  • The full list schema. You may not ever need this, but it’s great in cases where you find yourself doing a GetList/GetListItems pair of calls to set up further work in your script.
  • Server information (lines 17-31 above). This is really cool. I’ve never been able to figure out a consistent way to determine the language and locale settings, the time zone offset, etc. from the client, but here they are. Keep in mind that these are the *server* settings, not the client settings.
  • All of the list items. With large lists (not so large that they hit the 5000 item throttling limit – that’s another story entirely, and another post), you probably don’t want to get all of the list items. That’s where the contains parameter comes in handy. You might make the first call specifying that you only want the item with ID=1 or only items where the Title is null (generally this will result in no items) or something.

In the next call, you’re probably going to pass in the LastChangeToken value from the first call:

$().SPServices({
  operation: "GetListItemChangesSinceToken",
  listName: "Sales Opportunities",
  changeToken: "1;3;37920121-19b2-4c77-92ff-8b3e07853114;635243604504600000;33126"
});

If there have been no changes, the results will look something like this. Nice and simple.

<listitems MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask" xmlns:rs="urn:schemas-microsoft-com:rowset">
<Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635175524963570000;31260"> </Changes>
<rs:data ItemCount="0"></rs:data>
</listitems>

If there have been changes to items in the list, the results will look something like this:

<listitems MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask" xmlns:rs="urn:schemas-microsoft-com:rowset">
<Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635175524963570000;31260"> </Changes>
<rs:data ItemCount="2">
	<z:row ows_Attachments="0" ows_LinkTitle="MyItem1" ows_MetaInfo="3;#" ows__ModerationStatus="0"
		ows__Level="1" ows_Title="MyItem1" ows_ID="3" ows_owshiddenversion="2"
		ows_UniqueId="3;#{9153FDD3-7C00-47E9-9194-956BB20AAA8D}" ows_FSObjType="3;#0"
		ows_Created_x0020_Date="3;#2007-08-31T21:34:59Z" ows_Created="2007-08-31T21:34:59Z"
		ows_FileLeafRef="3;#3_.000" ows_FileRef="3;#sites/MyWebSite/Lists/MyList/3_.000"
		ows_ServerRedirected="0" />
	<z:row ows_Attachments="0" ows_LinkTitle="MyItem2" ows_MetaInfo="5;#" ows__ModerationStatus="0"
		ows__Level="1" ows_Title="MyItem2" ows_ID="5" ows_owshiddenversion="3"
		ows_UniqueId="5;#{5BDBB1C0-194D-4878-B716-E397B0C1318C}" ows_FSObjType="5;#0"
		ows_Created_x0020_Date="5;#2007-08-31T21:43:23Z" ows_Created="2007-08-31T21:43:23Z"
		ows_FileLeafRef="5;#5_.000" ows_FileRef="5;#sites/MyWebSite/Lists/MyList/5_.000"
		ows_ServerRedirected="0" />
</rs:data>
</listitems>

If there have been any item deletions, the Changes node will look a little different (you may see deletes, update, and adds depending on what’s been going on and how long it’s been since you asked):

<Changes LastChangeToken="1;3;641d61d7-b03e-4078-9e6c-379fa0208d6f;635243635233730000;33127">
  <Id ChangeType="Delete">1</Id>
</Changes>

This is just about the only place I know of where you can get information about deletes from the client side. Generally the only way to “catch” deletes is to write an event receiver to run on the server. Of course, you’ll only see the deletes since the last time you passed in a token value, but it allows you to reflect those deletes on the client.

If there have been any changes to the list settings (schema), we’ll get the schema again before the changed items. The results in this case look like what we get back from GetList and from the first call above. We receive everything we need to know to work with the list. For this reason, we would generally make a call to GetListItemChangesSinceToken first to get the list schema and all of the list items – at least those that match our filters – and then use either GetListItemChangesSinceToken, GetListItems, or GetListItemChanges for subsequent calls for our SPA.

Conclusion

As you can see, GetListItemsWithToken is incredibly powerful and returns a wealth of information. In fact, I’m not sure that you could get all of this from a REST call, but probably from CSOM. These old, crufty SOAP Web Services still have their place in the world.

Even better, they can provide you with what you need to get going on SPAs of you own. Hopefully you’re starting to see the possibilities.

jQuery Library for SharePoint Web Services (SPServices) 2013.02 Released

Yesterday I released SPServices 2013.02.

Here are the release headlines:

  • Improved compatibility with SharePoint 2013, whether on premises or on Office365, especially in the value-added functions (SPCascadeDropdowns, SPDisplayRelatedInfo, etc.)
  • Better API stability, including fixes for GetListItemChanges, GetListItemChangeSinceToken, and others
  • Bug fixes and performance improvements

While this is primarily a maintenance release, if you are using an earlier version of SPServices, I strongly suggest that you upgrade to this version, as you will see some performance improvements and there is some new functionality. Thanks to everyone who downloaded the beta and provided feedback.

Both releases in 2013 (2013.01 and 2013.02) work equally well – at least as best as I have been able to ensure – with SharePoint 2007, 2010, and 2013. My next planned release version will be 2014.01, which should make it obvious that the primary version number in my recent scheme is just the year of release.

One thing I wanted to mention that’s old news, but some people may have missed it. By far the most exciting thing in the 2013.01 release was jQuery promises, or deferred objects. All calls to the core operations return jQuery promises as of 2013.01. If you’re interested in increasing the efficiency of your code and getting ready for the way you are likely to work with the REST Web Services in SharePoint 2013, you should get familiar with using promises.

There are other goodies in this release, and you can see the full list of enhancements and improvements on the download page. Note the link to the Issue Tracker items for this release. For posterity, here are links to the release notes showing all the fixes and improvements from the Issue Tracker on Codeplex.

New Functionality

Alpha Issue Tracker Item Function Description
ALPHA5 10195 $().SPServices.SPGetQueryString Case insensitive SPGetQueryString?

Bug Fixes and Efficiency

Alpha Issue Tracker Item Function Description
ALPHA1 10152 $().SPServices.SPCascadeDropdown Multi-Select Lookup Cascading Dropdown’s Not Working
ALPHA1 10153 $().SPServices.SPComplexToSimpleDropdown Make SPComplexToSimpleDropdown Work in 2013 Publishing Pages
ALPHA1 10144 $().SPServices.SPXmlToJson userToJsonObject userId broken in 2013.01
ALPHA1 10146 $().SPServices ResolvePrincipals with addToUserInfoList=true requires SOAPAction
ALPHA2 10184 $().SPServices Lists.GetListItemChangeSinceToken operation input parameters not defined
ALPHA2 10183 $().SPServices Webs.GetColumns – Passing Unneeded webUrl Parameter
ALPHA2 10148 $().SPServices async on version 2013.01
ALPHA2 10177 $().SPServices First 2 arguments for the of addToPayload method for the GetTermSets are incorrect
ALPHA2 10154 $().SPServices SPCascadeDropdown: matchOnId not working with multi-select parents
ALPHA3 10165 $().SPServices SPServices() uses wrong server relative URL on SP2010
ALPHA3 10162 $().SPServices.SPRedirectWithID Bug in SPRedirectWithID
ALPHA3 10067 $().SPServices.SPAutocomplete SPAutocomplete dropdown issue
ALPHA3 10167 $().SPServices Caching bug if making same call to different webURLs
ALPHA4 10200 $().SPServices SPConvertDateToISO invalid TZD
ALPHA5 10147 $().SPServices.SPAutocomplete SPServices / jQuery ID Selectors in SP 2013, SPAutoComplete
ALPHA5 10189 $().SPServices.SPGetCurrentUser SPGetCurrentUser not being async
ALPHA5 10189 $().SPServices.SPGetCurrentUser SPGetCurrentUser data on iPhone
BETA2 10206 $().SPServices.SPCascadeDropdowns Multi-Select Dropdown Controls Don’t Have Name Attributes
BETA3 NA $().SPServices.SPDisplayRelatedInfo Fixed issues with lookup column handling

InfoPath 2013 Forms on Office365 with External REST DataSources

I’m building a reasonably complex form with InfoPath 2013 in which I need to look up data for some dropdowns from several external REST Web Services. The REST services are available publicly and anonymously, so authentication isn’t any issue.

For example, say I have a field on the form which displays values for Category in a dropdown. The form is housed on the domain https://mydomain.sharepoint.com at Office365 [in a subsite, but that shouldn't matter] and the REST service which provides the available values for Category is housed on http://www.someotherdomain.com. The latter domain is fully in our control, but it’s non-Microsoft.

I needed to be able to open the form in the browser for the users, but when I tried to do so, I got an error like this:

An error occurred querying a data source.

An error occurred querying a data source.

Warning
An error occurred querying a data source.
Click OK to resume filling out the form. You may want to check your form data for errors.A query to retrieve form data cannot be completed because this action would violate cross-domain restrictions.

If this form template is published to a SharePoint document library, cross-domain access for user form templates must be enabled under InfoPath Forms Services in SharePoint Central Administration, and the data connection settings must be stored in a UDC file in a data connection library in the same site collection.

If this is an administrator-approved form template, the security level of the form must be set to full trust, or the data connection settings must be stored in a UDC file by using the Manage data connection files option under InfoPath Forms Services in SharePoint Central Administration.

An entry has been added to the Windows event log of the server.
Log ID:6932

Correlation ID:c3e8619c-8c52-407e-34ca-9d5f1dc4564d

I understand all about cross-browser scripting, but this is SharePoint, which always makes things more complicated. Since we don’t have access to Central Administration, what was the right solution? Everything here is fully trusted, and the REST services are read only. There had to be a way.

I posted this as a question on the MSDN Forums at SharePoint forums > SharePoint 2013 – Using SharePoint Designer, InfoPath and Other Customizations. There were some good suggestions, but I finally stumbled on what seems to be the best fix.

When you look at your existing data connections, you can choose a data connection and Convert to Connection File…. You must have a Data Connection library in your Site Collection in which to store the UDCX files.

So, the steps seem to be:

  • Create a Data Connection Library in your Site Collection (I added it in the root of mine)
  • In the InfoPath ribbon, go to Data, then Data Connections. Your existing data connections should be listed.
  • Choose the data connection which is hosted on a different domain.
  • Clink the Convert to Connection File… button
  • In the location selector, specify the path to the Data Connection Library you created above. Mine is at https://tenantname.sharepoint.com/sites/Projects/DataConnections/ and append the name for the connection file, such as Categories.udcx
  • Leave the Relative to site collection (recommended) radio button selected.
  • Click OK

Once the operation completes, you’ll have a new item in your Data Connection Library, and the data connection should work in a browser-based form. In essence, what you’ve accomplished above is to tell SharePoint that the connection you would like to use is trusted.

All this without access to Central Administration!

Great SharePoint Books: ‘SharePoint Cheap Thrills’ by Ira Fuchs

SharePoint Cheap ThrillsIra Fuchs has been at it again, working to make SharePoint more interesting and accessible to Real SharePoint People. What I like about Ira’s books is that they are written for people who want to get something done from start to finish. In each book, Ira takes us through how he’s built real applications.

Ira’s latest is SharePoint Cheap Thrills. It’s no roller coaster ride, like some books are: it’s all good.

IHF PublishingIra self-publishes his books, so you get to buy directly from the author. Head on over to Ira’s imprint, IHF Publishing to learn more or buy his books.

I could say more, but I’ll let Ira tell you what he was out to accomplish this time around:

With so many SharePoint books on the market today it is difficult to determine which are truly informative and go beyond just enumerating the most fundamental features and functions of SharePoint.

It was my intent in writing SharePoint Cheap Thrills to deliver uncommon knowledge to people that empowers them to exploit the capabilities of SharePoint as fully as possible.

I  accomplish  this by identifying real-world functional requirements commonly found in most organizations and creating elegant solutions that address those needs using SharePoint’s most powerful and leveraged features, many of which are not well known or understood.

An example of one of these solutions is a workflow that can be created in just 20 minutes that generates documents and auto-populates them with customized information. There are numerous use cases for this type of application, such as employment package documents and service agreements. This is just one of the useful productivity enhancing solutions detailed in the book that will thrill people and make you look like a hero.

I am positive you will not be disappointed.

Regards,

Ira Fuchs
ira@ihfpublishing.com

If this book interests you, you should also take a look at Ira’s earlier opus: Enterprise Application Development in SharePoint 2010 – Creating an End-to-End Application without Code. (I’ve got to admit that I like the new, snappier title better.)

Enterprise Application Development in SharePoint 2010 – Creating an End-to-End Application without CodeFull disclosure: Ira has given me review copies of his books. The reason I wrote this post is that his books are good.