Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 4 – GetListItemChangesSinceToken
- Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 1 – Introduction
- Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 2 – GetListItems
- Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 3 – GetListItemChanges
- Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 4 – GetListItemChangesSinceToken
As I mentioned in the last part of the series, when we build a Single Page Application (SPA), we’ll usually want to keep the data we’re displaying up to date. GetListItems and GetListItemChanges are two of the operations that can help with this, but the more robust option is GetListItemChangesSinceToken (MSDN documentation). In fact, it’s becoming my favorite list-data-getting operation, even over the rock solid GetListItems.
GetListItems is great for simply grabbing items from a list, but if you want to monitor the list for changes efficiently, you’ll need GetListItemChanges and/or GetListItemChangesSinceToken. Of the two, GetListItemChangesSinceToken is the far more powerful. In fact, once you start using it, you may well stop using GetListItems altogether.
[important]GetListItemChanges and GetListItemChangesSinceToken do not work in versions of SPServices before 2013.02.[/important]
(I wish I had gotten these two operations working long ago, now that I realize how powerful they are.)
In this post, let’s take a look at the GetListItemChangesSinceToken operation.
GetListItemChangesSinceToken
GetListItemChangesSinceToken is clearly related to its siblings, GetListItems and GetListItemChanges, with a cross between the two of their characteristics, plus some extra goodness. We can be very specific about what we request, as with GetListItems, but we also can decide to only receive changes since a specific database token (more on this below).
[webURL]
See the GetListItems post in this series.
listName
See the GetListItems post in this series.
viewName
See the GetListItems post in this series.
CAMLQuery
See the GetListItems post in this series.
CAMLViewFields
See the GetListItems post in this series.
CAMLRowLimit
See the GetListItems post in this series.
CAMLQueryOptions
See the GetListItems post in this series. Note that the MSDN documentation for GetListItemChangesSinceToken provides far more parameters than the MSDN documentation for GetListItems. Most (if not all – I haven’t tested everything) of these parameters work with GetListItems as well.
changeToken
A string that contains the change token for the request. For a description of the format that is used in this string, see Overview: Change Tokens, Object Types, and Change Types. If null is passed, all items in the list are returned.
contains
A Contains element that defines custom filtering for the query and that can be assigned to a System.Xml.XmlNode object, as in the following example.
<Contains> <FieldRef Name="Status"/> <Value Type="Text">Complete</Value> </Contains>
This parameter can contain null.
In simpler terms, this can be a snippet of CAML to add an additional filter to the request.
Note
Contrary to the statements above about passing nulls for changeToken and contains, you cannot do so from the client (it may be allowable in server side code, but I’m not sure). If you pass an empty node in your SOAP request, the request will fail. SPServices – as of version 2013.02 – will handle empty values for you. This is the bug I fixed.
GetListItemChangesSinceToken Returns
In your first call to GetListItemChangesSinceToken, you won’t have a token yet. Or, you may choose not to pass a token that you do have. In that case, you get a robust amount of information returned to you.
The simplest call looks like this:
$().SPServices({
operation: "GetListItemChangesSinceToken",
listName: "Sales Opportunities"
});
Here, I’m just asking for the information for the Sales Opportunities list; I’m not providing any parameter values at all. On that first call, the results will look something like this (inside the SOAP envelope and GetListItemChangesSinceTokenResult):
<listitems xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask">
<Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635243604504600000;33126">
<List DocTemplateUrl="" DefaultViewUrl="/Intranet/JQueryLib/Lists/Sales Opportunities/AllItems.aspx" MobileDefaultViewUrl="" ID="{37920121-19B2-4C77-92FF-8B3E07853114}" Title="Sales Opportunities" Description="" ImageUrl="/_layouts/images/itgen.gif" Name="{37920121-19B2-4C77-92FF-8B3E07853114}" BaseType="0" FeatureId="00bfea71-de22-43b2-a848-c05709900100" ServerTemplate="100" Created="20090825 06:24:48" Modified="20131216 03:52:25" LastDeleted="20120627 07:54:25" Version="362" Direction="none" ThumbnailSize="" WebImageWidth="" WebImageHeight="" Flags="612372480" ItemCount="7" AnonymousPermMask="0" RootFolder="/Intranet/JQueryLib/Lists/Sales Opportunities" ReadSecurity="1" WriteSecurity="1" Author="3" EventSinkAssembly="" EventSinkClass="" EventSinkData="" EmailInsertsFolder="" EmailAlias="" WebFullUrl="/Intranet/JQueryLib" WebId="42f65d3f-343d-4627-a9a3-abf3d4d6491f" SendToLocation="" ScopeId="c8b78fea-7952-433a-be20-cda628ea6cbb" MajorVersionLimit="0" MajorWithMinorVersionsLimit="0" WorkFlowId="" HasUniqueScopes="False" AllowDeletion="True" AllowMultiResponses="False" EnableAttachments="True" EnableModeration="False" EnableVersioning="False" Hidden="False" MultipleDataList="False" Ordered="False" ShowUser="True" EnableMinorVersion="False" RequireCheckout="False">
<Fields>
<Field ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Type="Text" Name="Title" DisplayName="Title" Required="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="Title" FromBaseType="TRUE" ColName="nvarchar1"/>
<Field Name="Lead_x0020_Source" FromBaseType="FALSE" Type="MultiChoice" DisplayName="Lead Source" Required="TRUE" FillInChoice="TRUE" ID="{0ab3481c-a7b4-432f-8292-c1617744a167}" Version="6" StaticName="Lead_x0020_Source" SourceID="{f4ebb20a-91e0-4306-997d-208e1d1920b7}" ColName="ntext3" RowOrdinal="0">
<CHOICES>
<CHOICE>Newspaper Advertising</CHOICE>
<CHOICE>Web Site</CHOICE>
<CHOICE>Personal Referral</CHOICE>
</CHOICES>
</Field>
<Field Type="Currency" DisplayName="Potential Value" Required="FALSE" Decimals="2" LCID="1033" ID="{e9058e0c-b2e8-4af6-882c-9551a5f21451}" SourceID="{f4ebb20a-91e0-4306-997d-208e1d1920b7}" StaticName="Potential_x0020_Value" Name="Potential_x0020_Value" ColName="float1" RowOrdinal="0" Version="1">
<Default>0</Default>
</Field>
...
<RegionalSettings>
<Language>1033</Language>
<Locale>1033</Locale>
<AdvanceHijri>0</AdvanceHijri>
<CalendarType>1</CalendarType>
<Time24>False</Time24>
<TimeZone>300</TimeZone>
<SortOrder>2070</SortOrder>
<Presence>True</Presence>
</RegionalSettings>
<ServerSettings>
<ServerVersion>12.0.0.6421</ServerVersion>
<RecycleBinEnabled>True</RecycleBinEnabled>
<ServerRelativeUrl>/Intranet/JQueryLib</ServerRelativeUrl>
</ServerSettings>
</List>
</Changes>
<rs:data ItemCount="7">
<z:row ows_Attachments="1" ows_LinkTitle="Tue Feb 14 15:58:39 EST 2012" ows_Modified="2013-12-11 10:04:38" ows_Author="3;#Marc D Anderson" ows_Lead_x0020_Source=";#Newspaper Advertising;#" ows_Potential_x0020_Value="12.6100000000000" ows_Lead_x0020_Date="2013-04-20 00:00:00" ows_Country="1;#United States" ows_Region="1;#Northeast" ows_State="2;#Rhode Island" ows_City="706;#Providence" ows_Course_x0020_Title="1;#ss" ows_System="3;#Alienware" ows_StateID="North Dakota" ows_Domain="7;#baloney.com" ows_Notes="<div></div>" ows_Web_x0020_Service_x0020_Operatio="13;#GetList" ows_Radio_x0020_Buttons="C" ows_Sales_x0020_Rep="104;#Marc Anderson;#89;#Eric Mullerbeck;#91;#Renzo Grande" ows_a_x0020_b_x0020_bcb_x0020_n="" ows_CalcTest="float;#151.320000000000" ows_CalcTestDate="datetime;#2009-08-25 14:24:48" ows_CalcTestYesNo="boolean;#1" ows_YesNo="1" ows_UpdateList_Field_Sun_x0020_Oct_x="lk;k;lk;lk" ows__Level="1" ows_UniqueId="5;#{3345A2C2-0B2D-4D79-9CB8-4FEFA993F5FA}" ows_FSObjType="5;#0" ows_Created_x0020_Date="5;#2009-08-25 14:24:48" ows_Title="Tue Feb 14 15:58:39 EST 2012" ows_Created="2009-08-25 14:24:48" ows_ID="5" ows_owshiddenversion="256" ows_FileLeafRef="5;#5_.000" ows_FileRef="5;#Intranet/JQueryLib/Lists/Sales Opportunities/5_.000" ows__ModerationStatus="0"/>
...
</rs:data>
</listitems>
Yes, that is a heck of a lot of XML, but it’s not even all of it (note the ellipses). This may seem like overkill in many cases, and it may well be. In those cases, GetListItems is still the best way to go. But in the cases where you want to build an SPA, GetListItemsWithToken may be the ticket.
Like I said, you get a lot of robust information. Here are some more details on what’s there:
- A bunch of info about what frequency and size requests are allowed. Generally you won’t need to care about these values, but they are there when you get to that point.
- AlternateUrls – These are the URLs for each access path you might take to your list. Sweet! (The security folks will probably blanch at this.)
- In any call to GetListItemChangesSinceToken we get a new value for LastChangeToken whether we have passed a changeToken or not. This token isn’t something we’re supposed to pick apart or understand; it represents a certain point in the database life. If you really want to understand what the pieces of the token mean, you can check out the article Overview: Change Tokens, Object Types, and Change Types. You’ll want to grab that value, as you will use it in subsequent calls. You can decipher what each part of it means from the documentation, but you really shouldn’t care. Basically, it’s *like* a timestamp, but in database terms.
- The full list schema. You may not ever need this, but it’s great in cases where you find yourself doing a GetList/GetListItems pair of calls to set up further work in your script.
- Server information (lines 17-31 above). This is really cool. I’ve never been able to figure out a consistent way to determine the language and locale settings, the time zone offset, etc. from the client, but here they are. Keep in mind that these are the *server* settings, not the client settings.
- All of the list items. With large lists (not so large that they hit the 5000 item throttling limit – that’s another story entirely, and another post), you probably don’t want to get all of the list items. That’s where the contains parameter comes in handy. You might make the first call specifying that you only want the item with ID=1 or only items where the Title is null (generally this will result in no items) or something.
In the next call, you’re probably going to pass in the LastChangeToken value from the first call:
$().SPServices({
operation: "GetListItemChangesSinceToken",
listName: "Sales Opportunities",
changeToken: "1;3;37920121-19b2-4c77-92ff-8b3e07853114;635243604504600000;33126"
});
If there have been no changes, the results will look something like this. Nice and simple.
<listitems MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask" xmlns:rs="urn:schemas-microsoft-com:rowset"> <Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635175524963570000;31260"> </Changes> <rs:data ItemCount="0"></rs:data> </listitems>
If there have been changes to items in the list, the results will look something like this:
<listitems MinTimeBetweenSyncs="0" RecommendedTimeBetweenSyncs="180" MaxBulkDocumentSyncSize="500" AlternateUrls="http://204.144.120.200/,http://www.sympraxisconsulting.com/,http://172.29.4.39/,http://sympraxisconsulting.com/" EffectivePermMask="FullMask" xmlns:rs="urn:schemas-microsoft-com:rowset">
<Changes LastChangeToken="1;3;37920121-19b2-4c77-92ff-8b3e07853114;635175524963570000;31260"> </Changes>
<rs:data ItemCount="2">
<z:row ows_Attachments="0" ows_LinkTitle="MyItem1" ows_MetaInfo="3;#" ows__ModerationStatus="0"
ows__Level="1" ows_Title="MyItem1" ows_ID="3" ows_owshiddenversion="2"
ows_UniqueId="3;#{9153FDD3-7C00-47E9-9194-956BB20AAA8D}" ows_FSObjType="3;#0"
ows_Created_x0020_Date="3;#2007-08-31T21:34:59Z" ows_Created="2007-08-31T21:34:59Z"
ows_FileLeafRef="3;#3_.000" ows_FileRef="3;#sites/MyWebSite/Lists/MyList/3_.000"
ows_ServerRedirected="0" />
<z:row ows_Attachments="0" ows_LinkTitle="MyItem2" ows_MetaInfo="5;#" ows__ModerationStatus="0"
ows__Level="1" ows_Title="MyItem2" ows_ID="5" ows_owshiddenversion="3"
ows_UniqueId="5;#{5BDBB1C0-194D-4878-B716-E397B0C1318C}" ows_FSObjType="5;#0"
ows_Created_x0020_Date="5;#2007-08-31T21:43:23Z" ows_Created="2007-08-31T21:43:23Z"
ows_FileLeafRef="5;#5_.000" ows_FileRef="5;#sites/MyWebSite/Lists/MyList/5_.000"
ows_ServerRedirected="0" />
</rs:data>
</listitems>
If there have been any item deletions, the Changes node will look a little different (you may see deletes, update, and adds depending on what’s been going on and how long it’s been since you asked):
<Changes LastChangeToken="1;3;641d61d7-b03e-4078-9e6c-379fa0208d6f;635243635233730000;33127"> <Id ChangeType="Delete">1</Id> </Changes>
This is just about the only place I know of where you can get information about deletes from the client side. Generally the only way to “catch” deletes is to write an event receiver to run on the server. Of course, you’ll only see the deletes since the last time you passed in a token value, but it allows you to reflect those deletes on the client.
If there have been any changes to the list settings (schema), we’ll get the schema again before the changed items. The results in this case look like what we get back from GetList and from the first call above. We receive everything we need to know to work with the list. For this reason, we would generally make a call to GetListItemChangesSinceToken first to get the list schema and all of the list items – at least those that match our filters – and then use either GetListItemChangesSinceToken, GetListItems, or GetListItemChanges for subsequent calls for our SPA.
Conclusion
As you can see, GetListItemsWithToken is incredibly powerful and returns a wealth of information. In fact, I’m not sure that you could get all of this from a REST call, but probably from CSOM. These old, crufty SOAP Web Services still have their place in the world.
Even better, they can provide you with what you need to get going on SPAs of you own. Hopefully you’re starting to see the possibilities.
Marc,
Great post. I learned a few things about what is returned which I had not noticed in the past (alternate urls, locale info) .
Like you alluded to, this operation is now my favorite and I now use it almost exclusively over the old trusty GetListItems. Its very powerful and allows you to build some very sophisticated data driven UI’s using only client side technologies.
Re: Deletes From the Query
These are not actually deletions of items from the List… Rather, they are items that no longer matches the criteria used with the operation. Example: if you have an initial call to GetListItemChangesSinceToken for Tasks with a Status of ‘In Progress’ and someone else moves a Task from ‘In Progress’ to ‘Completed’, that task will be reported as “deleted” the next time GetListItemChangesSinceToken queries the list with the change token.
The reverse is also true. If any other items fall within the defined criteria since the last token, they will be returned in the response. The “sinceChange” is really against the Filter (Query) and not the entire List.
These two behavior are what sold me on this Operation becoming my new favorite. They are invaluable when wanting to keep client side data up to date with changes going on by others. The only thing I have not yet tested is if the behaviors I mention above (the deletion/addition) holds true when using Paging. I wonder if this Operation is truly that good and also keeps track of the pages that are returned and reports additions and deletions for that page only.
/Paul
Really useful series Marc, thanks! Fixing that bug in SPServices is really going to pay off methinks :-)
Paul – useful clarification, so this means we can build SPAs that are not just ‘list change’ responsive but ‘list change for context’ responsive so that each user will get a view on things that is relative to their activity and accessing context so that we can more accurately represent why things might be changing as well as just the fact that they have. That’s really powerful, especially if it transpires that it holds true for paging as well!
– rob
Paul and I went back and forth a bit about how this operation actually works elsewhere, and we reckoned that the two of us may be the only people who have ever tried to fully figure it out. Given that, our observations are simply attempts to understand That Which Is Not Documented.
However you look at it, GetListItemChangesSinceToken is an incredibly useful operation, and one which Paul and I will continue to dive into.
It *is* true that the operation will return the fact that an item is deleted. You can test this simple situation by calling it repeatedly while deleting an item and watching the results. This will factor into the demo that I am building for this series. Stay tuned!
M.
Thanks for the post Marc – a useful tool! Are you working with the XML the whole time on your SPA or converting to JSON to work with the data? If JSON, will your SPXMLtoJSON method work to get a JSON object for the token or have you written your own code to parse? I’ll stay tuned for the rest of the series if you’re going to address later – thanks!
Seph:
I sometimes use SPXmlToJson and sometimes just build out something on the fly. As usual, it depends on what I’m trying to accomplish.
The SOAP Web Services *always* return XML, so there’s overhead to convert to JSON if you don’t need to do it. You’ll need to parse the XML to get the token. That would look something like this in jQuery:
var changeToken = $(pledgesChangesPromise.responseXML).find("listitems Changes").attr("LastChangeToken");Note that I’ve pulled this line from inside a bunch of script so it’s a little out of context. It’ll be a part of the demo that I’ve got in the works.
M.
Awesome, Thanks! I currently use ListData.svc endpoints for a lot of my code which allow for direct JSON gets – this works well for frameworks that use JSON.
While using SPXmlToJSON does impose a cost, it would allow me to more easily plug and play SPServices and this ChangesWithToken capability, which could be extremely useful for two-way data binding in SPAs. Thanks again for the reply.
Thanks much – I used your sample to make a “slicer” similar to Excel’s – it shows a Sharepoint list next to a list of the available values. When you click the value, it toggles whether it is visible in the list.
We’re trying it on iphone. There seems to be a bug in the iphone browser where it chokes on the null verification. I had to put this line into a try-catch block to make it work.
thisField = (typeof thisField !== “undefined”) ? thisField.replace(/(^[\s\xA0]+|[\s\xA0]+$)/g, ”) : null;
Hello Marc,
Thanks and question: We are trying to use the CAMLQuery but from the below only the last one seems to work:
CAMLQuery: “CalorieConsumption”,
CAMLQuery: “2084”,
CAMLQuery: “0”,
Can you advise?
Kind regards,
Mario
Hello Marc,
I have to excuse myself. That the last did work put me on the wrong track.
I had to add:
CAMLQueryOptions: “”,
Kind regards,
Mario
Great info! Are you still performing this work within SPD 13 or does this require going down the path of Add-ins (app web)? Also, are there any limitations with building single-page applications using SharePoint Designer that you know of?
Text and date columns I can wrap my head around but I’m particularly curious how I might be able to implement a people-picker type of control to allow users to look up valid accounts prior to my insert or update of the selected user’s ID integer.
Thanks!
JPL