Unexpected Error from GetListItemsChangesSinceToken Using SPServices

This is another post in the vein of “it happened to me and it took me a long time to figure it out, so why should you go through the same pain”?

I’ve got an application running at a client that sits on top of SharePoint 2007. It uses KnockoutJS and lots of other modern Web development goodies, even though we’re on a very old SharePoint version. In fact, most of the time, I’m debugging in Internet Explorer 8, because that’s still the corporate standard.

Fun stuff, right? Well, the application serves them great, and I still get to have fun because it’s modern development.

Over the last few weeks, I’ve been trying to figure out why some of my calls to the SPServices function SPGetListItemsJson haven’t been returning what I expected. The calls were happening and I was getting valid responses in Firefox and Chrome, but in IE8, the data wasn’t all there. I was using localStorage to store a local cache of the data so I had data from weeks back there, but more recent data wasn’t coming back from my calls. (See Caching SharePoint Data Locally with SPServices and HTML5’s Web Storage for information on storing data locally in the browser cache.)

You can say what you want about my skills, but debugging Web Services calls in IE8 is well-nigh impossible. There’s simply no straightforward way to see the packets going out and back. Adding to the difficulty, Fiddler won’t run properly in this environment and the machine I remote into is locked down so that I can’t update it. Yup, three hands tied behind my back.

I finally got into the bowels of one of the calls using the IE8 Developer Tools (ick) and grabbed the innerHtml from the call. Here’s what I saw:

<xml:namespace prefix = soap />
<xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <?XML:NAMESPACE PREFIX = [default] http://schemas.microsoft.com/sharepoint/soap/ NS = "http://schemas.microsoft.com/sharepoint/soap/" />
    <GetListItemChangesSinceTokenResponse xmlns="http://schemas.microsoft.com/sharepoint/soap/">
      <GetListItemChangesSinceTokenResult>
        <listitems xmlns:rs="urn:schemas-microsoft-com:rowset">
          <Changes>
            <Id ChangeType="InvalidToken">
            </Id>
          </Changes>
          <?xml:namespace prefix = rs />
          <rs:data ItemCount="0">
          </rs:data>
        </listitems>
      </GetListItemChangesSinceTokenResult>
    </GetListItemChangesSinceTokenResponse>
  </soap:Body>
</soap:Envelope>

Note the highlighted lines. I’d never run into that response before, so off the to MSDN documentation for GetListItemsChangesSinceToken, which underlies the SPGetListItemsJson function.

The basic documentation page for GetListItemsChangesSinceToken doesn’t say anything about InvalidToken and I’d never run into it before, so I had no idea what it meant. But in an article marked as

This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

(Hey, we’re working with SharePoint 2007 here) there’s an explanation of several other possible responses to calls to the function:

Table 4: Change Events

In the first two cases above, the client should ignore other changes and do a full reconciliation of the list.

(See: https://msdn.microsoft.com/en-us/library/cc264031(v=office.12).aspxThe )

Like I said, I’ve never seen this response, so I had never built anything into SPGetListItemsJson to handle it.

The issue really came down to the fact that I was using localStorage rather than sessionStorage and I’d not built in a capability to occasionally throw away the localStorage cache and fully refresh it. Yes, my bad.

The quick fix was to switch to storing the cache in sessionStorage instead. On a lot of levels, that’s a good idea, anyway. Using localStorage means that I have a persistent cache, but if something goes wrong with that cache – corruption, or something exactly like this – there’s no quick fix. By switching to sessionStorage, I’ve made the cache volatile in that it will be purged at the end of each browser session. Now when someone has a problem with their cache (something they won’t ever understand from a user perspective), I can simply say “Please shut down your browser and restart. Let me know if that fixes the problem.”

I hope this helps someone else along the way.