Single-Page Applications (SPAs) in SharePoint Using SPServices – Part 2 – GetListItems

As I mentioned in the first part of the series, we have several workhorse operations at our disposal in the SOAP Web Services with which we can build our Single-page Applications (SPAs). (Of course, which API you use is somewhat unimportant. All of the techniques here should work using REST or CSOM, too. I’m just choosing to focus on SPServices and the SOAP Web Services.) There are many more SOAP operations that can prove useful, of course, but at first we’ll focus on the key set below which help us get data out of or push data into SharePoint lists and libraries.

While each of these operations is documented on the SPServices Codeplex site, I’m going to repeat some of that documentation here for anyone who is learning about them for the first time.

[important]GetListItemChanges and GetListItemChangesWithToken do not work in versions of SPServices before 2013.02. I’ll talk more about this in the next part in the series.[/important]

First, let’s review the primary workhorse operation, GetListItems.

GetListItems

GetListItems is generally the first operation that people try out when they start working with SPServices. It does exactly what its name implies: it enables you to get items from a list. In this sense, lists and libraries are the same thing. All useful SharePoint content that is available through the UI is stored in lists, even much of the configuration data that we set up. Document Libraries are lists. Publishing pages are are in lists, too. They just happen to be in a special flavor of Document Library, which is a list, called Pages.

GetListItems takes a set of parameters that makes it extremely powerful. If you can imagine a way to pull data from a list, you can probably do it, or get very close and then process the data client side into the form you need.

What this operation doesn’t do for you is any sort of aggregation. Once you have the data client side, you can work on aggregation from there in your code. This means that it’s important to carefully think about the data volume you might be pulling from the list and whether munging through it on the client side is a good idea or not based on what you need to accomplish.

The available parameters are listed below.

[webURL]

If you’d like to get content from a list in a different site than the current one (a different context), this parameter is what you need. You can specify any valid path. I recommend relative paths wherever possible so that you don’t run into cross site scripting issues moving from one environment to another.

listName

You must specify a listName at the bare minimum, as this tells the Web service where to get the content for you. You can provide either the list’s GUID (“{3bf9d2f3-78ec-4e92-a4c0-7c5f7ed51755}”) or the list’s name (“Announcements”). Note also that if you use the GUID, you do not need to specify the webURL if the list is in another site.

viewName

By default, GetListItems retrieves the items which are displayed in the default view for the list or library. If you specify a view’s GUID for the viewName parameter, you can retrieve the items shown in any existing view. In most cases, I’d recommend specifying what you’d like to retrieve using the CAML parameters below, as views can be easily changed through the UI and you may not retrieve what you’d expect.

CAMLViewFields

In the CAMLViewFields, you specify which list columns you’d like to retrieve. By default, you’ll get the columns defined in the default view. In many cases, you’ll want to request different columns, and you should also request as few columns as possible in order to reduce the number of bytes coming across the wire.

As an example, this is what you would specify in the CAMLViewFields to retrieve the Title and Created By (Author) columns.

CAMLViewFields: "<ViewFields><FieldRef Name='Title' /><FieldRef Name='Author' /></ViewFields>",

The column names should be the InternalNames for the columns, which might look like “My_x0020_Column”, as special characters like spaces are encoded. In this case, the space becomes “_x0020_” because a space is ASCII character 20 (hexadecimal). See http://www.asciitable.com/ for more info.

The easiest way to identify the InternalName for a column is to go to the List [or Library] Settings, click on the column name, and look on the end of the URL, which will end with something like this in SharePoint 2007:

/_layouts/FldEditEx.aspx?List={37920121-19B2-4C77-92FF-8B3E07853114}&Field=Sales_x0020_Rep

or this in SharePoint 2013:

/_layouts/15/FldEdit.aspx?List={F3641DF3-80A2-4BBA-A753-E6BFB3FD98E4}&Field=ImageCreateDate

Your column’s InternalName is at the end of the URL after “Field=”.

Note that regardless what you specify in the CAMLViewFields, you will get some additional columns even though you don’t want them, including the item’s ID.

CAMLQuery

In CAMLQuery, you specify any filters or sorting you’d like to apply to the results. Yes, it’s CAML, and no, most people don’t like CAML much. I find it great because it’s an easy language to build up programmatically in SPServices, but that’s me.

The syntax for the CAMLQuerycan be complicated, but CAML is well-documented. Here’s a simple example:

CAMLQuery: "<Query><Where><Contains><FieldRef Name='Title'/><Value Type='Text'>and</Value></Contains></Where><OrderBy><FieldRef Name='Title'/></OrderBy></Query>",

This CAMLQuery will retrieve items which contain “and” in their Title and sort those items by Title.

CAMLRowLimit

CAMLRowLimit allows you to specify the number of items you would like to retrieve. You can specify any positive integer.

CAMLRowLimit: 10,

CAMLQueryOptions

There are a number of CAMLQueryOptions, some of which work more as advertised than others. I’ll leave it as an exercise for the reader to read up on the available options in the MSDN documentation. Note that the options listed on the GetListItems page are not complete; see the GetListItemChangesSinceToken documentation for a more complete list.

Notes for GetListItems

When I first started building SPServices, I named the parameters [CAMLViewFields, CAMLQuery, CAMLRowLimit, CAMLQueryOptions] differently than the actual parameter names, which are [viewFields, query, rowLimit, queryOptions]. I’ve since set it up inside SPServices so that you can use either name, but most people continue to use the naming I came up with originally with the CAML prefix.

If you specify any of the parameters [CAMLViewFields, CAMLQuery, CAMLRowLimit, CAMLQueryOptions] explicitly, you will override the default view. One trick I use frequently to force the override is to just pass 0 for the CAMLRowLimit, which tells the GetListItems to return all items. I’ve seen quite a few other people pass in a CAMLQuery like:

<Query><Where><Gt><FieldRef Name='ID'/><Value Type='Counter'>0</Value></Gt></Where></Query>

This says to retrieve all items where the ID is greater than zero, which is true for all items since IDs start at one and increase from there. That seems like more code than “0″ to me, though.

GetListItems supports paging, though I rarely take advantage of it. I find that it often makes more sense to request all of the relevant items and do the paging on the client side. This makes sense because the more “expensive” action is the request for data from the server. However, if you are requesting items from large lists and may not need the majority of them, be sure to use tight filters or paging for efficiency.

GetListItems Returns

The GetListItems operation returns XML, as do all of the SOAP Web Services operations. The results take the following form:

<listitems xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882"
   xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"
   xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema"
   xmlns="http://schemas.microsoft.com/sharepoint/soap/">
   <rs:data ItemCount="4">
      <z:row ows_Number_Field="6555.00000000000"
         ows_Created="2003-06-18T03:41:09Z"
         ows_ID="3" ows_owshiddenversion="3" />
      <z:row ows_Number_Field="78905456.0000000"
         ows_Created="2003-06-18T17:15:58Z"
         ows_ID="4" ows_owshiddenversion="2" />
         ...
   </rs:data>
</listitems>

There are some important things to note about the results:

  • There is an rs:data element which contains an ItemCount element, telling us how many items have been returned
  • Each of the returned items is contained in a z:row element
  • Each item’s columns are provided as attributes on the z:row elements
  • All column names are preceded with the prefix “ows_”
  • All values are text (everything in both directions is text)

Processing GetListItems Results

Processing the results from GetListItems is fairly straightforward, but there are a few nuances. Assuming you have a container in the page with its id set to announcementsContainer, the following code will create an unordered list (bullets) showing the titles of each announcement in the Announcements list.

var out = "<ul>";
$().SPServices({
  operation: "GetListItems",
  async: false,
  listName: "Announcements",
  CAMLRowLimit: 0,
  CAMLViewFields: "<ViewFields><FieldRef Name='Title' /></ViewFields>",
  completefunc: function (xData, Status) {
    var itemCount = $(xData.responseXML).SPFilterNode("rs:data").attr("ItemCount");
    $(xData.responseXML).SPFilterNode("z:row").each(function() {
      out += "<li>" + $(this).attr("ows_Title") + "</li>";
    });
    out += "</ul>";
    $("#announcementsContainer").html(out);
  }
});

Here I’m making the call to GetListItems to get all of the items in the Announcements list. (By passing a CAMLRowLimit of zero, I’m requesting all items.) When the results come back, I’m iterating through all of the items (the z:row elements), building up my out string to contain the list bullets. Finally, I set the innerHTML of the announcementsContainer to what I’ve built up in the out variable.

Conclusion

GetListItems is a workhorse, all right, but we need more for our SPA work. In the next three parts of the series, I’ll introduce its siblings: GetListItemChanges, GetListItemChangesWithToken, and UpdateListItems. These four operations together will help us to build a highly interactive SPA and provide valuable functionality for our users.

Rogue IT Horror Story vs. Opportunity

harmon.ieOne of the big players in the SharePoint space, harmon.ie, is currently running a contest to collect the best “Rogue IT Horror Story”. If you have a story and want a chance to win a Samsung Galaxy S IV or a Trip to the Microsoft SharePoint Conference 2014 in Las Vegas in March, by all means head on over and submit your story.

Rogue IT users aren’t intending to harm their companies, they’re just trying to find the most efficient ways to get their work done. But this ‘efficiency’ comes at a high price: a recent survey found that 27% of workers who went rogue — and used consumer or unsanctioned apps for work — reported immediate and direct repercussions to the tune of $2 billion in penalties, lost business, data leakage and clean-up costs.

Mark Fidelman has a post about the contest on EndUserSharePoint today which prompted me to comment and write this post.

The problem that I have with the phrasing of contest and the thinking around it is it sets up more of the us vs. them thinking that ruins IT’s reputation in any organization.

These “horror stories” shouldn’t be water cooler jokes for IT (the most common thing I see in large organizations). They should be seen as opportunities for IT to offer improved services and support. Instead, they are usually met with increased lockdowns and policing which simply causes more effort expended to go around IT.

Every well-intentioned person who circumvents the “right” way to accomplish something:

  • Is trying to get something done in performance of their job
  • Most likely has met with a “no” – or the dreaded “no budget” – response from IT
  • Doesn’t trust that IT will help them or understand their needs
  • Has turned to the best approach they can figure out

We technologists should see these instances as excellent opportunities, not horror stories. We should strive to help people in our organizations before anything ever gets near some potential $500 million loss (which I frankly question as anything more than a great story).

Customizing Search Refiners in SharePoint 2010

Search RefinersI wanted to add some custom refiners to each of the custom search result pages I’ve been working on in a project. The steps to do this aren’t complex, but since I needed to read a bunch of posts out there to get it just right, I figured I’d write it up.

Create Custom Managed Properties

Only Managed Properties are available for use as refiners. While Site Columns automatically become Crawled Properties, they must be added as Managed Properties to use them as refiners (or in scopes, etc.).

To do this, we go to Central Administration -> Application Management -> Manage Service Applications -> Search Service Application -> Metadata Properties and whatever new Managed Properties we need.

Note that we cannot use the existing ContentType Managed Property in the refiners. If we do, we get the error

Property doesn’t exist or is used in a manner inconsistent with schema settings.

In order to use the Content Type as a refiner, we must create our own Managed Property which mirrors the existing one. I created a new Managed Property called ContentTypeRefiner and mapped it to ows_ContentType(Text).

Perform a Full Crawl

After creating the Custom Managed Properties, we must do a full crawl so that the new Managed Properties will be in the active index.

To do this, we go to Central Administration -> Application Management -> Manage Service Applications -> Search Service Application [you may have named this differently when you created it] -> Content Sources. From the dropdown for the relevant Search Scope, choose Start Full Crawl.

A full crawl can take a significant amount of time depending on the amount of content you may have, so it’s best to create all of the new Managed Properties before kicking one off.

Add Managed Properties into Refiners

To add the new Managed Properties as refiners in the custom search results pages, we do the following:

  • Navigate to the search results page
  • Go into Edit Mode
  • In the Refinement Panel (left side of the page) select Edit Web Part
  • In the Tool Pane, under the Refinement section, edit the Filter Category Definition XML to include the new refiners
  • Uncheck the Use Default Configuration box, otherwise your changes will disappear!

Note that any of the existing refiners can be removed. The order in which the refiners are included in the Filter Category Definition determines the order in which the refiners are shown, if there is enough content which contains that refiner’s metadata.

Here’s an example of what the XML for your own refiners might look like. This is the XML I added for the ContentTypeRefiner Managed Property. Most of the attributes are pretty self-explanatory.

<Category Title="Content Type"
  Description="Type of resource"
  Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator"
  MetadataThreshold="5"
  NumberOfFiltersToDisplay="4"
  MaxNumberOfFilters="20"
  SortBy="Frequency"
  SortByForMoreFilters="Name"
  SortDirection="Descending"
  SortDirectionForMoreFilters="Ascending"
  ShowMoreLink="True"
  MappedProperty="ContentTypeRefiner"
  MoreLinkText="show more"
  LessLinkText="show fewer" />

Useful Reference Links

Finding the SharePoint 2007 / 2010 Thesaurus Files

If you go to TechNet to find out where the thesaurus files for search are so that you can add in your own synonyms, you may be as confused as I was earlier today. It only took me about 20 minutes to figure out, but if three or four people find this post, we’ll have saved enough time for lunch.

The TechNet article you want is Manage thesaurus files (SharePoint Server 2010), though the one for SharePoint 2007 (Edit a thesaurus file (Office SharePoint Server)) is pretty much identical.

In the article, it says

By default, SharePoint Server 2010 installs the thesaurus files for all supported languages at %ProgramFiles%\Microsoft Office Servers\14.0\Data\Office Server\Config. When a search administrator creates a Search service application, the search system automatically copies the thesaurus files from the installation location (including any thesaurus files there that an administrator has edited) to %ProgramFiles%\Microsoft Office Servers\14.0\Data\Office Server\Applications\GUID-query-0\Config, where GUID is the GUID of the new Search service application. The search system performs the same operation on every query server that is running the new Search service application. Thus there is a copy of each thesaurus file on each query server that is running that Search service application.

When I looked in %ProgramFiles%\Microsoft Office Servers\14.0\Data\Office Server\Config, well, there was no %ProgramFiles%\Microsoft Office Servers\14.0\Data\Office Server\Config. Instead, because my client had decided to change the location of the index to another drive, I have to figure out where that actually was. Here’s the trick.

In complex farms, you may have multiple Search Service Application, multiple indices, etc. but these steps should work in most cases.

  • In Central Administration, go to the Search Application -> Central Administration/ Manage service applications / Search Service Application (or whatever you called it)
  • At the bottom of the page, you’ll see a section called ‘Search Application Topology’
  • Click the Modify button and on the next screen look for the ‘Index Partition’ (you may have more than one)
  • Click on the ‘Query Component 0′ link and Edit Properties
  • The field called ‘Location of Index’ contains the root location for the thesaurus files

image

Looking in that folder, you should find folders that look something like this:

image

As noted above, the thesaurus files you want to work with are in the GUID-query-0\Config folder. In my case above, it’s E:\Data\SearchIndex\Office Server\Applications\0f78bae4-05b9-417f-b533-43326409dfcc-query-0\Config

Happy equivalency!

One side note: it boggles my mind that there is no UI to manage synonyms in the thesaurus, but there you go.