Create a Simple SharePoint 2013 Employee Directory on Office365 – Part 4 – Search Schema

In the prior posts in the series, we’ve seen how to set up a page for our employee directory and then create Display Templates to render the information we want for each person, along with a nice alphabetical filtering capability.

Up to this point, things have worked pretty much the same as they would in an on premises installation of SharePoint. With Office365, though, this is where things get a lot more complicated. I want to thank my search guru Mikael Svenson (@mikaelsvenson) for his assistance with this part of the work. This is another instance with SharePoint where the steps should be simple, but they aren’t when you add Office365 into the mix: it only gets harder.

In the listing we’ve made so far, we’ve had little trouble displaying the User Properties we want to see in the directory. Adding in some slicing and dicing requires a bit more fancy footwork. After all, if we can’t find people, it’s not much of a search-based solution, is it?

Each property we want to use in a filter we build needs to be “Sortable”. If we want to use a property in a refiner – those lists of values on the left side of the Search Results page – those properties must be “Refinable”.

For some unfathomable reason, out of the boxbthe LastName property is not “Sortable”, nor is it “Refinable”.  The FirstName is “sortable”, but to me it’s a lot less likely that you’d want to sort or filter on FirstName than LastName. If we want to sort by Lastname, we’re out of luck. In Ari Bakker’s (@aribakker) post that shows how to set up a simple a Employee Directory on SharePoint 2013: How to: Create a Simple SharePoint 2013 People Directory, he shows how to tweak the Lastname property to make it sortable.

If you go to the Search Schema settings (Admin / SharePoint / search / Manage Search Schema), you’ll see that this is the case.

2015-02-06_14-45-42

In Office365, we’re not able to change the attributes of the out of the box User Profile properties, though. If we try to, they are all simply grayed out; we can’t touch them. This is where our path diverges from on premises installs.

On Office365, there is a very big set of dummy properties named RefinableString00, RefinableString01, etc. There are 100 of these String properties. There are also sets for Date (20), Decimal (10), Double (10), and Int (50). If you need any more of any of these, you’re stuck, so use them wisely.

Search Schema - Refinable Strings

Because the Lastname is a a string-valued property, we’re going to use one of the RefinableString dummy properties. What we do is map the RefinableString property to a crawled property. Here I’ve chosen RefinableString00 because I haven’t used it yet. Here are the steps to set up the mapping:

  • Click on the RefinableString00 property in the search schema listing
  • Scroll down to the section for Mappings to crawled properties
  • Click on the Add a Mapping link
  • Find People:LastName by typing “Lastname” in the search box and clicking “Find”

2015-02-06_14-52-51

 

  • Select the People:LastName property and click OK
  • You can only map to one Crawled Property, even though the UI will allow you to select several. As much as I wanted to include People:SPS-PhoneticLastName to match the LastName Managed Property, I couldn’t. I had to settle for just People:LastName, (which should be fine).

Lastname Mapped Properties

  • Scroll to the Alias setting and give the property a name you’ll recognize. I’ve used LastnameSortable.

Add Alias

  • Save the RefineablerString00 property by clicking OK

Now you still have a property named RefinableString00, but it has an alias of LastnameSortable (if you used the same name as I did) and it is mapped to the People:LastName property, meaning that RefinableString00 will get the same values as People:LastName.

RefinableString00 Configured

Perfect, right? Now we can just use that LastnameSortable property in our slicing and dicing tools and we’ll be all set!

Not so fast, Kemosabi. On Office365, we have no control over search crawling. We can’t just fire off a crawl to update the index like we can on premises. (In either case, we have to be admins, but that’s not the difference here.)

A User Profile will only be re-indexed if a value in that profile changes. For example, if I change my MobilePhone or a new value syncs over from Active Directory, then the next crawl will pick up that change and the value will be available in the search index. We’ve mapped the People:LastName property to the RefinableString00 property, but since no User Profiles were changed in the process, it makes no difference. We can’t just push the re-index button on Office365.

The only way we (Mikael, and therefore I) know to change every User Profile so that it will be indexed is to run a Powershell script that “touches” every profile. This is down and dirty stuff, folks, and not for the squeamish. You might want to enlist your local Admin Superhero to help you with this part.

2015-02-06_14-58-03Mikael built a script that loops through all of the User Profiles; copies the SPS-Birthday property value; sets the SPS-Birthday property to an arbitrary value; saves the profile; sets the SPS-Birthday back to the original, saved value; and saves the User Profile. Yes, for every single User Profile in the User Profile Store.

The one problem I had was that SPS-Birthday was almost never available in the User Profiles in the organization I was working with. Mikael adapted his Powershell script to also work with Department, which ought to be there more often. For the company I originally muddled through this with and their 100 or so employees, this wasn’t a big deal; the Powershell script ran through in a few minutes. If you are in a larger organization, the script might take hours and could possible timeout along the way. That said, it will work. Eventually.

I’m not going to go into all of the details on how to run the Powershell script. Instead, head on over to Mikael’s post “How to trigger re-indexing of user profiles in SharePoint On-line” and follow his instructions.

Once you’ve run the script, you’ll need to wait for some period of time – we don’t have any way of knowing when these crawl jobs actually happen. Experience shows that this will be 2-8 hours, but it can depends on the load in your tenant’s hardware. Once the values are indexed, you can start to use them in your Employee Directory.

The next step is to add some of the slicing and dicing capabilities. We couldn’t do that before we set up the RefinableString00 aka LastnameSortable property. If we had tried to use the Lastname property, we’d just get errors in the page I know this from experience). Errors that tell us precious little about what the actual problem is. Correlation ID!

Ari’s post shows some slick additional sorting capabilities that it would be nice to add at the top of the page. Sorting by Lastname or Firstname might make finding the person in the middle of the list by default a little easier to find. In the next article in the series, I’ll show you how to set those sorting capabilities up.

Synchronous XMLHttpRequest Warning with SPServices and Recent Browsers

If you’re working in the latest versions of Chrome (~40+) – and maybe Firefox – and you use SPServices, you may start to see an warning:

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help, check http://xhr.spec.whatwg.org/.

Vigilant SPServices user frankhale reported this to me the other day in the SPServices discussions on Codeplex, which is – at least at the moment – the best place to get help with SPServices. You can also add an issue on Github (sympmarc / SPServices) if that’s your fancy.

The warning is thrown in jQuery, not SPServices, but it’s an SPServices issue.

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience.

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience.

First off, it’s a warning, not an error. Your code will continue to work in the near term, at least.

Because of some backward compatibility concerns, I’ve left a few synchronous calls internally to SPServices in place. Those calls are what are causing the warning. In particular, it is most likely that the warning is being thrown for you because of a synchronous call on the $().SPServices.SPGetCurrentSite function. The reason for this is that early in the SharePoint 2007 days it was difficult to determine the current site without a call to the Webs.WebUrlFromPageUrl operation. Unfortunately, it seems that I’m making that call in SharePoint 2010 and 2013, even though the current site is available in JavaScript variables.

So, the bottom line is: “Carry on.” I’ll get a fix into the next release of SPServices for this. In the meantime you should be fine.

Get the InternalName for a SharePoint List Column

This comes up all the time in the SPServices Discussions on Codeplex. When you are making most Web services calls to SharePoint, you have to use the InternalName rather than the DisplayName. This is true whether you are using SPServices, CSOM, or REST. You may see the InternalName referred to as the StaticName as well. They are a little different, but that doesn’t matter here.

When you create a column in a list or library, you give it a DisplayName. That name may be encoded to some degree – depending on the characters you use – to become the InternalName. The InternalName never changes, no matter how many times you change the DisplayName. (This is why we end up with abominations like a column with an InternalName of “Salary” which has a DisplayName of “Name of Customer”. Rapid prototyping can paint us into these messes if we aren’t careful.)

The DisplayName is the name you’re used to seeing for columns in lists and libraries. You see these names all over the place: on list forms, in list views, etc.

There are a few [relatively] easy ways to get the InternalName if you only know the DisplayName.

  • Go to the List Settings and hover over each column’s name. You’ll see the InternalName after “&Field=” in the Status bar of your browser. Note that the name will be double encoded. For example, if a field’s DisplayName is “My Country”, you’ll see “My%5Fx0020%5FCountry”. The actual internalName is “My_x0020_Country”.
  • Call GetList, which will return the list’s schema and inspect the results in a debugger. Example:
    $().SPServices({
        operation: "GetList",
        listName: "MyList"
    });
    
  • Call SPGetStaticFromDisplay, with the list name and column name and SPServices will look up the Internal Name for you. Example:
    var thisStaticName = $().SPServices.SPGetStaticFromDisplay ({
      listName: "MyList",
      columnDisplayName: "My Country"
    });
    

Here’s a little table with some examples of how some column names end up:

DisplayName InternalName Double Encoded Name Notes
Title Title Title “Title” doesn’t have any special characters in it, so all three versions are the same.
My Country My_x0020_Country My%5Fx0020%5FCountry The space is encoded as _x0020_
The underscores are encoded as %5F
My_Country My_Country My_Country No extra encoding! Underscores are not special characters.
My-Country My__x002d_Country My%5Fx002d%5FCountry The dash is encoded as _x002d_
The underscores are encoded as %5F

 

Using RequireJS to Load the Right Version of jQuery Depending on Internet Explorer Version

Here’s a cool trick you can use with RequireJS. I found it in a post by @rnsloan called Conditionally Loading jQuery 2.x.

Since environments with SharePoint can often have a mixed bag of browser versions, this conditional setting lets us load jQuery 2.x when the browser can handle it (generally IE 9+) or jQuery 1.x when it can’t. The function document.addEventListener is undefined in IE8 and earlier – sometimes called oldIE – and defined in IE9+ – also known as modern.IE – so it’s a simple little test that ought to work.

If you don’t know why this matters, check out the Browser Support page on the jQuery site.

requirejs.config({
  paths: {
    "jquery": (document.addEventListener) ?
      ['//ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min','jquery-2.1.3.min']
      :
      ['//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min','jquery-1.11.1.min']
    }
});

Note: The versions of jQuery above were current as of this writing, but may well have already moved on by now.

Calculate How Much Web Storage You’re Using

This is a quick one, but it’ll be a post I return to over and over again.

When you use Web storage (a.k.a. DOM storage) – localStorage or sessionStorage – to cache data, you’ll often want to know how much you’ve used already. Each browser gives you a different amount of storage to work with, and you don’t want to run out for no good reason.

You might be surprised what the sites you visit are up to these days. Cookies are old hat; they just don’t give us enough to work with, as they are capped at 4k of data each.

From Wikipedia:

Web storage provides far greater storage capacity (5 MB per origin in Google Chrome, Mozilla Firefox, and Opera; 10 MB per storage area in Internet Explorer; 25MB per origin on BlackBerry 10 devices) compared to 4 kB (around 1000 times less space) available to cookies.

If you need to know how long each browser has offered Web storage, check out the Web storage page on Can I use. If you’d like to understand Web storage in a more practical sense, check out my post Caching SharePoint Data Locally with SPServices and HTML5’s Web Storage.

I’ve adapted some JavaScript I found out in a post on StackOverflow to display the storage used by each object in both localStorage and sessionStorage for the current origin

var storageTypes = ["localStorage", "sessionStorage"];
var x, log = [], total = 0;

for(var i=0; i < storageTypes.length; i++) {
	var thisStorage = window[storageTypes[i]];
	log.push("Statistics for " + storageTypes[i]);
	log.push("--------------------------------");
	for (x in thisStorage) {
		log.push(x + " = " + ((thisStorage[x].length * 2) / 1024).toFixed(2) + "KB / " + ((thisStorage[x].length * 2) / 1024 / 1024).toFixed(2) + "MB");
		total += thisStorage[x].length * 2;
	};
	log.push("Total = " + (total / 1024).toFixed(2) + "KB / " + (total / 1024 / 1024).toFixed(2) + "MB");
	log.push("================================");
	total = 0;
}
console.log(log.join("\n"));

The results will look something like those below, which I got by running the code in a console while I was on a Yammer external network.

Statistics for localStorage
--------------------------------
yj-chat-contact-list-1328414-1512241636 = 0.18KB / 0.00MB
yj-chat-contact-list-428797-1488346713 = 0.18KB / 0.00MB
Total = 0.36KB / 0.00MB
================================
Statistics for sessionStorage
--------------------------------
Total = 0.00KB / 0.00MB
================================

Here are the results on the home page of my Office365 tenant:

 Statistics for localStorage
--------------------------------
Ribbon.Document = 0.04KB / 0.00MB
Ribbon.Read = 0.04KB / 0.00MB
Ribbon.WikiPageTab = 0.05KB / 0.00MB
SPAnimationEnabled = 0.00KB / 0.00MB
SPMySiteLinks = 0.73KB / 0.00MB
SPSuiteLinksDate = 0.11KB / 0.00MB
SPSuiteLinksJson = 46.97KB / 0.05MB
SPSuiteLinksLanguage = 0.01KB / 0.00MB
SPSuiteLinksUserKey = 0.08KB / 0.00MB
SPSuiteNavHeight = 0.01KB / 0.00MB
SPSuiteThemeInfo = 0.74KB / 0.00MB
ShellCacheIndicator = 0.07KB / 0.00MB
Total = 48.86KB / 0.05MB
================================
Statistics for sessionStorage
--------------------------------
SPAnimationEnabled = 0.00KB / 0.00MB
SPCacheLogger0 = 0.26KB / 0.00MB
SPCacheLogger1 = 0.29KB / 0.00MB
SPCacheLogger2 = 0.29KB / 0.00MB
SPCacheLogger3 = 0.27KB / 0.00MB
SPCacheLogger4 = 0.28KB / 0.00MB
SPCacheLogger5 = 0.26KB / 0.00MB
SPCacheLogger6 = 0.26KB / 0.00MB
SPCacheLogger7 = 0.67KB / 0.00MB
SPCacheLoggerSize = 0.00KB / 0.00MB
SPSuiteLinksCached = 0.01KB / 0.00MB
SPUserPhotoToken = 0.03KB / 0.00MB
UserPhotoToken = 0.03KB / 0.00MB
Total = 2.64KB / 0.00MB
================================