Create a Simple SharePoint 2013 Employee Directory on Office365 – Part 5 – Sorting & Refiners

This post in the series was sitting in my “outbox” for a long time. Sorry to people like Nigel (@Nigel_Price) who probably gave up on me!

In a company with 100 employees, clicking on once of the letters are the top of the page will probably narrow things down well enough that you’ll see everyone with that letter at the start of their last name on one page. If you have many more employees, you may end up with too many people to see very easily. And since we’re limited to 50 results in the Search Results Web Part, you may need to page and/or scroll a lot, depending on what properties you’ve decided to show for each person (one line vs. two lines or more, for example).

There are a couple of out of the box capabilities we can use to sort or filter our results a bit more. Because we’re using search to drive the directory, we have all of the native search capabilities that SharePoint gives us. Let’s take a look at adding some sorting and improving the refiners.

Sorting

In the Web Part Properties for the Search Results Web Part, there is a checkbox labelled “Show sort dropdown”. It’s in the Settings section (which is full of other options as well).

2015-02-26_12-04-17

By checking the box and adding some of our own JSON into the field below it, we can control what sort options are available. The JSON should be an array of sort options taking this form:

{
    "name": "Last name (A-Z)",
    "sorts": [{
        "p": "RefinableString00",
        "d": 0
    }]
}

The values break down like this:

  • name – The text value you’ll see in the dropdown
  • sorts – The parameters for the sort that apply when the option is selected
    • p – The managed property to sort on
    • d – [0 | 1] If 0, the assort is ascending, if 1 it is descending.

Note that for the Last name sort above, I’m using the RefinableString00 managed property I set up in Part 4. If we tried to do this with the LastName managed property we’d get an error because that property is not Sortable.

Here’s a pretty basic set of sorting options that should at least get you started. I’m giving you four pretty basic sort options:

  • First name (A-Z)
  • First name (Z-A)
  • Last name (A-Z)
  • Last name (Z-A)
[{"name":"First name (A-Z)","sorts":[{"p":"FirstName","d":0}]},{"name":"First name (Z-A)","sorts":[{"p":"FirstName","d":1}]},{"name":"Last name (A-Z)","sorts":[{"p":"RefinableString00","d":0}]},{"name":"Last name (Z-A)","sorts":[{"p":"RefinableString00","d":1}]}]

You should be able to take this and enhance it for your own needs.

Refiners

Next we probably want to adjust the refiners a bit. Out of the box, the Refiners Web Part will display properties that look to a machine like useful ones, but they may not be the most useful in your organization. You may want to show refiners that are part of the search results already or you may want to show refiners that aren’t visible in the results.

Changing which refiners you see and in what order works like this:

  • Edit the page
  • Put the Refinement Web Part into Edit Mode
  • Click on Choose refiners
  • Change the settings
  • Save it

The key action here is to add or subtract from the Selected refiners in the Choose refiners dialog. For each refiner, you can change the settings on the bottom of the page. Note that refiners use Display Templates as well. I’m not going to go into building Refinement Display Templates, but you can see them in _catalog/_masterpage/Display Templates/Filters.

In my case, I’ve got refiners for Department, JobTitle, PeopleKeywords, and BaseOfficeLocation. You will probably want some others which work for you.

Refinement configuration

You should see the impact of these changes immediately upon saving the page. Assuming that the properties you have selected have been indexed, that is.

What Next?

There are a number of different directions we could go from here. We could build a similar page to show a Site Directory [Nigel]. We could enhance the display of each person to show additional information about them or metrics about their recent activities. We could add as second alphabetical listing row below the first so that if you selected the letter B, you could the choose the next letter (only lit up if there are people lurking there). As with most functionality like this, coming up with the innovative ideas can be even harder than good execution on them. In this case, some of the limitations in Office 365 make it even more difficult that it should be.

I’d love to see:

  • Ability to trigger a full crawl of the User Profiles when we change the schema. It wouldn’t have to happen right away, but it would at least remove the need for Powershell.
  • Allow us to build a more intelligible set of “RefinableType” properties so that we can manage them better. If we create more than a handful of these special mappings – as most organizations will do over time – then management is going to be cumbersome at best. We can’t even search by the Alias names.

Then basic process here should work for many different use cases. As you’ve probably gleaned from this series, building good metadata and taxonomy isn’t just an academic exercise. The whole point is to make content easier to find, and a People Directory is no different: it’s a customized page that allows us to perform a certain used case in an optimal way. What “optimal” means is going to vary based upon the characteristics of your organization.

  • How big is it?
  • Why would people be looking for each other (beyond the simple phone list idea)?
  • What information is important for people to see vs. that which must be secured?
  • etc.

I hope you’ve found the series useful. If you’d like to know more or have other ideas, I’d love to hear about them in the comments below.

References

SPC Adriatics 2016 Follow Up

One of the best parts about my job is that I get to travel to exotic locales to talk about SharePoint. My most recent jaunt was to the fine country of Croatia to attend SPC Adriatics. Every conference has its own personality, and SPC Adriatics is know for being fun and filled with meat. Who wouldn’t go?

20160530_191516407_iOS

The triumvirate of organizers – Toni Frankola (@ToniFrankola), Adis Jugo (@adisjugo), and Nenad Trajkovski (@ntrajkovski) sure know how to put on an event. They were ably helped this year by the talented Branka Obuljen Štritof.

SPC Adriatics Speakers

Unfortunately, I was sick in bed for Day One of the conference and still off my game for Day Two when I did my two sessions. I apologize to everyone who attended my sessions for not turning in a better performance. I hope I’ll get another chance to visit Croatia and do a better job.

Regardless, here are links to the two slide decks I presented.

Dear Microsoft: Remove the 5000 Item Limit for SharePoint List Views

It’s 2016. In 1993, I used Microsoft Access to process over 1,000,000 records of data to implement a product profitability model at Staples – on an IBM 486 machine.

SharePoint-List-View-ThresholdWe’ve been saddled with a 5000 item limit for views since SharePoint 2010 launched. Oddly, in SharePoint 2007, there was no such limit. When 2010 came out, with its requirements for heavier and beefier hardware, the limit showed up.

I’ve read (well, skimmed) the white papers about why this needs to be. I’ve had long conversations with people in the SharePoint Product Group about the technical debt involved and why it is a “hard” thing to fix. I’ve read the highly visited KB articles about it, like “The number of items in this list exceeds the list view threshold, which is 5000 items” error when you view a SharePoint Online list in Office 365 Enterprise.

Bill Baer (@williambaer) has posted some promising messaging about the thresholds (See: Navigating List View Thresholds in SharePoint Server 2016 IT Preview), but I still don’t see any outward evidence that the limit is going away anytime in the foreseeable future.

I think there have been mixed messages and misinterpreted messages (no blame to Bill). The auto indexing idea which was recently added to Office 365 and comes in SharePoint 2016 (I believe – but I’m not positive) was misconstrued as lifting the 5000 item limit, which it definitely doesn’t.

The 5000 item limit in the UI for lists and libraries is arguably a good thing. There is absolutely NO reason to show more than 5000 items on a view page. Ever. No one can digest that amount of data or make use of it. Views should show just enough information for people to make good decisions about what to do next. (This is a very common mistake in SharePoint information architecture.) There are times when we simply need to have a view which derives from 5000+ items, though. Remember what year it is; 5000 items is a speck, a mote, an iota of data in this Big Data era.

No 5000 item limit

When it comes to calling SharePoint’s APIs, the 5000 item limit simply has to go. It’s a crazy limit in this day and age. I don’t care about all the technical debt talk about it. The SharePoint Product Group should be able to slide a solution underneath, at least on Office 365 – but ideally in on premises versions of SharePoint as well.

That’s the whole point of APIs: you can fix stuff under the hood without breaking any “contracts”. As more and more processing moves off-SharePoint, the 5000 limit is stifling. It’s been a problem for me for years because I live on the client side. I don’t always need all 5000+ items, but I need to know stuff about them: how many items match some criteria, what the sales total is for 12,834 items, how many items fall into a set of groupings, etc.

Whatever approach the Product Group takes shouldn’t matter, as long as they uphold the API contracts. For years they have told us – very rightly – that touching the underlying SQL tables puts us in an unsupported state. As long as we’re calling the APIs, what happens under the covers doesn’t matter. In fact, most of us should care less; that’s Microsoft’s domain and we should leave it to them.

With the new SharePoint Framework (SPX?) coming along – which I think is a TREMENDOUSLY good thing – the 5000 item limit is going to be on the minds of every developer who works with SharePoint. There will be no more server side tricks to haul out to cheat your way around this stuff. Those tricks have been reduced in degrees anyway. If SharePoint is to truly be treated as a service (as I firmly believe it should) then it needs to behave like a service in 2016+, not a service in the 1990s – or earlier.

I’ve heard many people – my fellow MVPs included – say that Microsoft will never fix this. There’s no cool Marketing moment in it, no splash they can make at a conference, little to show in the UI for it. But think of the consequences if they DON’T fix it. How long can we continue to believe that we can pump millions of items into a list (which is the true capacity) only to be told that we can only retrieve fewer than 5000 of them from the client side?

Unfortunately, if we assume they won’t fix it or can’t do it right, then the SPX may be doomed, IMO. I’d prefer to believe that they know it has to be fixed and are working on it. As my friend Jeff Shuey (@shuey) is always saying “I want to believe!”


If you want Microsoft to hear your vote on this idea, check out any or all of the feedback below ion the SharePoint UserVoice site:

  1. Remove 5000 item limit
  2. Provide a list limit over 5000
  3. Remove the list view threshold (5000 by default)
  4. Default notification for lists and library’s hitting the 5000 items limit
  5. Remove the limit of 5000 items as result in a view
  6. Allow new library look for libraries over 5000 items (provided the structure in managed correctly)<

Getting Up and Running with Power BI and SharePoint Lists

Power BI

A few weeks back when I was at the Digital Workplace Conference in Melbourne – a most excellent conference – I saw Adam Cogan (@adamcogan) demonstrate Power BI in his session Improve Your Business Intelligence with Power BI. I’d seen Power BI before a few times, but the way Adam showed it, it finally “clicked” for me. I may be hooked, and I *know* I’m going to be able to do some amazing things for clients with it.

I had to try it out. I have a client where I think Power BI could be an excellent part of the over all solution, so I tried a few things out in their tenant. The client is a pharma startup and we’ve built a site where they can log all of their experimental data. There’s plenty of data stored across a set of lists and libraries we built to match their scientific processes. The main storage location is a SharePoint Document Library made up of hundreds of Document Sets.

If I tried to connect to the SharePoint Document Library using the “SharePoint Online List” connection, I end up with this as the query:

let
  Source = SharePoint.Tables("https://tenant.sharepoint.com/sites/siteName", [ApiVersion = 15]),
  #"57cb07a8-e7aa-4401-a778-699941bfe12b" = Source{[Id="57cb07a8-e7aa-4401-a778-699941bfe12b"]}[Items]
in
  #"57cb07a8-e7aa-4401-a778-699941bfe12b"

which gave me the following error (DataSource.Error):

DataSource.Error

Trolling the Power BI community forums, I came across the suggestion to switch from [ApiVersion = 15] to [ApiVersion = 14]. When I made that switch, the error changed, but still no joy, as I got this error (Expression.Error):

Expression.Error

I figured maybe since I was working with a Document Library instead of a list that was part of the problem. (It didn’t really make sense to me that it would, but…) I figured I’d try an OData call to the REST endpoint instead.  When I connected to the Document Library as an OData source without specifying anything for the $select, the query ended up looking like this:

let
  Source = OData.Feed("https://tenant.sharepoint.com/sites/siteName/_api/web/lists/getbytitle('List Name')/items")
in
  Source

…and I got the same error (DataSource.Error) as above.

DataSource.Error

The column in question here is a multi-select choice column and it became clear that Power BI doesn’t seem to like multi-select columns. I verified this in a conversation with my buddy John White (@diverdown1964). John is a fellow MVP and one of the best BI experts out there. I figured if he didn’t know how this worked, no one would. (Follow John’s Blog ‘The White Pages‘ for all sorts of BI goodness – and more.)

As John and I were talking, it occurred to me that I could just request the columns I really needed in the REST call. Not only might it solve the problem, it would certainly be more efficient. I’ve learned to only ask for the columns I need when I’m making Web Services calls in general.

When I added the $select clause and just asked for the Title column:

let
  Source = OData.Feed("https://tenant.sharepoint.com/sites/siteName/_api/web/lists/getbytitle('List Name')/items?$select=Title")
in
  Source

Joy! I saw the data in Power BI just fine.

Just the Title

So I think the moral of the story is to build a REST call to only retrieve the data you need, excluding any multi-select columns right up front. As I mentioned, the problem column was a multi-select choice, so there was no option there. I simply couldn’t use that column in my analysis. Luckily that column didn’t really matter to us – yet.

I had better luck with lookup columns – as long as they weren’t multi-select. By using $expand on those columns, I could retrieve the values from the associated lookup list.

Finally, since we only get 100 items back from a REST call by default, I added $top=5000 so that I could retrieve the maximum allowable number of items per request.

Unfortunately, there are some drawbacks to using Power BI right now, and the team will need to solve these for people to really start using it to its full potential in the SharePoint / Office 365 space. Multi-select columns are pretty common in SharePoint lists, and Power BI needs to handle them more gracefully, if not actually use them for analysis. The error messages I got were certainly misleading, and the suggestions about how to fix them were interesting, but not effective in the end.

And it’s very yellow.

SharePoint Framework – Initial Questions and Answers

The Future of SharePoint - May 4It’s been almost a week since the #FutureOfSharePoint event in San Francisco, and as usual, there are a lot of questions flying around. Wictor Wilen has started a post SharePoint Framework – Initial Questions and Answers and I thought I’d start my own. I’m not going to rehash Wictor’s answers, and will instead focus on the questions that I’m getting. Some questions have come to me publicly (perhaps as comments on my previous post Microsoft’s May 4 SharePoint Announcements and Client Side Development Future), some privately, or I’ve seen them in private forums. I’ll keep the latter two anonymous, of course.

As with Wictor’s post, this is entirely unofficial and will contain some of my opinions, as well as some things that may turn out to be rumors. I have no intention of misleading anyone, but a lot of things are in flux as the Product Group is working to finalize the first release of the SharePoint Framework (SPX). I also have no intention of pandering to Microsoft here. Some of the answers may not be exactly what we’d like and I’ll say so if I feel that’s the case. No crying and try to keep the teeth-gnashing to a minimum.

If you have questions, feel free to add them in the comments here (or on Wictor’s post if you’d like his take). I’ll add new FAQs as they seem useful.

Is the SharePoint Framework “Yet Another Development Model”?

I don’t see it as such. The SharePoint Framework is a new development option. Nothing else has been deprecated and the SPX doesn’t replace anything per se. There are times when it will be the right way to build things and other times when the other development models are more appropriate.

Can’t I just do this stuff in a Content Editor Web Part?

Absolutely, and many of us have been doing so for years. (I’ve been building things with KnockoutJS and AngularJS, even in SharePoint 2007!) Each of us has our own way of making this all work.

What the SPX will give us that’s new and good is:

  • Isolation between client side Web Parts. We’ll be able to run different frameworks on the same page without any cross- Web Part “pollution”.
  • We’ll all be using the same approach and we can improve upon it as a community.
  • The SharePoint Product Group is going to use the SPX to build their “experiences”. That means they will feel the same pain we do if it doesn’t work well; no more pulling rabbits out of hats for them by using secret tricks.

What versions of SharePoint will get the SharePoint Framework?

As with most SharePoint improvements and enhancements, we’ll see the SharePoint Framework first in Office 365. The plans don’t seem too firmed up yet, but I would expect that we’ll see the SPX in a SharePoint 2016 feature pack not long after it shows up in Office 365. I’m not sure about the SharePoint 2013 story, but there may well be one.

Won’t this just be replaced by another model in a few years?

Everything is replaced sooner or later. However, the SharePoint Framework will bring SharePoint development into the mainstream, allowing us to use the tools most Web developers have been using for years now. As goes the Web, so shall we go. We can “chase the shiny penny” of new tools and the SPX ought to support us in doing that, within reason. I expect the SPX will evolve rather than being replaced.

What do I need to do to get up to speed?

JavaScript. If you learn JavaScript, you’ll be laying the foundation of using the SharePoint Framework. No, JavaScript is not a baby tool, or a joke, or whatever other disparaging thing you’ve heard. People outside the SharePoint world have know that for years now. Get over it. See my comments in this video a few of us MVPs made up in Montreal at Sharegate: MVP Thoughts: What Should SharePoint Developers Focus on.

What about jsLink?

(See Marcel’s comment below for the question) I don’t recall hearing anything specific about this at the Dev Kitchen, but I may have. To me, jsLink and some of the other client side rendering (CSR) techniques were almost stop-gap measures. They moved people closer to client side development, but did it in often clunky ways. (Try editing JavaScript that has to be in a comment – no fun!) Using jsLink will still be a valid approach with the existing .NET style Web Parts, but I expect many people will decide to go all the way to the client side, calling SharePoint’s Web Services themselves instead. It gives you more control over time, and will become de rigeur, IMO.

SharePoint’s Future?

Ray Ban Shades