SharePoint and Office 365: The New Beautiful Cookbook Series

Most of us are “meat and potatoes” people when it comes to the technology we use. We like what we know and we know what we like. (Yes, there are vegan “seitan and potatoes” people, vegetarian “sprouts and potatoes” people, pescatarian “cod and potatoes” people, etc. I’m not trying to leave anyone out.)

Every once in a while, though, someone hands us a new ingredient – something we’ve never seen before, something we’ve never cooked with.

Image from the Netflix show Chef’s Table S3E6 – Virgilio Martinez

That new ingredient becomes a part of our pantry, and we want to try to cook with it. We’ve probably heard how delicious it is or how it can make an ordinary dish taste amazing.

Sometimes, we get a whole new palette of ingredients. (Many of us love to watch cooking shows for just this reason: we see novel dishes and decide if we’d like to try them at home.)

Image from the Netflix show Chef’s Table S3E6 – Virgilio Martinez

We need to take a ton of time to figure out what the new ingredients are, how we can work with them, and what we can cook. If we don’t cook with the ingredients pretty often, then we lose the knowledge of how to use them, what ripeness is best.

Writing off something because it tastes bad in one context means we may miss a great use of the ingredient later – a ripe plum tastes so much better than an unripe one. Once someone has eaten an unripe plum, they may decide they hate plums.

But if we can overcome these hurdles and learn about the new ingredients, we can make some incredible dishes.

Image from the Netflix show Chef’s Table S3E6 – Virgilio Martinez

This is what I think we are going through with SharePoint and Office 365 right now. Microsoft is offering us an entirely new set of ingredients with which to make our stew.

Let me give you an example…

In my “meat and potatoes” way of looking at the world, which has been pretty consistent for the last ten years or so, even though SharePoint and my approaches have evolved, I might use this set of ingredients:

  • A Single Page Application (SPA) written with AngularJS or KnockoutJS – or even just plain old JavaScript
  • A dollop of values passed on the query string to a…
  • Standard list form, with a little JavaScript mixed in to pre-populate some columns in the form
  • A SharePoint Designer workflow to add notifications on top (Substitute Alerts if your local market doesn’t carry SharePoint Designer)

But there are new ingredients now. Instead we could whip something up with these:

  • A SharePoint Framework Web Part (still maybe written with AngularJS or KnockoutJS)
  • Creating list items using REST based on the values in our SPFx Web Part
  • Microsoft Flow to add in the notifications and any process
  • Stir in a pinch of PowerApps – until they are ready

That’s quite a shift. We’re being asked to think about cooking in a very different way. We’ve been through stages of evolution before – new cooking techniques like sous vide (Sandbox Solutions), gelification (Add-In Model, nee App Model), etc. – but this time it’s really different. We’re not even sure if we’re supposed to like everything we taste. Is it just the next wave of kale frenzy or is it an ingredient that will last?

At this point, Microsoft is asking us to dream big, and reach for the previously unimaginable. I think we need to try to do it.

Image from the Netflix show Chef’s Table S3E6 – Virgilio Martinez

Some of us will be able to cook up truly amazing solutions on the “modern” platform. Don’t be afraid to give it a taste.

Image from the Netflix show Chef’s Table S3E6 – Virgilio Martinez

In case you didn’t figure it out, this post was inspired by the Netflix show Chef’s Table S3E6, which profiles the Peruvian chef Virgilio Martinez. It’s an outstanding series, and this particular episode was stellar.

Also see any volume in the Beautiful Cookbook series.

Let’s Capture Missing or Insufficient SharePoint REST Endpoints

Today I got an alert that the SharePoint UserVoice suggestion from Corey Roth (@coreyroth) entitled Add managed metadata term store operations to REST API got the coveted “Thinking About It” tag from the Product Group. I like to tweet out changes like this to let people know the Product Group is listening and acting on our feedback – beyond saying “That’s good feedback!” It’s not all wine and roses, though:

Thank you for your feedback! Just letting you know that we absolutely have this in our backlog, but unfortunately this currently is not included in our short term engineering tasks. We absolutely understand the request and seeing vote counts around this, will help to further prioritize this work for next sprints.

I got a couple of tweets back right away pointing out some other current holes in the REST APIs.

If you think there are other endpoints the REST APIs need or endpoints that don’t work well, please add them to the comments here. I’ll work them up into a list for the Product Group and let’s see what we can get moving! We’ll play by the rules and add the list to UserVoice, but I think all the individual suggestions get lost and it’s harder to see the bigger picture. For each item on the list, I’ve tried to capture related UserVoice suggestions.

The list so far:

Dear Microsoft: Confusing Failures in “Modern” SharePoint Libraries

Moving documents around in the “modern” Document Libraries in SharePoint Online (Office 365) has certainly gotten easier. Instead of opening libraries in File Explorer, downloading the files, and then uploading them into the new location, we have nice options like Move to, Copy To, and Rename right in the toolbar. That’s a big upside.

Sadly, there are downsides. While the new capabilities are awesome, I’m finding that the UI is often confusing – both to me and my clients.

When you use one of these new file manipulation capabilities, you get a little feedback message ion the right side of the toolbar telling you how it went. Unfortunately, the message location isn’t all that obvious, and it usually feels as though the operation succeeded even when it didn’t.

Here’s and example of a Move to operation that failed for me. Note that the message says “1 item wasn’t moved”.

There’s no other feedback, and on my wide monitor, the message is way over to the right side. I don’t usually notice it.

In the case above, I did see the message, so I clicked on the message to see what was up. The reason I saw it was that I was looking for something wrong. A client of mine told me that a Move to wasn’t working. Every time she went back to the library, the documents were still in the same place, no matter how many times she moved them.

As you can see from the expanded message below, there was indeed an issue: there was a file with the same name in the destination location. I have two logical options, either to Keep both files or Replace the original.

 

The message really isn’t obvious enough, and I’ve been caught many times because I didn’t see that something I did failed. Even worse, to my client “SharePoint is broken”. I’m hoping the Product Group can come up with a more informative way to provide the feedback that something has gone wrong – which happens surprisingly often!

There is yet another “unfortunately” here, though. In my client’s case, even when there is no error in a Move to operation, the files are boomeranging right back into the original position after a few screen refreshes.

I also tried the Content and Structure page to see if moving the document that way would help, but still no dice. I checked to see if perhaps there was a workflow in play here causing issues, but that’s not the case. The only other thing I see which MIGHT be a problem is that the library has 4988 documents in it, which is pretty close to the 5000 list view threshold. I hate that threshold with a steaming passion (have I mentioned that here and here and  here and here and here and probably tens of dozens of other places?), but I can’t think why it would matter in this case.

So we’re at an impasse. The error messages aren’t so great, and the Move to operations are failing when when there aren’t errors. Maybe SharePoint really is broken.

Anybody? Anybody?

Yes, Virginia, You Can Get More than 5000 SharePoint Items with REST

If you haven’t been paying any attention, you might not know that I loathe the 5000 item limit in SharePoint. I may have mentioned it here and here and here and a bunch of other places, but I’m not sure. But given it’s there, we often need to work around it.

No 5000 item limit

I’ve written this little function before, but alas today I needed it again but couldn’t find any previous incarnations. That’s what this blog is for, though: to remind me of what I’ve already done!

In this case, I’m working with AngularJS 1.x. We have several lists that are nearing the 5000 item level, and I don’t want my code to start failing when we get there. We can get as many items as we want if we make repeated calls to the same REST endpoint, watching for the __next link in the results. That __next link tells us that we haven’t gotten all of the items yet, and provides us with the link to get the next batch, based on how we’ve set top in the request.

Here’s an example. Suppose I want to get all the items from a list which contains all of the ZIP codes in the USA. I just checked, and that’s 27782 items. That’s definitely enough to make SharePoint angry at us, what with that 5000 item limit and all. Let’s not get into an argument about whether I need them all or not. Let’s just agree that I do. It’s an example, after all.

Well, if we set up our requests right, and use my handy-dandy recursive function below, we can get all of the items. First, let’s look at the setup. It should look pretty similar to anything you’ve done in an AngularJS service. I set up the request, and the success and error handlers just like I always do. Note I’m asking for the top 5000 items, using "&$top=5000" in my REST call.

If there are fewer than 5000 items, then we don’t have a problem; the base request would return them all. Line 32 is what would do that “normal” call. Instead, I call my recursive function below, passing in the request only, even though the function can take two more arguments: results and deferred.

The recursive function simply keeps calling itself whenever it sees that the __next attribute of the response is present, signifying there is more data to fetch. It concatenates the data into a single array as it goes. In my example, there would be 6 calls to the REST endpoint because there are 27782 / 5000 = 5.5564 “chunks” of items.

Image from https://s-media-cache-ak0.pinimg.com/564x/16/bb/03/16bb034cb3b6b0bdc66d81f47a95a59f.jpg

Image from https://www.pinterest.com/pin/323062973239383839/

NOW, before a bunch of people get all angry at me about this, the Stan Lee rule applies here. If you have tens of thousands of items and you decide to load them all, don’t blame me if it takes a while. All those request can take a lot of time. This also isn’t just a get of of jail free card. I’m posilutely certain that if we misuse this all over the place, the data police at Microsoft will shut us down.

In actual fact, the multiple calls will be separated by short periods of time to us, which are almost eternities to today’s high-powered servers. In some cases, you might even find that batches of fewer than 5000 items may be *faster* for you.

In any case, don’t just do this willy-nilly. Also understand that my approach here isn’t as great at handling errors as the base case. Feel free to improve on it and post your improvements in the comments!