Let’s Capture Missing or Insufficient SharePoint REST Endpoints

Today I got an alert that the SharePoint UserVoice suggestion from Corey Roth (@coreyroth) entitled Add managed metadata term store operations to REST API got the coveted “Thinking About It” tag from the Product Group. I like to tweet out changes like this to let people know the Product Group is listening and acting on our feedback – beyond saying “That’s good feedback!” It’s not all wine and roses, though:

Thank you for your feedback! Just letting you know that we absolutely have this in our backlog, but unfortunately this currently is not included in our short term engineering tasks. We absolutely understand the request and seeing vote counts around this, will help to further prioritize this work for next sprints.

I got a couple of tweets back right away pointing out some other current holes in the REST APIs.

If you think there are other endpoints the REST APIs need or endpoints that don’t work well, please add them to the comments here. I’ll work them up into a list for the Product Group and let’s see what we can get moving! We’ll play by the rules and add the list to UserVoice, but I think all the individual suggestions get lost and it’s harder to see the bigger picture. For each item on the list, I’ve tried to capture related UserVoice suggestions.

The list so far:

Lists

  • Ability to use $filter on Calculated column and Single value Taxonomy column label (@gautamdsheth)
  • Enable support for lookup columns in other webs in the REST API (Me!) https://sharepoint.uservoice.com/forums/329220-sharepoint-dev-platform/suggestions/9065329-enable-support-for-lookup-columns-in-other-webs-in
  • Create list item in a folder (@jfj1997)

Calendars

Libraries

  • Version history (@merill) – Provide the Version History feature on list items with CSOM and/or REST, i.e. provide which metadata has changed with the previous value. Currently, the only way to retrieve versions is through the FileVersionCollection/FileVersion objects which provides only the VersionLabel, CreatedBy, CreationDate and Version Url. Ideally, we would need a ListItem object holding all the attributes and values for each version. (John Gunning)

Permissions

Managed Metadata / Taxonomy

Search

  • Recurring events via the Search endpoint (Derek Gusoff)

Publishing

  • Publishing – PublishingPageContent, PublishingPageImage (@gautamdsheth)

Workflows

  • Starting a site workflow – StartWorkflowOnListItemBySubscriptionId & StartWorkflow are only for list items (@BradOrluk)

Microsoft Graph

  • Easily get a graph access token (anonymous)

Groups

  • SharePoint groups – Changing group owners has a bug such that you can’t change them using REST (Mike)

User Profiles

  • Allow custom user profile property to be filtered using the $filter parameter (@gautamdsheth)

Other

  • Edit and manage web parts (anonymous)
Advertisements

Beware the Office 365 Group -Based Site Regional Settings!!!

This is a quick post, yet it’s still an important one. We’re using more and more Office 365 Group -based SharePoint sites these days. Even when you know you aren’t going to use some of the goodies you end up with, this type of site is making more and more sense.

<addendum data-datetime=”Sun May 14 2017 10:56:53 GMT-0400 (Eastern Daylight Time)”>
Several people have asked me in other forums the basic question: “So what?” If all dates and times are stored in UTC, does it really matter what the site’s Regional Settings are? Frankly, that’s got me a little bit stumped.

It certainly feels wrong that the site’s settings don’t match its primary locus, but since team members can span the globe, what’s the impact?

I know I struggle as a developer to show everyone things in the right date/time based on their settings, and it feels like the platform doesn’t give us great tools for this. What other issues does it raise? Please add your thoughts and issues in the comments. I’m interested in things other than the usual “The people in Redmond don’t realize that we’re not all in their timezone” stuff – which is basically all I’ve pointed out here.
</addendum>

BUT, there’s a simple problem that can have longer-term ramifications. The default time zone for every new Group-based site we create is PDT, also known as UTC-08:00. You have to go into Site Settings to change it manually for every site you create this way. Since a lot of my clients are in EDT, this is tedious.

I’m guessing no one in Redmond even notices this, because PDT is their time zone. I spot it every time I create a new Group-based site during a migration because Sharegate warns me the time zones of the source and destination sites are different when I start to copy content across. (Yay, Sharegate!)

If you happen to be a non-US person, then ALL of the regional settings are likely to be wrong for you. I’ve checked, and there is no way to change the default here – unless it’s a VERY recent change.

Here are some Office 365 UserVoice suggestions you can run off to vote for:

 

Dear Microsoft: Confusing Failures in “Modern” SharePoint Libraries

Moving documents around in the “modern” Document Libraries in SharePoint Online (Office 365) has certainly gotten easier. Instead of opening libraries in File Explorer, downloading the files, and then uploading them into the new location, we have nice options like Move to, Copy To, and Rename right in the toolbar. That’s a big upside.

Sadly, there are downsides. While the new capabilities are awesome, I’m finding that the UI is often confusing – both to me and my clients.

When you use one of these new file manipulation capabilities, you get a little feedback message ion the right side of the toolbar telling you how it went. Unfortunately, the message location isn’t all that obvious, and it usually feels as though the operation succeeded even when it didn’t.

Here’s and example of a Move to operation that failed for me. Note that the message says “1 item wasn’t moved”.

There’s no other feedback, and on my wide monitor, the message is way over to the right side. I don’t usually notice it.

In the case above, I did see the message, so I clicked on the message to see what was up. The reason I saw it was that I was looking for something wrong. A client of mine told me that a Move to wasn’t working. Every time she went back to the library, the documents were still in the same place, no matter how many times she moved them.

As you can see from the expanded message below, there was indeed an issue: there was a file with the same name in the destination location. I have two logical options, either to Keep both files or Replace the original.

 

The message really isn’t obvious enough, and I’ve been caught many times because I didn’t see that something I did failed. Even worse, to my client “SharePoint is broken”. I’m hoping the Product Group can come up with a more informative way to provide the feedback that something has gone wrong – which happens surprisingly often!

There is yet another “unfortunately” here, though. In my client’s case, even when there is no error in a Move to operation, the files are boomeranging right back into the original position after a few screen refreshes.

I also tried the Content and Structure page to see if moving the document that way would help, but still no dice. I checked to see if perhaps there was a workflow in play here causing issues, but that’s not the case. The only other thing I see which MIGHT be a problem is that the library has 4988 documents in it, which is pretty close to the 5000 list view threshold. I hate that threshold with a steaming passion (have I mentioned that here and here and  here and here and here and probably tens of dozens of other places?), but I can’t think why it would matter in this case.

So we’re at an impasse. The error messages aren’t so great, and the Move to operations are failing when there aren’t errors. Maybe SharePoint really is broken.

Anybody? Anybody?

Sharegate to the Rescue – Getting Rid of Weird Taxonomy (Managed Metadata) Columns

Yet another entry for the “Things that make you go ‘Huh?’ in SharePoint” file.

We’ve got a Document Library in SharePoint Online (Office 365) that contains Document Sets. As per usual, each Document Set can contain several different Content Types, as we’ve defined. It’s nothing out of the ordinary, really. We’re using SharePoint capabilities the way they are intended, and it’s all “out of the box”.

One day last week, we started to see odd things when we looked at the documents using the list forms. Fields we didn’t create were showing up in between those we did. For example, we saw Sector_0 above the field Sector, ServiceArea_0 above Service Area, and a new field called Taxonomy Catch All Column. Obviously something was amiss. None of those red boxed fields should be there.

Weird Taxonomy (Managed Metadata) Columns

Turning to them InterWebz, I found a number of posts about this happening in SharePoint 2010 when the content was moved around using the Content and Structure UI. That was a long time ago, plus we were pretty sure we hadn’t used the Content and Structure page for anything. Even if we had, this was weird.

It turns out the fields are part of the [poorly built – please give it some love, Microsoft] Managed Metadata infrastructure. The fields are supposed to be there when you are using Managed Metadata columns, but you should never see them. The solution back in the SharePoint 2010 days was to use Powershell to hide the fields again.

Ok, so we didn’t know why or how this had happened, but Powershell seemed like a good bet to fix it. I headed on over to Gary Lapointe’s (@glapointe) most excellent SharePoint Online Custom Cmdlets, installed them (I don’t so a lot of Powershell – but perhaps I should do more), and adapted the old 2010-era Powershell examples from the forums for SharePoint Online. Here’s the important part:

Connect-SPOSite -Url https://$orgName.sharepoint.com -Credential "myCreds@$orgName.com"
$list = Get-SPOList -Web "/" -Identity "Proposals"
$fieldstoupdate = @("Sector_0","Taxonomy Catch All Column") #ENTER THE LIST OF FIELDS TO BE HIDDEN

if($list -ne $null) {
 Write-host -f black "List"$list.Title "on"$web.Name "at URL"$web.Url
 
 #GET ARRAY OF FIELDS TO UPDATE
 foreach ($field in $fieldstoupdate) {
  $fieldname = $list.GetFieldByTitle("$field")
  $fieldname.Hidden = $true
  $fieldname.update()
  Write-Host -f green "Updated field" $fieldname
 }
}
...

Basically, connect to SPO, find the list, and then loop through the fields which should be hidden and hide them again.

But that wasn’t getting me there. When I looked at the weird fields, I saw they were already hidden. Here’s the info about the ServiceArea_0 field, which I got by looking at $fieldname in the foreach loop. As you can see, Hidden is already set to True. Yet the field was showing up in the forms nonetheless.

So Powershell wasn’t going to solve it for us. What else could I try?

Well, Sharegate is definitely my Swiss Army Knife for SharePoint. (Yes, CEWPs and DVWPs are also Swiss Army Knives for me – I like great tools.) I use it just about daily for many things, most of which have nothing to do with migration. I looked in the main Sharegate app, wondering if there were some other settings for the fields I could look at. Great functionality from Sharegate, but I couldn’t find anything.

Then I wondered if simply copying the content into a new library might fix things. I set up a new Document Library named – aptly – Temp, and copied over one Document Set. Yay – everything looked fine!

Then I copied all of the other content from Proposals over to Temp, deleted Proposals, and then copied the Temp library to Proposals again, so I’d have the same URL.

Suffice it to say, all is well in Proposal land now. When I look at any of the documents, I see what we’d expect, which is certainly prettier and more useful.

Whew - Back to Normal!

Sharegate: it’s not just a migration tool.

References


This is blog post 1000 for me! I considered making some big deal or some special post out of it, but I decided I should just keep chugging along posting stuff that might help people. It’s just a number and today is just another day. If you want to, go out and celebrate for me.

Image from https://commons.wikimedia.org/wiki/File:Boston_Harbor_Fireworks_-_Composite_(21189670832).jpg

 

Dear Microsoft: In Office 365, Groups are Groups are Groups – Unless They Aren’t

One of the problems with using common English words for things like “groups” or “teams” is that we end up trying to figure out the difference between things with very common names. When Sue Hanley (@susanhanley) and I were building our session Lions and Tigers and Teams, Oh My! – Sorting through the options to connect and collaborate in Office 365 for SPTechCon Austin recently, I realized just how confusing this could be – yet again.

The Problem

On the SharePoint home UI, when I want to create a “group”, I am told I will “Get a team site connected to Office 365 Groups”. So based on that one sentence, I might quite reasonably assume I’m creating a “Team” and a “Group” – which it so happens I am – sort of.

In the Groups UI in the Admin Center, we see these options:

  • Office 365 Group
  • Distribution list
  • Mail-enabled security group
  • Security group

In the Exchange admin center when we start to create a “group”, we’re given these choices:

  • Office 365 Group
  • Distribution group
  • Security group
  • Dynamic distribution group

But don’t fret. When you go to create a “group” in the Azure AD UI, you can create a “group” with Membership type of:

  • Assigned
  • Dynamic Device
  • Dynamic user

We can also check a box to “

Selecting ‘Yes’ will turn on Office 365 features for this group

I can even try to add the “group” – it’s not clear which type of “group” I’ve actually created – to the membership of another “group”. “Group Type” seems to be one of the following:

  • Distribution
  • Security
  • Office
  • Mail enabled security

When I create a Team in Microsoft Teams, there’s not even a clue that I’m also creating an Office 365 Group behind the scenes. Maybe that doesn’t really matter to an end user, but the fact that a “Team” is also a “Group” is another terminology SNAFU.

So What?

Some people might say that as an “IT Pro”, one should always understand all of these terms intrinsically, but I doubt that it’s often the case that this is true. The terms cross different “workloads” in Office 365 and can vary in terminology from their on premises counterparts, especially across server versions. Add to that the fact that many “admins” are the lone SharePoint/Exchange/Office 365/Chief Cook/Bottle Washer in their organization, and there is little hope they will understand the nuances. Heck, I do this stuff for a living and I find it confusing. Sometimes things are hard for no good reason.

Let’s assume we have an organization which is totally sold on the value of Office 365 Groups (note the capitalization of “groups”). If they have been using SharePoint and Exchange either on premises or in Office 365, they probably already have a mix of all the different types of “groups” shown above. When someone in that organization tries to create an Office 365 Group for – say – the Executive Team, it’s highly likely they will get an error like:

Note that “group alias” is not one of the terms used above, nor does the error give even an inkling of how to solve the problem. Not even a worthless “Contact your administrator” as a fine how-do-you-do. So, what is the user likely to do?

Well, I’ll posit that they will create a Group named “Executive Group” or “Executives” or something else spelled a little differently. That Group won’t be linked to any of the previous artifacts and thus will begin the route to more chaos that we had before.

Of course, this is a real use case I’ve run into at an actual client, as is often the case with my beefs. As far as I can tell, there is no way to convert a “Mail-enabled security group” to an “Office 365 group” without some sort of Powershell tomfoolery. That’s just bad. I’ve reached out to my MVP network for help here and I’ll update this post if I’m wrong – and I’d love to be.

This is where the old Microsoft line would have been that it’s a great “partner opportunity” or that there are “$9 spent in consulting for every $1 spent on server products”. Those were always crappy answers, and they were often driven by confusing things just like this. The “avoid a mess” or “clean up the mess” roles fell to partners who were more than happy to lap up the dollars agonized organizations were willing to toss them to fix things.

The new Microsoft doesn’t want to work this way, but in many cases, their own large size and competing teams (small “t”, generic use) still lead us to these sorts of problems.

One of the things we as consultants spend a *lot* of time doing is working with organizations to build a common set of terminology. Sometimes you may hear this called taxonomy or ontology, but the bottom line point is to get everyone to agree to a set of words and phrases that they can use consistently to express the same thing. If I came out of one of those efforts with as many terms as I’ve listed above, I’d consider the effort a failure, especially if several terms meant essentially the same thing.

The Ask

So, Dear Microsoft, please tighten up the terminology as quickly as you can. Next we need to know how to convert each of the old, legacy types of “groups” to an Office 365 Group – without resorting to Powershell. Ideally, we should just be able to click a button on any of the old things to make it a new thing, which would take us through a process (if we need one) to get it done.

Even better, when I try to create that Executive Team Office 365 Group, walk me – as a user – through fixing the issue. In other words, help me solve the problem BEFORE I’ve created the mess, so I don’t need to call a consultant to fix things. (Yes, nose, face, etc., but I think most of us consultants would rather be building valuable functionality than cleaning up messes. If your consultants relish this type of work – beware.)

Make it easy for us to help ourselves. Cut down on those support calls. Stop making your customers hire expensive consultants to clean up messes. Make Office 365 the best it can be.


Post 999! Sadly, I may have just used up my allotment of quote characters for any future posts.