Dear Microsoft: Confusing Failures in “Modern” SharePoint Libraries

Moving documents around in the “modern” Document Libraries in SharePoint Online (Office 365) has certainly gotten easier. Instead of opening libraries in File Explorer, downloading the files, and then uploading them into the new location, we have nice options like Move to, Copy To, and Rename right in the toolbar. That’s a big upside.

Sadly, there are downsides. While the new capabilities are awesome, I’m finding that the UI is often confusing – both to me and my clients.

When you use one of these new file manipulation capabilities, you get a little feedback message ion the right side of the toolbar telling you how it went. Unfortunately, the message location isn’t all that obvious, and it usually feels as though the operation succeeded even when it didn’t.

Here’s and example of a Move to operation that failed for me. Note that the message says “1 item wasn’t moved”.

There’s no other feedback, and on my wide monitor, the message is way over to the right side. I don’t usually notice it.

In the case above, I did see the message, so I clicked on the message to see what was up. The reason I saw it was that I was looking for something wrong. A client of mine told me that a Move to wasn’t working. Every time she went back to the library, the documents were still in the same place, no matter how many times she moved them.

As you can see from the expanded message below, there was indeed an issue: there was a file with the same name in the destination location. I have two logical options, either to Keep both files or Replace the original.


The message really isn’t obvious enough, and I’ve been caught many times because I didn’t see that something I did failed. Even worse, to my client “SharePoint is broken”. I’m hoping the Product Group can come up with a more informative way to provide the feedback that something has gone wrong – which happens surprisingly often!

There is yet another “unfortunately” here, though. In my client’s case, even when there is no error in a Move to operation, the files are boomeranging right back into the original position after a few screen refreshes.

I also tried the Content and Structure page to see if moving the document that way would help, but still no dice. I checked to see if perhaps there was a workflow in play here causing issues, but that’s not the case. The only other thing I see which MIGHT be a problem is that the library has 4988 documents in it, which is pretty close to the 5000 list view threshold. I hate that threshold with a steaming passion (have I mentioned that here and here and  here and here and here and probably tens of dozens of other places?), but I can’t think why it would matter in this case.

So we’re at an impasse. The error messages aren’t so great, and the Move to operations are failing when there aren’t errors. Maybe SharePoint really is broken.

Anybody? Anybody?


Sharegate to the Rescue – Getting Rid of Weird Taxonomy (Managed Metadata) Columns

Yet another entry for the “Things that make you go ‘Huh?’ in SharePoint” file.

We’ve got a Document Library in SharePoint Online (Office 365) that contains Document Sets. As per usual, each Document Set can contain several different Content Types, as we’ve defined. It’s nothing out of the ordinary, really. We’re using SharePoint capabilities the way they are intended, and it’s all “out of the box”.

One day last week, we started to see odd things when we looked at the documents using the list forms. Fields we didn’t create were showing up in between those we did. For example, we saw Sector_0 above the field Sector, ServiceArea_0 above Service Area, and a new field called Taxonomy Catch All Column. Obviously something was amiss. None of those red boxed fields should be there.

Weird Taxonomy (Managed Metadata) Columns

Turning to them InterWebz, I found a number of posts about this happening in SharePoint 2010 when the content was moved around using the Content and Structure UI. That was a long time ago, plus we were pretty sure we hadn’t used the Content and Structure page for anything. Even if we had, this was weird.

It turns out the fields are part of the [poorly built – please give it some love, Microsoft] Managed Metadata infrastructure. The fields are supposed to be there when you are using Managed Metadata columns, but you should never see them. The solution back in the SharePoint 2010 days was to use Powershell to hide the fields again.

Ok, so we didn’t know why or how this had happened, but Powershell seemed like a good bet to fix it. I headed on over to Gary Lapointe’s (@glapointe) most excellent SharePoint Online Custom Cmdlets, installed them (I don’t so a lot of Powershell – but perhaps I should do more), and adapted the old 2010-era Powershell examples from the forums for SharePoint Online. Here’s the important part:

Basically, connect to SPO, find the list, and then loop through the fields which should be hidden and hide them again.

But that wasn’t getting me there. When I looked at the weird fields, I saw they were already hidden. Here’s the info about the ServiceArea_0 field, which I got by looking at $fieldname in the foreach loop. As you can see, Hidden is already set to True. Yet the field was showing up in the forms nonetheless.

So Powershell wasn’t going to solve it for us. What else could I try?

Well, Sharegate is definitely my Swiss Army Knife for SharePoint. (Yes, CEWPs and DVWPs are also Swiss Army Knives for me – I like great tools.) I use it just about daily for many things, most of which have nothing to do with migration. I looked in the main Sharegate app, wondering if there were some other settings for the fields I could look at. Great functionality from Sharegate, but I couldn’t find anything.

Then I wondered if simply copying the content into a new library might fix things. I set up a new Document Library named – aptly – Temp, and copied over one Document Set. Yay – everything looked fine!

Then I copied all of the other content from Proposals over to Temp, deleted Proposals, and then copied the Temp library to Proposals again, so I’d have the same URL.

Suffice it to say, all is well in Proposal land now. When I look at any of the documents, I see what we’d expect, which is certainly prettier and more useful.

Whew - Back to Normal!

Sharegate: it’s not just a migration tool.


This is blog post 1000 for me! I considered making some big deal or some special post out of it, but I decided I should just keep chugging along posting stuff that might help people. It’s just a number and today is just another day. If you want to, go out and celebrate for me.

Image from


Dear Microsoft: In Office 365, Groups are Groups are Groups – Unless They Aren’t

One of the problems with using common English words for things like “groups” or “teams” is that we end up trying to figure out the difference between things with very common names. When Sue Hanley (@susanhanley) and I were building our session Lions and Tigers and Teams, Oh My! – Sorting through the options to connect and collaborate in Office 365 for SPTechCon Austin recently, I realized just how confusing this could be – yet again.

The Problem

On the SharePoint home UI, when I want to create a “group”, I am told I will “Get a team site connected to Office 365 Groups”. So based on that one sentence, I might quite reasonably assume I’m creating a “Team” and a “Group” – which it so happens I am – sort of.

In the Groups UI in the Admin Center, we see these options:

  • Office 365 Group
  • Distribution list
  • Mail-enabled security group
  • Security group

In the Exchange admin center when we start to create a “group”, we’re given these choices:

  • Office 365 Group
  • Distribution group
  • Security group
  • Dynamic distribution group

But don’t fret. When you go to create a “group” in the Azure AD UI, you can create a “group” with Membership type of:

  • Assigned
  • Dynamic Device
  • Dynamic user

We can also check a box to “

Selecting ‘Yes’ will turn on Office 365 features for this group

I can even try to add the “group” – it’s not clear which type of “group” I’ve actually created – to the membership of another “group”. “Group Type” seems to be one of the following:

  • Distribution
  • Security
  • Office
  • Mail enabled security

When I create a Team in Microsoft Teams, there’s not even a clue that I’m also creating an Office 365 Group behind the scenes. Maybe that doesn’t really matter to an end user, but the fact that a “Team” is also a “Group” is another terminology SNAFU.

So What?

Some people might say that as an “IT Pro”, one should always understand all of these terms intrinsically, but I doubt that it’s often the case that this is true. The terms cross different “workloads” in Office 365 and can vary in terminology from their on premises counterparts, especially across server versions. Add to that the fact that many “admins” are the lone SharePoint/Exchange/Office 365/Chief Cook/Bottle Washer in their organization, and there is little hope they will understand the nuances. Heck, I do this stuff for a living and I find it confusing. Sometimes things are hard for no good reason.

Let’s assume we have an organization which is totally sold on the value of Office 365 Groups (note the capitalization of “groups”). If they have been using SharePoint and Exchange either on premises or in Office 365, they probably already have a mix of all the different types of “groups” shown above. When someone in that organization tries to create an Office 365 Group for – say – the Executive Team, it’s highly likely they will get an error like:

Note that “group alias” is not one of the terms used above, nor does the error give even an inkling of how to solve the problem. Not even a worthless “Contact your administrator” as a fine how-do-you-do. So, what is the user likely to do?

Well, I’ll posit that they will create a Group named “Executive Group” or “Executives” or something else spelled a little differently. That Group won’t be linked to any of the previous artifacts and thus will begin the route to more chaos that we had before.

Of course, this is a real use case I’ve run into at an actual client, as is often the case with my beefs. As far as I can tell, there is no way to convert a “Mail-enabled security group” to an “Office 365 group” without some sort of Powershell tomfoolery. That’s just bad. I’ve reached out to my MVP network for help here and I’ll update this post if I’m wrong – and I’d love to be.

This is where the old Microsoft line would have been that it’s a great “partner opportunity” or that there are “$9 spent in consulting for every $1 spent on server products”. Those were always crappy answers, and they were often driven by confusing things just like this. The “avoid a mess” or “clean up the mess” roles fell to partners who were more than happy to lap up the dollars agonized organizations were willing to toss them to fix things.

The new Microsoft doesn’t want to work this way, but in many cases, their own large size and competing teams (small “t”, generic use) still lead us to these sorts of problems.

One of the things we as consultants spend a *lot* of time doing is working with organizations to build a common set of terminology. Sometimes you may hear this called taxonomy or ontology, but the bottom line point is to get everyone to agree to a set of words and phrases that they can use consistently to express the same thing. If I came out of one of those efforts with as many terms as I’ve listed above, I’d consider the effort a failure, especially if several terms meant essentially the same thing.

The Ask

So, Dear Microsoft, please tighten up the terminology as quickly as you can. Next we need to know how to convert each of the old, legacy types of “groups” to an Office 365 Group – without resorting to Powershell. Ideally, we should just be able to click a button on any of the old things to make it a new thing, which would take us through a process (if we need one) to get it done.

Even better, when I try to create that Executive Team Office 365 Group, walk me – as a user – through fixing the issue. In other words, help me solve the problem BEFORE I’ve created the mess, so I don’t need to call a consultant to fix things. (Yes, nose, face, etc., but I think most of us consultants would rather be building valuable functionality than cleaning up messes. If your consultants relish this type of work – beware.)

Make it easy for us to help ourselves. Cut down on those support calls. Stop making your customers hire expensive consultants to clean up messes. Make Office 365 the best it can be.

Post 999! Sadly, I may have just used up my allotment of quote characters for any future posts.

Deleting a Very Large SharePoint List

I’ve complained about the 5000 item list threshold limit so many times, I’ve lost count. If you don’t believe me, you could read my posts here and  here and here and here.

Today’s difficulty was caused by needing to delete a Very Large List. This isn’t just a Large List – it’s Very Large. The number of items is way over 5000 at almost 300000. This list is part of a well-used application I built for a client about 3 years ago in SharePoint 2007. It’s been ticking away just fine, as there was no list threshold in 2007.

I’m migrating the application to SharePoint Online, so I’m totally rewriting it for several reasons. I copied across the list “shell” with Sharegate and brought along a few hundred items in the big list just for testing and building purposes.

But now I need to test “at volume”, so I brought across all 300k items. Unfortunately, I forgot to add an index to one of the columns where I need it to make some queries work. I figured “no big deal”. I’d just delete the list and start again.

Not so fast! When I tried to delete the list, I got this error:

I’m trying to DELETE the list, so what gives? Well, I found a post from Mike Smith (@TechTrainNotes) entitled SharePoint 2016: List View Threshold Limit to Delete a List is 99,993 Items??? which pretty much told me why this wasn’t working. It’s crazy, but you can’t delete a list or library which has more than 100k objects in it. (Read Mike’s post to see where he ran into a few wrinkles on this.) One would think that deleting a list would have nothing to do with any thresholds, but one would be wrong.

I tried a number of things – deleting the list in SharePoint Designer, writing some code to delete items one by one, etc. – but either my attempts didn’t work or they were taking forever.

Luckily, I have no pride and I complained on Twitter. Kelvin Hoyle (@kelvin_hoyle) came to the rescue!

By connecting to the list with Microsoft Access, I was able to select all the items and hit delete. Yes, this is taking a while as well, but it’s clearly faster.

The steps to do this are:

  • Open Access – I’m using Access 2016 – and create a blank database.
  • In the ribbon, go to External Data and in the Import & Link section, click on the More dropdown.
  • Choose SharePoint List
  • Provide the URL to the SharePoint site which houses the list
  • Be sure to leave the default radio button selected, creating a link between Access and the SharePoint list.

  • Click Next, and choose the list you want to work with.
  • Click OK.

Access will set up a new table which is linked to your list. You can open it like any other table in Access. Open the table and – voila – you can select any items you’d like to edit or delete.

Yet another workaround for something which I think shouldn’t be a limitation in the modern age. Sigh.


It took overnight, but my list finally was whittled down below the 100k object level and I was able to delete it. Innovation: 1. Productivity: 0.

After I posted this, I heard from another corner of the Twittersphere. In SharePoint no one can hear you scream there are usually at least several ways to do anything.

Ryan Yeats’ Client-side SharePoint PowerShell on Codeplex looks like another good option. He tweeted some more details and let me know there’s a function called Clear-ListContent which does the heavy work. I’ve always avoided CSOM and I’ve spent enough time of this today but from a quick look, Ryan’s library looks useful here.