Naming New Office 365 Groups Intelligently Is !Important

Sympraxis is starting a new Office 365 -based Intranet project with Sue Hanley (@susanhanley). Julie (@jfj1997) and I are really going to enjoy working with Sue!

As always when we’re starting a project, we want to start collaborating with the client project team in the tools we’re rolling out for their organization.

It’s a funny thing, but I often get push back on this, especially if the project team is IT-based. If the project team won’t use the tool set, then it’s not reasonable to expect everyone else to embrace it. As I always talk about in my Creating a Great User Experience session at conferences, excuses like “it’s too slow” or “I don’t like the UI” are serious problems that need to be addressed right up front. (And no, I’m not making this up.)

Anyway, Sue and I discussed this, and imagine the conversation going something like this…

Me: I think we should spin up a site where we can work with the project folks. Suggestions on the name and location? I would go with a subsite from https://[tenant], maybe “New Intranet”, with custom permissions.

Sue: Do you mean a team site for the intranet project? Why not a Group so we get a separate site collection?

Me: Doh! Of course! I’ll set it up. [Note: I’m still getting the hang of this whole “Groups” thing, clearly!]

Sue: Thanks! I am using one for [my other client] and it is actually great.

Me: Public or Private? And can you change that after the fact? (I sure hope so.) I’m thinking “Intranet Project 2017”, in case they want to do another Intranet Project later. Naming these Groups for good governance is a pretty tricky thing, IMO.

Sue: Private. That sounds good for a long name, but short name for the URL, right?

Me: Looks like I can shorten the Group ID. “IP2017”?

Sue: Works for me!

Me: This little exchange shows me that picking a good Group name and ID is REALLY important. Blog post!

Obviously, my first thought was not a new Office 365 Group, and I should really start thinking of it as at least one of the first options in cases like this. I’m simply too used to spinning up plain old Team Sites, which still serve their purpose well. A Team Site with a Document Library, Task list, and Calendar, is often enough to manage a SharePoint project, with other options added along the way. (See my older posts Simple Best Practices for Using SharePoint Task Lists and Recent Changes to Task Management Conventions on Office 365 for how I tend to use the Task lists.)

Office 365 Groups are really cool (they definitely didn’t strike me that way early on) and they make a lot of sense for project-based work. Because of the tight integration with Exchange (most important to end users via Outlook) and other Office 365 services, they really do make a lot of sense.

But the last point is what I want to dwell on most here.

One of the best – and to some people, worst – things about Office 365 Groups is that pretty much anyone can create them. You can shut things down to gain control, but then you lose the value of people spinning up a group whenever they need one to be productive. Your organization’s culture and governance will determine which way you go with this.

Let’s assume you keep things loose. Each time someone creates a new Group, they sort of “burn” the name they use for the group. Remember that creating a Group does a whole bunch of things behind the scenes that can’t realistically be “undone”, like creating a Site Collection for the Group, creating an Exchange mailbox for the Group, etc. This means we can run into all sorts of weird scenarios. For instance…

  • A couple of people could be going to a conference about marketing, so they create a Group called Marketing. Now the Marketing department can’t create a Group called “Marketing”.
  • We have a company meeting every year called the Extravaganza for the Company, so we create a group called EC so it’s simple to type. Now the Executive Committee can’t use that “handle”.
  • In the conversation above, Sue and I settled on the “short name” for the URL of IP2017. That’ great unless there’s a rotating Intellectual Property committee that starts work anew each year.
  • etc.

It’s not quite this cut and dried, but I’m exaggerating the problem a little bit to keep your attention.

A little planning can go a long way here. While you probably want to let people have the flexibility to create their own groups. making a few simple guidelines clear should avoid a lot of headaches down the road. Yup, another place where the dread governance word comes into play.

When you create a Group, there are two things to think about: the Group Name and the Group ID. The Group name can be changed at any time, but the Group ID cannot. Below, you can see that as I’ve typed the Group Name as testing, the Group ID has automatically been populated as testing as well. You get a hint of the significance of the Group ID because the email address the mailbox for the Group will get is listed in green below. (It’s a pretty yucky green, IMO.)

If you click on the pencil icon next to the Group ID field, you can edit the value. You’ll see the change to the email address as you type. As you type each character, there’s also a check to see if that Group ID is in use already. This is where things can get interesting.

In the screenshot below, you can see that when I type moderngroup, that Group ID has already been used. My example isn’t great, because I would be unlikely to want the URL to be moderngroup for a Group named testing, but hopefully you can see the point.

If you create the Group in the Admin interface, the UI is a bit different (Why, Microsoft???) but the result is the same: if the Group Id (note the different capitalization) has already been used, you can’t use it again.

So how can you keep things from going “off the rails”?

Odds are you’ll know about a big bunch of Groups you’ll need up front. For instance, if you want to create a group per customer team, you could use the customer number for the Group ID (12345) and then the Group could be named something like Big Realty Company (12345) or 12345 – Big Realty Company. You’d certainly want to create Groups for your departments and offices up front to reserve those names, etc.

So while there aren’t any hard and fast rules here, make sure you do some thinking about it. One of the reasons SharePoint became popular in the early days was that it grew like kudzu across the organization – it started out great, but things usually got out of hand. I feel that Groups may take the same path – and in early adopter organizations probably already have – without some rational thinking on the part of the planners in each organization.

Do you have a useful way for people in your organization to think about naming Groups? If so, please add a comment!



Greetings from Oslo, Where I’m Co-Authoring a Document

Today Julie (@jfj1997) and I were working on a document together. As usual, since we’re “cobbler’s kids”, we were emailing a Word document back and forth.

No way! We decided to share the document from one of our OneDrives and started editing online. I’m in Oslo, Norway at the Arctic SharePoint Challenge (more about that in another post) and Julie is back home in New Hampshire, so we were working with both a geographical and time zone difference, just like many people do these days.

The co-authoring experience seems a lot better than either of us remembered it. (In fact, there have been improvements to the experience over the last few months – it’s hard to keep up!)

Here are the things we really liked about this little experience…

By clicking on Review / Show Edit Activity, we could see what the other had been up to. Because I’m at ASPC2017 as a judge, I was interrupted several times (which was totally appropriate), and when I turned back to the document, I could see what Julie had been up to.

As we were editing, we could easily see what the other was doing, as there were little colored flags which moved along with the edits as they happened. In essence, it meant that we could see each other’s cursors in the document. This helped not just to see what the other was doing, but also as we were each adding new ideas into the document, we were able to avoid stepping on each other’s toes.

At one point, it felt like it would be a good idea to discuss a certain point we were making in the document. By clicking on the Chat button, we got an embedded chat window within the editing experience where we could have that discussion (and give each other a little crap when it was merited).

As with many of the new things that roll out to Office 365, I have been skeptical of the utility of this co-authoring experience. However, having been through this single experience working on a document with Julie, I think both of us are likely to use it many more times. It’s not necessarily useful in every editing scenario, but in this case it moved us forward much more quickly than any of the previous ways we might have worked.

Get Julie’s take on her co-experience co-authoring with me in her post Greetings from New Hampshire, Where I’m Co-Authoring a Document


SharePoint Online Search Isn’t Displaying What I Expect – Part 1 – Trimmed Duplicates

This entry is part 1 of 1 in the series SharePoint Online Search Isn't Displaying What I Expect

For years, I’ve been skeptical about search indexing in SharePoint, especially in SharePoint Online in Office 365. The fact that we can’t know when a search crawl has run – thus updating the indices – is a huge part of the problem. In the early days, before Content Search Web Parts (CSWPs) were available in SharePoint Online, we routinely saw delays between content creation and that content showing up in search results of days or even weeks. Later the CSWP was enabled on SharePoint Online, and it is a fantastically powerful tool, far better than the Content Query Web Part (CQWP) which it nominally replaced.

But the value of any search-driven mechanisms in SharePoint is directly tied to the recency and frequency of updates to the search index. While the CQWP is quite inefficient – since it actually goes out to look for content at the source every time it runs (though there may be some caching) – the CSWP uses the search index and can thus return results using fewer server resources in some cases. (One downside is that you can only retrieve up to 50 results with the CSWP.) Since we don’t know when the search crawls run in SharePoint Online, and we often seem to not see the results we expect, we tend to blame to indexing for the problem.

There are many things that can contribute to the indexing issues. Load on the indexing servers can mean that your tenant isn’t crawled as frequently as you might want – taking hours or in the worst cases days to display items you know should be there. Unfortunately, there is no way to know if the index is the issue. Based on what I’ve heard, there are usually multiple indexing servers per tenant, and those indices can supposedly get out of sync. Search is also a very complex beast: probably way too complicated for most use cases, as it is based on the old FAST search platform. Most people simply want to be able to see content they add to a SharePoint list or library right away if they search for it. Period. So simple, yet often not what happens.

The other day, Julie and I were certain we had an ironclad instance where search indexing simply wasn’t working correctly. The scenario seems to be a very common one, and we stripped it down to as minimal an example as possible and sent it off the the Product Group.

In the example, we were making a REST call to the search API:

to retrieve all of the events in calendars across an Intranet. The use case is a very common one: we have a calendar per department, office, etc., and in some cases we’d like to promote those calendar events to display on the home page of the Intranet. Put aside the governance questions here, and just assume that everyone who can create events gets to decide whether to check the Show on the home page box. To make this all work, we have a few custom content types which inherit from Event with a few extra columns. We have some nice, fancy display mechanisms on the home page using AngularJS and on a Company Calendar page using fullcalendar. But most of that doesn’t matter: we were seeing the issue in the call to the search API.

Our query looked something like this:

This query will return all (we thought!) list items which inherit from the Event Content Type, because its ContentTypeId is 0x0102. Of course, our actual query was more complicated: we requested specific fields with selectproperties, asked for more items by setting the rowlimit to 50, etc. But again, at the core we were simply asking for a bunch of events.

But we weren’t seeing all the events we expected. We assumed that the search index wasn’t working correctly, just like most site admins would.

There was a series of meetings going on about some HR changes, and the company was giving employees a set of webinars from which they could choose one to attend. The events were at four different times during the day. In our call to the search service, we were only getting one of those events. It happened to be the first one added to the calendar; all events had been added over 24 hours before.

When we tried searching for the title of the events in the regular old search box, we still only saw one result. At least that was consistent, and it showed we weren’t doing something stupid in our REST call. I’ve had to blur a number of things out in this screenshot, but here’s what we saw on the search results page. In this case, the results were coming to us in /_layouts/15/osssearchresults.aspx for the particular subsite where the events were, but it didn’t matter if we tried using the search center.

I included Mikael Svenson (@mikaelsvenson) in my email to my Product Group friends because if there’s something about search Mikael doesn’t know, it isn’t worth knowing. I probably should have just asked him in the first place, but we truly believed we had found a bug.

Mikael spotted that all four of the events we expected to retrieve had the same title. This isn’t so unusual: a couple of meetings on a given day with the same title. Maybe we have five interview slots set up for a new candidate, or have several different times when the Red Cross is running a blood drive on the same day, or exactly the example we have here.

We probably should have realized something was wrong when we looked at the search results above. As you can see below – now that I’ve highlighted it – there were three plus one items shown in the histogram for the Modified date.

It turned out that because the four events were so similar, search was considering them duplicates. Of course they weren’t duplicates to us: each is a unique event with its own value to end users.

By adding trimduplicates=false to our REST call, we were able to retrieve all of the events we expected. It was a very simple fix, but given the black art of SharePoint search, not necessarily an obvious one. Perhaps we should have known better, but I don’t think this is an unusual problem. Add to that the fact that the standard SharePoint search results UI doesn’t give you any way to see the duplicates, and I think there is a significant issue.

I’ve made a suggestion on the SharePoint UserVoice to Allow easier management of the trimduplicates setting.

When we search for content, we often (some might say “usually”) need to see all results for our search query. It seems that by default, trimduplicates is set to true in SharePoint Online. This seems to be true in the search Web Parts and in the search API.

My suggestion is that we have far better and clearer ways to manage this setting, including:
* An easy toggle in the Content Search Web Part (CSWP)
* A clear way on the search results pages to choose to show duplicates where there are any. This was present in earlier versions of SharePoint, and I’m not sure when it was removed.

Deciding when duplicates are appropriate is a complex thing, and it varies greatly by use case. Giving people setting up SharePoint pages simpler control over the setting will both help build compelling user experiences on the platform and help confirm that search is indeed working properly. When someone searches for something they know is there and it doesn’t show up in search results, it undermines faith in the entire platform.

If you feel that my ideas have merit, please vote this suggestion up! Suggestions at UserVoice with enough votes truly do get the SharePoint Product Group’s attention.

If you find yourself in a situation like this, there is a tool that can be helpful to solve whatever might be going on. If you do much work with search, check out the SharePoint Search Query Tool on Codeplex. Mikael has contributed to this tool and it basically allows you to issue REST calls through a UI that “understands” SharePoint search very well.

In the screenshot below I’ve done a search against our Sympraxis tenant using the querytext='test'. That’s certainly nothing fascinating, but it points out a few of the useful aspects of the tool:

  • Simple querytext configuration
  • Easy on/off switches for the various query options; in this case unchecking Trim Duplicates was the winner.
  • An easy way to see the effect of your settings on the REST call on the URL (right at the top of the screen)
  • On the right side, a clear view of the results returned, using a number of useful formats. If you’ve written JavaScript to parse search results, you’ll know that this is really helpful.

As a little bonus, here’s the key JavaScript we use to parse the RelevantResults table from the REST call to the search API. Because we’re requesting items  which are all based on the Event Content Type, we can treat all search results the same way. In this example, we’re using AngularJS and jQuery, preparing the data for use with fullcalendar, as I mentioned above.

Hopefully this post gives you a little more insight into the inner workings of SharePoint search. To me, these little eddies and backwaters of search are what turns it into a black art. I’d love to see things get even simpler that the so-called “Quick Mode” in CSWPs.

Thanks again to Mikael and the Product Group folks who engaged on this with me. Notice that this is Part 1 – I expect to add more entries to this series.


Mikael pointed me to these two articles on duplicates and “shingling” in case you’d like to understand the underlying principles more fully.

Mikael’s post about duplicate trimming

SharePoint Search Query Tool

SharePoint UserVoice suggestion:

Fixing the “Bad Request – Request Too Long” Error with Office 365 in Chrome

Bad Request - Request Too Long

This one really bugs me when it happens, and it’s pretty frequent. If you try to navigate to your Office 365 Settings or the Admin Portal in Office 365 with Chrome, you may well get this error instead of arriving at your regularly scheduled destination. I believe you may have a similar experience in Firefox as well.

I’ve gone Bingling for clues more than once on this one and I find lots of complaints about it happening, but the only solution seems to be “delete your cookies”. This is not an acceptable answer by any stretch of my imagination, but apparently it actually does work.

Eagle-eyed Twitter follower Harold Gale (@1wisegeek – aptly named!) spotted my compaint about it today and came to the rescue with a post with a very targeted fix.

In the hopes that it will help a few more people (and maybe rise a little higher on the Bingling charts so that they can find it!), here are the steps to the solution, temporary as it may be.

  • Go to the the Chrome Settings – usually this will be under the three vertical dots (sideways elipsis? leaky |?) in the top right of your window.

  • At the bottom of the page, click “Show advanced settings…”
  • Under Privacy, click the “Content Settings” button
  • Click on the “All cookies and site data…” button

  • In the search box in the upper right, search for “”


I found all these cookies after that search. Note the *5* “testcookie” entries. Enterprise-class!

You can also search for “office365” and/or “” and delete some of those cookies as well.

Note that I have no idea what each of those cookies does, and you will want to be thoughtful about what you delete!

My hope is that this post gets few reads because Microsoft solves the issues here, but the Office 365 login “experience” has been less than stellar for many years now. I’m not going to hold my breath, and at least I know I have this post to help future me out more easily.

Dear Microsoft: Please Fix Retrieving SharePoint Lookup Columns with REST When the Lookup List is in Another Web

I love SharePoint. I really do. I especially love writing client side code to build awesome applications for my clients.

Today’s annoyance, though, comes while I am in the process of rewriting an application I built on SharePoint 2007, porting it to SharePoint Online in Office 365. This ought to feel like a huge leap forward technologically, and in some ways it does. I’m changing all my SOAP calls with SPServices to REST calls. I’m switching from KnockoutJS to AngularJS, which will simply perform better given the profile of the applications. (KnockoutJS was the right choice years ago when I first built the applications, but the data and feature requirements have outgrown it.)

Unfortunately, I’m running into a simple constraint that makes my life a lot harder. When I first started building these applications five years ago, I created what I’ve got to say is a very solid information architecture. It’s withstood shifting needs and requirements in the interim, and I stand by it. One of the aspects of this good information architecture is storing commonly used reference lists in the root site of the Site Collection. By creating a Site Column which is a lookup into each reference list, I can reuse those common reference values throughout my subsites.

This works great in SharePoint 2007 with SOAP calls. When I retrieve items with one of these lookup Site Columns from a list in a subsite, I simply get the ID and Title values, separated by a “;#”. However, when I try to do the same thing with “modern” REST calls, I get an error like this:

I’ve been a good team player, and I’ve suggested they fix this on the SharePoint User Voice in my suggestion Enable support for lookup columns in other webs in the REST API. The votes are up, and it’s been a while.

There’s a workaround, but it’s not very pleasant. (The easiest workaround is to simply stick with SOAP calls and SPServices – I’ve done that in several cases in other projects. But SOAP is officially “deprecated”, so…)

Here’s a specific example. The client I’m working with is in financial services, and they issue recommendations on securities. Those recommendations are very standard, and predictable: Hold, Buy, Sell, etc. In other words, perfect to store in a list in the root site called Recommendations. Why not a Managed Metadata column, you might ask? Well, I also wanted to store several other columns in the Recommendations list, like Description (e.g., “The analyst expects the security to outperform their coverage universe.”), a SortOrder value so I could rearrange the values in dropdowns using SPArrangeChoices, and several other fields which drive configuration of some reports. In other words: great information architecture. The values are all consistent across the various subsites, I store them once, etc. Nice setup.

I created a Site Column back in the beginning called Recommendation, which is a lookup into the title column of the Recommendations list (Hold, Buy, Sell, etc.). I used that Site Column in many Content Types defined on the subsite level. Those Content Types are mainly used in a list I’ll call Notes.

In SOAP with SPServices, I can make this [simplified] call:

This retrieves the items and returns nice JSON for me. Because Recommendation is a lookup column, it comes back as something like “1;#Buy” and that’s easy to turn into a JSON object like:

Easy, peasy.

However, when I try the analogous call in REST:

I get the error:

In other words, there’s no way to $expand the Recommendation column because it comes from an other Web, even though that is ideal information architecture!

The workaround, which André Lage (@aaclage) pointed out in my UserVoice suggestion (but I clearly didn’t get at the time), is to simply ask for the Recommendation column’s ID instead. This isn’t obvious at all:

This doesn’t follow the syntax we’d expect: we need to append “Id” to the end of the lookup column’s InternalName. Of course, this just gets us the ID of the item in the Recommendations list; it doesn’t fetch us the Title value, which is what we really want. Because of this, I need to do a *separate* REST call to get the items from the Recommendations list and merge the values in my client side code.

Now, one could argue that this is more efficient. I don’t ask the server to $expand the values across thousands of notes (yes, there are way more than 5000; I’ve written enough about that lately – I may have mentioned it here and here and here and here), so it gets a break. Retrieving the 5-10 values in the reference list (in this case) is no big deal.

But I have a half dozen or so of these lookup columns to deal with in this application, which means a half dozen extra REST calls, plus the code to merge the values. More work for me, but more importantly a longer wait for the application user when they load the page. I believe that poor UX is what has doomed many a SharePoint roll out, and I loathe creating a poor UX myself. In this case, I’ll make it work, but I’d really like to see this change.