Looping Through Content in a SharePoint 2013 Site Workflow – Part 1 – Introduction

Site Workflows let us run logic independent of lists and libraries. By combining them with REST calls, we can traverse a Site Collection and process multiple lists and their contents by looping though them when we want. In this article, part 1 of the series, we’ll describe the problem we are trying to solve, but the solution can apply to many SharePoint scenarios.

SharePoint 2013 Site Workflow

As with most things SharePoint, setting up a workflow to loop through items across subsites seems like it should be easier. What I was trying to do seemed like a common requirement: get some data from lists in SharePoint subsites and loop through it, taking some actions on each item that meets some criteria.

I’ll warn you that this series of articles is not for the faint of heart, the new-to-workflow initiate, or the person without a lot of permissions. My solution involves reaching into Office 365’s workflow guts a bit here.

In my current case, I’m building an opportunity management solution for a sales organization. Each sales partner – these are outside companies – will have a subsite only they can access where they will enter sales opportunities as they arise. The sales folks for the host company want to be able to manage that “basket” of opportunities as well as send out emails to the partners as the opportunities move through various stages in a process. We won’t cover all of the possible permutations here as I want to focus more on the chassis of the workflow.

I need to check a specific list in a bunch of subsites for new items and send an email about each. I don’t want to create a separate List Workflow that sits in each of the lists because it would be an administrative nightmare to change the workflow down the road (we have over a dozen subsites, and the number will grow). Instead, I want to create one central SharePoint 2013 Site Workflow that finds all the subsites, finds the list in each subsite (if it exists), and iterates through the items in the list, taking different actions based on the data in each item. We’ll set this workflow to run every x minutes (probably once an hour or once a day) by adding a wait state at the end of processing.

You could accomplish this with “code” or Powershell, but that takes it out of the hands of the Site Admin who might want to change things down the road. By using a workflow, we can give them some control over things like the text of the emails without making them learn how to code to change them. As a consultant, I want to be able to leave my client with a solution they can maintain on their own as much as possible. They can still call me for the heavy lifting, but most changes should be something they can handle themselves.

On Office 365 in SharePoint Online, this feels a LOT trickier than it may have been in the past because of the complexity (nay, swamp) of authorization we must navigate. I expect that similar authentication issues may apply on premises, too, but Office 365 is more of a moving target.

In this series of articles, I’ll show you the steps I went through and let you know where I got stuck, waylaid, and confused. I believe that only one of us should have to suffer through this stuff so that everyone else can avoid it.

We’ll go through the overall concept of how the process works, as well as:

  • Setting up the workflow with the proper permissions so it can traverse content across the Site Collection
  • Initializing the workflow based on values stored in a user-maintainable list
  • Making the REST calls to discover all of the existing subsites without having to hard-wire where they are
  • Parsing the data returned from each REST call
  • Discovering whether each subsite contains the list we are looking for
  • Processing the items in each list based on a set of criteria that can change over time
  • Sending out emails appropriate for each set of criteria

This isn’t totally new ground; there are many blog posts out there that cover bits and pieces of this, and I’ll provide the links I found helpful and give credit where credit is due. I didn’t find anything that showed me how it all worked together, though.

Thanks right off the bat to Matt Bramer (@ionline247) and Fabian Williams (@fabianwilliams) for getting me up and rolling when I first start whining about this on Twitter.

This article was also published on IT Unity on 1/19/2016. Visit the post there to read additional comments.

SPServices into 2016!

SPServicesOver the break, I managed to get SPServices up and running on GitHub. You may wonder what that means and why it matters.

When I started moving SPServices away from Codeplex, it was due to several things:

  • Codeplex seems to be dying a whimpering death. Microsoft is clearly not maintaining it anymore, and its tech is falling behind – when it works.
  • I wanted to understand GitHub better. I’ve had to get too much help from my friends in the SharePoint community even just to push changes to cdnjs. Thanks Josh McCarty (@joshmcrty)!

So, mission accomplished on those goals – but it took me an embarrassingly long time to get here!

Of course, in the process I also wanted to improve SPServices. Here is what I am/was aiming at:

  • The next version will be SPServices 2.0 – This is just a number, and the current builds have this number already.
  • Make it (already is!) AMD-enabled using RequireJS
  • Convert from a monolithic file to modules (done, though perhaps more to do)
  • Enabled to take advantage of SharePoint’s REST APIs – where available – for internal calls to get list data in the value-added functions (still to do)

As you can see, I’m well on the way to meet my self-imposed goals; I think the hard work is behind me.

There are some other goals and here’s where you can help:

  • Test the “pre-alpha” builds of SPServices 2.0. If you’re familiar enough with the library to drop builds into your test environments, that would be a great help. I’ve tested using the same lists and pages I always use, but more real-world testing would be good. Report any issues you find using GitHub issues.
  • Write some tests. I’ve started writing tests with QUnit, but I’ve only scratched the surface. Writing good tests here is difficult, as we have to be sitting on top of SharePoint; in greenfield development, we can test anywhere. You’ll find some instructions for how to use the existing tests on the GitHub pages.
  • Migrate the documentation from Codeplex to GitHub – Since Codeplex is falling apart, there’s no reason to leave the documentation there, either. There are a few dozen pages (I can’t actually count the pages on Codeplex easily) of documentation, and it’s probably easiest just to move the over to GitHub manually.
  • Move the discussions off Codeplex – This one is hardest, I think. IMO, one of the big values to SPServices is the historical discussions about how to use it. But those discussions have covered many other things as well, and I’d hate to lose any of it. I’m not sure how to go about this, so if anyone has some experience moving forums like this, I’m all ears.
  • Propose improvements – I ask the community for suggestions all the time, but I don’t get a lot of them. If you’ve solved some gnarly SharePoint UI problem and would be willing to submit your code or just wish that someone would fix the darn _____, then let me know in the GitHub issues. Consider the issues our own UserVoice for SPServices.

A HUGE thank you goes out to Paul Tavares (@paul_tavares) for guiding me along this move to GitHub. He’s written most of the build code, the test chassis, etc. I couldn’t have done it without him. If you don’t know Paul, he’s one of the sharper tacks in the SharePoint community, and he only does SharePoint work on the side these days. Take a look at his SPWidgets project. I promise you – you’ll be impressed.

Thanks also to the folks over at BitHound for their code tools. If you check out the dashboard for SPServices, you’ll see that I’m doing OK, but there’s always room for improvement.

Happy 2016 to all of you, and happy coding!

An Unpleasant Refiner Bug in SharePoint Online

As always, I don’t know where to report issues with Office 365. So here I am carping in a public channel in the hopes that the right person sees it. This also may happen with SharePoint 2013 on premises, but I can’t recall.

Unpleasant SharePoint search refinerWhen I use a refiner that is based on a Lookup column, I’ll often see something like the attached image. As you can see, there are two clients listed: Anchor Glass and 29. Anchor Glass is the title of the item in the Clients list with the ID of 29, but it still doesn’t make sense (especially to those poor users). It’s as if the indexer isn’t smart enough to handle Lookup columns properly – but only some of the time.

This happens on multiple tenants, with different Lookup columns to different lists, but not consistently, or in any pattern I’ve been able to discern. Generally it’s the Title column we’re looking up into. Using a Lookup column into a list is almost always preferable to using Managed Metadata (IMO) because we can store additional data about the values in the list. For instance, in the Clients list above, we have the client contact info, who works with the client, when we started working with them, what the billings are, etc. Managed Metadata is worthless for that.

But the bug.

Has anyone else seen this? Any ideas on how to work around it?

SharePoint Online Search: Add a Refiner for Content Type

2015-12-21_14-00-52This is yet another in the “Things that Make You Go Hmm…” category. If I only had a nickel…

I would think that one of the most common needs on a search results page would be to add a refiner by Content Type. After all, we all have robust, sensical information architectures in place to improve performance in our organizations, right?

When we create a new search results page in a Search Center, we get a few refiners out of the box that look oh-so-promising.

First there’s ContentTypeId. We all know that Microsoft uses big, ugly ids and GUIDs under the covers all the time. Sometimes they translate easily into whatever they represent. This isn’t one of those times.

2015-12-21_13-37-48

Well, what about contentclass? That looks sort of almost useful. But it isn’t. Values like STS_ListItem_DocumentLibrary won’t really help anyone.

2015-12-21_13-37-35

Oh, I know. Let’s add the ContentType Managed Property. That’s what works almost everywhere else. Well, except here, as the values don’t make sense to humans, as they are things like application/pdf Document or application/vnd.openxmlformats-officedocument.presentationml.pres… Yes, the ellipses are appropriate here. Who know what that means?

2015-12-21_13-38-05

Finally, I found a blog post from Henri Merkesdal that retrieved my sanity. There’s a property way down the list called SPContentType that does exactly what we want.
2015-12-21_13-38-20

Ahhh. That’s much better. Thanks, Henri!

2015-12-21_14-00-52

CollabTalk Episode 12: Top 10 2015 News and 2016 Predictions

CollabTalk Episode 12 - December 2015We wrapped up our twelfth episode of CollabTalk for ITUnity yesterday, which means we’ve been at it for a full year now. Boy, how time flies! When I say we, I of course mean my illustrious friends Christian Buckley (@buckleyplanet), Naomi Moneypenny (@nmoneypenny), and Benjamin Niaulin (@bniaulin). Oh, and me.

In this episode, we did something a little different than in the prior eleven. Since it’s the end of the year – the time when everyone takes the time to look back on a year (hopefully well-lived) and look forward to what might come – we went through what we considered the big Office 365 news from 2015 and then gave a few predictions for 2016.

I proved in this exercise at least one of the following things: I am bad at making lists, I’m not highly motivated to do things that are important, I’m way too busy, I can’t count, and/or I’m lazy. You decide. Whichever it is, I came nowhere close to 10 items per list.

Big News from 2015

I think Office Graph was a 2014 thing, but people are really starting to see how cool it is

Modern organization dynamicsLast year at this time, we knew what the Office Graph was, but most people weren’t convinced of its importance. Over the course of 2015, we saw tremendous advances with Delve and the available analytics on Office 365, all driven by Office Graph data. I’m convinced that the Office Graph is one of the most important tools in the Microsoft toolbox.

People seem to be seeing this promise more and more now. Part of this is that some of the sharpest tacks in the drawer, like Waldek Mastykarz (@waldekm) with posts like these and Mikael Svenson (@mikaelsvenson) with posts like these have made the concepts more accessible. We’re also seeing Microsoft building more capabilities on top of the Office Graph to show its importance. In a larger organization, those “edges” of connection that live outside the normal organizational hierarchy are the ones where Things Get Done.

News of the next gen Office 365 dashboard to make admin tasks simpler

office-365-admin-center-preview-tour-3To me, one of the weak points of SharePoint has always been the back end management tools. In the old days, then mindset was “It’s for IT Pros and they don’t care how it works.” Well, on Office 365, many admins are people inside small or medium sized businesses (SMB) who don’t want to have to work to understand the admin tools – they want them to work and be easy to use. The SMB market is a sweeter spot for Office 365 then the enterprise market to me. If Microsoft can make it work well for the “IT not-Pro” [I’ll keep my editorial comments about the “Pro” moniker for professions out of this post. Sort of.], then they will clobber that part of the market. That’s the very market that Google has been creeping on on them through with their far-better admin tools.

We’ve seen a preview of the new admin dashboard in First Release tenants. I’m looking forward to seeing it fleshed out to cover more and replace the gobbledy-gook that has been SharePoint administration in the past. We can only hope that those improved tools com back to on premises in SharePoint 2016 as well. As much as IT Pros like to say they’d rather us e a command line, it’s far cheaper and reliable to just give the damn simple tools.

Platform independent apps – we now can do Outlook, and all the other Office things on iOS and Android(?)

Microsoft Apps on iOSThis move has been brilliant. Microsoft has realized that they shouldn’t try to drag the people back to the desktop; they should take their productivity tools where the people are. And the people are on their mobile devices. Microsoft decided it doesn’t matter who made the device – they want to own the productivity software business regardless what the hardware is.

Now on iOS we have better tools from Microsoft than we do from Apple. Not only that, but they are all backed by the data store that is OneDrive – in this case, two of the three OneDrives. Say what you want about OneDrive sync – this content integration on mobile is killer . (And besides, the sync client on iOS seems to work flawlessly – it’s Microsoft’s home turf of the desktop where the problems seem to be the biggest.)

Purchases of Accompli, [that task thing]

Microsoft hasn’t been on an acquisition tear, but they have made some important purchases to get their foot in the door on the iOS platform. Accompli and Wunderlist (that’s the one I couldn’t remember!) are two examples of this. Sure, Microsoft could have just built their own versions of these two products, but instead, they bought a *very* loyal and hip user base. In the case of Accompli, it meant they could launch a new app – re-branded as Outlook – in a very short time. (I’m an iPhone guy, so I can’t speak to other phone OSes. YMMV.)

Microsoft won’t be able to buy its way into the hearts and minds of the average consumer, but by starting with these two apps and keeping them hip they made big strides forward in the app wars.

And finally…what Ben said

Ben had a really good list. Unfortunately, I saw it before I wrote mine. That’s a motivation killer – at least to me.

Predictions for 2016

Microsoft will be seen as cool again

Is Microsoft cool now?Most people know that Microsoft really lost its mojo in the decade or so that Steve Ballmer was in charge. The products were ho-hum, and the attitude was very much “You’ll buy what we tell you to buy.” It was a sales-driven organization and it showed.

In the last year (or a bit more), Microsoft has found better mojo than it has had for years. When has anyone thought of Microsoft as hip? I’ve been around for the entire lifespan of the company, and I can’t recall that time. With the things they are doing now on Office 365, in mobile, and in hardware, even some of the stalwart cynics are sitting up and taking a good, hard look. I’ve even heard a few Apple fanboys say they have considered jumping over to Microsoft – either again or for the first time. This is almost unheard of.

Office 365 will become a $7B businessOffice 365 Logo

I’m not a financial guy and you should never take my investment advice. That said, I see Office 365 growth rates heading north; the integral of that curve is going to grow. As I said above, to me the sweet spot of the market for Office 365 is the SMB market. With the improvements to user experience Microsoft is putting in place, more and more small business are going to trust them to take a lot of the headache of IT off their plate for them. Whatever the numbers turn out to be, revenue is going to climb significantly in 2016.

Microsoft will write off a significant part of its Yammer investment

Yammer LogoYammer is a dog. I’ve been saying it for a long while now. There are members of the CollabTalk panel who disagree with me vehemently, but I’m sticking to my guns. I know it doesn’t make me a popular guy with the political people in Redmond either, but many Redmond-ites have to feel this, too. Microsoft bought Yammer for some of the technology it had, IMO, and even that technology hasn’t proven to be all that stellar. Add to that a “we know better then you” attitude in Yammer organization, and it’s a dog that don’t hunt. Things that virtually every user struggles with and wants fixed (unread counts, anyone?) are still a problem. Yammer doesn’t fit into the portfolio as it is. Perhaps it has some value if it’s cut up for parts, but even that seems like a stretch to me. Again, I’m not a finance guy, but I see some sort of write off or write down or whatever on Yammer. At the very least, we should see it slink away and hide under the barn.

Conclusion

So there you have it. My big news for 2015 and predictions for 2016. You probably don’t agree with everything I have on these lists or what I’ve said. I welcome your thoughts and feedback in the comments.

If you’d like to watch the entire CollabTalk episode, you can do so here. If you register, the good folks at IT Unity will let you know about upcoming episodes and maybe send you some other goodies via email as well.