Looping Through Content in a SharePoint 2013 Site Workflow – Part 3 – High Level View of the Workflow

This entry is part 3 of 3 in the series Looping Through Content in a SharePoint 2013 Site Workflow

As I discussed in the first part of the series, my goal is to find all the sales opportunities in a Site Collection and process them based on their characteristics. To do this I’ve created a Site Workflow – SharePoint 2013 flavor – that makes a set of REST calls to traverse the Site Collection to get the items we want to process.

Site Hierarchy
I’m working in a Site Collection with a fairly simple hierarchy, most of which I inherited. There’s a root site, of course, then a number of subsites – one per business partner – and one subsite for application management. The workflow will run in the management site (only Site Owners are allowed in there) and reach out across the rest of the sites in the Site Collection.

Looking at the details of a workflow like this right off the bat is probably a little too detailed, so let’s look at a conceptual picture. If you’ve used SPServices or REST calls to traverse sites before, this may not seem too complicated for you. If not, it may look like a nice picture (maybe – the colors are a little much!), but otherwise not make a lot of sense.

Conceptual overview of the workflowWorkflow Steps

The orange boxes represent App Steps. App Steps are only available to us if we give the workflow app permissions, as we did in the previous article. We need the App Steps so that we can make REST calls into sites other than the one where the workflow is running. When I tried to run my workflow without setting up the app permissions, each of the REST calls (now contained in App Steps) failed due to an authorization issue.

Stage: Get Authorization info

The first thing we’re doing is getting a digest token, which amounts to a way to authenticate the workflow when it does writes back to any of the lists. When we read information in the workflow, we don’t need this authentication info, but we do if we want to write anything. I’ve included the step in my workflow because I’m pretty sure I’ll want to be able to update some of the sales opportunities based on their status.

Stage: Get Webs

The next stage is where we make a REST call to retrieve the list of subsites (also known as Webs) which sit directly below the root site of the Site Collection. To do this, I need to make a call to the Webs REST endpoint. This is why we needed to set the workflow up with app permissions at the Site Collection level.

Stage: Process Webs

Once we have the list of subsites we can loop through them to process them.

Loop: Process Web

This loop contains the logic I use to process anything I need within each subsite. In the first few actions in this loop, I parse out the details about each subsite (title, relative URL, etc.) into variables.

Step: Process Opportunities

Using the variables I set up in the beginning of the Process Web loop, I make a REST call into the subsite to see if the Opportunities list is there and has sales opportunities in it which meet our criteria. (For example, we might filter out the opportunities which are already sold.)

Loop: Process Opportunity

If I do get a list of opportunities back from the call above, I process each of the individual opportunities in the Process Opportunity loop. This loop is where the “meat” of the workflow lives. All of the other steps simply exist to get us here so that we can take actions on each of the opportunities that meet whatever criteria we have defined. As the business process gels more (we’re designing it as we build the workflow), we may end up with multiple loops like this. For instance, we could have one loop which sends out emails about opportunities that are late; another loop to alert the sales team about big opportunities of note; and another loop that bumps up the commission percentage based on the quality of the opportunity, etc.

Building and Debugging

Thinking through how your workflow should work at a high level like this is sometimes useful. I’m more of an iterator, though. I tend to build one part of things and test it, then bolt on some more logic and test it, etc.

One thing I’m a huge fan of is emitting some sort of debugging messages along the way. This probably harkens back to the good old days where that was the only debugging technique we really had.

In any case, each of the phases, steps, and loops above emits something into the workflow history list using the Log to History List action.

Log workflow messages

As I build up the workflow, having a clear log of what’s happening in the workflow history list is a great debugging tool. The screenshot below shows the kind of thing I tend to do.

Workflow log messages

In this case, I can see that I found 15 subsites. Then when I processed the first subsite, I found 3 opportunities to work with. For each opportunity, I can see the customer name (this is test data, of course!), and so on.

The workflow will get more and more complex, but by emitting these messages, I have a fairly decent way to see what’s going on. Debugging these workflows is not a lot of fun – it’s slow going and sometimes hard to see what’s gone wrong – but at least I can look at my log and see where things might have failed.

In the next few articles, we’ll look at each of the phases, steps, and loops I’ve outlined above to see what’s going on in more detail.

Looping Through Content in a SharePoint 2013 Site Workflow – Part 2 – Setting Up App Permissions

This entry is part 2 of 3 in the series Looping Through Content in a SharePoint 2013 Site Workflow

Site Workflows are great, but if you want them to reach across into other sites you’ll probably have permission issues. By giving these workflows App Permissions, you’ll be able to let your workflow access content across your Site Collection.

Add XML

In part one of this series I described the basic task we’re trying to solve. We want to create a SharePoint 2013 site workflow that can iterate through subsites and then take some actions on each item that meets some criteria. At the end of that article, I said that the first step is that we’ll set up the workflow with proper permissions so it can traverse content across the Site Collection. So now you’re thinking, “Great. The problem is defined. Let’s pop over to SharePoint Designer and create a workflow.” Unfortunately, that isn’t the right thing to do. I know because I tried that, and I’m writing this series to pass on the lessons learned.

Before we can even think about writing the workflow, we should set up the authorization side of things. It wasn’t enough to get the digest token from the server for the REST calls. (More on this later.) When I tried to make the REST calls I needed from one subsite into the root of the Site Collection or another subsite, I got authorization errors.

Instead, there’s more heavy-duty permissions set up required to make the REST calls into other Webs (sites to normal people). We need to give our workflow App Permission, which isn’t obvious as you’re working on the workflow in SharePoint Designer (SPD), nor is it possible to do so in SPD.

Luckily I have good friends in the community. Matt Bramer (@ionline247) was the first person who replied to one of my tweets, and he gave me some good tips. Through him I found that Fabian Williams (@fabianwilliams) had written several blog posts about the trials and tribulations he went through so that I  wouldn’t have to. (I added links to Fabian’s blogs at the end of this article.) But setting up workflow App Permission is still confusing, in my opinion.

You’ll go through two two main steps to set these permissions. I found the steps generally documented in the MSDN article, Create a workflow with elevated permissions by using the SharePoint 2013 Workflow platform, but here’s my version.

Step 1 – Allow the workflow to use app permissions

Go to Site Settings in the site where you want the workflow to run and turn on the Workflows can use app permissions feature. You can do this by going to Site Settings and in the Site Actions section, choose Manage Site Features. Toward the bottom of the list of available features, you’ll see Workflows can use app permissions. To activate it, (figure 2) simply click the Activate button and wait for the activation to complete.

Workflows can use app permissions

Figure 2: In Site Settings / Site Actions / Manage Site Features you need to Activate the feature to allow workflows to read from and write to all items in the site.

Step 2 – Grant full control permission to a workflow

This step gets pretty weird. We have to give this workflow the permissions to run as if it’s an app add-in. (Though one might say that it’s an app add-in already – everything is an app add-in, right?)

In the site where you want the workflow to run, navigate to Site Settings and then under Users and Permissions, go to Site app permissions. If things have gone according to plan, you should see Workflow as one of the two or more items listed here.

What you want to grab is the yellow text in figure 3. It follows a bar character – “|” – and precedes an “@” character. I’m not sure exactly where this GUID comes from, but it’s what you need.

2016-01-12_11-05-44

Figure 3: Copy part of the Workflow App Identifier

Now you have to navigate to a hidden, super-secret page at:

http://yourtenant/sites/yoursitecollection/_layouts/15/appinv.aspx

In the example on MSDN, they send you off to the /sites/AppCatalog Site Collection for this. That’s only because they are showing an example interacting with a list there. You should go to the secret page in the Site Collection where you want the workflow to run. (Yes, I was down a rat hole on this for a while. Maybe I was just being stupid. It seemed like the good folks who wrote the MSDN article were telling me I needed to give permissions on the App Catalog site for this to work – but that wasn’t the case.)

The should look something like figure 4. The page doesn’t have a title like most SharePoint pages (hey, it’s super-secret), but the hover text for the info icon at the top of the page says “Grant permission to an app.”

Set the app permissions

Figure 4: You’ll paste the Workflow App Identifier into the Add Id field on the secret page.

Now as you can see in figure 5, you paste in the yellow text from above, and click the Lookup button. The next three fields will be filled in. You shouldn’t need to change anything in those fields.

Paste in the App ID

Figure 5: Paste in the App Id that you copied from the workflow in Site app permissions (figure 3).

Next, you need to paste some XML into the last field called Permission Request XML. Don’t try to get all smart about it like I did: paste in EXACTLY this XML— don’t make any changes.

You have two options here (the first option adds /web in line three) and I’ll explain the difference below the XML snippet:

<AppPermissionRequests>
  <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="FullControl" />
</AppPermissionRequests>

…or…

<AppPermissionRequests>
  <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
</AppPermissionRequests>

In this case, I went with the latter; I want my workflow to have permissions on the entire Site Collection. The former would give my workflow permissions only in the subsites of the Site Collection; I needed permission on the root of the Site Collection because I wanted to iterate through the subwebs (subsites) from there. So with both the App Id and the XML pasted, the app identity and permission look like figure 6.

Add XML

Figure 6: We’ve identified the workflow’s app’s identity and indicated where it has permissions to be used.

Click Create.

Next you’ll be asked if you trust the workflow. One would think this would be a given, but click Trust It.

 Site App Permissions

At this point, you have the permissions set up that give you the possibility of looping through the subsites off the root, even though you haven’t created an actual workflow yet. (Or, if you’re like me, you beat your head against the workflow first and then realized you had to figure this part out.)

In the next article, we’ll talk through what the workflow needs to do on a high level to be useful.

References for this step

This article was also published on IT Unity on 1/25/2016. Visit the post there to read additional comments.

Looping Through Content in a SharePoint 2013 Site Workflow – Part 1 – Introduction

This entry is part 1 of 3 in the series Looping Through Content in a SharePoint 2013 Site Workflow

Site Workflows let us run logic independent of lists and libraries. By combining them with REST calls, we can traverse a Site Collection and process multiple lists and their contents by looping though them when we want. In this article, part 1 of the series, we’ll describe the problem we are trying to solve, but the solution can apply to many SharePoint scenarios.

SharePoint 2013 Site Workflow

As with most things SharePoint, setting up a workflow to loop through items across subsites seems like it should be easier. What I was trying to do seemed like a common requirement: get some data from lists in SharePoint subsites and loop through it, taking some actions on each item that meets some criteria.

I’ll warn you that this series of articles is not for the faint of heart, the new-to-workflow initiate, or the person without a lot of permissions. My solution involves reaching into Office 365’s workflow guts a bit here.

In my current case, I’m building an opportunity management solution for a sales organization. Each sales partner – these are outside companies – will have a subsite only they can access where they will enter sales opportunities as they arise. The sales folks for the host company want to be able to manage that “basket” of opportunities as well as send out emails to the partners as the opportunities move through various stages in a process. We won’t cover all of the possible permutations here as I want to focus more on the chassis of the workflow.

I need to check a specific list in a bunch of subsites for new items and send an email about each. I don’t want to create a separate List Workflow that sits in each of the lists because it would be an administrative nightmare to change the workflow down the road (we have over a dozen subsites, and the number will grow). Instead, I want to create one central SharePoint 2013 Site Workflow that finds all the subsites, finds the list in each subsite (if it exists), and iterates through the items in the list, taking different actions based on the data in each item. We’ll set this workflow to run every x minutes (probably once an hour or once a day) by adding a wait state at the end of processing.

You could accomplish this with “code” or Powershell, but that takes it out of the hands of the Site Admin who might want to change things down the road. By using a workflow, we can give them some control over things like the text of the emails without making them learn how to code to change them. As a consultant, I want to be able to leave my client with a solution they can maintain on their own as much as possible. They can still call me for the heavy lifting, but most changes should be something they can handle themselves.

On Office 365 in SharePoint Online, this feels a LOT trickier than it may have been in the past because of the complexity (nay, swamp) of authorization we must navigate. I expect that similar authentication issues may apply on premises, too, but Office 365 is more of a moving target.

In this series of articles, I’ll show you the steps I went through and let you know where I got stuck, waylaid, and confused. I believe that only one of us should have to suffer through this stuff so that everyone else can avoid it.

We’ll go through the overall concept of how the process works, as well as:

  • Setting up the workflow with the proper permissions so it can traverse content across the Site Collection
  • Initializing the workflow based on values stored in a user-maintainable list
  • Making the REST calls to discover all of the existing subsites without having to hard-wire where they are
  • Parsing the data returned from each REST call
  • Discovering whether each subsite contains the list we are looking for
  • Processing the items in each list based on a set of criteria that can change over time
  • Sending out emails appropriate for each set of criteria

This isn’t totally new ground; there are many blog posts out there that cover bits and pieces of this, and I’ll provide the links I found helpful and give credit where credit is due. I didn’t find anything that showed me how it all worked together, though.

Thanks right off the bat to Matt Bramer (@ionline247) and Fabian Williams (@fabianwilliams) for getting me up and rolling when I first start whining about this on Twitter.

This article was also published on IT Unity on 1/19/2016. Visit the post there to read additional comments.

Error Saving JavaScript Files to SharePoint Mapped Drive – Minor Version Overload

Can't save JavaScript file from Sublime Text

This is a first. I was editing away in Sublime Text today and suddenly I couldn’t save my code to the mapped S: drive I was using in SharePoint 2013. S: is for SharePoint. Get it?

The error message on the PC side wasn’t all that helpful, as one might expect. (Sorry for the crappy captured image.)

Can't save JavaScript file from Sublime Text

The files I’m editing are stored in:

http://[Server Name]/sites/[Site Collection Name]/_catalogs/masterpage/_[Client Name]

I’ve found that putting all of my project artifacts in that one location works well because I can easily copy and paste from one environment to another, as needed. I have a folder per artifact type, such as

Master Pages, html, js, css, Display Templates, Page Layouts

etc. I know someone is going to tell me what a horrible idea that is, but it’s not horrible if it works. And it works.

I’m editing within a virtual machine in VMWare Workstation running Windows 8.1 because I can’t connect to the client’s VPN with Windows 10. I originally was using a HyperV VM, but when I installed the production release of Windows 10, that went defunct on me, too. (Yeah, all that’s a mouthful.) Given all that, a restart of the VM seemed like a good idea.

No joy.

The next thing I thought of was that the disks might be full on the SQL server. I’ve seen that wreak all sorts of havoc in the past. But my admin buddy told me everything was humming along and healthy.

My next move was to download SharePoint Designer onto the VM and try editing the files there.

Aha! When I tried to save my edits, I got this message, which I had never seen before:

Minor version limit exceeded

Because my files are in the _catalogs/masterpage folder, we inherit versioning and Check In/Check Out. That’s a good thing because I can always restore an older version if something dreadful happens.

Clearly this has been a project with a lot of Ctrl-S action. It turns out that I had 510 minor versions on one file, and that’s the limit. Nada mas.

Too many minor versions

As soon as I published a major version (in this case, the magical 1.0), I was back up and running.

Yeah, I probably should clean out all those old versions at some point, but disk is cheap.

BTW, this is a really cool project where I’m using KnockoutJS, KO,mapping, fontawesome, jQuery, jQueryUI, jQuery.cookie, fullcalendar, MomentJS, and more. It gets us a pretty and highly useful veneer over basic SharePoint functionality to match the business needs as exactly as possible.

Simple Expand/Collapse for the Quick Launch in SharePoint 2013 or SharePoint Online

Yesterday I helped my pal Marcel Meth (@marcelmeth) with a pretty simple little task, but everything’s simple if you know how to do it! We needed to add simple accordion-like expand/collapse functionality to the Quick Launch on the pages in a Site Collection. Because there were going to be a lot of links in the Quick Launch (mostly links to different views of lists), it was getting pretty cumbersome to wade through. By adding this expand/collapse capability, we could improve the user experience (UX). It’s little stuff like this that makes SharePoint usable. Never underestimate the power of a small tweak like this.

This code works on a Team Site on SharePoint Online in Office365 with its Quick Launch exposed in the normal way. Any other customization you have in place may interfere with this working. (Yadda, yadda, disclaimer, YMMV, etc.) I’m sure it will work for you – possibly with some tweaks – in any SharePoint 2013 or SharePoint Online site. You can add this script to a single page or to the master page if you’d like it to work on all pages in the Site Collection.

// Find all the top level links in the Quick Launch that have children
var topLevelLinks = $("div[id$='QuickLaunchMenu'] > ul > li:has('ul') > a");

// Prepend the little "twiddle" icon to each top level link
topLevelLinks.prepend("<span class='ms-commentexpand-iconouter ql-icon'><img alt='expand' src='/Comm/Comms2/_themes/1/spcommon-B35BB0A9.themedpng?ctag=2' class='ms-commentexpand-icon'></span>");

// We're starting with all of the sections collapsed. If you want them expanded, comment this out.
topLevelLinks.closest("li").find("> ul").hide();

// Set up for the click even of on the top level links
topLevelLinks.click(function(e) {

  // We're going to stop the default behavior
  e.preventDefault();

  // Find the elements we need to work with
  var childUl = $(this).closest("li").find("> ul");
  var isVisible = childUl.is(":visible")

  // If the section is visible, hide it, and vice versa
  if(isVisible) {  

    // Replace the icon with its antitheses
    $(this).find(".ql-icon").replaceWith("<span class='ms-commentexpand-iconouter ql-icon'><img alt='Expand' src='/Comm/Comms2/_themes/1/spcommon-B35BB0A9.themedpng?ctag=2' class='ms-commentexpand-icon'></span>");
    // Hide the child links by sliding up. Note: You could change the effect here.
    childUl.slideUp();

  } else {

    // Replace the icon with its antitheses
    $(this).find(".ql-icon").replaceWith("<span class='ms-commentcollapse-iconouter ql-icon'><img alt='Collapse' src='/Comm/Comms2/_themes/1/spcommon-B35BB0A9.themedpng?ctag=2' class='ms-commentcollapse-icon'></span>");
    // Show the child links by sliding down. Note: You could change the effect here.
    childUl.slideDown();

  }

});