Sympraxis’ SharePoint Client Side Development Pipeline – Where Should We Put Our Stuff?

As I mentioned in my previous post, when Julie Turner (@jfj1997) joined Sympraxis, we quickly realized we needed to get smarter about managing our code and our overall development process.

One of the first things Julie and I discussed was where we should store our code. Even way back before SharePoint entered my life, I’ve thought of code as content. It’s just a different type of content that has different content management requirements. For this reason, I’ve never fretted too much about storing my code in a SharePoint Document Library or in the Master Page Gallery.

Code as Content

That works fine when you’re a development team of one. But as Julie and I started working on a project together, we knew we needed to do better. Plus, angst and all that. One thing that is key here – and is key in many conversations about stuff like this – is there is no one-size-fits-all answer. Managing code is like managing any other content in that there should be some governance around it. But the governance doesn’t have to be – nor should it be – the same in every instance.

So we thought about what we wanted to be able to accomplish. Basically, what our requirements were to get rolling.

  • We needed an offsite (meaning not just on our machines) repository. This would make us worry less about disaster recovery. We were both doing backups to the cloud (me with Crashplan and Julie with Acronis), but we wanted a repository that belonged to the company, not to either of us personally.
  • We wanted to improve our code reuse. It’s not that we build the same thing over and over, but like the functionality in SPServices, there are some things we do fairly often. In other words, our tricks of the trade. By storing all of our code in one place, we hoped we would make reuse easier.
  • We work with clients on project-based work. Sometimes we work with a client for a while, then they take over for a while, and we reengage with them when a new need arises. We wanted to make it easier for ourselves when that re-engagement happened: basically improve our speed-to-useful again.

As Julie mentioned in her post, she had used Team Foundation Server (TFS) Online at a previous job. I had touched TFS at a previous gig, but like many Microsoft tools it seemed way overblown for our needs. Plus, it’s really tuned for Visual Studio, which I never use.

Julie decided to set up a private Github repository, figuring it would be more palatable to me. I’m always a fan of using things that are simple (Github confused me for years, so I’m not sure it qualifies as simple!) and I liked the fact that we would be using something the wider tech community had stamped with a seal of approval.

GitHub_Logo

We went with an Organization Bronze Plan – which now seems to be obsolete. (Here’s a link to the old plans courtesy of the Internet Archive Wayback Machine.) This gave us up to 10 repos and unlimited users for USD$25/month. Not much money, really, and we figured 10 repos was plenty.

Once we had the repo, we thought about how to organize it. We started with three repos for our organization: clients, admin, and samples. Our thought was that we would put all of the client work artifacts into that one clients repo. It certainly addressed our requirements to work that way. The admin and samples repos would be for Sympraxis administrative stuff and demos or samples we used for speaking sessions, respectively.

This got us up and running. We were both storing our code in a central repository and we could use whatever code editor (or IDE, depending on what terminology you find acceptable) we wanted. I’m using WebStorm a lot these days, but I also use Sublime Text and SharePoint Designer, and… Yeah, whatever works. Julie came from using Visual Studio, but these days she’s mainly using Visual Studio Code.

One of the greatest things about this Brave New World is that what we use to edit code really doesn’t matter that much. I really started liking WebStorm when I realized that its tooling for Github actually made Github make sense to me. I love the ecosystem of plugins for Sublime Text. And no IDE understands SharePoint as well as SharePoint Designer does. So I get to use whatever I need for the task at hand, and so does Julie. We’re just putting the results of that work into the same place.

As we moved forward with this setup, we started to see a few flaws in our thinking. The great thing about being a learning organization (the learning group is YUGE at Sympraxis) is that we comfortably revisit our decisions whenever it makes sense. The clients repo quickly became unwieldy. (Some of you would say “Duh!” here, but we’re fine with the way this went.) We were still manually copying code into our clients’ environments or editing in place and taking copies to dump into Github. We were very happy with Github, but not the mechanics of how we were using it.  So all wasn’t rosy yet.

In my next post, I’ll talk about the next steps we took to tune things…

 

The SharePoint Framework (SPFx) Is Here!

There’s no official blog post to point to yet (I’ll add it when it appears later), but today is a big day for SharePoint! The new SharePoint Framework is ready for its developer preview.

UPDATE: Here’s the REAL post.

SPFx Workbench To Do Web Part

I’ve been watching the Twitter feed from SharePoint Fest Seattle, and Jeff Teper (@jeffteper) announced the preview in his keynote – or someone else did when they were talking.

In case you missed it, the SharePoint Framework was announced publicly at the big May 4 The Future of SharePoint event in San Francisco.

When it’s available, you will see it on Github/SharePoint:

SharePoint on Github

In case you get confused by all the new terminology for “software release” these days, I’d call the developer preview a pre-alpha. The Microsoft folks would say it’s more solid than that, but they do say that things may change, you should not use it in production (and truthfully, you can’t unless your Office 365 tenant supports it), etc. In other words, it’s ready, but not THAT ready.

As someone who can realistically claim to be one of the early client side developers for SharePoint, I’m really excited about the possibilities here. When we build client side code, we’re giving users of SharePoint a much better user experience (UX) – if we do things right. Rather than the old school “conversation” with the server – and the attendant postbacks – we bring the responsiveness right behind the glass of their device. The UX becomes far more intimate. People are used to this sort of UX from everywhere but classic SharePoint. The consumer Web and most other Web-based enterprise products have been there for a while.

If you’re coming from server side development, I think this Brave New World is going to feel pretty weird for you. What I’d ask is that you give the SPFx more than one chance. If you dig into it today and find things confusing, or the documentation a bit light, or the tooling pretty weird, or the examples not quite up to snuff, stick with it. These are early days.

This is the first time Microsoft has put a development model out there and agreed that it will be how THEY build things in the future. No chicanery; no trickery. In fact, they claim that many of the new “experiences” we are seeing are built with the SharePoint Framework. I have some doubts that they are playing by the rules on this quite yet – the SPFx seems a bit lean to get to the types of “experiences” they are giving us – but they will abide by this pact.

Those of us who were able to attend one of the the Developer Kitchens earlier this year and play with the early versions of the SPFx came away energized and impressed. Not ecstatic and blissful – most of us a realists after all – but energized and impressed. There are many holes to fill and we will encounter bumps in the road. But given the open source foundations of the SPFx, we can help drive this Brave New World forward with Microsoft. Now that’s a new thing, for sure.

 

If you’d like to know more about the SharePoint Framework, check out my previous posts:

Sympraxis’ SharePoint Client Side Development Pipeline – Introduction

As promised in our August Sympraxis Newsletter, Julie and I wanted to start a new blog series to explain our new SharePoint client side development pipeline. We’ll each put our own spin on the idea, as our ways of thinking, backgrounds, and blog focuses are a bit different (but totally compatible!), so be sure to check Julie’s take as well.

We worked a lot in July and early August to improve our development practices in general. Julie joined Sympraxis with some great ideas, and she’s managed to teach this old dog quite a few new tricks. We’ve got all of our current client projects up on private Github repositories and we’ve got a really nice workflow going now that both improves our efficiency and gives us far better disaster recovery capabilities.

GitHub_Logo

This isn’t the pipeline we’ll use when the new SharePoint Framework (SPFx) is released in Preview (hopefully soon!), but it borrows from some of the tech used in that process. If you’re still doing client side development that you plant in SharePoint pages with the Content Editor Web Part (CEWP) or the Script Editor Web Part (SEWP), then this pipeline process will prepare you for some of the things you’ll need to understand when you get to SPFx.

We edit our client side code on our devices (laptops, tablets, even phones!) in local copies of the repos, use gulp tasks with spsave to push changes to the Office 365 or on premises tenant, and then commit blocks of changes to the Github repos.

Gulp logo

We’re even using different IDEs to work on our code. Right now my favorite is WebStorm, whereas Julie prefers Visual Studio Code. Because the tooling we used all sits on top of nodejs – which is truly cross platform, cross IDE, cross everything – it doesn’t really matter that we’re using different IDEs.

Webstorm logo

If you’ve been using source control for a long time, this may seem like old hat to you. But for many client side devs (like this old goat), there haven’t been very good ways to do it effectively. The old “map a local drive and edit the JavaScript/HTML/CSS in place” method has been good enough for years. But with the newer Document Library “experiences” on Office 365, the trusty Open with File Explorer is starting to work unreliably. Besides, it was time to get with the program.

For years now, my “pipeline” has been to map a drive to my code repository in the client tenant or installation, edit the files in place, and all was good. I’ve gotten in the habit of storing my code artifacts in one of two places:

  • ScriptsCSS – This was my practice for quite a few years, and is simply a Document library – usually in the root of the Site Collection – where I put things. Because it’s a Document Library, I can turn on versioning if I choose, restrict access for write (though everyone needs read permissions to access the code), etc. It’s also in “plain sight” for the people I work with at the client. This is usually a *good* thing, but more recently I switched to using…
  • /_catalogs/masterpage/_ClientName – This is a bit more out of the way, as it means putting my code in the Master Page Gallery. If permissions are set correctly, this means that few people can accidentally wander into it, and everyone has read access by default.

Either of these locations work and have worked for me regardless the version of SharePoint, whether it be 2007, 2010, 2013, 2016, or any flavor of SharePoint Online. My original goal with all of this was to avoid deploying ANYTHING to the server. It just so happened that this goal ended up meshing with where Microsoft has come over the years.

My reversion technique was simply Ctrl-Z (undo) in my editor of choice. If I had versioning on in the Document Library, then the version history was available for me, too. As for IDEs, sometimes I use SharePoint Designer, sometimes Sublime Text, sometimes WebStorm.

One of my other approaches to “working in production” (a collective gasp goes up from most people) was to create copies of the “prod” files I was working with – maybe something like HomePageNEW.ASPX, HomePageNEW.html, HomePageNEW.js, etc. This gives me a way to work on the next version of everything while still using the same content base. In cases where there are content updates, I may even have a whole shadow set of lists and libraries for testing. Having them in the same location makes it easy to copy content back and forth – usually using Sharegate, of course! I still do this, but now with better source control I have a better record of whence I’ve come.

None of these practices are really source control. Sure, I would ZIP everything up once in a while and store it away, but I couldn’t get back to a specific point in time. My approaches *worked* but weren’t as robust as I would have liked.

In the rest of the series, Julie and I will explain how we arrived at our new approaches in our posts, how they work – in detail, with code! – and what we get out of it. If you have specific questions, please feel free to add them in the comments and we’ll attempt to cover them. And don’t forget to read Julie’s spin on it, too!

 

Save Your SharePoint Online Tenant: The SharePoint Sandboxed Solutions Inspector

If you’ve been following the “code-based sandbox solutions on Office 365” saga, you know that there is little time left to fix your existing sandbox solutions in Office 365. See: Microsoft Is Removing Code-Based Sandbox Solutions in SharePoint Online – Be Prepared!

Last week, Vesa Juvonen (@vesajuvonen) released a script (New Script Available from Microsoft PnP: Generate list of sandbox solutions from SharePoint Online tenant) that can help you find your sandbox solutions. Surprisingly, what was missing from Vesa’s script was identification of the solutions that contain code. You’d get a list of all your sandbox solutions, but not specifically the ones that were going to cause you problems.

The Rencore Team

Some of the great looking folks at Rencore. Where’s Waldek?

My awesome friends at Rencore – the SFCAF folks – were kind enough to make a free tool available this week to help with even more with your diagnosis and even some of the cures. In Erwin van Hunen’s (@erwinvanhunenpost Introducing the Rencore SharePoint Sandboxed Solutions Inspector, you can learn more about the free tool and how it can help.

On August 31st, 2016 Microsoft is going to shut down support for Sandboxed Solutions with code.

Sandboxed Solutions containing code will be deactivated and this might impact your Office 365 tenant big time!

If you know you need it, just head right on over to the download page.

The SharePoint Sandboxed Solutions Inspector

They have already released several updates to the tool, and are keeping it current based on feedback from the folks using it. Now that’s service – and for a free tool!

But I think the best thing is that the Rencore tool can fix some of the most common issues – most notably the “empty DLL” issue that makes Office 365 think you have code in your sandbox solution when you don’t.

We heard “30 days” when all this started, and now people seem to be taking that as August 31. Don’t leave your users in the lurch – get going on handling this situation.

Oh, and if you’re a vendor or consultant who has written a sandbox solution with code over the last few years: reach out to your client and own it. Get them back on the right road and you’ll be the better for it.

Clean report!

Clean report!

New Script Available from Microsoft PnP: Generate list of sandbox solutions from SharePoint Online tenant

As I wrote on Monday, the decision to begin removing already deprecated code-based sandbox solutions on SharePoint Online took many people by surprise. Even though the news about the deprecation has been out there since 2014, the abrupt move – especially during a time when many people are on summer vacation – caused consternation for some. Others welcomed the move, in essence saying “good riddance” to a model that never really reached maturity.

But what if you manage a tenant on Office 365 that might have code-based sandbox solutions? Maybe you’ve used outside vendors to build solutions for you and you’re not sure what techniques they have used. Or maybe your own team built some things a few years back, you’ve had some turnover, and you source control isn’t so great. (Not so unusual, frankly.) How do you know what you have and what to do about it? You certainly don’t want functionality your users actually need to stop working unexpectedly. Some of these solutions could be InfoPath forms with code-behind, for example.

Office 365 Dev Patterns & Practices (PnP) Vesa Juvonen (@vesajuvonen) – one of my true Microsoft heroes for what he has done with the PnP set of tools – has come to the rescue, apparently with some help from Karine Bosch (@kboske). They have released a PowerShell script today that promises to “Generate [sic] list of sandbox solutions from SharePoint Online tenant“. (Far be it from me to correct Vesa’s Finglish!)

Generate list of sandbox solutions from SharePoint Online tenant

Generate list of sandbox solutions from SharePoint Online tenant

This script can be used to generate list of sandbox solutions in SharePoint Online tenant. You will need to use tenant administrator account to connect to SharePoint Online and script will generate a list of sandbox solutions to separate txt file, which can be imported to Excel for further analyses.

Note: This script is relatively simple and does not use multi-threading, so execution in larger tenants might take a while. We are looking for further enhancing the script with multi-threading support, if there’s demand for this. Also community contributions on this side are more than welcome.

Output file has following columns

  • URL of the site collection
  • Name of the sandbox solution
  • Author field from the sandbox solution – who uploaded the file
  • Created field from the sandbox solution – when solution was uploaded
  • Status field – 1=Activated, 0=Not activated

What seems to be missing here – at least to me – is the “and this one contains ‘code'” indication, but it’s still going to be very useful. Maybe it’s not simple to tell which solutions contain code? If you have ideas about this, it’s an open source project, so head over there and enhance it!

IMPORTANT: Please note that this script lists ALL sandbox solutions. But only code-based sandbox solutions have been deprecated and are being removed from Office 365. So don’t panic when you see all of your no-code solutions and site templates listed. This is a first step in inventorying your solutions.

The script requires – not surprisingly – that you have the SharePoint Online cmdlets installed. My bet is that there are plenty of Office 365 customers that have never really figured out how to download, install, and use PowerShell against Office 365. Many tenants are run by business users rather than technical types, as befits a powerful SaaS offering. In fact, in many cases, IT doesn’t need to be involved at all. That said, one would hope that those tenant administrators would know whether they have sandbox solution installed. However, see my mention of possible situations above, even if IT was in charge.

Here’s a quick tutorial on how to install those cmdlets in case you need it. In writing this section, I’m stealing the TechNet article Connect to Office 365 PowerShell. (I fear this article may not be available to everyone, as TechNet is part of subscriptions? I’m not really sure.) To do this, you have to be a tenant administrator. If you are, you’re probably the one wondering what you have in any case. It’s a pretty painless process, but if you haven’t used PowerShell – think batch files for servers – then it might be intimidating. I’m just copying the instructions from TechNet here, but I’ll add in some graphics and additional comments over the course of the day, so come back if you have questions. If you want to add any tips, please do so in the comments.

As Vesa always says: Sharing is Caring!

Step 1: Install required software

These steps are required once on your computer, not every time you connect. However, you’ll likely need to install newer versions of the software periodically.

  1. Install the 64-bit version of the Microsoft Online Services Sign-in Assistant: Microsoft Online Services Sign-in Assistant for IT Professionals RTW.

Microsoft Online Services Sign-In Assistant for IT Professionals RTW

2. Install the 64-bit version of the Windows Azure Active Directory Module for Windows PowerShell: Windows Azure Active Directory Module for Windows PowerShell (64-bit version).

Windows Azure Active Directory Module for Windows PowerShell (64-bit version)

Step 2: Open the Windows Azure Active Directory Module

  1. Find and open the Windows Azure Active Directory Module for Windows PowerShell by using one of the following methods based on your version of Windows:
    • Start menu   On the Start menu, enter Azure in the Search programs and files box.
    • No Start menu   Search for Azure using any of these methods:
      • On the Start screen, click an empty area, and type Azure.
      • On the desktop or the Start screen, press the Windows key+Q. In the Search charm, type Azure.
      • On the desktop or the Start screen, move your cursor to the upper-right corner, or swipe left from the right edge of the screen to show the charms. Select the Search charm, and enter Azure.
  2. In the results, select Windows Azure Active Directory Module for Windows PowerShell.
Here's what it looks like on my laptop running Windows 10

Here’s what it looks like on my laptop running Windows 10

Step 3: Connect to your Office 365 subscription

  1. In the Windows Azure Active Directory Module for Windows PowerShell, run the following command.
    $UserCredential = Get-Credential
    
    Windows Azure Active Directory Module for Windows PowerShell running...
    Windows PowerShell Credential Request 
    

    In the Windows PowerShell Credential Request dialog box, type your Office 365 work or school account user name and password, and then click OK.

  2. Run the following command.
    Connect-MsolService -Credential $UserCredential
    Connect-MsolService -Credential $UserCredential - Success!

    Connect-MsolService -Credential $UserCredential – Success!

How do you know this worked?

After Step 3, if you don’t receive any errors, you connected successfully. A quick test is to run an Office 365 cmdlet—for example, Get-MsolUser—and see the results.

If the Get-MsolUser cmdlet runs successfully, you'll see a list of your users

If the Get-MsolUser cmdlet runs successfully, you’ll see a list of your users

If you receive errors, check the following requirements:

  • A common problem is an incorrect password. Run Step 3 again. and pay close attention to the user name and password you enter.
  • The Windows Azure Active Directory Module for Windows PowerShell requires that the Microsoft .NET Framework 3.5.x feature is enabled on your computer. It’s likely that your computer has a newer version installed (for example, 4 or 4.5.x), but backwards compatibility with older versions of the .NET Framework can be enabled or disabled. For more information, see the following topics:
  • Your version of the Windows Azure Active Directory Module for Windows PowerShell might be out of date. To check, run the following command in Office 365 PowerShell or the Windows Azure Active Directory Module for Windows PowerShell:
    (Get-Item C:\Windows\System32\WindowsPowerShell\v1.0\Modules\MSOnline\Microsoft.Online.Administration.Automation.PSModule.dll).VersionInfo.FileVersion
    

    If the version number returned is lower than the value 1.0.8070.2, uninstall the Windows Azure Active Directory Module for Windows PowerShell, and install the latest version from the link in Step 1.

  • If you receive a connection error, see this topic: “Connect-MsolService: Exception of type was thrown” error.