The Middle Tier Manifesto: An Alternative Approach to Development with Microsoft SharePoint – An Early Response

My white paper, The Middle Tier Manifesto: An Alternative Approach to Development with Microsoft SharePoint (see my prior post) has been garnering a decent amount of attention over the last few days, as I hoped it would.  I’m enjoying the discussions it is provoking, as the discussions were the whole point in the first place.

I felt that one particular exchange was worth reproducing here, if only so that I have easy access to it via a link.  A gentlemen who calls himself “Jay” has caused a bit of a kerflufffle over at EndUserSharePoint.com over the last few days with his comments on Jason MacKenzie’s useful post jQuery vs Webparts solution.  I’m assuming (I think reasonably) that it’s the same “Jay” who posted the comments below on my cross-post at EUSP of my original manifesto post.

Here are “Jay”‘s comments followed by my response.

I agree that there is value to be gained from what you call ‘Middle tier development’, which from a developer’s point of view is nothing new technology wise (it is after all CSS, JavaScript and html).

I do take issue that this is the solution to managing important business data. You touch on some of my main concerns:

1. SharePoint designer cannot produce reusable components (at least not in MOSS 2007). By this I mean you need to manually reproduce your steps through your test environments, all the way to production. Any time tedious processes need to be followed by a human, mistakes happen.

2. Code management, storing code in CEWP or a list is not the same as version controlled software such as TFS, a proper development process needs to have change tracking, not so much copies of files. Like the steps above, a manual process to deploy to each environment needs to be followed, a whole bunch of moving parts. On the flip side a well developed wsp needs only some features to be activated .. very quick and easy to repeat by anyone. Don’t under estimate the tacit knowledge needed to deploy ‘middle tier’ components.

I have to completely disagree with this statement in your document:

“Middle Tier solutions are far more adaptable in a large corporate environment
than the deployment of managed code solutions.”

So we are taking about a large corporate environment, given the concerns above, I can’t see this being true at all. ‘Middle tier’ solutions might cut it for keeping a list of lunch orders, it’s simply not going to work for anything more serious for all the reasons mentioned above.

I also disagree with how you trivialize testing:

“Because nothing is deployed to the server, testing can be done in a much more ‘unit testing’ mode”.

Stuff is deployed to the server, all that JavaScript you’ve pasted in a CEWP. Or maybe you’ve added it to the master page? In that case have you tested it across your whole site? You may have pasted code that pollutes the JS global namespace and causes errors on some pages. The fact is your trying to say “look I’m really not doing any development, so I don’t want to waste time testing”, which is fine for your lunch order site, but not for a large corporate install. Your still doing development, so follow development practices. I won’t touch on your ‘Unit test” misconception, I figure your not a developer anyway (or you wouldn’t have called it ‘middle tier’ development, it’s not, its client side development).

Your whole argument revolves around ‘Middle tier’ development being quicker, therefore cheaper.
Headings like ‘Faster Development lifecycles’ under the context of ‘Middle tier’ development sends off warning bells to me. Its says you don’t want to follow best practice around deployment and testing. Lets face it, you’ve already read the stuff above and dismissed it as baseless. You want to cut corners. If everyone does this in an organisation, you will soon end up with half baked solutions that become brittle and hard to maintain (if the organisation even knows about it), a proper governance model shouldn’t allow it.

To me it seems like people are preferring these approaches because they haven’t been exposed to good SharePoint developers. Touching system files shouldn’t be a concern with a good developer, so concerns around this are baseless. So isn’t the big issue the fact that good SharePoint devs are hard to find?

I really do understand what your trying to do, you feel empowered but don’t want the responsibility of deployment and proper testing. Its fine for little ‘toy’ applications, but just doesn’t cut it for serious applications. Overall this is a good place to start a proper dialog around ‘Middle tier’ solutions.

Marc Anderson says:

• April 16, 2010 at 11:08 am

  Jay:
  Thanks for your comments, and no, I don’t find them baseless at all. Every situation is different, and what I am trying to point out is that there is a hugely useful set of options that many business people and developers are unaware of. I’m not trying to say that these methods are new, just that they tend to be overlooked. When I’m using the term Middle Tier, I’m capitalizing it for a reason. I don’t intend it as the generic ‘middle tier’ you may learn about in university, but as something specific to SharePoint.

  Every organization has different needs, and no one approach ought to work for everyone. I’ve consulted to organizations using SharePoint which are anywhere from a one person services firm using WSS in the cloud to a tens of thousands person global bank running the Enterprise CAL. The approach in those two extremes damn well better not be the same or no one is getting anything that they want.

  The Middle Tier methods ought to be one set of methods in your portfolio, as I point out in the white paper. Every business requirement should be looked at against the set of available approaches in the portfolio and the optimal approach should be used in each case. Sometimes the answer ought to be notecards, other times SharePoint, and other times it ought to be some other toolset altogether. When it *is* SharePoint, there should be more options than “We have to write managed code.”

  Speaking of cloud computing, the Middle Tier is going to be even more important as more and more organizations move their use of SharePoint to the cloud. With SharePoint 2007 (WSS *or* MOSS) and cloud hosting at the low end, there is zero capability to touch the server on the back end. The Middle Tier methods are all that there are to do real customizations. I expect that this will increasingly be the case. In these situations, the Middle Tier ought to have extreme usefulness.

  SharePoint Designer cannot produce reusable components (nor does Visual Studio or Notepad, for that matter), but the people using it can. In all cases (and yes, I’ll be absolute here), it’s the people using the tools that either get it right or wrong. With the right mindset, tools, and practices, anyone can create reusable objects with SharePoint Designer. I do it all the time.

  When we deploy script in a CEWP or in an aspx page, we aren’t deploying it to the server, per se, but into the database. That script is therefore ‘content’ as opposed to ‘code’ in many ways. I can write script that damages SharePoint sites and content (assuming that I have the permissions to do so), true, but I cannot damage the server file structure itself if the server is set up and administered correctly. (There’s another place where people doing what they ought to is critical.)

  As for my credentials, they are available in many places (see the About page on my blog), but let me give you a synopsis. It’s very possible that I was writing code before you were born. My first paying programmer job was in 1983. I’ve worked for quite a few world-class companies, and I’ve probably spent time working in the IT departments of far upwards of 100 different companies at one time or another through my consulting. I’ve also managed real-time 24×7 mission critical systems. The best example of this is my role at Staples in the early 1990’s when I managed (and developed on a daily basis) the systems that ran the delivery business, which included everything from catalog management, order taking, warehousing, fulfillment, to billing, etc. In short, the whole ‘ball of wax’. Through all of this experience, I’ve seen the good, the bad, and the ugly when it comes to software development.

  The only reason I’ve raised to the bait on my credentials is to say that I believe that I know what I am talking about. In my experience, overly formalized systems (the generic term ’systems’) lead to stagnation. Having a great, formal SDLC *can* mean that nothing of use ever really gets done. I’ve seen this time and time again in large organizations with their approach to SharePoint development. I’ve also seen solutions that went through the full, formal SDLC that were absolute crap. I’ve seen real business needs go unmet because the SDLC made it impossible to produce a useful result in any sort of reasonable timeframe or at a reasonable cost.

  Finally, you are absolutely right that good ‘SharePoint developers’ are hard to find. Good *developers* are hard to find. People who are truly good at *anything* are hard to find. All skill distributions follow a bell curve, and everyone wants to think that they sit in the top tail. That’s statistically impossible; the large majority of people will be clustered around average. We all want to hire the people in the top tail and pay what we’d pay someone in the bottom tail. Too often, the reverse ends up being the case.

  Again, thanks for your comments, and I hope my additional comments help to clarify what I’m trying to say in the white paper. There’s a reason I wanted it to be the first edition. I wanted to hear from folks like you to get your reactions so that I can hone the messages. Then people who want to really get valuable work done can take the white paper to their IT people and see if they can’t come up with a way to make good things happen for their organizations.