Improving the quality of your Custom SharePoint 2010 Solutions via continues Integration(CI) with Team foundation Server 2010

This is a follow up blog post based on the content of my presentation on “SharePoint2010-ALM” presented at Melbourne SharePoint User Group meeting (#MOSSIG) on 23rd March 2011. Since then I’ve received number of requests from the community for sharing the content and the references. Therefore I thought of writing a little detailed post about this topic and here we go.

Continues integration is a critical process in modern day product development in improving the quality of your solution from 1st Sprint (iteration) of your product development if you are following an agile approach like Scrum.  This allows you to early detect and rectify the issues in your solution which increases the chances of success without having to go through painful integration issues and surprises at the last minute in your product development cycle.

How does it improve the quality of my custom SharePoint solution?

Well if you have a mature fully automated continues build (CI build) and a Nightly build you are in the correct path and your SharePoint development team enjoys the following in every sprint.

  1. It enforces you to promote the code from Dev -> Test-> Staging -> Production in the correct way in the first place. Otherwise automation will not be possible.
  2. Every code check in by a developer will kick-off an automated code validation, run unit tests (if any) and build the SharePoint solution (.wsp files) in the Team foundation server. 
  3. A Nightly build will do every thing in step 2 plus Add, Deploy and Activate your SharePoint solution packages into your test farm and can run and automated coded UI tests and automated load tests (if any) for you.

here is the sketch workflow.

image

Why was this approach not so common in SharePoint Development?

The Lack of SharePoint development tooling support with VS2008 contributed significantly. This is also coupled with the natural complexity arises when you develop on a platform rather than a development Framework. When it comes to SharePoint development, you may not necessarily develop with Visual Studio environment. You can develop quite powerful collaborative solutions and branding work with SharePoint Designer as no code solutions. But unless we convert those customisations into solution artefacts we will not be able to cleanly source control and maintain the solutions which breaks the fundamental concept of Application Lifecycle Management (ALM).

The Challenge

SharePoint 2010 has evolved and positioned as a single unified platform for most originations playing a mission critical role for day to day operations. With the introduction of Business connectivity Service (BCS), Client OM, Sandbox solutions, WCF based Service Applications and Claims based authentication SharePoint 2010 has also significantly evolved as application development platform which opens up a number of extensibility options for the developers.

We also have integrated set of tools built into Visual Studio 2010 for SharePoint Development. So the challenge is how to use these tools effectively and efficiently in a team based environment to build your product or solution and deploy into those shared mission critical environments confidently with consistent quality? This post describes how to implement an automated continues build with the help of following tools to validate, build and deploy SharePoint solutions.

Step1: We need to get all your customisations as solution artefacts. Reverse engineer your SharePoint designer customisations and featurize them. CKSDev Branding SPI can help you to deploy master pages and CSS as Features. Save all SharePoint Designer authored workflows as Reusable workflows as .wsp from SharePoint Designer and bring it to Visual Studio environment.

image

Step2: Create a new Team project and add your solution into source control under TFS. For the purpose of this post I am using a simple VS2010 solution name SPHelloWorld. For Code analysis, SPDisposeCheck ruleset needs to be in source control and can be shared across multiple projects with in your repository. 

image

Step3: Enable Code analysis for you solution via project poperties and select SPDisposeCheck RuleSet from your local TFS workspace as shown below.

image 

Step4: Note that Code Analysis now runs SPDisposeCheck rules as part of compilation and both Dispose and Do not Dispose rules are fired when detecting the the poor code.

image

 

Step5: Now open the Default Build Template for the project. We are going to extend the default build definition file to deploy our solution to the Test SharePoint Farm remotely. TFS 2010 is using windows workflow foundation 4 based

.xaml template.  Drag the activities from the left side bar tool box and assemble the activities as shown below just after packaging is finished.

The high level tasks performed by this segment of the workflow are

  1. Build the Solution
  2. Run Code Analysis with SPDisposeCheck on the TFS Server. If this code analysis fails our build would fail at this point.
  3. Create .wsp solution package
  4. Check the user specified deployment flag (This is a work initiation parameter)
  5. Write a log message to the log file
  6. Map a Admin share drive to the Target SharePoint Server (Test Farm) with the user specified credentials.
  7. SET the STSADM path on the target server
  8. Stop V4 Admin service in the target farm. (This uses PSEXEC to remotely execute STSADM on the target server)
  9. Copy each .wsp file to the target SharePoint  server and add, deploy and execute the timer job in user specified sequence.
  10. Finally turn on the V4 Admin timer service.

1

If you are interested the customised TFS2010 workflow template for building and deploying .wsp solutions can be downloaded from my SkyDrive here

Now define the additional parameters required for the workflow initiation that needs to be supplied by the user when starting a new Team build.

image

Now set activity properties for each activiry. As a sample I’ve shown below the properties set for the  “Add solution” InvokeProcess workflow activity. Once all properties are set WF4 validates your workflow. Once all validations are passed, checkin your custom build definition template into TFS.

image

 

Step6: Author a new Build definition based on our updated workflow template.

image

Some important parameters that you need to set are shown below.

image

image

Step7: It’s time to Queue a new Build. Now assume a scenario where the developer rushes to checkin by overriding the Check-in Policies set by the build master.

image

and here is the result : TFS build agent decided to fail your build because your code did not pass the code analysis. No .wsp’s produced.

image

So developer now fix the code with proper disposing as shown below and check-in the code again.

 

image

And after few seconds we get this TFS Build status. All green!

image

Opening the build log gives us the workflow execution log as shown below. Note how STSADM has been invoked remotely via psexec.exe on the TFS build agent.

 

image

and see below the end result in the central admin in our test SharePoint Farm.

image

On a final note “the right tools make all the difference”. But tooling it self may not helps you to build a quality product. The process matters. TFS 2010 support you with your chosen process and allows you to tailor the default process templates if required. You have the chance to select the process template when you create a new Team project as shown below. I highly encourage you to check out the new “Microsoft Visual Studio Scrum 1.0” template. This template has been co-developed by Microsoft in collaboration with the Scrum community. The template guides you with scrum based agile product development. I’ve highlighted a few below.

  • Creating effective user stories
  • Creating prioritizing grooming product backlog
  • Agile estimation and Sprint planning
  • Scrum artefacts such as sprint burndown and release burndown chart

image

And here is the high level  process behind the template.

image

 

The correct tools coupled with the correct process will take your product/solution to the next level faster and smoother and make you a successful SharePoint developer and hence a successful product/solution delivery Team.

Where to go from here?

Enjoy!


Creating a custom site Definition with a custom master page

Site definition can be considered as a  blueprint of a SharePoint site. When developing business solutions on WSS platform sometimes we will have to create our own site definitions to incorporate functional business modules. Site Definitions gives you lots of flexibility in terms of maintaining and upgrading your solutions compared to Site Templates. Its also gives us the flexibility of choosing custom layouts and the ability to pre program the sequence of the feature activation order when  you create a new site based on the definition.

In this post I will outline the steps that we have to follow when Creating a custom Site definition with a Custom master page. I am a fan of WSPBuilder as a development tool by  Carsten Keutmann because it light to use over WSeWSS 1.3 and does not abstract too much from how the solution artifacts get deployed to your farm environment. Also its great for team development plays nice with TFS and Nightly Build Automation with MSBuild.

I will not be able to cover the details of how to create a custom master page here. So I will be using one of the out of the box default master page as a sample.

Lets get Started.

1. Create a WSPBuilder Project

I am using Visual studio 2008 with WSPBuilder extensions 1.0.6. Once extensions are installed VS will light up with new WSPBuilder project types  as shown

below. Select WSPBuilder project and give your project a name.

image

2. Create 12 hive folder Structure within your solution.

One of the nice feature I like about WSPBuilder is that we can develop against 12 hive folder hierarchy in dev environment.

image

Standard 12 hive folder structure is shown in left. The folders that we are interested for this task of deploying a custom site    definition are highlighted. All standard and custom site definitions with their default pages/schemas lives in SiteTemplates folder. All Features and feature manifests resides in FEATURES folder. So we will create this structure within our solution relative to 12 hive holder (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\)

Once folders are created inside the solution it should look as follows.

image

3. Create the site definition.

Create a sub folder inside SiteTemplates folder. This folder will contain all out custom site definition files. I call mine DevSiteTemplate. Instead of creating the definition files from scratch i will copy standard team site definition from 12 hive (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\SiteTemplates\sts) into my DevSiteTemplate folder. You can delete Defaultdws.aspx file as we do not need it for our requirement.

image

4. Update master page reference in default.aspx file

We are going to use our own master page in our site definition. Therefore we need to change default.master -> custom.master in our default.aspx page. Our custom master page will be created little later and SharePoint will replace custom.master reference with our master page during runtime page rendering based on the site definition schema (onet.xml) which we defined later on.

image

5. Update Configurations in onet.xml file

A Site definition can have many configurations. Because we have used Standard Team site definition as our base definition we can remove unused configurations. Std. Team site definition contains three configurations

0 : Default     1:Blank   2:DWS

We are going to use the Default Configuration (0) . Therefore we can remove Blank and DWS Configurations as well as their module sections

Following code segment shows the original onet.xml file contents

image

After removing the configurations onet.xml file should look as follows

image

6. Create the Custom Master page Feature

We need to add new feature to apply the master page to our site definition. The advantage of creating a separate feature for master page instead of depend on the site definition is that it can be operated independent from our site definition. In other words its all about loose coupling and reusability.

Right Click the project and Add a new “Web” Scope blank feature. I call it DevSiteMaster. WspBuilder will create feature folder for us with in the FEATURES folder along with the feature.xml file and element.xml file.

feature.xml describes the feature and its id,scope and metadata. feature.xml file reference the element.xml file which describes the real work load for the feature.

image

Create a folder named MasterPages inside the feature folder to add our custom master page. Now we need to add our custom master page to our solution. For this demo I will be using one of the out of the box master page (default.master) from 12 hive Global folder (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\GLOBAL). Copy default.master page into the MasterPages folder. Then rename default.master to devsite.master.

Now our solution folder structure should look like below.

image

Now we need to add our custom master page to our solution. I will be using one of the out of the box master page (default.master) for this example from 12 hive Global folder (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\GLOBAL).

7. Update Feature.xml and Element.xml files

Update the WspBuilder generated feature file with the settings required for our master page feature. Add the ElementFile section which reference our custom master page.

<?xml version="1.0" encoding="utf-8"?>

<Feature  Id="a349b379-6fda-41bf-9fe2-cfad70c99f0b"

          Title="DevSiteMaster"

          Description="Custom master page feature for  DevSiteMasters"

          Version="12.0.0.0"

          Hidden="FALSE"

          Scope="Web"

          DefaultResourceFile="core"

          xmlns="http://schemas.microsoft.com/sharepoint/">

  <ElementManifests>

    <ElementManifest Location="elements.xml"/>

    <ElementFile Location="MasterPages\devsite.master"/>

  </ElementManifests>

</Feature>

Update element.xml file and a module element to copy our custom master page from our MasterPages folder path to master page gallery of the underlying web. Now when DevSiteMaster Feature get activated the job defined in our element.xml file get executed. As a result our custom master page get added into the master page gallery of the underlying sub web.

<?xml version="1.0" encoding="utf-8" ?>

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

    <Module Name="DevSiteMasterPage" Url="_catalogs/masterpage" Path="MasterPages"  RootWebOnly="FALSE">

        <File Url="devsite.master" Type="GhostableInLibrary"/>

    </Module>

</Elements>

8. Update onet.xml file and add the DevsiteMasterPage feature into the list WebFeatures.

The Features specified in the SiteFeatures and WebFeatures section that get activated as part of site provisioning process based on the site definition. Also they get activated in the order specified. We need to define our custom master feature under WebFeatures. If our master page feature had been a “Site” scoped feature we would specify under siteFeatures section.

<WebFeatures>

<Feature ID="00BFEA71-4EA5-48D4-A4AD-7EA5C011ABE5" />

  <!-- TeamCollab Feature -->

  <Feature ID="F41CC668-37E5-4743-B4A8-74D1DB3FD8A4" />

  <!-- MobilityRedirect -->

  <!-- devsite custom master page-->

 <Feature ID="a349b379-6fda-41bf-9fe2-cfad70c99f0b" />

</WebFeatures>

9. Update onet.xml default configuration to use our Custom master page.

Update default configuration and specify the that our confiugration should used our custom master page by setting the CustomMasterUrl and MasterUrl points to our custom master page living in the master page galley. By setting these values we effectively telling SharePoint that when ever .aspx refers to custom.master page replace that with devsite.master page which is our custom master page.

<Configuration ID="0" Name="Default"

 CustomMasterUrl ="_catalogs/masterpage/devsite.master"

 MasterUrl="_catalogs/masterpage/devsite.master">

10. Create WEBTEMP*.xml file for the Custom SiteDefinition.

WebTemp*.xml add our custom site definition into the catalogue of site definitions so that we can use pick the site defintion to provision from create site and workspace application pages.  It is important that the Template Name exactly match our site definition name as shown below. Create a folder called xml inside 1033 folder and create webtemp*.xml file inside the folder named xml. This file name must begin with “webtemp” prefix as required by SharePoint. In this demo I call it  WebTempDevSite.xml

<?xml version="1.0" encoding="utf-8"?>

<Templates xmlns:ows="Microsoft SharePoint">

    <Template Name="DevSiteTemplate" ID="75999" SetupPath="SiteTemplates\DevSiteTemplate" >

        <Configuration ID="0" Title="Dev Site"

                       Hidden ="false"

                       ImageUrl="/_layouts/images/stsprev.png"

                       Description="A Custom Site Definition based on Team site"

                       DisplayCategory="Custom Templates" >

        </Configuration>

    </Template>

</Templates>

Our final solution structure should now look as follows.

image

All done! Our solution is now ready for package up into .wsp solution file and deploy into the SharePoint Farm. WSPBuilder will take the heavy lifting  from here including creating manifest files,DDF files and packaging up our solution. Therefore Right Click our project and select Deploy option from WSPBuilder menu.

This action will build and package our solution, add it to the SharePoint solution store and deploy the solution by invoking STSADM.

image

Once succesfully deployed we should be able to create site based on our custom site definition as follows.

image

The visual studio solution associated with this post can be downloaded from here. Hope this helps with creating a custom site Definitions with a custom master page using WSPBuilder.