Improving the quality of your Custom SharePoint 2010 Solutions via continues Integration(CI) with Team foundation Server 2010

This is a follow up blog post based on the content of my presentation on “SharePoint2010-ALM” presented at Melbourne SharePoint User Group meeting (#MOSSIG) on 23rd March 2011. Since then I’ve received number of requests from the community for sharing the content and the references. Therefore I thought of writing a little detailed post about this topic and here we go.

Continues integration is a critical process in modern day product development in improving the quality of your solution from 1st Sprint (iteration) of your product development if you are following an agile approach like Scrum.  This allows you to early detect and rectify the issues in your solution which increases the chances of success without having to go through painful integration issues and surprises at the last minute in your product development cycle.

How does it improve the quality of my custom SharePoint solution?

Well if you have a mature fully automated continues build (CI build) and a Nightly build you are in the correct path and your SharePoint development team enjoys the following in every sprint.

  1. It enforces you to promote the code from Dev -> Test-> Staging -> Production in the correct way in the first place. Otherwise automation will not be possible.
  2. Every code check in by a developer will kick-off an automated code validation, run unit tests (if any) and build the SharePoint solution (.wsp files) in the Team foundation server. 
  3. A Nightly build will do every thing in step 2 plus Add, Deploy and Activate your SharePoint solution packages into your test farm and can run and automated coded UI tests and automated load tests (if any) for you.

here is the sketch workflow.

image

Why was this approach not so common in SharePoint Development?

The Lack of SharePoint development tooling support with VS2008 contributed significantly. This is also coupled with the natural complexity arises when you develop on a platform rather than a development Framework. When it comes to SharePoint development, you may not necessarily develop with Visual Studio environment. You can develop quite powerful collaborative solutions and branding work with SharePoint Designer as no code solutions. But unless we convert those customisations into solution artefacts we will not be able to cleanly source control and maintain the solutions which breaks the fundamental concept of Application Lifecycle Management (ALM).

The Challenge

SharePoint 2010 has evolved and positioned as a single unified platform for most originations playing a mission critical role for day to day operations. With the introduction of Business connectivity Service (BCS), Client OM, Sandbox solutions, WCF based Service Applications and Claims based authentication SharePoint 2010 has also significantly evolved as application development platform which opens up a number of extensibility options for the developers.

We also have integrated set of tools built into Visual Studio 2010 for SharePoint Development. So the challenge is how to use these tools effectively and efficiently in a team based environment to build your product or solution and deploy into those shared mission critical environments confidently with consistent quality? This post describes how to implement an automated continues build with the help of following tools to validate, build and deploy SharePoint solutions.

Step1: We need to get all your customisations as solution artefacts. Reverse engineer your SharePoint designer customisations and featurize them. CKSDev Branding SPI can help you to deploy master pages and CSS as Features. Save all SharePoint Designer authored workflows as Reusable workflows as .wsp from SharePoint Designer and bring it to Visual Studio environment.

image

Step2: Create a new Team project and add your solution into source control under TFS. For the purpose of this post I am using a simple VS2010 solution name SPHelloWorld. For Code analysis, SPDisposeCheck ruleset needs to be in source control and can be shared across multiple projects with in your repository. 

image

Step3: Enable Code analysis for you solution via project poperties and select SPDisposeCheck RuleSet from your local TFS workspace as shown below.

image 

Step4: Note that Code Analysis now runs SPDisposeCheck rules as part of compilation and both Dispose and Do not Dispose rules are fired when detecting the the poor code.

image

 

Step5: Now open the Default Build Template for the project. We are going to extend the default build definition file to deploy our solution to the Test SharePoint Farm remotely. TFS 2010 is using windows workflow foundation 4 based

.xaml template.  Drag the activities from the left side bar tool box and assemble the activities as shown below just after packaging is finished.

The high level tasks performed by this segment of the workflow are

  1. Build the Solution
  2. Run Code Analysis with SPDisposeCheck on the TFS Server. If this code analysis fails our build would fail at this point.
  3. Create .wsp solution package
  4. Check the user specified deployment flag (This is a work initiation parameter)
  5. Write a log message to the log file
  6. Map a Admin share drive to the Target SharePoint Server (Test Farm) with the user specified credentials.
  7. SET the STSADM path on the target server
  8. Stop V4 Admin service in the target farm. (This uses PSEXEC to remotely execute STSADM on the target server)
  9. Copy each .wsp file to the target SharePoint  server and add, deploy and execute the timer job in user specified sequence.
  10. Finally turn on the V4 Admin timer service.

1

If you are interested the customised TFS2010 workflow template for building and deploying .wsp solutions can be downloaded from my SkyDrive here

Now define the additional parameters required for the workflow initiation that needs to be supplied by the user when starting a new Team build.

image

Now set activity properties for each activiry. As a sample I’ve shown below the properties set for the  “Add solution” InvokeProcess workflow activity. Once all properties are set WF4 validates your workflow. Once all validations are passed, checkin your custom build definition template into TFS.

image

 

Step6: Author a new Build definition based on our updated workflow template.

image

Some important parameters that you need to set are shown below.

image

image

Step7: It’s time to Queue a new Build. Now assume a scenario where the developer rushes to checkin by overriding the Check-in Policies set by the build master.

image

and here is the result : TFS build agent decided to fail your build because your code did not pass the code analysis. No .wsp’s produced.

image

So developer now fix the code with proper disposing as shown below and check-in the code again.

 

image

And after few seconds we get this TFS Build status. All green!

image

Opening the build log gives us the workflow execution log as shown below. Note how STSADM has been invoked remotely via psexec.exe on the TFS build agent.

 

image

and see below the end result in the central admin in our test SharePoint Farm.

image

On a final note “the right tools make all the difference”. But tooling it self may not helps you to build a quality product. The process matters. TFS 2010 support you with your chosen process and allows you to tailor the default process templates if required. You have the chance to select the process template when you create a new Team project as shown below. I highly encourage you to check out the new “Microsoft Visual Studio Scrum 1.0” template. This template has been co-developed by Microsoft in collaboration with the Scrum community. The template guides you with scrum based agile product development. I’ve highlighted a few below.

  • Creating effective user stories
  • Creating prioritizing grooming product backlog
  • Agile estimation and Sprint planning
  • Scrum artefacts such as sprint burndown and release burndown chart

image

And here is the high level  process behind the template.

image

 

The correct tools coupled with the correct process will take your product/solution to the next level faster and smoother and make you a successful SharePoint developer and hence a successful product/solution delivery Team.

Where to go from here?

Enjoy!


Leverage and Extend Claims based identity in SharePoint 2010

Windows Identity Foundation (WIF) is the platform on which SharePoint 2010 claims authentication is based. WIF, which is fully supported in SharePoint 2010, ADFS 2.0, ASP.NET, Windows Communication Foundation (WCF), and any other .NET application you care to develop, provides the infrastructure necessary to generate, transmit, and process claims-based identity in a simple and straightforward manner. It removes the roadblocks imposed by legacy authentication schemes like NTLM and Kerberos and puts control directly into the hands of developers, users, and IT security professionals. long story short, it’s a framework written to help solve identity issues common in the of cloud computing and service-oriented architecture.

The idea of claims based identity is one that many people are willing to try. Getting accurate information out there to the public though does take time.

The important point is this is based on industry standards.  Many different entities are on board along with Microsoft in this matter. The digital world continues to give us new opportunities and those involved believe that this will help all of us to get the most out of it. There is a strong foundation in place to continue building upon. The use of AD FS v2, CardSpace, and Windows Identity Foundation are all important pieces of this puzzle.

As a demonstration of these capabilities, I’ll show how SharePoint 2010, WCF,and WIF can be put together to solve the identity delegation problem. In this demo session part 1 I start establishing the trust relationship between ADFS 2.0 and SharePoint with PowerShell and demonstrate how the claims get into SharePoint.Then we build and deploy a claims viewer Webpart with WIF programming model. In part 2 We start with a web service that is front-ending line-of-business information stored in a SQL database. Then, we’ll configure it to use WIF to request the calling user’s claims from SharePoint and process the token so that authorization decisions can be made. we’ll surface this information in SharePoint 2010 as an External Content Type using Business Connectivity Services (BCS).

This session was presented in SharePoint Saturday Melbourne 2010  in Oct 2010  and MOSSIG user group Nov 2010.

images

Click here to view a recorded screencast of this session. 

The slide deck of this session is shared here

The SharePoint Claims Viewer Visual WebPart VS2010 solution,  Claims based WCF Service VS2010 solution and necesssary powershell scripts to add and remove identity provider in SharePoint 2010 can be downloaded from here.
 
I encourage you to look at following links for further information about this topic

Building an interactive dashboard with SharePoint 2010 RESTful Services

REST (or Representational State Transfer) is an easy way to easily get or post items to a server-side web service.  It is used in situations where you would use a web service but do not want the overhead and tightly-coupled nature of SOAP.   Technically, REST is not a protocol, but an architectural style. It complies to Open Data Protocol (OData), a Web protocol for querying and updating data that provides a way to unlock your data and free it from silos that exist in applications today.

SharePoint 2010, being based on WCF in the .NET 3.5 Framework, has built-in RESTful web services which allow us to take advantage of this simplicity in custom applications we write..

The RESTful service can be accessed via  http://<site collection url>_vti_bin/ListData.svc

This blog post is based on my Demo #2 last week at MOSSIG user group meeting in Melbourne.

 

uuo

 

The Data for this dashboard is sourced from 2 custom lists “Products” and “Sales”  The two lists has parent-child relationship.

aas

 

Sales List

aas2

Edit Sale Item.

aas5

 

Any given list can be accessed via the RESTFul service. This demo application is using  WCF Data Services to read/write to sharepoint  list Data, then, WCF Data Services  was the first Microsoft technology to support the Open Data Protocol. It provides developers with client libraries for .NET, Silverlight, AJAX etc..

image

Once the service proxy is created from ListData.svc you will have typed access to all the list data with LINQ support.  The following code segment in the ViewModel  shows how sales data is obtained from the service along with corresponding product details and register for Unit of Work tracking when property change.

 

private void GetSalesData() { // Set up the data context if it hasn't been done already if (_dataContext ==null) { _dataContext = new HomeDataContext( new Uri("http://spdemo/_vti_bin/ListData.svc")); } Sales = new ObservableCollection<SalesItem>(); //Prepare REST service request var query = (from s in _dataContext.Sales.Expand("ProductCode") select s ) as DataServiceQuery<SalesItem>; query.BeginExecute( (ar) => { IEnumerable<SalesItem> _salesItems = query.EndExecute(ar); foreach (SalesItem item in _salesItems) { item.PropertyChanged += ((sender, e) => { var entity = (SalesItem)sender; //track changes to entities _dataContext.UpdateObject(entity); }); Sales.Add(item); } NotifyPropertyChanged("LineValue"); }, null) ; }

The viewModel implements INotifyPropertyChanged event for the purpose of Notifying the view of Data changes and visa versa. The ViewModel exposes two properties LineValue (bound to the slider) and ObservableCollection of SalesItems. These 2 properties are declaratively bound to both chart types (bar and pie). I am using few charting controls  from Silverlight Toolkit (http://silverlight.codeplex.com/)  with XAML Declarative bindings as shown below.  

 

            <TextBlock x:Name="ScoreTextBlock" Text="{Binding LineValue}" />
            <chart:Chart Height="200" LegendTitle="Item" Title="No of Units Sold By Product" 
                         x:Name="MyBarChart" Width="562" DataContext="{Binding}">
            <chart:ColumnSeries x:Name="MySalesChartData" Title="Sales of Products" 
                    ItemsSource="{Binding Sales}"
                    IndependentValueBinding="{Binding ProductCode.ProductName}"
                    DependentValueBinding="{Binding NoofUnits}"/>
            </chart:Chart>
            <chart:Chart Height="230" Title="Sales Distribution" x:Name="MyPieChart" Width="562" Margin="4,0">
                <chart:PieSeries ItemsSource="{Binding Sales}" 
                    IndependentValueBinding="{Binding ProductCode.ProductName}"
                    DependentValueBinding="{Binding NoofUnits}" Background="#FF2B1515" />
            </chart:Chart>

The VS2010  solution that contains source code for this demo is uploaded here.

Enjoy!


SharePoint 2010 Live Image Capturing WebPart packaged as a Sandbox solution

It was great speaking and networking with fellow members at MOSSIG July meeting in Melbourne last week. I presented a session on Effective use of SilverLight 4 with SharePoint 2010.  The Slide deck is uploaded here. I demoed a SilverLight 4 WebPart  taking a snapshot on demand and with the help of  client object model  automatically save the captured snapshot to SharePoint picture library.

This demo solution was created to showcase effective use of SilverLight 4  in SharePoint 2010 as well as to show how MVVM (Model-View-ViewModel)  pattern pattern is used in designed and developing maintainable and testable SilverLight applications with powerful declarative Data binding capabilities

I thought of writing a little elaborative blog post about this solution and here we go…

First lets have a look at the outcome.

Final Result:

Untitled3

By the way this is my little boy Bryan. He enjoyed this a lot  🙂

The picture library to be used for storing the captured images can be configured by the user as shown below as a WebPart property.

Untitled1

 

Solution Type:

This  is sandbox solution. Therefore as site collection administrator you can add this solution to your solution gallery and try out its functionality with minimal impact on your farm as compared to a classic farm solution. SharePoint perform resource monitoring of your sand box solutions  as shown below. If it doesn’t behave properly it will be kicked out by SharePoint before you even know about it at times . How cool is that ?

This is a great model for ISV’s to distribute their apps and Farm/Site Collection Administrators to trial out third-party solutions with minimum risk.

image

 

Solution architecture:

The basic solution archtecture is shown below.  The viewmodel is decorating the model/Repository  and sits between your model(domain objects) and the view.  MVVM is targeted at modern UI development platforms (WPF and Silverlight). You can read more about this simple pattern  here

Untitled44

The VS2010 solution structure for this application is as follows.

Untitled4 

SandBox WebPart properties:

One of the limitation with running in the sandbox is that all web parts must inherit from “System.Web.UI.WebControls.WebParts.WebPart” instead of “Microsoft.SharePoint.WebPartPages” (which is not allowed)  when running in the sandbox.  This means the traditional means of creating custom Web Part Properties using the ToolPart object in SharePoint is not available.  Thus one must now rely on how these custom properties are created when dealing with ASP.Net Web parts.  Which by the way (as is obvious in the sandbox restriction) is now the preferred way to create web parts.  ASP.Net Web Parts have an equivalent class to the “ToolPart” class called the “EditorPart”. So I’ve created my webpart property editor inheriting from EditorPart as shown below.

// Create a custom EditorPart to edit the WebPart control.
 [AspNetHostingPermission(SecurityAction.Demand,
     Level = AspNetHostingPermissionLevel.Minimal)]
 public class SLWebCamEditorPart : EditorPart
 {
     private DropDownList _partTargetLibrary;

     public SLWebCamEditorPart()
     {
         Title = "Image capture webpart settings";
     }

     private DropDownList PartTargetLibrary
     {
         get
         {
             EnsureChildControls();
             return _partTargetLibrary;
         }
     }

 
 
Passing initialize parameters into Silverlight from Sharepoint:
 
The CreateChildControls Method in the hosting webpart injects SilverLight host control  into the WebPart as shown below.  In this case we are injecting the current site URL and target picture library name as ‘initparams’ into the SilverLight host control.
NOTE: It is important to inject the current site URL in order to guarantee the ClientContext (in SharePoint client OM ) is not NULL during  runtime execution.
[Personalizable, WebBrowsable(false)]
public String PictureLibraryName { get; set; }

protected override void CreateChildControls() { var width = "400"; var height = "600"; base.CreateChildControls(); string source = SPContext.Current.Site.Url + "/_catalogs/masterpage/SLWebCam.xap"; Controls.Add(new LiteralControl( "<div>" + " <object data=\"data:application/x-silverlight-2,\" type=\"application/x-silverlight-2\" width=\"" + width + "\" height=\"" + height + "\">" + " <param name=\"source\" value=\"" + source + "\"/>" + " <param name=\"onerror\" value=\"onSilverlightError\" />" + " <param name=\"background\" value=\"white\" />" + " <param name=\"minRuntimeVersion\" value=\"4.0.50401.0\" />" + " <param name=\"autoUpgrade\" value=\"true\" />" + " <param name=\"initParams\" value=\"MS.SP.url=" + SPContext.Current.Site.Url + ",PictureLibraryName=" + PictureLibraryName +     

…..Continue..

The injected initparam  values in the serverside can be extracted from the client side and stored as application resource as shown below.

private void Application_Startup(object sender, StartupEventArgs e)
{   // Extract init params and inject into application resource dictionary
    foreach (var resource in e.InitParams)
    {
        Application.Current.Resources.Add(resource.Key,resource.Value);
    }
    this.RootVisual = new MainPage();
}

Declarative DataBinding:

The XAML based view was created using VS2010 enhanced UI designer. The ViewModel exposes the necessary bindable properties and bindable commands (using command pattern). In other words ViewModel is the datacontext for the View

ie: XAML markup for showing the live web cam video uses a Rectangle bound to a property named VideoBrush in the ViewModel as shown below.

<Rectangle Fill="{Binding VideoBrush}" Height="138" Width="170" />

private VideoBrush videoBrush;
public VideoBrush VideoBrush
{
    get { return videoBrush; }
    set
    {
        videoBrush = value;
        FirePropertyChanged("VideoBrush");
    }
}

The view Model implements INotifyPropertyChange interface. Notifies clients that a property value has changed. The snapshots taken are exposed as a ObservableCollection of Writable bitmaps from the ViewModel and bound to a ListBox with  a DataTemplate as shown below. ObservableCollection  Represents a dynamic data collection that provides notifications when items get added, removed, or when the whole list is refreshed.

public ObservableCollection<WriteableBitmap> Images { get; set; }
   public ICommand StartCaptureCommand { get; set; }
   public ICommand StopCaptureCommand { get; set; }
   public ICommand TakeSnapshotCommand { get; set; }
   public ICommand SaveSnapshotCommand { get; set; }

<ListBox x:Name="Snapshots" ItemTemplate="{StaticResource SnapshotDataTemplate}" ItemsPanel="{StaticResource SnapshotItemsPanelTemplate}" ItemsSource="{Binding Images}" SelectedItem="{Binding SelectedSnapshot, Mode=TwoWay}" Height="123"

Width="356" />

Saving the Captured Image to the Sharepoint picture Library:
In order to use sharepoint client OM all you need is referencing the following 2 assemblies from SP root folder. Web Server Extensions\14\TEMPLATE\LAYOUTS\ClientBin  and use the following namespace from silveright.

image
using Microsoft.SharePoint.Client;
I am also using SilverLight Image manipulation library “ImageTools” from   http://imagetools.codeplex.com/ for handing bitmap processing in the client side.

When the user select a snapshot and click the save button ClientContext is initiated and List information is obtained from the server via Client OM as a Asynchronous operation.  On the successful callback FileCreationInformation object is created with the image content , add the new file  to the list of files and commit is performed as a another asynchronous call to the server.
 
/// <summary>
 /// Obtain list instance with required minimum information via Client OM
 /// </summary>
 private void GetSpTargetListInfo(string listname)
 {
     //first build the workload 
     _targetlist = _ctx.Web.Lists.GetByTitle(listname);
     _ctx.Load(_targetlist, l => l.RootFolder.ServerRelativeUrl);
     //now execute it as a on single batch  
     _ctx.ExecuteQueryAsync(ListloadSucceeded, ListloadFailed);
 }

private void ListloadSucceeded(object sender, ClientRequestSucceededEventArgs e) { if (_img == null) return; using (Stream fs = _img.ToStream()) { var fileContent = new byte[fs.Length]; fs.Read(fileContent, 0, fileContent.Length); //create file object var file = new FileCreationInformation { Content = fileContent, Url = ApplicationContext.Current.Url + 

_targetlist.RootFolder.ServerRelativeUrl + "/" + Guid.NewGuid() + ".png" }; //add the new file to the target list var files = _targetlist.RootFolder.Files; files.Add(file); //Now execute the workload _ctx.ExecuteQueryAsync(UploadRequestSucceeded, UploadRequestFailed); } }

 

If you want to play with the webcam webpart a packaged Sandbox solution (SpImageCapture.wsp) can be downloaded here.  When activating this solution Silverlight .XAP file named SLWebCam.xap will be created in the site masterpage gallery and a hosting WebPart will be added to your WebPart gallery named SLWebCamWebPart.webpart  Drag this WebPart to your page and configure its web part properties by specifying a picture library to use for storing the images and you are done.   I’ve also uploaded the  full VS2010 solution here if you are interested in the code.

Enjoy!