Moving to SharePoint 2007:
The Architecture
Summary: Migrating from SharePoint 2003 to SharePoint 2007 is an extensive undertaking. There are many, many planning activities required and in most cases some significant mapping if not development work will be required. One question that needs to be answered before starting the process is: "Why move to SharePoint 2007?" This series address the many answers to that question.
I’m not a marketing maven by any means but with respect to Microsoft SharePoint 2007 adoption within the marketplace I would estimate that we are closing out the early adopter phase and moving into the late adopter phase. I think this explains why I have been seeing some strong interest recently in SharePoint 2007 from organizations currently using SharePoint 2003.
I have been asked quite frequently in the last two months why any organization should consider moving to SharePoint 2007 from SharePoint 2003. Most of the time, the requestors are interested in more than a simple bullet-point list of new and enhanced features. They are already users of SharePoint and understand its general feature set. They expect feature set enlargement and enhancement. They want the list, of course, but their real interest is in reasons that build a strong case for moving to the new version.
Why then? Why should an organization move to SharePoint 2007 especially if they are already using SharePoint 2003 successfully? In this, the first of a series of three blogs, I will provide some answers to this question. I’m certain you can find blogs written around the time that SharePoint 2007 was released that also answer this question, but I hope to provide a fresh take on this that will be valuable to those of you looking seriously at and planning for a move to SharePoint 2007 in the next year.
A listing of what are, in my personal opinion, the key SharePoint 2007 feature additions and enhancements can be found at the end of the last blog in this series. There you will also find links to a number of Microsoft documents presenting comprehensive lists and discussions of all the feature additions and enhancements.
Assuming, however, that you also are interested in more than a list of feature enhancements, this series offers a number of other reasons. I have grouped them according to three categories: architecture, technology and business in that order with no significance implied by their sequence. Also, I have intentionally not differentiated between Windows SharePoint Services (WSS) and Microsoft Office SharePoint Server (MOSS). If you are not familiar with the differences between the two, there are a number of documents available on the Microsoft web site that describe them.
In this blog I will focus on the architecture of SharePoint 2007 and why the changes in the architecture provide justifications for moving from SharePoint 2003.
Architecture
How we arrived here
To say that SharePoint 2007 provides an improved architecture over SharePoint 2003 is, well, an understatement. A bit of SharePoint history is useful here – very brief I promise!
SharePoint 2001, the first version of SharePoint, used the Web Storage System known as WSS (unrelated to Windows SharePoint Services, also called WSS). WSS was the central repository for content and was associated with Exchange 2000. The document store of SharePoint 2001 was a modified version of the Exchange 2000 web store. This meant that Exchange technologies could be used with SharePoint 2001. Exchange 2000 provided a workflow engine and designer, collaboration capabilities tied to event handling and other possibilities provided by Collaboration Data Objects (CDO). SharePoint 2001 provided its own interface PKMCDO (Publishing and Knowledge Management CDO) that allowed access to and extensions of its out-of-the-box functionality. It was also possible to tie into SharePoint Team Services which provided a collaborative engine and integration for Office XP clients. (The "sts" still found throughout SharePoint – stsadm, numerous folders prefixed with or including "sts" in their name, and so on – is a holdover from SharePoint Team Services)
The underlying architecture of SharePoint 2001 presented some issues however since every SharePoint Workspace (analogous to a site collection in SharePoint 2003 or 2007) had its own folder in the file system. Each workspace folder contained a complete set of all the files necessary to render the pages in the workspace. (Imagine the 6 or the 12 hive existing for each virtual server or web application and the issues become pretty clear!) Eventually Microsoft made some tools available that allowed SQL Server 2000 to be used as the repository for content but its use as the repository was unsupported. Still, there were some large enterprises I worked with that elected to use SQL Server 2000.
SharePoint 2003 incorporated major architectural and technology advances over the 2001 product. Architecturally SQL Server, initially 2000 and later 2005, became the central repository engine. Additionally SharePoint 2003 represented the advent of a single-instance "factory" if you will – best represented by ghosted pages which, despite all the issues associated with them, or better their unghosted versions, were a dramatic improvement over the duplicated files of 2001workspaces.
Additionally the technology foundation of SharePoint 2003 was .NET (with large doses of unmanaged code) and represented a substantial improvement over the confusing mix of technologies including Digital Dashboard, PKMCDO, WebDAV and CDO underlying SharePoint 2001. Web parts, introduced in SharePoint 2001, became .NET based as well in SharePoint 2003.
Finally, the Code Access Security capabilities of .NET-based web parts closed a very large security hole in SharePoint 2001, though many developers lost handfuls of hair successfully implementing custom web parts in SharePoint 2003.
Despite all these steps forward, however, feature/function loss accompanied the transition to SharePoint 2003. The add-back of many of these lost features as well as an immense increase in its functionality drove the initial interest in SharePoint 2007 to a large extent.
Enter SharePoint 2007
There are many areas where SharePoint 2007 incorporates architectural advances over SharePoint 2003. The principal ones include:
- Farm Topology
- Redesign of Shared Services
- Index/Search interaction
- Sites
- Services
- Integrated Publishing
- Security Model
- Site Definitions
- App platform
Flexible Farm Topology
It’s difficult to identify the most significant architectural improvement from among the many improvements in SharePoint 2007. Looking from a pure topology perspective, I would pick as the number one improvement the ability to configure and reconfigure the farm in numerous ways. There are very few unsupported topologies with SharePoint 2007. The consequence is that an organization can much more easily grow their SharePoint farm or farms as needed and not have to be concerned about moving a service to a different server in the farm. Scale up, scale out and reassignment of services to new or other servers in the farm are not only possible but generally very easy. A reconfiguration of a SharePoint 2003 farm often required a rebuild of the entire farm and all servers. This was usually prohibitively expensive which meant organizations were forced to make difficult decisions. SharePoint 2003 deployments could not be easily adapted to changing business requirements or events like an acquisition for example. SharePoint 2007 eliminates those shortcomings.
Shared Services Redesign
A redesign of how shared services, for example index/search, audiences, etc. are provided can be found in SharePoint 2007. The ability to have multiple shared service providers, important in some situations, was not possible in SharePoint 2003. Furthermore, MySites are always (not optionally) provided by shared services in SharePoint 2007 and can be kept in a web application (virtual sever in 2003 terms) dedicated to MySites. This latter capability isolates MySites so that business operations are not disrupted by the need to restore a deleted or corrupted MySite, which is not a site but rather a site collection. Usually this was not the case in SharePoint 2003 which meant that users might not have access to their MySite for days if it became corrupted or a sub-site within it was deleted unintentionally. Another related and important result of the redesign of shared services is that the ability to manage portions of the shared services can be given to knowledge managers without having to give them farm-level administrative rights. This means that those directly responsible for monitoring and improving the use of SharePoint have direct access to the tools that make this possible.
Index/Search Interaction
The design of index/search and the interface between the index server and search server(s) has been completely redone in SharePoint 2007. The number of crawl types is less (2 instead of 4); indexed items’ metadata is stored in a SQL Server database rather than in the index files; and propagation of indexed content to the search server(s) is done as the indexing occurs. These are just a few examples. These changes mean more easily managed indexing schedules, faster search results and more accurate search results respectively. Although search itself still requires configuration – mainly because the default configuration will generally not yield satisfactory search results – the architectural changes have made significant differences in the performance and manageability of indexing and search.
Those of you familiar with the details of the three-tiered structure of the SharePoint indexes (RAM, Shadow and Master), the Persistent Query Engine and the index management challenges of SharePoint 2003 can put all that behind. It’s all been redesigned and streamlined.
One final point before leaving index/search: the ability to manage indexed properties more easily thereby improving the search experience of SharePoint 2007 users is a direct result of the redesign of the indexing architecture. While it was possible to extend the properties indexed in a SharePoint 2003 environment, it was difficult and often required a trial and error approach. You might have considered or actually taken the plunge and extended the indexed properties before but with SharePoint 2007, I can virtually guarantee you will do so. It would almost be irresponsible not to do so.
Sites, Sites Everywhere
All SharePoint "webs" are based on sites – even central administration and search center(s). Unlike SharePoint 2003 that incorporated areas, which were sites with special properties and an unusual nesting characteristic (the "C" buckets: …/C2/C10/…), even portal webs are site-based in SharePoint 2007. This means consistent management and backup/restore operations, as well as common behaviors, apply throughout a SharePoint 2007 implementation. This results in more efficient administration and management of SharePoint and a more consistent experience for all users, administrators included. Additionally the improved ability to rearrange the structure of a site is a beneficiary of this design.
Services
A service-based architecture is used for SharePoint farm functions. Shared Services, Excel Services, Forms Server, the Business Data Catalog, a central catalog of XML-based data retrieval, are all service-based in their architecture. The ability to extend the capabilities of SharePoint 2007 using this service architecture is another major benefit of this design. The flexibility of the farm topology is largely due to this design as well.
Integrated Publishing
Integration of a publishing model, even coexisting with a collaboration model in the same web application if desired, is a significant architectural change that SharePoint 2007 incorporates. The same web application can be used to host collaborative sites and internal portal sites that follow the publishing model familiar to those who have used Microsoft CMS. The model used in SharePoint 2007 is not identical to that in CMS but its best aspects have been carried forward into SharePoint. Furthermore external facing web applications can follow web publishing best practice models and the staging environment can incorporate SharePoint collaboration capabilities that those involved in producing content find useful.
Security Model
SharePoint security has also been redesigned. Most important in the redesign is the ability to use Active Directory or other stores for authentication purposes; pluggable authentication can be used whenever required. Also either NTLM’s challenge-response or Kerberos’ ticket-based authentication mechanism can be used. This allows the ever-present delegation (double-hop) issue to be addressed at the system level. Other key changes in the redesign of SharePoint security include authorization and policies. Authorization is truly role-based and permissions are more fine-grained. Additionally application policies allow the assignment of rights, or the denial of rights, to accounts. These policies override local settings. This allows the denial of access to be assured regardless of what individual administrators might configure and is extremely important where compliance with regulatory restrictions are required.
Site Definitions Re-architected
The structure of site definition files and the use of supporting files have been significantly re-architected so that it is easier to use and to deploy new features, web parts or custom site definitions. Chief among the various components of this redesigned approach is the ability to construct "Features", which is a specific SharePoint term representing a unit or a set of features/functions/web parts that are deployed as a whole. Deployment is accomplished through stsadm – to one web front-end server. SharePoint updates all other web front-end servers in the farm. Additionally roll-back is just as easy. This greatly reduces the effort and the likelihood of error associated with deploying new functionality. Finally these features can be enabled or disabled through the browser UI at the farm, application, site collection or site level depending on how they are scoped during development. None of this is available in SharePoint 2003.
The real benefit of this redesign and the Feature-based approach it incorporates is the ability to add functionality incrementally to existing SharePoint sites and webs. The difficulty in doing this in SharePoint 2003 was a problem that many organizations found vexing. Again, Microsoft has made it much easier for an organization to adapt its SharePoint environment to changing business needs. This brings us to the final and for some the most significant architectural change in SharePoint 2007.
True Application Platform
SharePoint has been redesigned so that it can be used as an application platform. A number of the architectural changes discussed above make this possible, but so does the fact that almost every one of the components that comprise SharePoint is extensible. Your organization can use it like it is, out of the box, with very limited customization. SharePoint 2007, however, can also be nothing more than a set of services and classes that your organization uses to build a highly customized, perhaps even unrecognizable as SharePoint, application or set of applications.
I’m not talking about branding that masks SharePoint’s look nor revisions to master pages or page layouts that reposition content, nor web parts that present SharePoint data in a new way, nor even web parts that present external data as business performance indicators.
I’m talking about creating applications based on SharePoint that use the data of external databases, either legacy or newly designed, for read and write operations initiated from the collaborative SharePoint environment. I’m talking about applications based on SharePoint that use web services and services for portlets to present unified intelligence related to process and practice; applications that use system services and other applications as well as all that .NET 3.0 and now 3.5 provide to create a system that drives improvements in the effectiveness and the efficiency of the day-to-day work that takes place in every organization. It’s not just for collaboration anymore.
That might sound idealistic or even like an over-the-top marketing-oriented, hyperbolic, paragraph. It isn’t. If you want to see what I mean just take a look at how Microsoft themselves have done this. Take a look at how Office 2007, Microsoft OCS, Microsoft Dynamics and Microsoft Performance Point Server can be integrated into the SharePoint 2007 environment. Use your imagination and overlay on top of that what you know about .NET, about web services and about the need you have today for a truly integrated environment and I think you can see the possibilities. I just happened to give an example of how Microsoft has done this. I’m sure you can see ways in which you can do the same thing with your systems and applications.
Next up: Moving to SharePoint 2007: The Technologies
