I am truly inspired by the direction Microsoft has taken in recent times with the move to iterative development, open tooling and open source. At its core Office 365 is a suite of services providing OOTB experiences and a set of APIs ready for consumption. The experiences will satisfy many of the requirements for Portals, Intranets and fundamental productivity needs like File Management, Search and Sharing.
However, when you need to extend capability and customize, the ecosystem is really opening up. Microsoft will continue to focus on developing OOTB experiences but the Office 365 Developer’s world just became way more interesting!
During The Future of SharePoint event (May 4^th 2016) the new SharePoint Framework was announced. We’re yet to get our hands on this Framework but we already know a lot about what it will involve:

• Modern client-side development
• It will focus on lightweight web and mobile patterns
• Microsoft will use the same SharePoint Framework for development
• It will support open source tools and Javascript web frameworks

Here is what your Office 365 development world might look like:

The SharePoint Framework and a commitment to make more OOTB experiences open source is a further extension of Microsoft’s open stance on source code. We’ve seen a great community build up around the Office Patterns and Practices (PnP) group as well as open source solutions like Matter Center for Office 365.
As a consulting organization working on client projects it is not always straightforward to embrace open source. Ultimately most code we write will be owned by our clients and it will be at their discretion whether we utilize open source. There are huge benefits to be gained from working with open source and it’s great that we’ll now be able to do this with SharePoint Online and Office 365. Here are some core benefits to open source:

1. Cost – Providing there is a good fit, starting from a common foundation will be more cost effective than starting from scratch.

2. Completeness – Ultimately open source is likely to solve common problems better than a completely custom approach. Engaging thousands of users and developers in solving a problem is more likely to yield a comprehensive solution.

3. Quality – Linus’ Law, named after Linus Torvalds, the creator of Linux says that “given enough eyeballs, all bugs are shallow”; or more formally: “Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone.”. Open source will almost always have more “eyeballs”.

4. Security – The implementation of security by obscurity can be a risky business. How can a client be sure that a system is truly secure unless the source can be adequately reviewed? Open source can put more eyeballs on to an implementation to identify holes and thereby offer the opportunity to fix vulnerabilities.

5. Assurance – Working with open source ensures that commonly accepted patterns and practices will be adopted on a project. This will give a client greater assurance that their solution can be maintained as individual developers move on and off of projects.

 
A great early indicator of how open source is working with the Office 365 is the Office Patterns and Practices (PnP) group. It is hosted by Microsoft but is a community effort complete with a GitHub repository.
http://dev.office.com/patterns-and-practices
Since Perficient started to engage with PnP we have seen a steady increase in membership and attendance on the monthly call. During the May 2016 call I was very proud to see four Perficient names recognized in the top contributions for the month. I think this is a sign of times to come with SharePoint development. It’s great to be part of it!
 

OneDrive is leapfrogging the competition for document management, collaboration and sharing. It’s hard to fully quantify the benefits of using OneDrive for your file collaboration and sharing because it’s already part of Office 365. By virtue of being on the platform OneDrive already has the advantage of built in: Search, eDiscovery, Data Loss Prevention and the ability to discover content via Delve (to name a few core capabilities). However, if a 1:1 OneDrive to Box comparison is also relevant, please do read on…
Updates already released improve the synchronization client and the external sharing experience. Selective synchronization and a smoother External User experience were the most notable. Microsoft is further building upon these with more updates announced during The Future of SharePoint event (May 4^th).
In the web browser the OneDrive experience is consistent with the new Document Library experience. This will be great for consistency across the platform, ensuring that a user can interact with documents in the same way wherever they are on the platform. The new Grid view is a great example of how everything on the platform is becoming beautiful and mobile responsive.

The mobile App will fulfill the promise of being a one-stop-shop for all your document needs. Users will be able to browse for their personal and business documents on their mobile device. They will also be able to leverage Search and Delve right from their phone. Whether the document is located in a Team Site, in OneDrive or published on the Intranet the user can discover it via the mobile App.
It will most definitely be a first class experience when using the App on a phone. Features like the ability to share documents leveraging native controls for warning the user will be there. Even some analytics around a user’s own content will be surface in the App.

The concept of a document lifecycle is also being introduced. Whether part of a formal lifecycle or not, all documents usually have a lifecycle. I will usually create an initial draft version before starting share and collaborate on it with a small group. Once ready I might then publish the document somewhere accessible and discoverable by many people. The new Copy features in OneDrive and SharePoint will allow me to implement this, perhaps starting a document in my OneDrive before progressing it to a Team Site for collaboration and then finally publishing it on the Intranet. The new intelligent experience helping me find the right destination will smooth the whole process.

Any discussion of document management in Office 365 should include both SharePoint, OneDrive and the core capabilities of the platform. However, if your need is to compare OneDrive with the competition 1:1, then these new features significantly boost the case for OneDrive.

On Star Wars day the force is most definitely with SharePoint. SharePoint is here to stay and continues to form an integral part of Office 365. Let’s take a look at the Top 5 innovations announced today…
#1 Page Rendering Model
There is a radically new page rendering model. SharePoint pages previously relied a lot on ASP.Net with most of the rendering on the server. Instead, rendering is now on the client using modern web techniques. If the new Document Library pages are representative you’ll see that the page is constructed using Javascript, specifically using Knockout and a Require.JS pattern.
The new pages are mobile responsive by default and are built with a new canvas for WebParts. For some time we have been building mobile responsive experiences on top of SharePoint. However, these experiences often stopped being responsive when reaching non-Publishing pages. The new page model will be great news for building modern Intranets, Portals and Collaboration sites.
By divesting from ASP.Net and making the components available client-side developers now have an opportunity to reuse Microsoft components which were previously buried in .Net. By getting out of the .Net stack for rendering I think there are also great opportunities for optimizing page performance with less reliance on the server and new ways to tailor to mobile or low bandwidth scenarios.
#2 Document Libraries
Previously, SharePoint Document Libraries and OneDrive looked pretty different. This was counterproductive to adoption and it’s great they will now have the same UI. Like everything else, libraries are now mobile responsive. However, they also get some other important updates which will greatly enhance the experience.

• Thumbnail (‘Grid’) view of documents
• Inline editing of document properties in the Information panel
• Addition of columns inline in the view (without going back stage)
• Spotlight documents at the top of the page
• Easily link to documents and sites outside of the library
• Copy feature making it easy to copy to another location

 
#3 Team Sites
The Team Site template has been the most prolific of all the SharePoint site templates. However, it was usually where a modern experience started to degrade. Not anymore! Team sites get a new responsive look. Like all the other new experiences we expect the page to be rendered client-side and offer greater extensibility options than we saw before.

Team Sites will now come with the ‘hooks’ into other platform workloads like Office 365 Groups. We will now be able to use Office 365 Group membership to define the members of a Team site. This relationship will be reciprocal; an Office 365 Group will also have the capability to launch a Team Site.
#4 SharePoint Framework
A good friend of mine works in digital marketing. Whilst this industry is fickle-by-design I had become a little envious of his innovative work during the past couple of years. Over a beer, or milk as is his preference, we would discuss technology. He would tell me about the latest Javascript framework he was using whilst all I could report was that we had triumphed over Master Pages and Layouts. Well, I am looking forward to the next time I see him because SharePoint just became cool (again!).
The new SharePoint Framework will fully enable client-side Javascript development for 1^st and 3^rd parties. Microsoft will be using exactly the same framework to develop experiences as its partners and customers. They’ll be no ‘running down the hall’ in Redmond to get a special hook to build something. Microsoft will use the same tools as the rest of the community. What’s more is that a lot of what Microsoft does build will be available open source in GitHub.
The new SharePoint Framework is the realization of the ability to use open client-side tooling directly with SharePoint. Importantly, the ability to consume the Office Graph will be central to this as well. At Perficient we are aligning the tools, frameworks and libraries we will be recommending. For now, here are some you might consider, all centered around the Office Graph.

 
#5 SharePoint App
SharePoint will be getting its own mobile App for iOS, Android and Windows. Building mobile responsive experiences for SharePoint targeting mobile browsers has been great and I think will continue to be a good option. However, the new App for SharePoint will add a new dimension to the mobile experience.
The new responsive experiences will be neatly wrapped inside of the App. Accessing and working with a Team Site will now be easy via the App. Whilst we are not yet fully aware of the details, it has been indicated that the App will also facilitate the caching of the authorization, avoiding the user having to log in every time. This would be a great win over the mobile browser experience.
One could dismiss the App as a glorified browser wrapper. However, the future potential for leveraging push notifications and offline working is significant.
Conclusions
SharePoint is once again innovative and at the forefront of web technology. I am excited to see some foundational aspects of the new features drop into First Release tenants already. I understand that we expect to see further deployments in the next few days… I am looking forward to it!

It has been a busy couple of years working with SharePoint Online in Office 365. Perficient has been full throttle migrating organizations to the Microsoft Cloud adopting workloads like Exchange, SharePoint, OneDrive and Skype as well as making sense of new innovations like Delve and embracing Social with Yammer.
Particularly within the last year there has been a steady stream of new innovation deployed to Office 365. At times this has been a challenge to keep pace with and Perficient has been helping our customers digest and align with their business needs.
Microsoft show no sign of ‘letting up on the gas’. I am really excited about The Future of SharePoint on the 4th of May! I would encourage anybody engaged in SharePoint and productivity workloads to attend the virtual event. You can register here:
https://aka.ms/splaunch
 
 

The International Legal Technology Association (ILTA) Conference is off to an exciting start! The Perficient team is in Las Vegas this week to present our solutions for the Legal industry including our partnership with Microsoft on the new Matter Center for Office 365.

Matter Center was built for Microsoft’s own Legal and Corporate Affairs group and will now be offered at no additional cost to Office 365 customers. The solution provides document management using SharePoint Online and Azure as its foundation. Integration to Outlook and Word come via Add-Ins enabling all the major user stories an attorney would expect. By virtue of being deployed to Office 365 other major benefits of the platform such as: mobility, security, compliance, search, external sharing and discovery can be realized.
http://www.microsoft.com/mattercenter
Perficient has been working behind the scenes to deploy early versions of Matter Center and collect feedback from Law Firm customers. Our involvement builds upon our breadth and depth in SharePoint solutions, SharePoint-as-a-DMS and early experience with Office 365. All of which make Perficient a ‘go to’ partner for Matter Center.
https://blogs.office.com/2015/08/31/were-making-our-in-house-legal-software-matter-center-for-office-365-available-broadly/
In many ways Matter Center is a further extension of the patterns Microsoft is offering for Office 365 development. Rob Lefferts, Director of Program Management at Microsoft, presented at ILTA yesterday on this topic. In Perficient’s experience it’s a common misconception that Office 365 and SharePoint Online cannot be extended. In fact, the APIs have never been so rich, versatile and fresh.
The Office Patterns and Practices (PnP) group is a great starting point to help understand this very open, collaborative and progressive stance for the platform.
http://dev.office.com/patterns-and-practices
Matter Center will likely fulfill many of the requirements for a legal document management system out-of-the-box as well as offer new opportunities on the platform like Delve and the Office Graph. Where there are gaps we can rely on development against rich APIs to complete solutions in a supported fashion. Microsoft will even be releasing the source code on GitHub by the end of year.
If you are at the ILTA conference please do stop by the Perficient booth (#127).

SharePoint makes great use of its Search engine and it is pervasive throughout most SharePoint solutions today. Whether you are building Content Search WebParts leveraging Display Templates or creating a custom Search center you will be making use of SharePoint’s powerful and mature Search engine. However, what happens when things behave unexpectedly? How can you troubleshoot Search? This article will focus mainly on SharePoint Online (O365) but could also be applied to Search on premises.
Search is not magic!
When supporting others in their troubleshooting of Search issues I usually start by explaining that Search is not a black art. I would agree there are nuances but, for the most part, figuring out issues can follow simple scientific process. After all, SharePoint Search is just a collection of properties stored for query and retrieval by the Search engine.
The perception that Search is a black art often extends to users and their expectations. When search does not yield the results a user expects we should always investigate what they expect and why they expect it. Work directly with end users and try to help them understand why they are not getting what they expect.
Approach to Troubleshooting
Generally speaking it is advisable to try and make small but smart moves when changing configuration. It can sometimes take days or weeks to test the results of your configuration changes if long re-crawls are required. Above all, be scientific! Use the same test cases before and after changes and measure your results by tracking search results before and after.
The two sides of Search
There are two sides to Search, Crawl and Query. Potentially either side could be failing or not working as expected so it is important to investigate both.
SharePoint Online in O365 does not currently offer too much control nor insight into the Crawl side (this may change in future). SharePoint 2013 on premises offers lots of opportunity to troubleshoot the crawl. On premises we can:

• Inspect the crawl log
• Turn on verbose logging during crawls
• Attach crawls to Fiddler

The Query side can be inspected almost equally online as it can be on premises. It’s important to understand that Search queries are transformed as they are passed to the search engine. The user can type a query into a search box but there are number of places the query could be transformed or results influenced by search configuration:

• Search Results WebPart e.g. in a Search Center
• Content by Search WebPart
• Results Source
• Query Rules
• Synonyms

Inspect the Query using the Query Builder
There are at least two great ways to inspect the Query side of Search.
One, use the OOTB Query Builder to help you. From a Content by Search or Search Results WebPart you can open the Query Builder. Switch to Advanced Mode (you are most definitely Advanced now!) and then head over to the TEST tab. Once on the TEST tab click ‘Show more’ to see the full transformation of the query in the bottom text box (highlighted).

Seeing the full transformation of the query is important because there may well be transformations impacting results you had not thought of. In addition if any part of the query is using dynamic property values based upon the Page or User (e.g. {User.ProfileProperty}) these values will be expanded and displayed for inspection.
Inspect the Query using the 2013 Search Tool
The SharePoint 2013 Search Query Tool is now an essential part of my day-to-day work with SharePoint Search. It uses the REST API to retrieve results from SharePoint Search and can be used with SharePoint Online (O365) and On Premises SharePoint 2013.
http://sp2013searchtool.codeplex.com
The tool provides a fast, convenient and repeatable way to inspect Search results. A few tips for using this tool most effectively:

• Be scientific! Use fixed queries, analyze results before and after configuration changes.
• Turn off Trim Duplicates, it will be ON by default. Trimming duplicates can be really confusing when analyzing results, especially when you have repetitive test data in a system.
• Ensure you are using the correct Results Source. By default this will be set to Local SharePoint Results. However, if custom Result Sources are in play be sure to set them in the tool. You will need the GUID identifying the Results Source, this can be obtained easily by browsing to the Results Source in SharePoint and grabbing the ID from the querystring.
• Inspect the Rank values returned. The rank values will determine which results appear first. If any parts of the transformation use XRANK to boost results then this should be evident in the Rank value.

Check the basics
Are you writing valid Keyword Query Language (KQL)? SharePoint Search can only understand KQL and not syntax from other search engines such as Google, Solr, Westlaw, Lexis etc.
http://msdn.microsoft.com/en-us/library/office/ee558911.aspx
If the user is expecting a specific document to be returned has that document actually been indexed? This can be verified by performing a Path search e.g.:
Path:http://contoso.sharepoint.com/documents/document.docx
On Premises you could also check the crawl log to see if the item hit an error during indexing.
If the search query is expecting to hit document body content you should also check that the source text is readable by the search engine. In the case of PDF documents, they will need to be OCR’d.
Managed Properties
It’s likely that Search is not behaving the way users expect due to Managed Property configuration. Managed Properties are really the backbone of the search engine and our greatest opportunity for customization. They are also our greatest opportunity to make mistakes in configuration. We need to check the Managed Property settings. This article does not fully explore Managed Property configuration but here are the basic settings we should consider when troubleshooting:

• Searchable – Enables us to query against the content of the property, the content is stored in the full-text index. On premises we can also set the ‘Context’ of the property which will allow different weighting to be applied.
• Queryable – Enables us to query directly against the property e.g. Title:Hines . Note that this is different to simply being included in the full text index.
• Retrieveable – Enables us to retrieve the value of the property in search results. If a property is not retrievable it can sometimes be confusing when trying to inspect it using the search tool. The value may be Queryable and used in queries but not retrieveable so that we can inspect it.
• Refinable – Means that it can be used in a search by refinement. Note that this is different to querying properties.
• Sortable – Means that results could be sorted by this property.
• Mappings – Managed Properties will be mapped to one or more Crawled Properties. This relationship is worth reading more about and the necessity to re-crawl after changing these values in important to consider. When more than one Crawled Property is selected be sure that the configuration is resulting

It’s really important during any Search project to have consistent and well-understood Managed Property configuration. At Perficient we often build a Managed Properties Specification which allows us to collaborate on how properties will be configured. In addition it is highly recommended to script Managed Property configuration so that Development, QA and Production environments are the same. It’s very easy to let this slip and have misconfigurations confuse you when testing across multiple environments. More on script deployment to O365:
https://blogs.perficient.com/microsoft/2014/09/powershell-deployment-to-sharepoint-online
OOTB Title Managed Property
The ‘Title’ Managed Property is probably the single greatest point of confusion I see in SharePoint Search projects. Looking at the Mapping configuration it is already apparent that it’s complicated.

Unfortunately the MetadataExtractorTitle has, in my opinion, only added to this confusion. If your users are relaxed about seeing an interpreted Title in search results then you will most likely not get feedback. However, it appears most users get confused about this and require us to troubleshoot exactly why their Title is not displaying as expected in Search results. In response to this, you can of course look at the Crawled Property ordering and try to determine exactly why the Title Managed Property is populated as such. However, if it transpires this is a major problem for users I would look into creating your own separate Managed Property for handling Title.
Low level Debugging
Detailed debugging during a crawl or query is reserved for SharePoint on premises. During a crawl we can turn on VerboseEx logging and analyze the detail of what’s happening during acquisition. This could point to a number of problems including communication with the source during a crawl or an issue with security trimming when querying.
If you need to pursue a Search support incidents with Microsoft it is most likely Microsoft will ask for VerboseEx log. The only caution with this is that VerboseEx will grow the logs very quickly so it’s recommended you ensure it is only enabled for a short period of time.
Good luck!
SharePoint Search does have its challenges but it is also extremely powerful and offers great patterns for surfacing content. We see it making its way into new areas and services in Office 365 all the time. Embrace it! I think it’s here to stay.

Personal sites (formerly known as My Sites) are provisioned on demand in Office 365. Only when the user first clicks e.g. their OneDrive link in the suite bar is their personal site actually provisioned. This was a prudent architectural decision on Microsoft’s part to not provision space until it is actually needed. Anybody who managed pre-provisioned personal sites on premises will know that this can be unnecessarily expensive, especially when you have a very large number of users.
With this approach it is sometimes useful to know how many users have already provisioned their personal site, so as to get a measure of adoption.
Individually by User Profile
We can navigate to the ‘Manage User Profiles’ link and find this out individually for each user. When the user has a personal site we can click the drop down option to ‘Manage Personal Site’ and we are taken to the site settings.

When the user does not have a personal site, a message is displayed saying they don’t have one.

 O365 Reports 
For gathering a total count, there is a report under:
Admin > Office 365 > Reports > OneDrive for Business sites deployed

 
Use Search to report on Personal Sites
We can also use Search to find all the personal sites e.g.
Path:https://chrishines-my.sharepoint.com AND contentclass:STS_Site
This will get all the personal sites under the My Site application (chrishines-my.sharepoint.com). The query will naturally return only one page of results at a time. However, we can use the search REST API to get creative and return large pages (maximum 500) and iterate through all pages to get a count. The REST API call would look like something like this:
http://chrishines.sharepoint.com/_api/search/query?querytext=’Path:https:%2f%2fchrishines-my.sharepoint.com+AND+contentclass:STS_Site’&trimduplicates=false&startrow=4500&rowlimit=500
This particular request will get all the personal sites from count 4,500 – 5,000. In my case this returned 239 results telling me that 4,739 personal sites had been created thus far.
You may be comfortable writing REST API calls to achieve this. Alternatively, I would highly recommend using the SharePoint 2013 Search tool to help out.
https://sp2013searchtool.codeplex.com
Remember to set trimduplicates=false as identification of duplicates can cause a lot of confusion with this type of query.

As a technical architect I am used to the rapid evolution of language to describe an accelerating technical world. Only a couple of years ago using the word Cloud would most likely conjure images of the cumulonimbus variety. Today I rarely join a conference call where The Cloud is not mentioned and we can be confident in technical circles that everybody understands the term.

The Cloud

According to Wikipedia, references to The Cloud began as early as 1996 when Compaq used the term in an internal document. Much later Amazon began to use it as part of their Elastic Compute Cloud terminology. We now use the term to describe great new services like Azure and Office 365.

I like The Cloud and feel it is a very fitting term for describing the way we now host services. My compliments to whomever actually first coined the term! It makes a lot of sense.

Now that we have The Cloud we have the premise of On Premises and need a term to clearly refer to services hosted on site (as opposed to in The Cloud).

So, do you say On Premises or On Premise?

Premises

“A house or building, together with its land and outbuildings, occupied by a business or considered in an official context.”
– http://www.oxforddictionaries.com/definition/english/premises

Premise

“(British also premiss) Logic A previous statement or proposition from which another is inferred or follows as a conclusion.”
– http://www.oxforddictionaries.com/definition/english/premise

Was it a Mistake?

It seems clear to me that On Premises is correct whereas On Premise is derived from a mistake made and copied many thousands (millions?) of times.

I find the discussion interesting because I think it highlights the rapid adoption of terminology, correct or otherwise. As technical professionals I think we should always strive to communicate better. Describing technology more accurately, clearly and concisely is important and will help us serve the needs of decision makers and users better. I think we should always question the terminology we use and improve upon it whenever possible.

I was recently preparing a presentation for a Chicago SharePoint Saturday. As I built out my slides explaining some O365 DevOps best practice it struck me that an Agile methodology could be the only viable methodology to deliver and maintain SharePoint Online projects. Here’s why…
At Perficient we have embraced SCRUM for many SharePoint projects and it has proven to be very successful. I took the SCRUM Master Course and certification to solidify my understanding of SCRUM. I recall the tutor saying that the largest part of adopting Agile is to think in an agile way. Quite simply I have modified the way I think about projects and I think this has helped me lead projects in the cloud.
To contrast, I began to think about how hard it would be to deliver SharePoint Online projects using a more traditional waterfall methodology. When you consider the ‘Evergreen’ service and how quickly we are seeing new features appear it’s a paradigm shift in my field of work as a SharePoint Architect.
I have made it part of my weekly routine to check the Office 365 public roadmap to assess features being rolled out as well as those on the horizon. This helps me understand, from a feature perspective, what I need to keep a close eye on in coming weeks.

O365 Public Roadmap

O365 First Release

In my last blog post about DevOps for SharePoint Online the process I presented relied a lot upon scripted deployment to SharePoint Online (O365). I wanted to expand upon that a little and explain in a little more detail about how Perficient is using PowerShell to manage our deployments for our Development, QA and Production environments.
Automating any task which is repeated can be a productivity benefit providing the time invested in developing the automation takes less time than repeating the task itself. Automation also significantly reduces chance of ‘human error’.
Automating deployments is of little benefit to light users of SharePoint who do minimal customization of SharePoint in a single O365 tenant. However, as you begin to customize more and introduce the need for testing cycles then automation starts to become valuable. When you add multiple tenants into your DevOps and add multiple developers or administrators then automated deployment can really pay huge dividends.
I think it is fair to say we are in a period of emerging standards for deployment of customizations to SharePoint Online. When we worked on-premises with SharePoint the WSP provided great deployment options especially when you consider Feature stapling. This is basically off the table with O365 and we’re looking for new best practice.
I think that the combination of PowerShell and the SharePoint Server 2013 Client Components SDK is a strong candidate for best practice automation of deployment to SharePoint Online. PowerShell gives us the lightweight scripting we need in order to move rapidly through automated builds and deployments. The Client Components SDK gives us the full Client Object Model on the administrator’s desktop allowing them to execute on a huge variety of scripted tasks. Here are a couple of useful resources on this topic, one from my colleague Roydon Gyles-Bedford whom I credit with a lot of Perficient’s thought leadership in this area:
https://github.com/rgylesbedford
http://soerennielsen.wordpress.com/2013/08/25/use-csom-from-powershell
At Perficient we have invested in PowerShell Modules which use XML configuration to drive deployment of items such as:

• Master Pages
• Page Layouts
• Content Types
• Display Templates
• Term Store Terms

The XML configuration files are pseudo-CAML (Collaborative Application Markup Language!) which is wrapped in our own markup to help the Modules know what to do with it. The nice thing about CAML is that it is already defined and baked into SharePoint. We will often use the Client Browser Tool http://spcb.codeplex.com to browse existing artifacts like Content Types to understand how to define Content Types from scratch. E.g.

Aside from configuration defined in XML we also simply drive configuration through PowerShell modules using the Client Object Model directly. Here is an example function for adding a Web:

At this point in time the Client Object Model does lack functionality when compared to its server-side counterpart. However, this is improving all the time with new methods being added in every release.
In some cases it is possible inspect the server-side object model using a tool like IL Spy http://ilspy.net and find (unsupported) ways to get the job done. For example we found a way to add links to the Search Center Navigation via this technique. I must stress that using an unsupported method should be for convenience only and you should have a backup plan should it fail. We normally write this backup plan into our deployment documentation and it’s usually just a manual way to achieve the same thing albeit more slowly.
I am now also seeing lots of discussion and examples around HTTP Remote operations to help fill the gaps in the Client Object Model. This is of course also unsupported but can be effective as a convenience and time-saver. We’ve used this effectively to map Search Crawled Properties to the Refinable Managed Properties in SharePoint Online. This is not supported by the Client Object Model and can take a huge amount of time so is ripe for automating. Here is a snippet showing how we call a function to update RefinableString00 with Crawled Properties:

In conclusion, automation using scripted deployment can be an extremely versatile and effective way to support your DevOps for SharePoint Online. At Perficient, SCRUM has proven to be a very effective methodology for SharePoint Online projects. Typically we are making the scripted deployment of any new feature part of the ‘Done Criteria’ for any development work. Scripting the deployment then very much becomes part of feature development and will be effectively tested in development environments before progressing to QA and Production.

With the rapidly evolving migration to the cloud SharePoint teams are faced with a new challenge: How do we develop and deploy for SharePoint Online?
If your feet have been firmly planted with on-premises development for SharePoint it can be a little daunting trying to move your process to the cloud. Where and how should we conduct development? How can we implement release through development, quality assurance and production?
This article aims to help you get started and is based upon the hands-on experience of working with SharePoint 2013 Online during the past 18 months.
Perficient Model

Develop for the Service
Above all recommendations it is highly advisable to build new features for the service using SharePoint Online. Whether you are writing CSOM, customizing a Master Page or building an App you should do this for the service and not in a local (on-premises) development environment. SharePoint Online offers a very rich API which is very extensible but it can be extremely sobering to realize the feature you just spent the last few weeks building relies upon a feature not available Online. If you are developing features for both Online and On-Premises you can always bring things back on-premises later.
With a MSDN subscription developers can provision their own Office 365 tenant and begin development within a few minutes. How many hours would this have taken for the developer to build their own VM for on-premises development? If the developer does not have an MSDN subscription they could always use a trial tenant on a temporary basis or pay for a single user tenant for indefinite use. When provisioning any new tenant for development ensure that it is under the same license as QA and production (e.g. E3).
Once a developer is ready to deploy and review new features they can do this on a separate Development (Integration) tenant accessible to the team. This Development Environment is typically used for demonstrations of new features, in SCRUM Review meetings for example.
Tenant Isolation
Consistent with any mature software development it is important to ensure that Development, QA and Production are properly isolated and permissions configured accordingly. Developers will most likely have full administrative access to Development but will only have read or less access to QA and Production. Keeping your developers out of Production is a key principle for stability and ensures good consistent deployment techniques are employed. It also ensures that we maintain healthy disagreements between developers and administrators which is as old as time and ensures the project is fun!
It helps to name tenants consistently. We usually use the convention:

• https://<production tenant name>.sharepoint.com
• https://<production tenant name>DEV.sharepoint.com
• https://<production tenant name>QA.sharepoint.com

A key consideration with this isolation is how to maintain accounts across all three environments. Most likely the Production environment will have federated identities synchronized to the cloud with ADFS and DirSync or FIM. This allows us to work with corporate credentials in Production. However, a single domain can only be synchronized to one Office 365 tenant. So what should be configured for Development and QA? It is of course possible to build new domains (on-premises) and mirror the synchronization for Production. This is of course the most pure form of ensuring Development and QA are true representations of Production. However, this may be overkill for your development and testing needs.
It can be advantageous to use cloud accounts (onmicrosoft.com) in Development and QA, they are extremely lightweight and easy to manage as your team grows. Cloud accounts are particularly useful when working with professional services organizations as setup can usually avoid what might otherwise be a lengthy setup process. However, if your solution relies heavily on synchronized identities then it may be necessary to have Development and QA domains which mirror production.
Another key driver for isolating tenants in this way is that it ensures no global configuration changes during development can impact the production system. Consider the configuration of:

• Content Types
• Term Store Configuration
• Tenant-level Search Managed Properties
• Tenant-level Search Result Sources

One could argue that developing in a single Site Collection isolates development appropriately. However, the misconfiguration of these items alone could easily break a production system and take some time to recover from e.g. Search may need to re-crawl or the Content Type Hub will need to wait for a scheduled push.
Scripted Deployment
This article will not fully elaborate upon Scripted Deployment to SharePoint Online I will write another article shortly on this topic. However, it is an important principle of this model. Automating any task which is repeated can be a productivity benefit providing the time invested in developing the automation takes less time than repeating the task itself. Automation also significantly reduces chance of human error. It is less obvious how to automate deployments for SharePoint Online but the benefits are clear and have paid huge dividend for our teams working with the service.
What is Scripted Deployment? For SharePoint Online this means writing PowerShell with XML configuration and using the SharePoint Server 2013 Client Components SDK (currently v16). The PowerShell is run locally on the developer or administrator’s machine but connects to SharePoint Online using the Client Object Model. Through this script we can deploy most things required for SharePoint Online customization such as:

• Master Pages
• Page Layouts
• Content Types
• Display Templates
• Term Store Terms

It has taken some investment in the development of PowerShell modules but these become highly reusable across projects.
As developers work with their own tenant they develop the deployment scripts required for their feature. Those familiar with SCRUM will relate to ‘Done Criteria’. Our Done Criteria includes development of a feature and its scripted deployment to the Development (Integration) tenant where it can be reviewed. There are some exceptions which cannot be achieved by this technique but the Client Object Model does support a very wide range of common needs for deployment and configuration. Where exceptions exist these are documented in a deployment document for manual execution by an administrator.
Replication of Production Data
It is desirable to have recent data available in QA to ensure good and valid testing. For this replication it is advisable to use a third-party migration tool like Metalogix Content Matrix. When selecting a tool for this purpose ensure that it can migrate the data faithfully to ensure good testing but also that it can transform data as required. For example, if Production data uses synchronized identities but QA uses Cloud Accounts it will be necessary to perform some transformation. E.g.
chris.hines@contoso.com could be mapped to chris.hines@contosoqa.onmicrosoft.com
Happy development and deployment!

I started working with SharePoint 2013 RTM today on a Windows 2012 Server. My initial project is to do with search so I quickly created a Search Service Application and performed the minimal configuration to begin crawling the ‘Local SharePoint sites’ content source. Everything looked good so I started a crawl on this source. I quickly realized that it was stuck in the ‘Starting’ state.
I had failed to notice that I was missing the green check marks indicating that the search components were running correctly. Here is what they should look like:

Green check marks indicating search components are running.