To objectively quantify your ITIL process and implement Continuous Improvement using the Seven Step Process, an organization will need to measure their overall performance. Establishing meaningful organization metrics is a vital component of any ITIL process.
One of the first steps in identifying metrics to use, is to determine what the objectives are. The metrics must support the specific objectives of the process in the organization. For example, in the Incident Management process, if the desire is to stabilize production then the number of incidents or uptime metrics would be useful barometers of success. These type of metrics are extremely well documented and fairly standardizes. Sites like KPI Library have literally dozens of useful Key Performance Indicators (KPIs) for nearly every ITIL process with one notable exception – Knowledge Management.
Measuring the success of a Knowledge Management process is challenging. A Successful Knowledge Management system empower customers and IT staff with vital information that they require in order to perform their role. It is difficult to quantify incremental improvements in organizational intelligence. How do you objectively measure “Are you smarter today than you were yesterday?” at an organizational level?
Objective measurements like “Number of Pages of documentation per Release Unit” or “growth rate of Knowledge content database over time” are not truly meaningful. The challenge of these metric is it gives you a target for documentation of information, not for measuring the value of transforming information into intrinsic business knowledge.
Since you cannot effectively measure business knowledge based on fairly arbitrary measurements of output what can you do? Here are three simple techniques to measure Knowledge Management delivery utilizing components of your SharePoint infrastructure as well as other tools and processes that you may have in place:
- Measure Knowledge Management Through Proxies
- Utilize SharePoint Surveys to Measure Customer Satisfaction
- Measure the Effectiveness of Knowledge Management Delivery Through SharePoint Search Logs
Measuring Knowledge Management Through Proxies
One technique is to do what scientists do – measure something else. Scientists measure one physical quantity to use it as an indicator of the value of another. This is referred to as a proxy.
A famous example of a proxy for climatologists is to analyze ice cores from Antarctica and Greenland. There are no temperature charts that can be plotted in an Excel spreadsheet from 800,000 years ago. By analyzing trapped dusts in ice cores, scientists can gather data on global scale volcanic eruptions and major trends in desertification. By measuring deuterium, a form of the element hydrogen, scientists can map out the temperature of the last 800,000 years.
In much the same ways that ice cores can be used as proxies for temperature, other ITIL processes can be used to measure the overall effectiveness of Knowledge Management.
Effective Incident and Problem Management processes will result in concomitant improvements in Knowledge Management.
- Problems that generate workarounds and known errors will appear in the Known Error databases, now a subcomponent of the SKMS.
- Proactive analysis of request fulfillment trending data should denote the number of requests for knowledge content. Measure the number of Service Requests for information that can be answered by your published Service Catalog documentation. Effective Knowledge Management presentation should illicit a reduction in the number of request fulfillments. If it does not, you may have a presentation layer issue of your SKMS that needs to be corrected (i.e. your SharePoint site needs to better lay out the knowledge). This approach will also help you identify opportunities for Knowledge Management content creation. If customers are asking for the same information over and over again that is not available via your SKMS, you have identified an opportunity to create it.
Release and Change processes should identify Knowledge Management opportunities. Each release unit should have a valid Knowledge Management component and a Technical Services Catalog component.
- In general terms, each major release unit should generate a technical service catalog entry along with IT technical Knowledge Management components. For example, the release of a new WSP will generate a catalog entry reflecting the new update as well as instructions for installing the WSP to the farm.
- Knowledge Management may be generated for the customer. There may not be a 1:1 relationship of catalog entry and customer Knowledge Management documentation. For example, the release of an administrative service pack may result in no new meaningful customer functionality. However, the release of the service pack should initiative a review, if only cursory, of the existing documentation from a customer perspective. Utilizing a SharePoint wor
kflow to trigger a review and sign off from the identified Subject Matter Expert of content management would be useful.
Utilize SharePoint Surveys to Measure Customer Satisfaction
And, as with any ITIL processes, measure the effectiveness of Knowledge Management delivery through customer surveys. Be sure that your questionnaires address:
- The knowledge management content itself
- the ability to find the content intuitively through the presentation layer (i.e. your SharePoint site) and
- the ability to find the content through a search engine.
Measure the Effectiveness of Knowledge Management Delivery through SharePoint Search Logs
Nobody wants to search for anything. Everybody wants to find what they are looking for. SharePoint Server 2010 collects meaningful information on what your customers are looking for with regards to Knowledge Management. Your Knowledge Manager should work with your SharePoint infrastructure team and review your Search logs on an ongoing basis.
- Results returning zero click-throughs where Knowledge Management content exists indicates that your users are not finding the Knowledge Management Content that you are presenting. These are opportunities for Best Bets.
- Results returning zero click-throughs where no Knowledge Management content exists should be evaluated as knowledge gaps. If the search string count is high, an opportunity to create Knowledge Management content exists.
