Metrics

“If you can’t measure it, you can’t manage it”

This is about as accurate a phrase as I have ever heard in my life and yet we have a multibillion dollar industry where a standardized set of metrics is generally not in place.

Consider a room in a house that is 10 ft by 20 ft. The area of the room is therefore 200 sq ft. Now if a carpet installer comes in to measure up for new carpeting, he will measure the room and come up with 200 sq ft. Sure, there may be a slight deviation in his measurements: he might come up with 199 sq ft or 201 sq ft but he will be in the vicinity of 200 sq ft. Next, if a painter is brought in, he too will measure the room to be 200 sq ft. Any other workmen will also measure the room to be 200 sq ft in area. The reason for this consistency is that the world of “areas” has a standard of measure. This is square footage in our example but could be square nanometers for nanotechnology or acres for measuring farmland. The point is that there is a standard of measurement which is universally followed and therefore leads to consistency across the board.

However, in the world of IT and particularly software centric IT organizations, we have a great deal of confusion and chaos. A set of requirements generated by the customer produce a wide range of responses from one company to the next. One company might state that they can complete the project and deliver the solution to the client in 2 months while another might forecast 6 months. After which, both companies will end up being delayed and deliver late and over budget anyway. No wonder then that 7 out of 10 IT projects “fail” in some respect.

Now there are many reasons for this astonishingly high failure rate in IT, but the attribute that would assist the most in obtaining a clear picture of the situation and determining the steps to rectify the problems beforehand would be metrics and their proper application. Interestingly, metrics are also something that I, personally, as an IT process consultant have had the most difficulty in getting people/organizations to implement.

In my first post, I had touched on reasons for this difficulty in implementation of metrics and process improvements in general. It usually boils down to fear of negative impact to people’s career if the correct numbers were to emerge. It is the “cubicle dwellers” that generally fear the impact of metrics. Due to this short sightedness, however, they suffer working in a highly chaotic and stressful environment dealing with unnecessary emergencies and problems. My view is that if you are doing an honest day’s work, you have nothing to fear from metrics and your work being measured. It is, however, ultimately up to management to ensure that a positive atmosphere is created regarding metrics and their application in the workplace.

In my opinion, the IT department of any and every organization should implement an IT Balanced Scorecard, a derivative of the Balanced Scorecard (introduced by Nolan Norton and Robert Kaplan). For software centric organizations, I would also recommend the implementation of Function Point Analysis as a subset to the IT Balanced Scorecard.

The IT Balanced Scorecard consists of the following 5 categories:

• Financial Performance: This should include budgeting, accounting and charging (if applicable) metrics


• Project Performance: Metrics could include the ratio of investment to return for new projects as well as enhancement projects


• Operational Performance: This includes metrics that measure the performance of the product or service being offered such as availability, uptime, downtime and Mean time to Failure etc.


• Talent Management: Metrics that measure the IT Human capital such as Staff Satisfaction and Retention, Staff Training and attractiveness to external job seekers


• Customer Satisfaction: Metrics that provide feedback of the customer satisfaction levels

It is up to each organization to populate each of these categories with the appropriate metrics that would provide a clear picture of the IT department and its alignment with the company’s mission and strategy.

The paradigm shift of the Balanced Scorecard from previous metrics techniques is the focus on alignment with customer’s requirements and satisfaction as opposed to short term focus on cost reduction and low-price competition.

For organizations that create and release software, Function Point Analysis (FPA) provides a standard of measure for requirements. Past attempts at measuring software development included Lines of Code which suffered from lack of standardization between development environments (java vs. C# for example) and Man-Hours which simply isn’t enough information. To describe FPA simply, it is an adjusted count of the transactions as well as data access operations that a requirement necessitates within the software hardware environment of the application. When all the function counts for the requirements are added a standardized measure of the software requirements is obtained for the project. The beauty of this is that if one certified function point counter adds up the function points for a project’s requirements and another counter independently recounts the function points for the same project, the number of function points that would emerge from both counters would be the same. I hope I am not the only one excited by the positive ramifications of this technique! With a combination of the Balanced Scorecard and FPA in place, a fundamental basis of measurement can be made available in order to evaluate the situation, plan for the future and correct mistakes before they get out of hand.

The implementation of new metrics within an organization need not be a traumatic event. A sequence of steps that make the implementation a walkover are:

• Initial training and education of relevant stakeholders


• Analysis and development of the strategy of the metrics to be created (usually derived from the company’s high level corporate strategy)


• Selection of the metrics (care must be taken here to select metrics that can be accurately measured)


• Definition of the measurement technique and the data collection technique
  Assigning ownership to the appropriate stakeholders


• Actual data collection and quality assurance


• Review by management and analysis of success of implementation

It is recommended that this process be repeated regularly in order to introduce new metrics or phase out irrelevant metrics as necessary. As IT is an ever changing environment it is inevitable that the metrics being collected and analyzed will also be subject to change.

To summarize, without measurement, management is impossible. The implementation of an IT Balanced Scorecard and Function Point Analysis allows any organization to be in a superior position of knowing where they are on the map. A logical process of metrics implementation allows no reason for organizations to recoil from setting metrics in place so it is baffling to the author why so few organizations have this vital technique engaged worldwide.