Testing Maturity & Improvement

Testing is an important and significant part of the product or service lifecycle. This is true of any industry but more so in the case of IT where the sheer complexity of the trillions of bits and bytes zipping around bring about an incredible permutation and combination of possible ways things could go wrong. To counter for this complexity the implementation of a high level of testing maturity is essential within the IT organization.

For overall organizational maturity, organizations can avail of CMMI and its 5 maturity levels. Testing also has 5 levels of maturity within the Testing Maturity Model (TMM) that integrates well with CMMI and other methodologies. Furthermore, the Test Process Improvement model also exists that has been developed based on practical experiences and knowledge of test process development.

TMM was developed in 1996 at the Illinois Institute of Technology, and was designed to be a counterpart to the CMMI model. The 5 maturity levels are similar in definition to CMMIs levels which can be easily viewed online. TMM advocates the implementation of various test processes that increase testing maturity within the organization.

Similarly, TPI offers 20 Key areas with increasing levels of implementation for each area. Some of the key areas (not all) include:

• Test Strategy


• Moment of Involvement


• Estimating and Time Planning


• Metrics


• Test Tools


• Evaluation


• Communication


• Reporting

As may be deduced, this is a far more structured approach than the old fashioned write a few test cases at the last minute and frantically test till midnight strategy that some organizations are utilizing to this day. Moving beyond simply preparing for testing by creating test cases and test plans is simply not enough. It is now imperative to be optimizing the test processes and continuously improving to be at the right combination of efficiency and quality.

Simply put, organizations should make themselves aware of the latest in testing techniques and methodologies like TMM and TPI and implement the recommended processes before the competition does. Not doing so will only put the organization at an unnecessary disadvantage that is a great handicap in today’s difficult times.

The Rain in Spain

When Dr. Higgins attempts to improve Eliza Doolittle’s speech in My Fair Lady, he starts with the basics: practicing speaking with marbles in her mouth, repeating basic sounds and words, the most famous being “the rain in Spain is mainly in the plain”. The parallels with an organization seeking to improve its processes are similar in that the basics must be mastered first before one can be the belle of the embassy ball.

What are some of the basics that an organization can put into place while attempting to improve? Some choices are:

Strategy: Easily the most neglected area of organizations worldwide and IT organizations in particular, certain basic techniques of Strategy should be implemented. While full blown strategy methodologies might be a bit much for the beginning effort towards improvement, fundamental techniques of demand analysis, financial management and portfolio management should be implemented.

Customer Point of Contact for Negotiation: While organizations do have this in place in some fashion, it is rarely enacted formally enough to bring its true value and benefits to the table. ITIL’s Service Level Management process is a well defined methodology for achieving this objective. The ability to not merely interact and form a point of contact with the customer but to build a relationship and understand their needs allows for superior alignment of IT with customer’s requirements. This effort returns rich rewards and is definitely much value for money.

Change & Configuration Management: Again implemented by most organizations but not adequately. A good first step for organizations committed to improvement would be to evaluate what they have in place and tighten up and further align with what users require. At an organization that I consulted for in the past, they had a home grown Change/Configuration management tool that had fields and options that users did not need or use and did not have needed fields and options. Clearly they could have benefitted immensely with a properly thought out tool that fitted with their needs better.

Service Desk, Incident and Problem Management: Another set of those processes that most organizations do have in place but could desperately use an overhaul and update of. Common service desk shortcomings are lack of current information made available to service desk personnel, increasing call volumes and increasing and more complex changes to the service. Incident and Problem management also suffer from lack of communication from change and configuration management typically.

Continuous improvement: While there may not exist an organizational maturity to reach six sigma levels at the present, certain basic improvement techniques can certainly be implemented. A basic technique of Root Cause Analysis and resolution to prevent similar mishaps occurring in the future is easy and requires minimal investment. Therefore, there is no reason to not implement a RCA system of continuous improvement, no matter how limited resources are available in the organization.

It is often argued that times are too challenging or resources not available to implement process improvements by those not enthusiastic about improvements. However, there are small and simple steps that can be carried out that yield rich returns for the effort expended. It is possible to get started without a great deal of investment and disruption. With the improvement and stability gained with these initial steps, further and more complex process improvement endeavors can then be undertaken. Even if an organization is dedicated to a large scale process improvement effort, the basics must first be completed successfully. Remember, the rain in Spain is mainly in the plain.

Continuity

Service continuity is now an expected feature in any organization’s portfolio whether IT or non-IT. In the past, customers were sympathetic and understanding regarding disaster events that unexpectedly disrupted services. However, nowadays, organizations are expected to have accounted and planned for possible disaster events and to prepare and execute continuity plans in the event of the disaster actually occurring. Finally, after the dust clears, the operations should be brought back to a normal state.

IT organizations are expected to manage service continuity and this is generally included in the Service Level Agreements when the services are being negotiated and agreed upon with the customer. An IT Service Continuity Process with a Service Continuity Manager as the process owner should be established to carry out this activity on an ongoing basis. The process should then create a set of IT Service Continuity Plans that support the overall business continuity plans of the organization. The plans should identify possible disaster events and the contingency and continuity activities that should occur if the disaster does strike. Furthermore, the plans should include a description of how a return to normal service operation should occur after the disaster is over and the contingency plan is no longer necessary.

After the creation of the continuity plans, regular Business Impact Analysis (BIA) activities should be carried out to ensure that all the plans are in sync with changes that have been made to the service and organization.

Other activities of the Service Continuity Process include assisting the Change Management in assessing changes for any possible impact to service continuity and working with suppliers and the Supply Management Process to ensure supplies are made during a disaster event.

Of course, during the occurrence of the disaster event, the IT SCM process comes into the forefront and initiates the contingency plan in order to continue service delivery to the customer. Service Continuity monitors the situation until the disaster event subsides and then presides over the transition back to normal operations. To conclude, the process records the success of the continuity event and makes notes for future improvement.

Disaster recovery and service continuity are no longer a luxury but a necessity in today’s market. Organizations must take service continuity seriously in order to maintain customers in the competitive environment we live in now.

Security

In the good old days, security meant a guard with a gun or a well trained Doberman that refused food from strangers. Now, we have hacking, phishing, identity theft, viruses, spyware, adware and a host of other malicious attack techniques. Over and above this, an aspect of security that is generally not considered as deeply, there exists the possibility of problems and issues occurring simply due to non-intentional, non-malicious errors. An example of this might be that due to a bug in the code, sensitive client’s information is available to view by everybody. This wasn’t a deliberate move on the programmer’s part but simply an error in the code. However, the net result was a compromise in the security level of the application.

The solution to security issues is, of course, a well defined and implemented security management process. The cornerstone of the security management process is the overall security policy for the organization. The Service Level Agreements of each service should also include security requirements that can then be individually addressed.

Security activities can be divided into the following steps:

• Planning


• Implementing


• Evaluating


• Maintaining


• Reporting


• Controlling

Security activities can also be broken down into the following types:

• Preventative – such as firewalls, login requirements, ID cards etc.


• Reductive – backups and testing etc.


• Detection – Antivirus and antispyware software, network intrusion monitoring etc.


• Repression – Blocked login after 3 failed login attempts, card retention after failed pin entry etc.


• Correction – restoring backups, removing viruses that have entered the system etc.

Therefore, it is clear that a lot of thought and work must be devoted to security in order to maintain the security requirements that are considered part and parcel of any product or service nowadays. Security must be a consideration right from the very beginning when a service is being conceived at the strategy stage and should be designed into the service. Too often, very superficial security considerations are made in the beginning which results in inadequate security of the final product. Organizations must now consider security as important and significant as any other aspect of their organization’s functioning.

Taking Stock

In my experience, most organizations do not have a good understanding of their capabilities. I do not mean that they have not taken a good inventory of what they possess. Sure, they probably have a list of how many laptops and desktops are scattered around the office and the number of employees pounding the keyboards. They know how many licenses of Windows and Office are out there and the number of desks and chairs. The problem is that they do not have a good understanding of their organization’s capabilities; what the organization can achieve in how much time and more importantly what it cannot achieve.

The definition of an asset in the context of IT is a combination of resources and capabilities. Resources are defined as direct inputs for production and some examples are financial capital, applications, infrastructure and people. Capabilities represent an organization’s capacity, competency and capability for action. Some examples of capabilities are management, knowledge and processes. Generally organizations maintain a good checklist of their physical resources but have a poor idea and understanding of the less tangible capabilities that they possess. This lack of understanding makes management more challenging and in particular, makes improvements difficult to implement. After all, how can you improve that which you don’t understand in the first place?

Improvement is by no means the only aspect that suffers when an organization does not have a good understanding of itself. The ability for IT to align itself with business and to support business processes also suffers. So does agility and the ability to make quick changes which is crucial in today’s world. Financial estimating is also highly inaccurate when the capabilities of an organization are not understood completely.

Therefore, it is clear that an organization must understand its capabilities completely and move beyond just an inventory stock keeping of its resources. How does an organization go about understanding its capabilities properly? The first step, of course, is to keep a good stock of the organization’s resources as they are the building blocks of capability. A well setup Configuration Management system is crucial in achieving the ability to keep tabs on the resource items. The configuration Management System should also maintain relationships between the items that allow for an understanding of how a change in one item will affect another item or a system.

Next, a reliable process of documentation must be setup and maintained. Arm in arm with the documentation process, a system of collecting and analyzing metrics must be created and maintained as well. Metrics must be carefully collected and archived for future reference.

Finally a system of modeling should be setup that utilizes all the aforementioned data to provide a realistic estimate of the organization’s capabilities. The modeling should be set up to predict finances and costs, schedules, technical complexities and project and service deliverables.

With all this setup and relevant information available, management can make crucial decisions with confidence. Furthermore, the organization will make accurate estimates and will be extremely agile and better aligned to customer’s requirements. Simply by understanding one’s own self.

Booms and Busts

The current financial conditions are unlike any previously seen in the history of the world. A perusal of the last 15 years indicates that cyclical patterns of high growth and frenzied activity alternate with periods of decline and layoffs. This pattern does not seem to be abating in the near future and it seems that booms and busts will be part of a way of life for all of us for the next 15 years as well, whether we like it or not. This situation only reinforces the need for both individuals and organizations to position themselves strategically for the turbulent future advancing upon us. While it may not be possible to exactly predict the timings and nature of the booms and busts, certain basic steps can be taken to ensure a smoother ride.

Firstly, for individuals, training and certifications in their chosen area of expertise should be undertaken in order to separate themselves from the herd. Continuous learning and self-improvement are no longer the activities of a few “nerds” but a necessary part of survival for everyone nowadays. Individuals must also keep up with the latest in industry innovations and stay aware of the latest tools, techniques, methodologies and standards. Those who have kept themselves at the cutting edge will be in a superior position for advancement as companies scramble to make themselves more efficient and competitive.

Which brings us to organizations and what they can do during financial swings from a process standpoint. During the boom periods, companies have a tendency to focus entirely on taking advantage of the business available and not caring much about the way the growth and the new business is being handled. This, then, translates to a skewed and mismanaged growth that is inefficient and costly. Furthermore, the profit generated during good times is rarely saved and kept aside for the rainy day. Companies, like individuals, must save and set aside revenue for use during lean times. The tendency to operate only for the quarterly result is not a good strategy for the long term and senior management and the board should understand and support this way of doing business.

During the busts, the companies should then call upon the revenue saved from the good times and instead of laying off people, put them to work in making improvements and efficiencies for the future. A lean period is a good time for a company or organization to become CMMI certified or ISO certified utilizing staff that are freed up due to diminished business. That way, when the good times roll in again, the company is now more efficient and better positioned to take advantage of the new business.

Granted that this is very theoretical and a bit on the Pollyanna, “in a perfect world” perspective, but what are the alternatives? Haphazardly growing frantically during the boom period and then laying of people and losing market share during the bust? Clearly, both individuals and organizations must plan for the cyclical market conditions that are now a way of life in the most intelligent manner possible. Assuming that things will go smooth and steady in the future is hazardous and foolhardy at best.

The Need for Strategy

In a study, it was determined that the area of strategy within IT organizations and for that matter even non-IT organizations) is the most undeveloped and under-utilized with the greatest scope of improvement and realizing benefits. I have certainly found this to be true in my own career and dealings with various organizations.

The word strategy instantly brings to mind the concept of long-term planning. A highly reactive response to solving a customer’s immediate problem as quickly as possible is not a strategic activity. However, deciding what new products and service to introduce 3 years down the line is an example of strategic activity. What I have noticed too often in the past is that organizations get into a constant state of firefighting and reactive problem solving which results in adequate strategy never being realized. It is up to management to ensure that sufficient resources are dedicated to strategic activities and kept free of the day to day firefighting tasks.

Strategy is important because it provides the initial roadmap or path to the organizations long term goals and objectives. A wrong decision taken in the initial plan can have disastrous consequences in the long term. Furthermore, possible risks and downturns need to be evaluated and accounted for in the future planning. Over and above all this, the strategy team should evaluate the current products and services and the customer’s happiness with respect to them and make course corrections based on this as necessary. Therefore, it is apparent that the strategy step is crucially important and should not be neglected.

So now that we are convinced of the importance of strategy, how do we go about strategizing? The different areas of strategy, in my opinion, can be broken down to three main components. Understanding of your organization (which includes current products and services, resources and capabilities etc.), understanding the customer (demand patterns, market conditions etc.) and financial information (including Budgeting, Accounting and Charging). These are found in the ITIL body of knowledge as the Portfolio Management, Demand Management and financial Management processes within the Service Strategy Module.

Therefore, with the information needed to adopt strategy for IT services being readily available, there is really no excuse for the implementation of poor strategy. All the greatest generals in history, considered strategy the most important part of their military campaign, beyond even the number and strength of their armies and the technological sophistication of the weapons being used. Indeed, Napoleon Bonaparte won numerous battles simply because of his superior strategic planning. In the battlefield of business, the implementation of correct strategy will ensure economic victory.