Measuring The Success Of Your Outsourcing

"If you can't measure it, you can't manage it." - Peter Drucker

Many people fear that outsourced software development means having little or no control over the development process. They think there is no need to measure while the programmers are there in the same room. Or is there?

When I worked as a programmer in the 1980s, my boss used to joke that he was going to hire a guy with a kettle drum and put him in the corner of the room. Every time the kettle drum was hit, we had to have written a line of code!

Today, outsourcing promises huge cost savings and executives are less concerned with lines per minute than with dollars per hour. But in the end, it is important to know the money you spend is fueling real progress in the development of your software. How can this be done?

Business Process Outsourcing (BPO) provides an example of outsourcing that can be successfully measured. Business processes such as accounts receivable and outbound sales calls can be so well defined that you can accurately measure how efficiently and effectively they are implemented. New software tools not only help you detect problems and inefficiencies, but can predict and fix the problems before they even arise.

To measure new software development you track how many new features are added over time. Some metrics split the programming required into work units and then track how many units are completed over time. It is best to measure results daily and at least weekly.

Engineers are notoriously optimistic about their ability to create working software. So another metric measures how accurate their estimates are for the time required to finish the software development. Initially, their ability to estimate will likely be poor. You can set a goal for the engineers to improve this skill as your development continues so you improve the predictability of your process.

For maintenance programming you need to track the work units or bugs fixed over time. In addition, you should measure the amount of re-work required for bugs that fail the QA step after bug fix attempts.

Your outsourced team should commit to a schedule for completing the programming work. As part of this their commitment, they must also agree to the definition of work units and the productivity level they believe they can achieve. Their commitment makes them independent and liberated from requiring specific instructions for all their daily work activities.

You typically measure the throughput of your outsourced team as a whole. A team is typically a combination of junior and senior members. Junior engineers will need guidance and mentoring from the senior engineers. This is normal and should be expected and encouraged. But it should also be measured over time. A senior engineer can be expected to spend from 5% to 25% of his/her time with junior engineers depending on the complexity of the project and prior experience of the junior engineer.

Today most people use simple software tools like spreadsheets and Microsoft Project to track the metrics of their outsourcing. More sophisticated tools are also available but are expensive and best applied when you have a large portfolio of software development projects. New tools are being developed to automatically compute your metrics as your software is developed.

For example the amount of time source files are checked out of your source code control system can be used to help measure the productivity of your engineers. Other on-line techniques to track progress and measure results will be coming soon.

You can use metrics as the basis for a Service Level Agreement (SLA) you’re your outsourcing vendor. But remember: the purpose of an SLA is to help guide your software development to success and to detect and correct problems as they arise. It is not to support micro management, a blame game or to create an adversarial relationship with your outsourced team.

Will software development become as predictable as BPO and enable you to fix problems before they occur? I doubt we will ever have this much control over the creative software development process... but who knows? That guy with the kettle drum may not be far off!