Measuring Your Progress: Application Portfolio Management

Application Portfolio Management helps decision-makers to match corporate priorities with IT resources– across operations, architecture, and development. In the first post in the series I looked at how an organization can define what those priorities are. In this post I’ll look at how you can measure the portfolio to find where goals aren’t being met.

Defining Questions

To achieve the goals that you have defined you have to ask questions. If you are a CIO and you want to reduce application management costs by 20%, you could start by asking questions like:

• What is the total hardware and infrastructure cost per application?
• What is the total development team cost per application?
• Where in the application portfolio do developers spend most of their time?
• How much effort is required to complete a change by application?
• How much does each outsourcer cost per application?
• How business critical is this application?
• Which platforms are the most expensive to maintain?

The answers to these questions can help determine how well you are reaching your goals – and where you need to do more work. Each of these questions should be drilled-into as executives determine areas of weakness and pass the need onto managers for resolution. This means that more specific questions should be asked at more focused levels in the organization.

Defining Metrics

You have your questions, but what are the answers? In order to spot issues answers should be quantifiable and trendable. For instance, in reply to the question “how business critical is this application?” your metric may be weighted scale from 1 to 10. Metrics must meaningfully answer the question at hand.

Metrics may be in the form of a snapshot where one-off measurements are used to find outliers. Or, more usefully, they can be trended over time to spot creeping issues that can be corrected before they become business critical.

Collecting Data

Once you have your questions and your measurements in place, the next step is to determine what data should be collected. This is the information that will be gathered to answer your questions and locate where goals aren’t being met. Ideally this data should be trended over time to spot service-levels that are eroding and should be corrected before they become significant issues.

Data gathering should err on the side of ease of collection. You do not want to establish a metrics collection regime that costs more time and effort than you can expect to save from improved management. In fact, you may stagger the level of data collection with less granular metrics collection conducted initially and more granular as savings accrue. This approach will be discussed in a later post on “maturity” levels of portfolio management.

For Application Portfolio Management, data typically comes from three sources:

Stakeholder Surveys
To effectively weight business priorities you need opinions from key members of your organization. For instance, you may want to re-architect applications that reach a certain threshold for complexity. But if two have equal levels, which should come first? This is where measurements like “value to the business” and “perceived riskiness” become important.

Typically, these kinds of value metrics are collected by surveying stakeholders in the organization. Be careful to choose an efficient and repeatable approach. Browser-based surveys that can be distributed and collected in an automated fashion are a preferred method.

Related Technologies and Sources
Within IT are numerous data sources that can help answer the questions you’ve posed. For instance, we may try to answer the question of which applications drain the most resources. In that case, frequency of change, bug counts, and time to complete a work item may all be metrics that matter. This data may be instantly accessible via integration with your lifecycle management tools.

Other data sources may be equally important, depending on the question. An HR system can help determine costs and time spent on a given activity. A PPM technology may have insight into project costs. Regardless of the source, it is important that data collected from these sources can be drawn automatically without significant manual effort. This helps ensure that real-time measurements can be presented to end-users.

Application-Specific Measures
The application portfolio itself is a rich source of data points that are useful for decision-making. Details like application (or more granular) size and complexity are important. The challenge, as always, is how to collect these data points. Today’s application analysis tools provide these measurements quite handily out of the box. But often they will focus only on one specific language. Look for coverage across a range of languages to avoid a patchwork of tools.

There are hundreds of industry-standard metrics that can be collected. Cyclomatic complexity, dependency levels, and program volume are just a few. Naturally, you will want to determine which make the most sense for your team -- you don't need hundreds of metrics, just those that answer your questions. Also, be aware that many measures are language-specific and don’t make sense cross-portfolio.

Mixing Metrics
You are collecting metrics to answer the questions that you are asking. So, in some cases you will need metrics only from survey information – in other cases, only from code analysis. In some cases it is important to combine metrics, for instance, dividing “bug counts” by “code complexity” provides a clearer picture of which applications need re-factoring.

Metrics should also be adapted to suit your company’s specific needs. If your goal is to align the application portfolio with corporate security standards, then there may be specific measurements that would track your unique security standards. Again, the metrics you collect should be only those that match the overarching goal-question-metric paradigm that you have defined.


In the next posting I’ll take a look at how goals, questions, and metrics differ by level in the organization.

APM Best Practices

An interesting discussion has kicked off on LinkedIn’s “Application Portfolio Management” Group discussion board. The discussion looks at best practices for running an APM initiative. Because APM directly affects strategic decisions around where IT resources should be applied, the topic is an important one.

There are a few interesting branches to pursue that look at APM best practices. Over the next few weeks, I’ll explore each in some detail:

• Goal: Constant fire-fighting is no way to run a development organization. Especially in today's era of tight budgets and fast change. In this post I'll summarize the goal of APM and set the stage for a discussion of best practices. Read the post.
  
  
• Questions and Metrics: APM data should answer questions that address a specific goal. Say, ‘why is this business process inflexible?’, ‘where can I cut costs?’, or ‘where is my software architecture flawed?’. To answer these different questions requires different combinations and weightings of data (user surveys, application code, or external sources). Sometimes more of one source, sometimes another. Read the post.
  

• Decision-Makers: APM data needs vary based on where you are in the organization. Higher level managers require higher level abstractions, particularly of technical metrics. Also, different types of users will have different data needs. An architect may want technical complexity data, but it may only be meaningful to him if it is filtered by architectural models. Read the post.
  

• Maturity: There are different levels of maturity for decision-making. This maturity directly affects which metrics are accessible in the first place and also indirectly because it determines the kind of business goals that an organization is prepared to address.
  

• What’s next: As a particular initiative moves from “decision” to “action”, different data may be needed. More “bottom-up” data may be necessary to implement the decisions at this stage. Further, different metrics can be monitored to ensure the success of a given development or modernization project as it is executed.

New Release of Modernization Workbench

A new release of the Modernization Workbench has just been launched. The new version 3.1 is "3 in 1". It combines best-in-class functionality from Micro Focus' analysis and application portfolio management capabilities into a single platform.

• Single Platform for Managers: Modernization Workbench provides the only business-centric solution for measuring and managing the application portfolio. Its integrated Enterprise View module delivers browser-based dashboards that help prioritise development projects via metrics like application cost, complexity, value, and risk.
• Single Platform for Assessments: Modernization Workbench provides the richest technical information about your applications. Powerful queries, visualizations, and specialized assessment tools, including for platform migrations, are available.
• Expanded Language Coverage: Modernization Workbench provides deep and now even broader language coverage. This new release expands its coverage of Java, JEE, additional job schedulers, and more.
• Mass Change Activities: Modernization Workbench dramatically accelerates the execution of projects that require numerous changes to application code -for instance, to adhere to regulatory requirements like ICD-10.
• Major Relational Database Support: Modernization Workbench 3.1 adds support for Microsoft SQL Server, ensuring that all three major relational databases are supported (adding to existing support for Oracle RDBMS and IBM DB2).

For more information, please see this release here:

Webinar: Application Understanding

You already know that applications automate your core operations. But do you have adequate control over these systems? The answer is often 'no' because of the sheer size and complexity of your application portfolio.

Find out how you can regain control over the applications that run your business at a webinar on May 27. Register here:

http://www.microfocus.com/promotions/wwwcwwmw0509/default.aspx?page=email

Planning for Application Modernization

eWeek just published a strong piece on how to plan Application Modernization initiatives. In the first paragraph industry expert Tim Pacileo identifies the key for any modernization activity: justify it. CIOs face a multitude of competing interests when determining where to allocate resources. The pressures are more intense now that IT has become so tightly interwoven with very visible business processes. As a result, it is critical that the CIO has the right information available to determine which priorities should be selected and why.

This is the role of Application Portfolio Management. It provides a framework to collect measurements and place them into business context. This allows CIOs to quickly determine which projects make sense to act on based on the strategy of overall organization and not just on the narrower needs of IT. Justifying projects is vastly simplified when the interests of IT are tied to the business.

Interestingly, Pacileo suggests using a chargeback model to ensure that costs are properly associated with the correct IT activity. The best approach for this method is via business-centric portfolio management. By placing application portfolios into business context (by operational unit, geography, etc) CIOs can match costs to the exact software (and IT infrastructure) that is incurring the costs.

Pacielo also points out that in the rush to solve problems, IT often will product multiple overlapping applications that drain resources. A business-centric Application Portfolio Management solution is again the ideal approach. When IT compels itself to match business-value, cost, and risk to IT assets these kinds of glaring issues become readily apparent. He cites the example of one company having three overlapping systems. We've seen cases with more than 2 dozen duplicate systems being maintained separately.

Application Portfolio Management is an increasingly rigorous discipline. Executives should turn to this framework when determining where priorities lie and how to justify them.

Portfolio Management is Top Priority for 2009

This week Baseline published its top IT trends for 2009. Among the usual contenders for top spot (such as virtualization and Software as a Service) was Project and Portfolio Management. The reason cited for its selection was the need to oversee IT activities and allocate resources toward high-priority tasks – which stands to reason in these challenging economic times.

The linkage between Project Portfolio Management (PPM) and Application Portfolio Management is an interesting one. PPM focuses on helping development managers to monitor and control in-flight development projects. Application Portfolio Management concentrates on the allocation of development and related infrastructure resources toward business priorities.

As discussed previously, Application Portfolio Management is at its most effective when measurements are placed into their appropriate business context. This could include managing applications by the business process they automate or by the geography that manages the applications, or other contexts or combinations of contexts. So, we could uncover complexities within a high-value business system managed in India, and decide to allocate resources to correct the issues.

We can think of PPM’s project entities as being another grouping through which we want to manage APM. This isn’t to say that APM should replicate PPM functionality, but rather that the technologies can effectively collaborate by sharing metadata. Much the same way that APM doesn’t replicate BPM functionality simply because it reuses business process models as a way to structure metrics reports.

Let’s look at how this can be deployed. A CIO of a bank is under pressure to improve responsiveness. Using APM, he looks at change request backlogs as organized by business process and spots an outlier in the customer management business process. He decides to investigate further. He looks at metrics for this process and sees that complexity levels are especially troublesome in the portion that is managed in Brazil. He decides to reallocate resources to attack complexity levels and whittle down the change request backlog.

This is clearly when PPM would become involved. Managers can use PPM to monitor deliverables, outcomes, and other elements that are part of a development project. But in parallel, APM continues to play a critical role. We could now overlay a new grouping onto our software; in this case the grouping would be by project. So, we are now tagging the portion of our application portfolio that is encompassed by the newly introduced project. Then, as we proceed we can collect metrics associated with the grouping of our project. This means that we can use APM to monitor the trend of value, cost, and risk of the software that is being enhanced by our project.

We can even start to combine business contexts, for instance by collecting metrics from software within Project A that is managed by Team A versus software managed by Team B. We start to see within APM very interesting metrics about adherence to SLAs as a result.

Best Practices for Application Portfolio Management: Gartner

An excellent piece of research by Jim Duggan came out from Gartner today. It details how companies should approach Application Portfolio Management – and of course, why you should be interested in APM in the first place. The why is clear: as the economy has slowed, companies must uncover and replicate efficiencies while slashing wasteful spending. The oft-quoted figure of 80% of IT budgets being dedicated to ‘lights-on’ activities is a primary reason why Application Portfolio Management has become such a hot topic.

But how should you discover where to focus rationalization and follow-on modernization activities? The paper relates a number of suggestions. The major thrust is that management should assess which portions of the application portfolio to rationalize based on different perspectives. That is, you should determine the ways in which you manage your business, and then rank your applications based on these views.

For instance, we could start with the most obvious perspective: cost. What are your most expensive applications and do you need to maintain these systems? Do they overlap and can be consolidated? Are they not used by the business? Is there a less expensive architecture?

You can then quickly move to other Application Portfolio Management perspectives. It could be by organization. Are applications that are managed by expensive / low-value providers that could be re-assigned? Or the perspective could be by business process. Are there business processes that could be better managed by an external service provider? Are there applications that support defunct business processes? You can see that executing APM from different perspectives allows you to rationalize based on KPIs that matter to your organization.

There is also a significant side advantage that comes from managing by these perspectives as you focus APM and continue to refresh APM. Once perspectives are in place – and rationalization decisions may have been made – you can focus modernization activities on sub-sets of the portfolio that matter to you. High-cost and low-business value areas? High-risk and frequently-changing applications? Now, resources can be applied to the right area. You may decide to deploy richer code analytics at this stage to get a complete picture about how developers should be concentrated.

Further, these perspectives –especially once placed onto the software – offer a ‘filtration mechanism’ for metrics as you collect them for APM. Looking for cost data on a business process, or risk information about applications managed by a particular organization? The perspectives provide the means to get these business answers. The result is highly business-centered development decisions.

Application Portfolio Management Resource Site

This new site exclusively is dedicated to best-practices for Application Portfolio Management is now online. The site explores what kinds of metrics that you should collect, how they should be combined into measures that have business utility, and what are the key attributes of an APM tool.

Application Portfolio Management is a methodology for identifying and prioritizing development activities. The aim is to locate misalignments between the application portfolio and business requirements, and then allowing managers and other IT professionals to adjust their resources accordingly.

APM is fundamentally about collecting business and technical metrics, combining them into useful KPIs in the right business context, and presenting them to decision-makers. On the site you'll find a wealth of relevant resources.