Technology Cockpit – The KPI Dashboard for the CIO

by Manuel Mörtl / September 12th 2018

In our day-to-day business at ProSiebenSat.1, we handle a huge number of different technology KPIs. These indicators span from organizational data to the purely technical ones. To keep the KPIs manageable, we developed a so-called Technology Cockpit, a comprehensive KPI dashboard for all the ProSiebenSat.1 technology-related KPIs.

Driver and main objectives of the Technology Cockpit

The Technology Cockpit is designed to support IT Management as a main stakeholder in steering IT organization within the ProSiebenSat.1 Group.

It serves as single source of information for all relevant technology KPIs:

  • Providing a basis for consistent and reliable reporting;
  • Archiving key performance metrics;
  • Consolidating all relevant information in one place;
  • Is accessible at anytime from anywhere.

It provides a basis for continuous improvement:

  • Enabling trend and root cause analysis;
  • Ensuring the focus of the organization is on most important drivers of performance and health.

The technology KPIs are provided to the Management on a quarterly basis, although some data is collated annually, depending on the dimension.

The Technology Cockpit currently covers six dimensions of the IT organization

The graphic below gives an overview of KPIs that are currently active, with potentially more dimensions to come. Technology Cockpit Overview of dimensions

1. Operations: Covers all operations-related data with the key question “How available are our applications?” Presents availability measurement from a user perspective for all main applications (e.g., TV/digital booking systems, playout center, corporate website, Intranet) on a quarterly basis. Easily accessible applications are essential in ensuring business processes within the company. Having a transparent measurement over time helps to improve the availability of specific applications where necessary and is the basis for continuous improvement. For details on performance measurement of applications please see: The hitchhikers guide to better web performance

Another key area is “How good is our helpdesk availability?” Decisive for a good IT service is a fast-responding and reliable helpdesk. The main KPIs here are number of helpdesk calls, response times, and incident resolution times.

2. Projects: Key question is _“How good is our project delivery performance?” _Projects to change and transform IT are necessary to ensure a future-proof IT landscape. We take all IT-related projects into account and measure them against well-established, market-standard criteria: Time, budget, scope, and customer satisfaction. We assess our projects in 5 categories:

  • more than 20% positive deviation
  • 10% - 20% positive deviation
  • +/- 10% deviation
  • 20% - 10% negative deviation
  • more than 20% negative deviation

Time, budget, and scope are relatively easy to measure since they can be evaluated against predefined values. Customer satisfaction depends on the customer’s subjective perception. Comparability between projects is not necessarily given. We suggest using a customer survey for all project-related stakeholders to evaluate the project’s customer satisfaction in a meaningful way.

3. Organization: The organizational dimension covers all employee-related KPIs such as number of employees, internal and external fluctuation, as well as hiring data (avg. time to hire, number of applications, share of open positions, etc.). Ensuring a good hiring pipeline is a particularly key success factor in healthy IT organization.

4. Security: IT security has gained more and more relevance over the last years due to an increase in cyber security threats. This is why we established a group-wide IT security program to secure our business. For further information on this topic, please also see our blog article on cyber security: 10 actions to increase cyber security

Within the Technology Cockpit, our target is to track some of the initiatives in cyber security, like awareness training, implementation of ISO standards, risk analysis of business processes, and email encryption. We also track our findings from internal audits.

5. Architecture: This dimension gives an overview of all applications, servers, databases, and programming languages currently in use. It shows, for example, which applications are using prohibited or unsupported databases, servers or, programming languages. The key question is “How many applications run on outdated technologies?” On the one hand, it helps to improve those applications and make them future-proof. On the other, it gives an indication of which applications need replacing. In addition, it also shows the degree of automation, including the number of fully automated interfaces and the share of virtualized servers.

6. (Cost-) Efficiency: This finance-oriented dimension covers the cost structure of the IT department. The main aim is to evaluate cost efficiency. Also important is the IT cost evolvement, the investment ratio, and the change/run split, meaning how much do we need to invest to change or transform our organization, compared to simply running it?

The road to the Technology Cockpit

Our first prototype was a simple Excel sheet with five dimensions (Organization, Operations, Projects, Security, Architecture); one dimension per tab. To define the most relevant dimensions and KPIs for our Technology Cockpit, we combined external research (like Gartner, McKinsey) with our internal knowledge. We focused on using market-standard KPIs to ensure comparability and for a common understanding. During several internal discussions, we finalized the KPIs and defined KPI owners. For every single dimension, there are one or more KPI owners in the organization who are responsible for gathering and processing the data. All KPIs are consolidated in one place by the Product Owner of the Technology Cockpit. Technology Cockpit Excel Sheet Excerpt from the Technology Cockpit Excel sheet, Operations dimension.

The next step was to professionalize the way we store data and replace the Excel sheets. So we set up a central database for all KPI data. We chose a MS SQL database because it works perfectly with Tableau and fits our purpose. Tableau was selected due to its breadth of use, both within our company and externally, its rich functionality, and its sheer power. The database structure is a critical part of the overall project and needed careful consideration as any changes made would require quite intense effort. We chose the concept of flat table for the data and defined tables for dimension, category, business unit, measurement, unit, KPI definition, and KPI entry for all KPI database entries. Using simple SQL statements, we inserted all data into the database.

Technology Cockpit database structure

Data visualization takes place in Tableau. Within the Tableau Desktop version, we defined the dashboards: One dashboard for every dimension and subsites for each specific KPI. After the dashboards were created, they were published to the Tableau Server, where every defined stakeholder within the organization can access the data. A proper user and rights management for every single dashboard is supported by Tableau.

To avoid shooting SQL scripts directly into the database and to reduce the risk of corrupting data, we developed a web GUI to easily insert new records into the MS SQL database or update existing ones. The web GUI can easily be accessed by the relevant KPI owners to insert or update their datasets. For implementing automated interfaces, we always consider economic aspects like costs and resources for implementation. If the update frequency is low and requires little effort, then it’s not worth automating the data import but rather sticking with a manual process.

The following image shows the current setup of the Technology Cockpit. Technology Cockpit Status overview

Next steps

The next steps for the Technology Cockpit are to:

  • Release the Tableau Server into a production environment with Active Directory connection, so that users can easily access the KPIs and to ensure an appropriate user rights management;
  • Potentially add further dimensions with new KPIs that support the IT Management in the better management of their organization;
  • Foster an automated database import of KPI data wherever it’s possible and economically viable.