How to use Tableau to analyze agile, devops, and website metrics

Developers use many productivity, coding, testing, and cloud-management tools as part of designing, developing, testing, deploying, and managing applications. Although most tools offer reporting and analysis capabilities, technology teams might have good reason to develop their own reporting capabilities to better analyze, predict, and manage aspects of their work.

Tableau is a popular and useful tool for delivering on those reporting needs.

Reasons to add a tool like Tableau to your portfolio might include:

• Analyzing, trending, predicting and reporting off more dimensions or aggregate metrics, or with different data visualizations than what out-of-the-box reporting your current tool provides.
• Cleansing data before reporting and analyzing.
• Integrating data from multiple tools to provide more comprehensive reporting.
• Centralizing reporting tools, standardizing data visualizations, and documenting data definitions to make it easier for technologists and their managers to use the analytics in decision making.

The good news is more technology tools today provide mechanisms to interface with the underlying data, and data-visualization tools and other analytics tools have methods to integrate different data sources and formats. Larger development organizations might find that the benefits of developing and maintaining analytics reporting capabilities outweigh the costs especially when using self-service business intelligence tools.

In this article are three examples of using Tableau to centralize dashboards and analytics from the tools used by technology organizations. While the examples were done using Tableau, they can be reproduced in other self-service BI tools that offer comparable integration options.

Example 1: Analyzing agile productivity and quality metrics

Teams practicing agile development often select a management tool such as Atlassian Jira, Microsoft DevOps, VersionOne, or to manage projects, organize teams, capture the user story backlog, and track sprints, epics, and releases. These tools typically have standard reports such as burndown reports showing work committed versus completed for a sprint, release or epic. Other typical reports include team velocity reports, control reports showing lead and cycle times, and time-tracking reports.

Although the reports built into agile tools are useful, they may be difficult to use in larger organizations. For example:

• Burndown and velocity reports may be more useful for forecasting when they use dimensions around development skills, types of work (such as new feature, defect, usability improvement, or technical debt), or technical components (such as UI, database, or data integration).
• Development teams might want to use reporting metrics on defects, production issues, and technical debt to diagnose root causes and implement process improvements or standards.
• Agile product owners may want to measure the business value or cost benefit of the features implemented by joining cost metrics on features from the agile tool with benefit metrics coming from other data sources.
• Larger organizations may have program management offices that are required to do longer-term forecasting around deliverables, resources, costs, and quality.

Many of the agile projects that I work on use Jira. To connect Tableau to Jira data,  I’ve used AIO Tableau Data Connector for Jira, which works well for analyzing Jira data as a single source and is relatively easy to connect, configure to the data objects, and extract the required data columns. If Jira data needs to be joined to other data sources, a second option is using an ETL (extract, transform, load) or integration tool such as StitchData, XPlenty,Tray.io, or Boomi to move Jira data to a database.

Once the data is accessible, there are multiple worksheets worth implementing. First, I like to create worksheets to enable drilling into key dimensions such as project, development team, development skills, types of work, and technical components. From there, I often create a burndown dashboard that enables switching between sprint, epic, and release scope with the key dimensional worksheets. Separate dashboards can be used to investigate agile velocity, defects, or forecast timelines.

Example 2: Reviewing AWS cloud spend

A second devops use case where additional analytics offers benefits is for researching cloud usage and related costs.

For example, while there are built in AWS cost and usage reports that administrators and devops engineers can use, it is often important to perform additional analytics or make the data accessible to a large number of users. For example:

• Larger enterprises may be in multiple clouds and charge back business units for utilization. These organizations may want to centralize cost data from their cloud environments and provide dashboards to business teams on costs.
• Developers may want to review usage reports to better understand how their architecture and implementation decisions might affect costs. This can be particularly useful in early stages of application development where cost information can be used to steer implementation decisions.
• Data scientists loading data sets and experimenting with machine learning algorithms should be exposed to the usage and costs of their experiments.
• Devops engineers should monitor cloud asset utilization and proactively optimize configuration parameters based on utilization.

Tableau recently published a recipe to connect AWS cost and usage reports to Tableau. It uses Amazon Athena to query cost data extracted on an hourly or daily schedule to Amazon S3. Once the connector is enabled, the recipe includes a sample Tableau workbook with dashboards to review monthly spend and information on Amazon EC2 assets.

These are just starting reports; organizations using AWS should consider developing dashboards suited to their architecture and user needs. Some of this can be done by creating user filters by usage type, so for example, data scientists might see costs associated with databases and machine learning. Other dashboards can help benchmark costs among applications and even among multiple cloud providers.

Example 3: Analyzing web and mobile site performance

A third potential area for using an analytics tool like Tableau is to better understand web and mobile user experiences.

Most companies with customer-facing web and mobile sites configure an analytics tool to capture user behavior. Common platforms such as Google Analytics, Adobe Analytics, and Heap come with built-in analytics, reporting capabilities, and integration methods.

Technologists can certainly use these tools to get basic metrics. For example, understanding page-view response times, peak usage periods, top devices and browsers, and top geographies are all standard reports in most web-analytics tools. But the more technologists want to segment patterns based on user types, access locations, and other dimensions, the more likely using tools like Tableau to drive the analytics can be beneficial.

In additions, engineers should consider integrating web analytics with other data sources. For example, integrating web analytics with system or cloud performance metrics can help correlate spikes in poor application performance with system-level information and application log data. Reviewing access patterns with system utilization metrics can also help optimize cloud and infrastructure resources.

Most important, getting more granular information on user behaviors is also important in determining where and how to improve the user experience.

Tableau has an out-of-the-box connector to Google Analytics that makes integration with this platform easy for users who want to develop dashboards from this one source. Users looking to join Google Analytics with other data sources, or users connecting to other web analytics platforms, may need other integration options. StitchData and Xplenty both support automated data synchronizations from Google Analytics and Heap to databases, and there are several options to integrate with Adobe Analytics.

Isaac Sacolick/StarCIO

Once Tableau is connected to the web analytics data and joined with other data sources, consider using Tableau’s data source publishing capabilities to share the data model with other users. Web analytics data is very important to business and marketing departments, so enabling easier access should help drive better decision making.

Example 4: Becoming a data-driven technology organization

Centralizing access to analytics and data visualizations is a first step to becoming data-driven. With data accessible, integrated, and shared, people in the technology organization should start to use it to drive better decisions. For example:

• What parts of an application should have its technical debt addressed at a higher priority? Application-usage metrics should be one factor to help prioritize focus areas.
• What application stacks are not cost-optimized? Consider using multiple applications to benchmark costs, find outliers, and research optimization options.
• How should development teams improve the consistency of delivery? Review user stories that have long cycle times or were the source of defects, seek patterns, and discuss process improvements.

Many technology departments facing increased demand to deliver more frequent application releases and improve both reliability and performance. Asking questions and using analytics to investigate options provides a data-driven approach to driving improvements.