Metabase Dashboards

Drill4J uses Metabase to display application metrics.

This guide will walk you through Metabase dashboards Drill4J provides, without going in deeper details on how metrics are calculated. For that refer to the Metrics overview page.

Prerequisites

1. Drill4J services deployed.
2. Application agent configured.
3. At least two different version of application under test launched (to enable difference-based metrics).

In case you don't have your application configured with Drill4J yet:

the easiest way to follow this guide is to deploy Realworld Demo, and execute steps up to tests launch for the build1 (including the tests launch).

How to use

Login and Home pages

1. Open Metabase UI in browser (http://localhost:8095 by default in case of local deployment)

   You should be greeted with authentication page.

   

   Metabase authentication form

   Enter the following credentials:

   user@user.user

   useruser1

2. Once logged in, you should see the home page. On the left side there is a sidebar.

   

   Metabase home page. "Builds" dashboard highlighted in green

   In Bookmarks section click on the link to Builds dashboard.

   Builds dashboard is the entrypoint to navigate to other dashboards.

   Otherwise, if you open any other dashboard directly, you won't have filter values filled in - so you won't see metrics.

Builds dashboard

3. Once on Builds dashboard page, you should see the table with all application builds Drill4J has information on.

   

   "Builds" dashboard. Link to the build metrics page is highlighted in green

   By default:

   • builds are sorted from latest to oldest (by date of addition to Drill4J database)
   • all builds are displayed, regardless of group and app. To filter click on group/app name or set filter value in page header.

   Click on Build ID cell value to navigate to Build Summary metrics page. For the goals of this guide click on the older application version (realworld:backend:0.1.0 in case of realworld demo).

Build Summary dashboard

4. The Build Summary dashboard contains various information and metrics regarding the particular build.

   

   "Build Summary" dashboard header

   There is a lot to take in. Lets go through the basics (please refer to screenshot for highlights):

   1. Filters area (highlighted area №1) - these filters define for which build metrics are displayed.

      Package Filter and Class Filter are set to *. This value sets "display all" behavior. Empty values will result in error and metrics won't be displayed.

   2. Build information (highlighted area №2) - this card display information for the build we are viewing metrics for right now. If there were integration with CI/CD, we would'be seen Git metadata, such as branch, commit, date message. That is purely informational and does not affect metrics.

   3. Tests information (highlighted area №3) - these numbers display how many test executions Drill4J has detected.

      Executed Tests value refers to overall number of launched tests

      Unique Tests value indicates non-repeating tests (so if you launch same tests multiple times these values will differ.)

      You can click on the numbers to open Build - Tests dashboard with more details. This dashboard is somewhat self-explainatory so we'll skip it in this guide.

   Some components display "There was a problem displaying this chart." message. This is not an error. These components depend on Baseline Build filter value. We'll go through that later in sections below.

Aggregate and Isolated metrics

5. Scrolling down on Build Summary dashboard reveals two more sections - Aggregate Metrics and Isolated Metrics

   

   "Build Summary" dashboard - Aggregate and Isolated Metrics

   Aggregate Metrics (highlighted area №1) - are metrics derived from combined data from various application versions. For instance, if you tested some code in other version and that code remains unchanged in this version, the coverage from that test will be included in these metrics.

   Isolated Metrics (highlighted area №2) - are metrics calculated from data collected exclusively from a single, specific application version.

6. Lets go trough the specific metrics (these are available both in Isolated and Aggregate variants)

   

   "Build Summary" dashboard - Isolated Metrics

   1. Testing Pyramid Chart (highlighted area №1) - this chart shows the coverage grouped by tests metadata.

      In a fully integrated Drill4J setup, encompassing all testing stages, it will show percentages from unit, integration, API, end-to-end (e2e), and manual tests.

      TOTAL category indicates the overall coverage.

      UNGROUPED category refers to coverage without test metadata.

      Other categories are formed based on tests metadata associated with coverage. In order for these to work you have to integrate Test Agent.

   2. Code Coverage (highlighted area №2) - shows overall coverage percent for the whole application.

      Coverage percentage values are based on per-probe coverage, which is more granular than per-line coverage. For example, if multiple statements are on a single line, this metric reflects coverage for each statement individually.

   3. Methods (highlighted area №3) - this metric calculates coverage percentage based on the number of methods.

      A method is considered:

      “fully tested” at 100%

      “not tested” at 0%

      “partially tested” if it falls between 0% and 100%

   

   Metrics - Clickable piecharts

   Click on either the Code Coverage or Methods charts (highlighted in green in the image above) to navigate to the Build - Code Coverage dashboard.

Code Coverage dashboard

This dashboard displays detailed coverage in 3 tables - Packages Table, Classes Table and Methods Table

7. Packages Table displays coverage grouped by packages.

   

   Code Coverage dashboard - packages table

   • Packages column contains name of the package.
   • Probes column displays package size in probes. You can think of probe as the smallest unit of coverage possible.
   • Covered probes column displays number of probes covered in tests.
   • Coverage column displays % of covered probes to overall number of probes.

   Packages are sorted by coverage and size (in probes). This means:

   • Bigger and less covered packages will appear at the top
   • Smaller and more covered packages will appear at the bottom

   This way you'll immediately see least tested application areas.

   Clickin on package name sets the Package filter, affecting tables below.

8. Classes Table displays coverage grouped by classes

   

   Code Coverage dashboard - classes table

   It is very similar to packages table, so we'll skip repetitive explanations.

   Clikcing on the classname sets Class filter, enabling you to see coverage for class methods in the table below

9. Methods Table displays coverage for methods of particular class.

   

   Code Coverage dashboard - methods table

   It won't display data if Package and Class filters aren't set. You have to set both in order for methods table to work.

In order to reset filters click "x" on the respective filter and then enter * (asterisk) value

Set Baseline Build

Metrics described above are, in essense, different ways to display Coverage.

Now lets see other two key metrics: New Or Changed Methods and Recommended Tests.

But before we can do that, we have to set Baseline Build

10. First, lets open dashboard with metrics for more recent application version.

Following the Realworld Demo:

1. Open Builds page (bookmarked on the left sidebar)
2. Click on Build ID realworld:backend:0.2.0
   

   Click on Build ID for realworld:backend:0.2.0

11. Now, set Baseline Build page filter value.

    Since we're vieing more recent application version, it has some code changes compared to the previous version. Set Baseline Build to compare two versions and show the difference.

    In a typical development workflow, it’s best to set the Baseline Build to the most tested application version, which is usually the previous release.

    

    Two ways to set Baseline Build filter value

    Click on Baseline Build filter dropdown and find earlier version (highlight №1, seach by 0.1.0) or select it from the table with other builds (highlight №2)

    Now you should see additional metrics - Recommended Tests and Changed Or New Methods (in both Aggregate and Isolated metrics)

    In case of Realworld Demo: note that Aggregate and Isolated metrics are now different.

    Drill4J has selected coverage from 0.1.0 applicable to 0.2.0 and incorporated it in the report.

Recommended Tests

Baseline set - Recommended Tests table appears

12. Once the Baseline Build filter is set, three new components become available:

    • Baseline Build Info card (highlight №1) - displays metadata of the Baseline Build.
    • Recommended tests table (highlight №2) - contains tests that Drill4J suggests you execute to test changes between two application versions.
    • Changed Or New Methods - shows the comparison results between the Build and the Baseline Build.

Changed Or New Methods

13. Scrolling down reveals Changed Or New Methods card both in Aggregate and Isolated metrics

    

    Changed or new methods metric

    This metrics (highlight №1 in the screenshow above) indicates how many changes in methods Drill4J has detected. Similarly to Methods chart, it groups methods into “fully tested” (100% coverage), “partially tested”(some coverage) and “not tested” (0% coverage)

    Clicking on piechart opens dedicated page, where you can see these methods.

    

    Dedicated page for new or changed methods

    Methods are sorted by both size and coverage, so you will see the biggest gaps in coverage at the top of the page.

    This chart and corresponding page allows you to quickly understand whether you tested all changes between two application versions, or still have some work to do.

    

    Tabs to switch between Aggregate and Isolated reports

    Note that this report has Aggregate and Isolated variants as well (highlighted on the screenshot above). Aggregate report is very handy, whenever your testing efforts are spread across multitude of versions. It most often happens in case with Manual tests, when its unlikely that all manual testing will happen on the same version.

Conclusion and further reading

This covers basics of using Metabase dashboards provided with Drill4J out of the box.

Automating Reports

Although this paged walked you through manually setting filters to get to necessary reports, this process could be automated. Refer to guides below to learn how to link reports to your Pull/Merge requests and branches updates:

1. GitHub Integration Guide
2. GitLab Integration Guide
3. CI/CD Plugin Reference