GitHub Integration

Overview

This guide explains how to modify GitHub Actions workflow file to enable Change Testing reports in Pull Requests and workflow artifacts.

This is a step-by-step instruction, make sure to fulfill prerequisites and then follow steps for integration

Change Testing report highlights the number of new and modified methods detected by Drill4J, their coverage status, and identifies untested methods that may pose potential Risks.

Report on Pull Requests

1. Detects changes between the source and target branches by comparing the latest commit in the source branch with the merge base commit in the target branch.
2. Displays metrics on how thoroughly changes are tested.
3. Handy for feature/*, fix/* branches and other shorter-lived branches
4. Attached to Pull Requests as comments

GitHub Pull Request Report

Report on branch push

1. Identifies changes in a branch between the latest commit and the configured baseline commit. Generally, the baseline commit is a tagged commit that follows the tagPattern defined in the configuration file.
2. Best used in long-living branches such as main and development
3. Attached to artifacts of workflow triggered on branch push event

GitHub Workflow Report

Prerequisites

1. Familiarity with the GitHub Actions
2. Familiarity with GitHub secrets and variables
3. Drill4J core services deployed.

Steps for integration

1. Configure Drill4J CI/CD Integration plugin
2. Configure build step to send Git metadata to Drill4J
3. Configure deployment step to enable Drill4J Application Agent
4. Configure tests to enable Drill4J Test Agent
5. Push initial data to Drill4J
6. Generate Change Testing report on Pull request
7. Generate Change Testing report on branch push event

Configure Drill4J CI/CD Integration plugin

1. Follow CI/CD Integration Plugin - Adding Plugin section to add plugin to your Maven or Gradle project.

2. Generate API key using Drill4J UI. Save it to GitHub secrets and variables as DRILL_API_KEY

3. Create DRILL_API_URL variable in GitHub secrets and variables. Set its value to Drill4J Admin Backend service API address (e.g. https://drill4j.my-host.foo/api). Remember to add /api at the end

4. Create a personal GitHub access token (classic).

   • Select repo scope - that is necessary to post comments on Pull Requests.
   • Save it to GitHub secrets and variables with the name DRILL_BOT_GITHUB_TOKEN. Reports will be posted in comments from the corresponding user

5. Set the following environment variables to GitHub workflow file env section:

   env:

   DRILL_API_KEY: ${{ secrets.DRILL_API_KEY }}

   DRILL_API_URL: ${{ vars.DRILL_API_URL }}

   DRILL_BOT_GITHUB_TOKEN: ${{ secrets.DRILL_BOT_GITHUB_TOKEN }}

Modify build step to send Git metadata to Drill4J

Modify build step to call send build info command. It submits commit hash, author, and branch associated with build to Drill4J. It is necessary to identify application build in the Drill4J reports and dashboards.

Gradle

jobs:

build:

runs-on: ubuntu-latest

steps:

- name: Checkout

uses: actions/checkout@v4

with:

# Add this only for Pull Requests, see explanations below

ref: ${{ github.head_ref }}

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

- name: Setup Gradle

uses: gradle/actions/setup-gradle@v4

# Your build step

- name: Build app

run: ./gradlew clean build

# Execute command to send data to Drill4J

- name: Send build info to Drill4J

run: ./gradlew drillSendBuildInfo

Maven

jobs:

build:

runs-on: ubuntu-latest

steps:

- name: Checkout

uses: actions/checkout@v4

with:

# Add this only for Pull Requests, see explanations below

ref: ${{ github.head_ref }}

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

# Your build step

- name: Build app

run: mvn clean install

# Execute command to send data to Drill4J

- name: Send build info to Drill4J

run: ./mvn drill:sendBuildInfo

The github.head_ref property instructs checkout@v4 action to check out the last commit of the branch.

By default GitHub Pull Requests create local merge commit from target branch to the source branch. It exists only inside workflow environment and is never pushed to remote. That makes it impossible to map it to the rest of repo Git history.

See GitHub documentation (#1 and #2) and checkout@v4 Readme for more info

🚧 TODO: add link to Metabase dashboards to verify Git metadata is submitted and detailed reports are avaialble 🚧

Deploy the app with Drill4J Application Agent

Option 1 - app is launched with Maven or Gradle

1. Setup Application Agent with using CI/CD Integration Plugin - Application Agent section

2. Add DRILL_COMMIT_SHA env varible following the instruction below

Option 2 - application is launched as a standalone Java process

1. Add DRILL_APPLICATION_AGENT_URL to GitHub secrets and variables.

   Set its value to URL of release's .zip file from https://github.com/Drill4J/java-agent/releases

   For example:

   https://github.com/Drill4J/java-agent/releases/download/v0.9.4/agent-linuxX64-0.9.4.zip

2. Add the following step to download agent files (above the app launch step):

   - name: Download Drill4J Application Agent

   run: |

   # Download

   curl -L ${{ vars.DRILL_APPLICATION_AGENT_URL }} -o agent.zip

   # Prep folders

   mkdir -p ./temp-agent-dir /agent

   # Unzip

   unzip agent.zip -d ./temp-agent-dir

   # Move files to /agent folder

   find temp-agent-dir -mindepth 2 -exec mv -t /agent/ {} +

   - name: Deploy the application

   run: # your command to deploy application

   env:

   JAVA_TOOL_OPTIONS: -agentpath:/agent/libdrill_agent.so # libdrill_agent.so for linux, libdrill_agent.dylib for Mac, drill_agent.dll for Windows

   # Set Application Agent parameters, see link to reference below

   DRILL_APP_ID: # ...

   DRILL_GROUP_ID: # ...

   DRILL_PACKAGE_PREFIXES: # ...

   See Agent Parameters Reference for a full list of parameters

3. Set DRILL_COMMIT_SHA env varible following the instruction below

Add DRILL_COMMIT_SHA to env variables

For Pull Request, set it to the github.event.pull_request.head.sha:

- name: Deploy the application

run: # your command to deploy application

env:

# ...

DRILL_COMMIT_SHA: ${{ github.event.pull_request.head.sha }}

For branch push event, set it to github.sha:

- name: Deploy the application

run: # your command to deploy application

env:

# ...

DRILL_COMMIT_SHA: ${{ github.sha }}

🚧 TODO: add link to Metabase dashboards to verify classes and methods information is available 🚧

Run tests with Drill4J Test Agent

1. Configure Maven or Gradle project to use Test Agent.

   Once configured, Test Agent is launched with tests tasks automatically.

2. Make sure your tests are launched in GitHub Action workflow. Usually it looks something like this:

   - name: Run tests

   # Gradle

   run: ./gradlew test

   # or Maven

   # run: ./mvn test

Push initial data to Drill4J

By now, both the Application Agent and Test Agent should be configured.

As the name suggests, the Change Testing report identifies changes, so the first step is to push data to Drill4J to establish a baseline for future comparisons. Ideally, this should be an application version built from your primary branch. We'll call it main, but the branch name doesn't matter — it could be develop or any other long-living branch.

1. Push to the main branch to trigger the GitHub Action workflow to launch the application.
2. Wait for the workflow to complete.
3. Check Metabase to verify that information about the new application build is available.

Generate report on Pull Request

Next, you need to configure the command to generate reports on Pull Requests. These reports will include metrics for changes introduced only in the specific Pull Request, allowing you to verify that the changes are tested before merging.

1. Add an additional step to your workflow file using either the Gradle or Maven configuration provided.
2. Push the changes and follow the instructions to verify that the reports are being generated.

Gradle

jobs:

# ... all kinds of build and test jobs above ...

# Add this at the very end of workflow file

reportByPR:

if: github.event_name == 'pull_request'

needs: test # ensures that tests are completed _before_ this step starts. Adjust name of test step according to your workflow file # ensures that tests are completed _before_ this step starts. Adjust name of test step according to your workflow file

runs-on: ubuntu-latest

steps:

- name: Checkout

uses: actions/checkout@v4

with:

ref: ${{ github.head_ref }}

fetch-depth: 0 # required to fetch complete Git history. Report won't work correctly otherwise

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

- name: Setup Gradle

uses: gradle/actions/setup-gradle@v4

- name: Send Change Testing report

run: ./gradlew drillGithubPullRequestReport

Maven

jobs:

# ... all kinds of build and test jobs above ...

# Add this at the very end of workflow file

reportByPR:

if: github.event_name == 'pull_request'

needs: test # ensures that tests are completed _before_ this step starts. Adjust name of test step according to your workflow file

runs-on: ubuntu-latest

steps:

- name: Checkout

uses: actions/checkout@v4

with:

ref: ${{ github.head_ref }}

fetch-depth: 0 # required to fetch complete Git history. Report won't work correctly otherwise

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

- name: Send Change Testing report

run: ./mvn drill:githubPullRequestReport

Confirm PR Report is Working

Now it's time to create a new branch from the main branch.

1. Checkout a new branch (e.g., feature/xyz).
2. Make some changes to your application code.
3. Push the branch and open a Pull Request from feature/xyz to main.
4. Wait for the workflow to complete.

The comments section of the Pull Request should include a new comment with the report.

GitHub Pull Request Report

Generate report on Branch push

In addition to reports on Pull Requests, it’s useful to have reports on individual branches. These reports are attached as workflow artifacts. To configure them:

1. Add the baseline section to CI/CD integration plugin configuration section:

   Gradle

   drill {

   groupId = "my-group"

   appId= "my-app"

   // ... and other agent parameters ...

   // new section

   baseline {

   searchStrategy = "SEARCH_BY_TAG"

   // Adjust to match your tag naming scheme

   tagPattern = "v[0-9].[0-9].[0-9]*"

   }

   }

   Maven

   <plugin>
       <groupId>com.epam.drill.integration</groupId>
       <artifactId>drill-maven-plugin</artifactId>
       <version>0.1.6</version>
       <configuration>
           <groupId>my-group</groupId>
           <appId>my-application</appId>
           <!-- ... and other agent parameters ... -->
           <!-- New section -->
           <baseline>
               <searchStrategy>SEARCH_BY_TAG</searchStrategy>
               <!-- Adjust to match your tag naming scheme -->
               <tagPattern>v[0-9].[0-9].[0-9]*</tagPattern>
           </baseline>
       </configuration>
       <executions>
           <execution>
               <goals>
                   <goal>generateChangeTestingReport</goal>
               </goals>
           </execution>
       </executions>
     </plugin>

2. Add additional step to your GitHub workflow file:

Gradle

jobs:

# ... all kinds of build and test jobs above ...

# Add this at the very end of workflow file

reportByBranch:

if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'

needs: test # ensures that tests are completed _before_ this step starts. Adjust name of test step according to your workflow file

runs-on: ubuntu-latest

steps:

- name: Checkout sources

uses: actions/checkout@v4

with:

fetch-depth: 0 # required to fetch complete Git history. Report won't work correctly otherwise

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

- name: Setup Gradle

uses: gradle/actions/setup-gradle@v4

- name: Generate Drill4J Change Testing Report

run: ./gradlew drillGenerateChangeTestingReport

- name: Attach report

uses: actions/upload-artifact@v4

with:

name: change-testing-report

path: ./build/drill-reports/drillReport.md

Maven

jobs:

# ... all kinds of build and test jobs above ...

# Add this at the very end of workflow file

reportByBranch:

if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'

needs: test # ensures that tests are completed _before_ this step starts. Adjust name of test step according to your workflow file

runs-on: ubuntu-latest

steps:

- name: Checkout

uses: actions/checkout@v4

with:

fetch-depth: 0 # required to fetch complete Git history. Report won't work correctly otherwise

- name: Setup Java

uses: actions/setup-java@v4

with:

# adjust for desired Java distribution and version

distribution: 'temurin'

java-version: 17

- name: Generate Drill4J Change Testing Report

run: ./mvn drill:generateChangeTestingReport

- name: Attach report

uses: actions/upload-artifact@v4

with:

name: change-testing-report

path: ./target/drill-reports/drillReport.md

Confirm its working

Push some changes. Once workflow is complete, check the attachments for uploaded report:

GitHub Workflow Report

Examples

🚧 TODO: cleanup example action yml file 🚧

You can see example GitHub Drill4J integration in a demo repository with the following GitHub Action