Data collection

This section describes how Drill4J implements data collection. There are lots of similarities in implementations, regardless of platform, frameworks or type of tests.

note

It is important to understand concepts laid out here for those, who will perform Drill4J deployment and configuration.

note

This is not an installation guide. For direct instructions please refer to the Getting Started section.

The abstract task

There are are two key points of integration: tests and target application.

Supporting any kind of tests for any application includes following tasks:

1. Tests integration:
   • Extract test context - get test's metadata - things like test name, type, parameters and status.
   • Propagate test context - attach test context to actions performed on target application.
2. Target app integration:
   • Extract code structure - scan which packages, classes, methods and functions comprise the app.
   • Recognize test context - identify test that triggered code execution.
   • Track code execution - record which parts of code are executed in response to each action.

These tasks are implemented in different ways, depending on:

• Language and Platform;
• Testing Framework and Test runner;
• Transport and Clients.

Using Harmony library and Coverlet framework for data collection in .NET

For .NET tests and applications, the Harmony library is utilized to propagate the test context, and the Coverlet framework is employed to extract the code structure and identify the test context. The usage is illustrated in the diagram.

The concrete example

Let's look how these concepts map to some of the real world cases.

API tests (Java + TestNG) for Java HTTP API service

A diagram. On the left there is 'API Tests' entity with 'Autotest Agent' box attached to it. On the right there is 'HTTP API' entity with 'Agent' box attached to it. An arrow labeled 'HTTP requests + TestID added to headers' connects them

1. Autotest Agent
   • Instruments TestNG classes to extract current test context. Each test is assigned with unique TestID
   • Instruments HTTP Client classes to inject TestID to request headers
2. Agent - code structure analysis
   • Hooks into class loading process to extract code structure
   • Instruments classes handling HTTP Requests to recognize TestID header
   • Instruments application classes to track code execution

Both Java Autotest Agent and Java Agent also send data to Drill4J Admin Backend, but that is omited for simplicity.

One thing to note is that the whole Test Context is not added to headers for performance reasons. Instead, Autotest Agent hashes this data to produce TestID:

• TestID + TestContext are sent to Admin Backend;
• TestID is attached to headers.

That allows both to identify which test triggered code execution and to avoid overhead of stuffing too much data into request headers.

Automated UI tests (Java + Selenium)

A diagram: On the left there is 'UI Tests' entity with 'Autotest Agent' box attached to it. In the middle there is 'web browser' entity with 'frontend application' nested inside. They are connected with two arrows: one is labeled 'WebDriver Commands', other is labeled 'Additional commands to inject TestID' On the right there is 'HTTP API' entity with 'Agent' box attached to it. An arrow labeled 'HTTP requests + TestID added to headers' connects browser and HTTP API Service

As you can notice, the picture is pretty similar to the API tests, with a slight difference.

This time, Autotest Agent:

• does everything described earlier;
• but also instruments Selenium classes, to ensure that requests made from Web Browser page are injected with TestID header.

Manual tests in Web Browser

A diagram depicting: - Manual QA Engineer performing actions on Frontend Application in Web Browser. - backend HTTP API service handling requests - additional 'Browser Extension' entity is attached to the web browser

This time tests are executed by a person - Manual QA Engineer. Task of context propagation is fulfiled by browser extension:

• Extension allows user to enter the test name and press button to signal the beggining and the end of manual test execution.
• It generates TestID based on test name and injects it to HTTP request headers.

Summary

Overall integration principles are similar regardless of the platform and stack. Typical integration involves:

• test context propagation;
• app structure analysis;
• code execution tracking.

Out of the box, Drill4J provides ready to use components for Java / JVM applications and Web JavaScript applications.

For the complete list refer to the Supported Technologies section.