Skip to main content

Glossary

This section establishes meaning for both general terms used in this documentation and terms specific to Drill4J.

note

1.a / (forward slash) character in term's definition means that terms are used interchangeably.

2.a word emphasized in Italic implies this is a term defined in this glossary. Click to navigate to definition. If not available, try manual search.

General terms#

This section describes meaning behind general terms. The goal is to avoid ambiguous wording. These definitions do not claim absolute correctness outside of this documentation.

  • System Under Test / SUT - the complete system which is being tested. It may be comprised of multiple components, or a single component.

  • Service / Application - a distinct part of the SUT. Examples: a monolithic backend service, a microservice, a web frontend.

  • Automated tests - are any kind of automated tests.

  • API tests - tests which call API endpoints directly.

  • UI tests / E2E tests * - tests which manipulate web browser by simulating user actions.

  • Manual tests - tests performed manually in the web browser.

  • Unit tests - tests which check isolated parts of a code.

  • Test Gap Analysis / TGA - the process of identifying parts of the codebase not covered by any test. Drill4J covers this area with the Coverage and Risks features.

  • Test Impact Analysis / TIA - a process of speeding-up testing phase by understanding which tests should be run after a change to the production code. Drill4J achives that via test-to-code mapping and test recommendations.
note

*Drill4J does not support desktop applications at the moment. For simplicity's sake it's assumed that "UI tests" means "WEB UI tests".

Drill4J terms#

Components#

This list is not exhaustive and intended for introductory use.

  • Drill4J instance - a collection of Drill4J components, mainly Admin Backend and PostgreSQL database, which stores the data about the Agents. It also may include other components, depending on the setup.

  • Admin Panel / Admin UI - a Drill4J's web UI.

  • Admin Backend - the core Drill4J backend service which stores and processes information about all Agents.

  • Agent* - a component (or set of components) responsible for tracking code execution during tests. Each such Agent has a corresponding Agent entity.

  • Autotest Agent** - a component that integrates with the testing framework. Learn more in data collection section.

  • Browser extension - a Drill4J's Chrome Browser extension for recording Manual Tests coverage.

note

Don't get discouraged if something sounds a bit confusing at the moment and keep reading. The further sections will expand on terms and topics described here in greater detail.

* Note on agents


** Note on autotest agents


Features#

This section briefly defines terms for main Drill4J features.

  • Test-to-code mapping / test2code mapping - the mapping data that shows which tests executed which code, and visa-verse. In the more technical terms, it is the association between Coverage data and the particular Test context this data is received from. Drill4J represents this in a two-way relation - see methods tab and tests tab in Admin Panel Basisc section.

  • Coverage - data identifying to what extent packages, classes and methods were executed during tests. On a lower level it is the information on which probes were triggered (or "hit"). It is also the coveredProbestotalProbes\frac {coveredProbes} {totalProbes} percentage calculated based on this data. Refer to Metrics Calculation and Coverage Implementations sections for in-depth explanation.

  • Risks - modified or new Methods detected in Build. Drill4J tracks if you actually covered these changes and reports the result via Quality Gate API.

  • Recommended tests / tests to run - a minimal set of Tests suggested by Drill4J that will allow to cover Risks.

  • Quality gate - a configurable set of conditions for Coverage and number of not-covered Risks. Allows to check if Build meets target metrics such as: Coverage percentage and number of Risks.

Internal#

This section describes terms used specifically in Admin Backend and Admin Panel.

  • Agent - is an entity representing an individual Service within Drill4J.

  • Service group - a group that includes multiple Agents, which allows to record metrics for them at the same time. E.g. collecting metrics for microservices; collecting metrics for web frontend and Java backend.

  • Agent Registration - a one-time configuration process. Whenever the Agent, previously unknown to Drill4J, is added, user must navigate to Admin Panel and go through registration procedure.

  • Build - is a concrete version of the Service, identified by Build version string. Build is always associated with the Agent.

  • Packages, Classes and Methods - for Java and JVM languages (as well as .NET) these are exactly what they sound like. However for JavaScript/TypeScript they do not make much sense. But to keep UI consistent, we stuck with them, with the following caveats.

  • Probes - programmatic counters which track code execution. The exact implementation differs depending on the platform, you can learn more in Coverage Implementations section.

  • Tests - the set of tests which Drill4J recorded data about. It includes Test context and the detailed Coverage data for each package, class and method. Notably, coverage data is stored separately for each test.

  • Test Context - test's metadata, including but not limited to: test name, type, parameters, status, duration. Depending on the exact autotesting framework Drill4J extracts additional information.