reporter

package module
v0.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2025 License: Apache-2.0 Imports: 17 Imported by: 0

README 

= Reporter

ifdef::env-github[]
:note-caption: :information_source:
:important-caption: :warning:
endif::[]

Reporter is a simple CLI tool that provides a convenient way to synchronize test results to Jira.

You can leverage this tool to keep stakeholders and adjacent teams informed about the status and health of features, releases or any other deliverable tracked in Jira. One possible use case for this tool is to sync pipeline status from various CI/CD systems, and display that information on a Jira dashboard.

Reporter works by processing one or more JUnit test reports given as input, and uploading the results to dedicated Jira Issues managed by Reporter, which then can be queried for information using the Jira Query Language (JQL).

IMPORTANT: This tool is not supported commercially by Red Hat.

== Getting started

=== Installation

NOTE: Ready-to-use binaries for various architectures can be found on the https://github.com/redhat-eets/reporter/releases[Releases] page.

To download the latest available binary for an `x86-64` linux-based system, issue the following commands:

[source, text]
-----
$ wget https://github.com/redhat-eets/reporter/releases/latest/download/reporter_linux_x86_64 -O /tmp/reporter
$ chmod +x /tmp/reporter
-----

*(Optional)* To make the `reporter` binary easier to discover, move the binary to one of the directories defined in your `$PATH` such as `/usr/local/bin`:

[source, text]
-----
$ mv -n /tmp/reporter /usr/local/bin/reporter
-----

Once you download the binary, see the section below to understand how to run Reporter for the first time.

=== Dry run

The easiest way to learn how to use the tool is to enable the `-n/--no-sync` flag and start experimenting with the CLI.

If this flag is present, no requests are sent to Jira when executing the command of your choice. This is great for testing various CLI options and custom configuration files.

Assuming you have a JUnit test report in the current working directory named `test-report.xml`, you can let Reporter process the file, display the results (test counts, etc) and simulate uploading the results to Jira with the following command:

[source, text]
-----
$ reporter upload -i test-report.xml --no-sync
-----

Once you verify that Reporter reads and processes your JUnit test reports correctly, you can start uploading the results. For that you will need to configure access to Jira, which is covered in the next section.

=== Authentication

To upload test results to Jira with Reporter, you must first https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html[obtain a Personal Access Token (PAT)] to authenticate the API requests with.

NOTE: If you are using Reporter in a CI/CD pipeline, consider creating a new, dedicated Jira service account with limited permissions. Refer to your organization's internal guidelines for information on how to create such an account.

*(Recommended)* The easiest way to provide the Personal Access Token to Reporter is by setting an environment variable named `REPORTER_JIRA_TOKEN`. The token will then be read automatically.

[source, text]
-----
$ export REPORTER_JIRA_TOKEN=<access-token>
$ reporter upload -i test-report.xml
-----

NOTE: If you are using Reporter in a CI/CD pipeline, consider saving the `REPORTER_JIRA_TOKEN` environment variable using the dedicated functionality of your CI/CD system for storing secrets instead of `exporting` the value in the job itself.

*(Not recommended)* It is also possible to provide the PAT using the `-t/--jira-token` option. Be careful when using this method as you can accidentally log the token causing a potential data leak.

[source, text]
-----
$ reporter upload -i test-report.xml -t <access-token>
-----

=== Configuration [[getting_started_configuration]]

To tweak the behavior of Reporter, you can create a custom configuration file.

By default, the tool will look for a `config.yaml` file in the current working directory. As config files are completely optional, Reporter will not terminate even if it does not find one.

See link:config.yaml.default[config.yaml.default] to review the default values used by Reporter and link:config.yaml.sample[config.yaml.sample] for a full example of a configuration file that you could create.

NOTE: Loading a custom configuration file does not replace all default values, just the ones that you have defined in the custom config. This means that if you are mostly happy with the default values Reporter ships with, you can maintain a tiny custom configuration file that overwrites just one or two values.

To specify a path to a custom configuration file, set the `-c/--config` option:

[source, text]
-----
$ reporter upload -c custom-config.yaml
-----

== Basic usage

=== Selecting input data

To upload test results to Jira, specify one or more paths to JUnit test reports with the `-i/--input` option. You can also provide a path to a directory instead of listing individual files.

By default, Reporter will look for test reports in the `input/` directory (if such directory exists). +
This means that this command:

[source, text]
-----
$ reporter upload -i input/ --no-sync
-----

Is equivalent to:

[source, text]
-----
$ reporter upload --no-sync
-----

Place the test reports in the `input/` directory and run the command listed above.

=== Uploading results

Now, if all your JUnit test reports can be parsed correctly by Reporter, specify the destination where the results should be uploaded to.

The destination can be either one of the following:

* *Jira Story*. Reporter will first analyze the Story and check if it contains the required summary and labels (as defined in the default configuration file). Then, it will search for any Sub-tasks under the Story that match the Reporter configuration. If no valid Sub-task exists, Reporter will create a new Sub-task and post the results there. If the Sub-task already exists, it will update it with the latest test results.
* *Jira Sub-task*. Reporter will first analyze the Sub-task and check if it resembles a Sub-task that the tool would create (as defined in the default configuration file). If it is a valid Sub-task, Reporter will update it with the latest test results.

To upload test results from reports stored in the `input/` directory to a Jira Issue `EXAMPLE-15`, issue the following command:

[source, text]
-----
$ reporter upload -d EXAMPLE-15
-----

In some scenarios, specifying one destination for all test results might not be practical. Refer to the next chapter to learn how to configure more advanced routing for test reports.

=== Annotating results with custom metadata

If you want to provide additional information such as artifact links, source commit hashes, release versions, etc you can use the `-m/--metadata` option. This CLI option can be repeated multiple times to enter as many metadata strings as needed.

[source, text]
-----
$ reporter upload -d EXAMPLE-15 \
    -m "CI Job URL = https://prow.ci.openshift.org" \
    -m "Commit hash = a1b2c3d"
-----

== Advanced usage

=== Configuring routing rules

NOTE: Make sure you have read the <<getting_started_configuration>> section first.

You can control which test suite (and test case) result will be reported to which destination by creating a custom configuration file and defining routing rules in the `reporting.routing` section.

Assume you want to upload just one test suite named _"Example end-to-end tests"_ to Jira Issue `EXAMPLE-15`, and filter out everything else. This could be achieved with a custom configuration file like the following:

[source, yaml]
-----
apiVersion: v1
spec:
  reporting:
    routing:
      - destination: EXAMPLE-15
        testSuites:
          - name: "Example end-to-end tests"
-----

It is also possible to choose which test cases should be included in the final result. To that end, define a `testCases` section under the relevant test suite entry and list the test cases by name.

To create a match-all rule, use the `"*"` symbol.

The following example defines a match-all rule for test cases of _"Example end-to-end tests"_, which makes it functionally identical to the previous example.

[source, yaml]
-----
apiVersion: v1
spec:
  reporting:
    routing:
      - destination: EXAMPLE-15
        testSuites:
          - name: "Example end-to-end tests"
            testCases:
              - name: "*"
-----

You can define as many destinations as you want, and as many test suite or test case configurations as you want. The following example defines three destinations: `EXAMPLE-15`, `EXAMPLE-20` and `EXAMPLE-25`, where only the selected test suites and test cases will be uploaded, as defined in the configuration listed below.

NOTE: Writing configuration files by hand is not required -- for more complicated setups, you could consider writing a script that would generate this file automatically.

[source, yaml]
-----
apiVersion: v1
spec:
  reporting:
    routing:
      - destination: EXAMPLE-15
        testSuites:
          - name: "Example end-to-end tests"
            testCases:
              - name: "[rh] Service is reachable"
              - name: "[rh] System backup can be scheduled"
              - name: "[rh] Service can be rolled back"

      - destination: EXAMPLE-20
        testSuites:
          - name: "Example feature A unit tests"
          - name: "Example feature A system tests"

      - destination: EXAMPLE-25
        testSuites:
          - name: "*"
-----

To upload test results from reports stored in the `input/` directory to the destinations defined in the `config.yaml` file, issue the following command:

[source, text]
-----
$ reporter upload
-----

=== Matching by property

Instead of defining routing rules using test suite and test case names, you can directly reference JUnit properties to do the matching.

Assume you want to upload a set of test reports to Jira `EXAMPLE-15` where each applicable test suite is label with a property `report_results` set to `True`. This could be achieved with a custom configuration file like the following:

[source, yaml]
-----
apiVersion: v1
spec:
  reporting:
    routing:
      - destination: EXAMPLE-15
        testSuites:
          - property: "report_results=True"
            testCases:
              - name: "*"
-----

If you apply this custom configuration to a test report structured like in the example below, then the test cases `one`, `two` and `three` will be uploaded to `EXAMPLE-15`, while test cases `four` and `five` will be ignored as their parent test suite lacks the required property.

[source, xml]
-----
<testsuites>
  <testsuite>
    <properties>
      <property name="report_results" value="True"></property>
    </properties>
    <testcase name="one" status="passed"></testcase>
    <testcase name="two" status="passed"></testcase>
    <testcase name="three" status="passed"></testcase>
  </testsuite>

  <testsuite>
    <testcase name="four" status="passed"></testcase>
    <testcase name="five" status="passed"></testcase>
  </testsuite>
</testsuites>
-----

Matching individual test cases by property is also possible. Consider the following example for a scenario where each test case is labeled with a property referencing an entity in an external system (in this case, this system is Polarion).

The custom configuration listed below would allow you to upload test results to Jira by mapping the relationship between the external system and your destinations in Jira.

[source, yaml]
-----
apiVersion: v1
spec:
  reporting:
    routing:
      - destination: EXAMPLE-20
        testSuites:
          - name: "*"
            testCases:
              - property: "polarion-testcase-id=POL-100"

      - destination: EXAMPLE-25
        testSuites:
          - name: "*"
            testCases:
              - property: "polarion-testcase-id=POL-105"
-----

When applied to a test report structured like in the example below, test cases `one` and `two` will be uploaded to `EXAMPLE-20`, while test case `three` will be uploaded to `EXAMPLE-25`.

[source, xml]
-----
<testsuites>
  <testsuite>
    <testcase name="one" status="passed">
      <properties>
        <property name="polarion-testcase-id" value="POL-100"></property>
      </properties>
    </testcase>

    <testcase name="two" status="passed">
      <properties>
        <property name="polarion-testcase-id" value="POL-100"></property>
      </properties>
    </testcase>

    <testcase name="three" status="passed">
      <properties>
        <property name="polarion-testcase-id" value="POL-105"></property>
      </properties>
    </testcase>
  </testsuite>
</testsuites>
-----

Documentation

Index

Constants

View Source 
const MatchAllSymbol = "*"

MatchAllSymbol defines which symbol will be used to represent a rule that accepts any string as input.

Variables

View Source 
var (
	InfoLog  = log.New(os.Stdout, "[INFO] ", logFlags)
	WarnLog  = log.New(os.Stdout, "[WARN] ", logFlags)
	ErrorLog = log.New(os.Stderr, "[ERROR] ", logFlags)
)
View Source 
var EmbeddedDefaultConfig []byte
View Source 
var EmbeddedTemplatesFS embed.FS

Functions

func LogAggregateReports 

func LogAggregateReports(logger *log.Logger, reports []AggregateReport)

LogAggregateReports dumps the AggregateReports in a human-readable form to a given logger.

func LogMetadataEntries added in v0.1.5 

func LogMetadataEntries(logger *log.Logger, metadata []MetadataEntry)

LogMetadataEntries pretty-prints all system and user-provided metadata entries using a given logger.

func LogUploadSummary 

func LogUploadSummary(logger *log.Logger, uploadedCount int, reports []AggregateReport)

LogUploadSummary logs a simple summary message displaying how many AggregateReports were successfully uploaded.

func RenderEmbeddedTemplate 

func RenderEmbeddedTemplate(path string, data any) (buf bytes.Buffer, err error)

func RenderLocalTemplate 

func RenderLocalTemplate(path string, data any) (buf bytes.Buffer, err error)

func UploadAggregateReports 

func UploadAggregateReports(reports []AggregateReport, metadata []MetadataEntry, config Config, token string) error

UploadAggregateReports takes multiple AggregateReports and uploads them all to their corresponding destinations.

func UploadSingleAggregateReport 

func UploadSingleAggregateReport(report AggregateReport, metadata []MetadataEntry, config Config, token string) error

UploadSingleAggregateReport uploads a given AggregateReport to its destination.

Types

type AggregateReport 

type AggregateReport struct {
	Destination string
	TestSuites  []TestSuite
	Counts      Counts
}

AggregateReport stores information about all Test Suites and Test Cases that should be reported to a selected Jira Issue (Destination).

func ProcessJUnitReports 

func ProcessJUnitReports(paths []string, config ReportingConfig) (reports []AggregateReport, err error)

ProcessJUnitReports loads and analyzes JUnit Test Reports according to the routing config defined by the user.

func ProcessJUnitSuites 

func ProcessJUnitSuites(suites []junit.Suite, route ReportingRouteConfig) (report AggregateReport)

ProcessJUnitSuites processes all loaded Test Suites according to a given routing configuration. A single AggregateReport will be created for each route defined by the user. If any Test Suites or Test Cases match any of the rules defined for this route, they will be added to the Report.

func (*AggregateReport) AggregateCounts 

func (r *AggregateReport) AggregateCounts()

AggregateCounts takes all Test Suites contained in the report and calculates the total sum on all Counters.

type Config 

type Config struct {
	APIVersion string     `mapstructure:"apiVersion"`
	Spec       ConfigSpec `mapstructure:"spec"`
}

type ConfigSpec 

type ConfigSpec struct {
	Jira      JiraConfig      `mapstructure:"jira"`
	Reporting ReportingConfig `mapstructure:"reporting"`
}

type Counts 

type Counts struct {
	Passed  int
	Failed  int
	Errored int
	Skipped int
	Total   int
}

Counts provides a container for storing information about test counts.

func (*Counts) Add 

func (c *Counts) Add(test junit.Test)

Add increases test counts based on the status of the Test Case given by the user.

type IssueDesiredStateFields 

type IssueDesiredStateFields struct {
	Summary     string
	Description string
	Labels      []string
}

IssueDesiredStateFields stores values that are expected to be sent to Jira in order for the Jira Issue to reach the desired state.

type JiraClient 

type JiraClient struct {
	ServerURL   string
	AccessToken string
}

JiraClient manages communication with the Jira REST API.

func (JiraClient) CreateSubtask 

func (c JiraClient) CreateSubtask(parent string, summary string, description string, labels []string) (string, error)

CreateSubtask sends a request to create a Sub-task under a given parent Issue.

func (JiraClient) GetIssue 

func (c JiraClient) GetIssue(id string) (issue JiraIssue, err error)

GetIssue sends a request to Jira REST API to fetch an Issue with the given ID.

func (JiraClient) UpdateIssue 

func (c JiraClient) UpdateIssue(id string, summary string, description string, labels []string) error

UpdateIssue sends a request to Jira REST API to update an Issue with the given ID.

type JiraConfig 

type JiraConfig struct {
	Server       JiraServerConfig            `mapstructure:"server"`
	Discovery    JiraIssueDiscoveryConfig    `mapstructure:"discovery"`
	DesiredState JiraIssueDesiredStateConfig `mapstructure:"desiredState"`
}

type JiraIssue 

type JiraIssue struct {
	ID          string
	Type        string
	Parent      *JiraIssue
	Summary     string
	Description string
	Labels      []string
	SubTasks    []*JiraIssue
}

JiraIssue represents an Issue as returned by the Jira REST API.

func (*JiraIssue) IsLabeledWithAnyOf 

func (issue *JiraIssue) IsLabeledWithAnyOf(ls []string) bool

IsLabeledWithAnyOf provides a convenience method to check whether the Jira Issue is labeled with at least one of the labels given by the user.

type JiraIssueDesiredStateConditionalConfig 

type JiraIssueDesiredStateConditionalConfig struct {
	Labels []string `mapstructure:"labels"`
}

type JiraIssueDesiredStateConfig 

type JiraIssueDesiredStateConfig struct {
	Summary     JiraIssueDesiredStateSummaryConfig     `mapstructure:"summary"`
	Description JiraIssueDesiredStateDescriptionConfig `mapstructure:"description"`
	OnSuccess   JiraIssueDesiredStateConditionalConfig `mapstructure:"onSuccess"`
	OnFailure   JiraIssueDesiredStateConditionalConfig `mapstructure:"onFailure"`
}

type JiraIssueDesiredStateDescriptionConfig 

type JiraIssueDesiredStateDescriptionConfig struct {
	TemplatePath string `mapstructure:"templatePath"`
}

type JiraIssueDesiredStateSummaryConfig 

type JiraIssueDesiredStateSummaryConfig struct {
	Contents          string `mapstructure:"contents"`
	IncludeTestCounts bool   `mapstructure:"includeTestCounts"`
}

type JiraIssueDiscoveryConfig 

type JiraIssueDiscoveryConfig struct {
	Summary JiraIssueDiscoverySummaryConfig `mapstructure:"summary"`
	Labels  JiraIssueDiscoveryLabelsConfig  `mapstructure:"labels"`
}

type JiraIssueDiscoveryLabelsConfig 

type JiraIssueDiscoveryLabelsConfig struct {
	RequiredAnyOf []string `mapstructure:"requiredAnyOf"`
}

type JiraIssueDiscoverySummaryConfig 

type JiraIssueDiscoverySummaryConfig struct {
	RequiredPrefix string `mapstructure:"requiredPrefix"`
}

type JiraServerConfig 

type JiraServerConfig struct {
	URL string `mapstructure:"url"`
}

type MetadataEntry added in v0.1.5 

type MetadataEntry struct {
	Key   string
	Value string
}

MetadataEntry represents a single key-value pair of metadata set by the user.

type ReportingConfig 

type ReportingConfig struct {
	Routing []ReportingRouteConfig `mapstructure:"routing"`
}

type ReportingRouteConfig 

type ReportingRouteConfig struct {
	Destination string                     `mapstructure:"destination"`
	TestSuites  []ReportingTestSuiteConfig `mapstructure:"testSuites"`
}

type ReportingTestCaseConfig 

type ReportingTestCaseConfig struct {
	Name     string `mapstructure:"name"`
	Property string `mapstructure:"property"`
}

type ReportingTestSuiteConfig 

type ReportingTestSuiteConfig struct {
	Name      string                    `mapstructure:"name"`
	Property  string                    `mapstructure:"property"`
	TestCases []ReportingTestCaseConfig `mapstructure:"testCases"`
}

type TestSuite 

type TestSuite struct {
	Name   string
	Counts Counts
}

TestSuite represents a Test Suite from a test report.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.