Using DCI components in partner hooks

Ramon Perez

15 minutes read

Back to all posts

Introduction

DCI components are really useful for establishing efficient CI workflows with partner workloads. This blog post will cover the full workflow that a partner should follow to:

  1. Properly create a component.
  2. Be able to visualize the component created on DCI GUI.
  3. Use the components data in partner hooks.
  4. Launch a DCI job using components.

To show this workflow, we will follow the tnf_test_example from dci-openshift-app-agent, which is a good example that contains all the configuration needed to run components in DCI jobs, then retrieving the data needed from the component and using it in the hooks to run Red Hat Best Practices for Kubernetes test suite over the deployed workloads.

Some other examples will be shown to see the different ways we can use DCI components.

Please check How to automate DCI components creation blog post to know more details about the description and general usage of DCI components.

DCI components workflow

Creation of components

Components are created with dci-create-component utility from python-dciclient project. After launching it, DCI Feeder is in charge of creating the Red Hat components continuously on DCI.

To create the component with this tool, you need to specify the following configuration:

For the case of tnf_test_example, let's see what we have to do to meet the following requirements:

For each remoteCI, we would execute the following (component data must be provided in JSON):

    #!/bin/bash
    DCI_CLIENT_ID='<omitted>'
    DCI_API_SECRET='<omitted>'
    DCI_CS_URL='<omitted>'
    export DCI_CLIENT_ID
    export DCI_API_SECRET
    export DCI_CS_URL

    JSON_DATA_V1='<data_in_json_format>'
    JSON_DATA_V2='<data_in_json_format>'

    # iterate over all current OCP versions available
    for i in {7..12}; do
      echo OCP-4."$i"
      dci-create-component --format json OCP-4."$i" tnf_test_example v0.0.1 ga --data "${JSON_DATA_V1}"
      dci-create-component --format json OCP-4."$i" tnf_test_example v0.0.2 ga --data "${JSON_DATA_V2}"
    done

Finally, validate that the execution has provided a successful response, showing the id provided to your component and verifying that the data you have introduced is saved there.

Visualization of components in DCI GUI

If you go to DCI GUI > Topics > <select the OCP topic>, you can see all the components (example for OCP 4.10) that are currently available for that OCP topic in the Components section.

Once there, you can also filter the components available for that OCP topic based on its name. This is a good way to verify that your components have been created correctly and that they are present in DCI.

For example, if we search tnf_test_example components created for OCP 4.10, four components are retrieved: two different versions (v0.0.1 and v0.0.2) that correspond to the components created on each team used.

tnf_test_example components retrieved from OCP 4.10

By clicking in any component, we can also check the details of the component. Here you have a couple of examples to take a look of the information provided, also checking that the data saved in the component can be visualized here.

Usage of components in partner hooks

Firstly, we need to reference the component in the configuration file we use to launch the job. This depends on the configuration used:

After this, we need to update the hooks to make use of the component.

Both dci-openshift-agent and dci-openshift-app-agent use job_info variable to save context information about the DCI job currently running. This variable can be used to look for the component and all its information.

For example, in tnf_test_example, that variable is used to retrieve the data contained in the component:

    - name: Get variables from tnf_test_example component
      set_fact:
        tnf_app_image: "{{ item['data']['tnf_app_image'] }}"
        tnf_operator_to_install: "{{ item['data']['tnf_operator_to_install'] }}"
        tnf_helm_chart_to_install: "{{ item['data']['tnf_helm_chart_to_install'] }}"
      with_items: "{{ job_info.job.components }}"
      when: item["type"] == "tnf_test_example"

Thanks to this, we do not need to hardcode the configuration in the hooks or in the settings/pipeline: it is directly saved in the component and we only need to retrieve it.

Running DCI jobs using components

With all this done, you are now ready to run your DCI jobs using components!

Let's suppose the following examples of DCI jobs that come from tnf_test_example component, which show why we need two versions for this particular component:

  1. We started using v0.0.1, our super-first release of our component, to be tested with Red Hat Best Practices for Kubernetes test suite. But the job failed, the results from certification suite were not the expected ones! Note that, in the job, you can see the component under the Components list as Tnf_test_example v0.0.1
  2. After troubleshooting, it was discovered that the operator used was not certified by Red Hat, so the development team released v0.0.2, using a certified operator. We relaunched the job by just changing the component version (or directly using the latest, without specifying version), and now… it worked! Again, you can see that Tnf_test_example v0.0.2 component is present under the Components list, and that a success fix is calculated for the Red Hat Best Practices for Kubernetes test suite tests (because there is now one test that is passing and it was not passing in the previous component version) as the same component type is used for the same OCP topic.

And if we go to DCI GUI and return again to the component view, we will be able to see some statistics:

Statistics from v0.0.1

Statistics from v0.0.2

Also, if you go to Analytics section in DCI GUI, you can retrieve some interesting dashboards related to your component coverage! Here's an example from tnf_test_example:v0.0.2:

Analytics dashboards from v0.0.2

Other examples of DCI components usage

example-cnf hooks

For example, in example-cnf, we use a component called nfv-example-cnf-index. Here we have some components from OCP 4.10 as examples:

nfv-example-cnf-index components

The data embedded in the component contains the URL of the public catalog. Here you can see an example of this.

This data is, in fact used in the hooks to save the index image:

    - name: Get example-cnf component details from job.components
      set_fact:
        operator_version: "{{ item['name'] }}"
        example_cnf_index_image: "{{ item['data']['url'] }}:{{ item['name'] }}"
      with_items: "{{ hostvars.localhost.job_info.job.components }}"
      when: item["type"] == "nfv-example-cnf-index"

For example, in F5 SPK (Service Proxy for Kubernetes) hooks, we have components for each SPK version, without data embedded. These are examples for OCP 4.10:

F5 SPK components

Note that, in this particular case, we are not including any data in the component. Here you can see an example of this, which is v1.5.2 for OCP 4.10).

In the hooks, we retrieve the version used, and based on that, we use the SPK binaries, CRDs, configuration, etc. according to the version.

With this code, for example, we retrieve spk_version from the component name (e.g. v1.5.2):

    - name: "Get SPK version from job.components"
      set_fact:
        spk_version: "{{ item['name'] }}"
      with_items: "{{ hostvars.localhost.job_info.job.components }}"
      when:
        - spk_version is undefined
        - item["type"] == "f5-spk"

And using this variable, we load the configuration according to the version retrieved from the component. For instance:

    - name: Load SPK DNS46 configuration variables
      include_vars:
        file: "config/{{ spk_version }}/{{cluster}}.dns46.json"

This will check in config/<spk_version>/...:

F5 SPK repository structure

Conclusions

DCI components are the artifacts used to define software releases within DCI scope. They can embed data that can be used later on in partner hooks to automate the workload deployment process.

In this way, no hardcoded configuration is needed in the hooks or in the pipelines/settings; you only need to select the proper component version. As a result, they make evaluating and comparing job results easier.

References to have in mind

Please check the following documentation as references that supports this work:

Ramon Perez