Do you need to read this?
Perhaps the most important question you should be asking yourself right now is if you really need to follow the steps in this blog post. I can help you answering this a little bit, if you:
- Want to develop a custom hook for your specific dci-openshift-agent (d-o-a) setup or
- Identified a bug and the patch to fix it or
- Want to add a core feature to the agent or
- Just want to know more!
Then feel free to continue reading :)
All examples throughout this blog post assume we're working with RHEL/CentOS 8 Stream
What does it mean to have a development environment?
A development environment in this case refers to a complete setup where you can test your DCI OCP agent changes, this means a jumpbox and a System Under Test (SUT)
Let's face it, not everyone has an extra rack with 4 (minimum) servers laying around just to be used. If you do then great (I am a bit jealous even) but if you're like me, then you have at most one extra (albeit beefy) server where you can start tinkering with the DCI OCP agent.
For an everyday solution we created what we call an OCP on LibVirt setup which means you have your entire cluster emulated in a single server. Your jumpbox, your SUT, your mirrors, all of it, it can be done in a single machine.
What do you need to start?
The requirements are simple, you need:
- A server with a minimum of 200Gb disk and 64G of RAM
- RHEL 8 (CentOS 8 Stream works just as well)
- An SSH connection
That's all. Of course there are fancier ways to work on this but I will show you the most basic ways to test your libvirt environment and then you can go and explore on your own.
I am going to explain what the rough process to get a libvirt environment is, then at the very end I will point you in the direction of an experimental (at the time of this post) script to automate it all
The base operating system
As mentioned before, you need a server with RHEL/CentOS 8. Make sure your system has all updates installed and it's got an active subscription in the case of RHEL.
Make sure you have enough space (around 200G for an IPI
install) in the partition
holding your data, by default it is
/opt but it can be customized
Install Ansible 2.9 because currently the OCP agent is not compatible with
2.12. If you are using CentOS want to install the
centos-release-ansible-29-1-2 package first, then install
like to lock the version of ansible to avoid issues on future system updates,
1 2 3 4 5
Finally, install libvirt and all other packages needed:
1 2 3 4 5 6 7
One last piece is to enable nested KVM, but this is actually done by one of the playbooks you have to run down in the "Creating your virtual SUT" section. If you're curious how that is done, it's something like this:
1 2 3 4 5 6 7
Setting up DCI
The next step is to install the DCI repo and the required packages, easy enough
to do if you download the appropriate
dci-release package from the DCI repo at
https://packages.distributed-ci.io and then installing the
1 2 3
This creates a
dci-openshift-agent user in your system, which should be the
owner for all directories and processes that drive the OCP agent. As such, you
should install your public key(s) under
~dci-openshift-agent/.ssh/authorized_keys so you are able to login directly.
This also creates a directory
/etc/dci-openshift-agent which houses your DCI
remote authentication credentials. You can download your remote CI
authentication credentials as per the official
documentation and once you have
them, add them to your
The last step to configure DCI is to edit your
/etc/dci-openshift-agent/settings.yml file to suit your needs e.g. changing
the topic you want to test
Creating your virtual SUT
A SUT (System Under Test) in this case is composed of a provision host plus a mixed OCP control/data plane with 3 nodes, for a total of 4 VMs.
We have a playbook that will take care of that for you, it is located in
samples/ocp_on_libvirt/libvirt_up.yml under the
home dir, and you can execute it from your jumbox now that you have installed
ansible, like so:
1 2 3
This takes a little while, but at the end of the run you should be able to
check there are indeed 4 VMs running with
sudo virsh list --all
This also creates a file
templated out from your SUT definition. You have to copy that file to
/etc/dci-openshift-agent/hosts. This file should work as-is assuming a
connected environment, you can edit this inventory to suit your needs e.g.
creating a disconnected cluster and etc.
Along with the
libvirt_up.yml playbook there is a second
libvirt_destroy.yml playbook which you can use to re-create your virtual
SUT, should you need to.
Running the agent
Once you have all the pieces in place, you can run the agent in two ways:
- Using the packaged
/usr/bin/dci-openshift-agentbinary. This will use the stock/release codebase
- Manually running via
ansible-playbookif e.g. you're running from a local clone of the dci-openshift-agent codebase
For point #1 above there's not much to say, but let's say you're developing something with the OCP agent, you want to test that out and of course the released code doesn't have it. In such cases you need a copy of the codebase in an editable place and then you can run your changes like this:
1 2 3
And done, you can iterate as many times as you want now.
I have explained roughly the overall process to get you up and running, and you may be asking yourself "well, this should all be in a playbook somewhere" which you are 100% correct and, good news! It already is!
You may have noticed a suspicious
playbook. This performs all the steps I have explained automatically. It is
experimental as of yet and prone to changes in the near future, but if you want
to try your luck here's what you can do:
- Create a
~/.ocp-agent.ymlfile containing the following variables:
dci_client_id: use DCI_CLIENT_ID from your remote CI credentials
dci_api_secret: use DCI_API_SECRET from your remote CI credentials
rhn_user: If your lab is RHEL, you need this to automatically subscribe
rhn_pass: If your lab is RHEL, you need this to automatically subscribe
github_user: To add your public keys to the agent's authorized keys list
- Create an inventory file (or edit the default
/etc/ansible/hosts) and add an entry for your lab
- Execute the playbook. The playbook itself is executable thanks to the magic
of shebangs and bash, so you can just
samples/ocp_on_libvirt/ocp-on-libvirt.yml myhostor you can take a look at the shebang to figure out the proper call to execute it.
After this, you should end up with a system ready to execute the OCP agent.
Developing for/with OCP is not easy; we at DCI aim to make it as easy as possible to tighten the development->testing->release feedback loop, but there is and always will be room to improve. So feel free to get in touch with us if you see something not-quite-right or you have any questions, don't hesitate!