Working with TripleO Composable Roles!!

Problem:

Deploying heterogeneous compute nodes (NICs in this document) using TripleO Openstack deployment tool.

Solution:

Composable roles give a great flexibility to the deployment and are good way to have own customized environment while using Redhat director (Triple O upstream project).

Test deployment contains 2 compute nodes and 3 controller nodes. Both computes nodes are identical to each other except number of NICs and its supplier vendors.

Compute 1 is having

2 Intel X710 10 Gbps SRIOV capable 4 ports ethernet NICs

Compute 2 is having

2 Intel X710 10 Gbps SRIOV capable 4 ports ethernet NICs

• 1 Mellanox ConnectX-5 40 Gbps SRIOV capable 2 ports ethernet NIC

We need to have 2 separate roles for this deployment.

Role 1 deploys Compute 1 and Role 2 for Compute 2.

Remember, Single role is applicable where all the nodes are having identical hardware configuration. For example, if we have 3rd Node with identical hardware configuration as Compute 1, Role 1 should be used to deploy Compute 1 and Compute 3 and Role 2 will be used for Compute 2.

Let’s create both roles. Here Onwards, Role 1 = ComputeOvsDpdkSriov and Role 2 = ComputeOvsDpdkSriovMellanox.

openstack overcloud roles generate -o roles_data.yaml Controller ComputeOvsDpdkSriov ComputeOvsDpdkSriov:ComputeOvsDpdkSriovMellanox

Note: There are already predefined roles present in undercloud in Redhat OSP14. Here, “Controller”, “ComputeOvsDpdkSriov” are predefined and “ComputeOvsDpdkSriovMellanox“ derived from “ComputeOvsDpdkSriov” for our customization.

Detailed templates used in this deployment is found here.

After creation of roles_data.yaml, we need to map roles to its config file.

Following excerpt from network-environment file where i mapped Role 2 to its config file.

resource_registry:  # Specify the relative/absolute path to the config files you want to use for override the default.  OS::TripleO::ComputeOvsDpdkSriov::Net::SoftwareConfig: nic-configs/computeovsdpdksriov.yaml  OS::TripleO::Controller::Net::SoftwareConfig: nic-configs/controller.yaml  OS::TripleO::ComputeOvsDpdkSriovMellanox::Net::SoftwareConfig: nic-configs/computeovsdpdksriovmellanox.yaml

Next step is to create a flavor in undercloud machine to associate it to Compute 2.

openstack flavor create –disk 97 –ram 125000 –vcpus 39 mellanox

Associate this flavor to Compute node 2.

openstack baremetal node set –property capabilities=’profile:mellanox’’ compute-2

Associate “mellanox” flavor to “ComputeOvsDpdkSriovMellanox” role and its compute count.

OvercloudComputeOvsDpdkSriovMellanoxFlavor: mellanoxComputeOvsDpdkSriovMellanoxCount: 1

We can now configure required customized parameter under

ComputeOvsDpdkSriovMellanoxParameters:

Now, deployment here needs to be told that there is a difference in number of NICs. Else, it may try to config those extra NICs on Compute 1 and it wont find them which lead to deployment failure.

This is done in nic-mapping.yaml file.

We need to map out “ComputeOvsDpdkMellanox” role to os-net-config-mappings.yaml. This tells only this roles will have extra NICs and remain other won’t have it.

Excerpt from nic-mapping.yaml file:

resource_registry:  OS::TripleO::ComputeOvsDpdkSriovMellanox::NodeUserData: /usr/share/openstack-tripleo-heat-templates/firstboot/os-net-config-mappings.yaml

Lets deploy overcloud now.

openstack overcloud deploy –templates -r /home/stack/roles_data.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/host-config-and-reboot.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dpdk.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-sriov.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/collectd-environment.yaml -e /home/stack/api-policies.yaml -e /home/stack/nic-mapping-env.yaml -e /home/stack/network-environment.yaml -e /home/stack/containers-prepare-parameter.yaml –log-file overcloud_install.log