Installing on OKD/OCP

Istio Installation Guide

This guide provides instructions for installing Maistra into an existing OpenShift Container Platform (OCP) or Origin (OKD) cluster and for creating a standalone, all-in-one origin cluster with Istio

Tested Configurations

To use this on an OCP 3.11 cluster, the Red Hat registry must be configured.

Preparing the Installation

Before Istio can be installed into an existing installation it is necessary to make a number of changes to the master configuration and each of the schedulable nodes. These changes will enable features required within Istio and also ensure Elasticsearch will function correctly.

The master/node updates discussed below are no longer necessary in OCP/OKD 4.1

Updating the Master

If manual sidecar injection (i.e. kube-inject) is used this section may be skipped.

To enable the automatic injection of the Istio sidecar we first need to modify the master configuration on each master to include support for webhooks and signing of Certificate Signing Requests (CSRs). Then each individual Deployment requiring automatic injection needs to be modified.

First, make the following changes on each master within your installation.

  • Change to the directory containing the master configuration file (e.g. /etc/origin/master/master-config.yaml)

  • Create a file named master-config.patch with the following contents

admissionConfig:
  pluginConfig:
    MutatingAdmissionWebhook:
      configuration:
        apiVersion: apiserver.config.k8s.io/v1alpha1
        kubeConfigFile: /dev/null
        kind: WebhookAdmission
    ValidatingAdmissionWebhook:
      configuration:
        apiVersion: apiserver.config.k8s.io/v1alpha1
        kubeConfigFile: /dev/null
        kind: WebhookAdmission
  • Within the same directory issue the following commands:

cp -p master-config.yaml master-config.yaml.prepatch
oc ex config patch master-config.yaml.prepatch -p "$(cat master-config.patch)" > master-config.yaml
master-restart api
master-restart controllers

Updating the Nodes

In order to run the Elasticsearch application it is necessary to make a change to the kernel configuration on each node, this change will be handled through the sysctl service.

Make the following changes on each node within your installation

  • Create a file named /etc/sysctl.d/99-elasticsearch.conf with the following contents: vm.max_map_count = 262144

  • Execute the following command:

sysctl vm.max_map_count=262144

Installing Maistra

Maistra 0.12 supports plaintext and strict mutual TLS only. Permissive mode does not yet work.

Kiali does not currently support the Jaeger operator.

Installing the Kiali Operator

As of the Maistra 0.12 release, Kiali now relies on the presence of the Kiali operator being installed before Maistra. To install the operator, execute the following as a privileged user:

bash <(curl -L https://git.io/getLatestKialiOperator) --operator-image-version v1.0.0 --operator-watch-namespace '**' --accessible-namespaces '**' --operator-install-kiali false

When the Kiali operator is running, the output should look similar to the following:

$ oc get pods -n kiali-operator
NAME                             READY   STATUS    RESTARTS   AGE
kiali-operator-6575d54c6-9ptfv   2/2     Running   0          4m3s

Installing the Jaeger operator

As of the Maistra 0.12 release, Jaeger now relies on the presence of the Jaeger operator being installed before Maistra. To install the operator, execute the following as an privileged user.

$ oc new-project observability # create the project for the jaeger operator
$ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/crds/jaegertracing_v1_jaeger_crd.yaml
$ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/service_account.yaml
$ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role.yaml
$ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role_binding.yaml
$ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/operator.yaml

When the Jaeger operator is running, the output should look similar to the following:

$ oc get pods -n observability
NAME                               READY   STATUS    RESTARTS   AGE
jaeger-operator-5bcd7ff5df-59msz   1/1     Running   0          4m9s

Installing the Istio Operator

The Maistra installation process introduces a Kubernetes operator to manage the installation, update, and removal of the Istio control plane within the istio-system project.

The following steps will install the Maistra operator into an existing installation, these can be executed from any host with access to the cluster. Please ensure you are logged in as a cluster admin before executing the following

oc new-project istio-operator
oc apply -n istio-operator -f https://raw.githubusercontent.com/Maistra/istio-operator/maistra-0.12/deploy/maistra-operator.yaml

Verifying Installation

The above instructions will create a new deployment within the istio-operator project, executing the operator responsible for managing the state of the Istio control plane through the ServiceMeshControlPlane.

To verify the operator is installed correctly, wait for the operator to reach the running state

oc get pods -n istio-operator -l name=istio-operator

NAME                              READY     STATUS    RESTARTS   AGE
istio-operator-5cd6bcf645-fvb57   1/1       Running   0          1h

Deploying the Istio Control Plane

Single-tenant mode is deprecated and will be removed in a future release. Running in single tenant mode may cause issues with OpenShift upgrades and restarts.

Maistra supports both multi-tenant installations and single-tenant installations. In a single-tenant installation, a single control plane is used for the entire cluster. In a multi-tenant installation, multiple control planes can be deployed in a cluster, with each control plane allowing access to only projects listed in a ServiceMeshMemberRoll resource belonging to that controlplane.

Multi-tenant control plane installations cannot be used in conjunction with a cluster-wide control plane installation, i.e. all installations must be mult-tenant or a single, cluster-wide installation must be used.

The proxy-init image must be changed to maistra/proxy-init-centos7 if running on a RHEL 7 host. More details can be found in the custom installation documentation

Maistra supports a shorthand of smcp for ServiceMeshControlPlane and smmr for ServiceMeshMemberRoll.

Single-Tenant Installation

Single tenant mode is disabled by default in TP12. To enable it, set multitenant to false in your ServiceMeshControlPlane. In order to deploy the Istio Control Plane, we need to create a ServiceMeshControlPlane such as the one in the following example. In our examples, we use the istio-system project. For more information on the parameters and their configuration please see the custom installation documentation.

apiVersion: maistra.io/v1
kind: ServiceMeshControlPlane
metadata:
  name: minimal-install
spec:
  istio:
    global:
      proxy:
        # constrain resources for use in smaller environments
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 128Mi

    gateways:
      istio-egressgateway:
        # disable autoscaling for use in smaller environments
        autoscaleEnabled: false
      istio-ingressgateway:
        # disable autoscaling for use in smaller environments
        autoscaleEnabled: false

    mixer:
      policy:
        # disable autoscaling for use in smaller environments
        autoscaleEnabled: false

      telemetry:
        # disable autoscaling for use in smaller environments
        autoscaleEnabled: false
        # constrain resources for use in smaller environments
        resources:
          requests:
            cpu: 100m
            memory: 1G
          limits:
            cpu: 500m
            memory: 4G

    pilot:
      # disable autoscaling for use in smaller environments
      autoscaleEnabled: false
      # increase random sampling rate for development/testing
      traceSampling: 100.0

    kiali:
      # to disable kiali
      enabled: false

      # create a secret for accessing kiali dashboard with the following credentials
      # dashboard:
      #   user: admin
      #   passphrase: admin

    # disable grafana
    grafana:
      enabled: false

    # to disable tracing (i.e. jaeger)
    tracing:
      enabled: false
      jaeger:
        # simple, all-in-one strategy
        template: all-in-one
        # production strategy, utilizing elasticsearch
        #template: production-elasticsearch
        # if required. only one instance may use agentStrategy=DaemonSet
        #agentStrategy: DaemonSet

Once you have modified the ServiceMeshControlPlane to suit your installation you can deploy the resource using the following command, substituting istio-system if appropriate.

oc new-project istio-system
oc create -n istio-system -f <name of file>

Multi-Tenant Installation

Multitenancy is enabled by default in TP12. To use it, create a project to contain the control plane, and create the ServiceMeshControlPlane in that project.

ServiceMeshMemberRoll

The ServiceMeshMemberRoll resource configures which projects belong to a control plane. Only projects listed in the ServiceMeshMemberRoll will be affected by the control plane. Any number of projects can be added, but a project may not exist in more than one control plane. This resource must be created in the same project as the ServicemeshControlPlane resource and must be named default. An example resource can be seen below:

apiVersion: maistra.io/v1
kind: ServiceMeshMemberRoll
metadata:
  name: default
spec:
  members:
  # a list of projects that should be joined into the service mesh
  # for example, to add the bookinfo project
  - bookinfo

The control plane processes the ServiceMeshMemberRoll when: the ServiceMeshMemberRoll is created, updated, or deleted, the ServicemeshControlPlane in the same project is created or updated, or a project in the ServiceMeshMemberRoll is created or deleted.

Istio CNI plugin

The Istio CNI plugin allows a pod to join the service mesh without requiring additional privileges to be granted for each pod. To enable the plugin, set the enabled field in istio_cni section to true.

apiVersion: maistra.io/v1
kind: ServiceMeshControlPlane
metadata:
  name: basic-install
spec:
  istio:
    istio_cni:
      enabled: true

For more information on CNI, please refer to the Container Network Interface Specification (https://github.com/containernetworking/cni/blob/master/SPEC.md#container-network-interface-specification).

Uninstalling Maistra

Removing the Control Plane

Subsitute the proper project below if the controlplane was created in a project other than istio-system.

The following steps will remove Istio from an existing installation. It can be executed by any user with access to delete the CustomResource.

To get the name of the installed ServiceMeshControlPlane, type:

oc get servicemeshcontrolplanes -n istio-system

This resource can now be deleted as follows:

oc delete smcp -n istio-system <name_of_cr>
oc delete project istio-system

The removal of the CustomResource will tell the Istio operator to begin uninstalling everything it installed.

Removing the Operator

Removing the Kiali Operator

After all control planes have been removed, the Kiali operator can be removed as follows.

$ bash <(curl -L https://git.io/getLatestKialiOperator) --uninstall-mode true

Removing the Jaeger operator

After all control planes have been removed, the Jaeger operator can be removed as follows.

$ oc delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/crds/jaegertracing_v1_jaeger_crd.yaml
$ oc delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/service_account.yaml
$ oc delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role.yaml
$ oc delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role_binding.yaml
$ oc delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/operator.yaml
$ oc delete project observability

Removing the Maistra Operator

In order to cleanly remove the operator execute the following:

oc delete -n istio-operator -f https://raw.githubusercontent.com/Maistra/istio-operator/maistra-0.12/deploy/maistra-operator.yaml

The istio-operator project can now be removed.

oc delete project istio-operator

Upgrading from a Pre-Existing Installation

To upgrade Istio, please remove ServiceMeshControlPlane and then create a one. The operator will upgrade appropriately.

To upgrade the operator, please first remove the operator and then reinstall it. Note that Istio must be removed before the operator.

If the operator was removed before the ServiceMeshControlPlane, you can uninstall the control plane manually. Using the instructions below

oc delete csr istio-sidecar-injector.istio-system
oc get crd  | grep istio | awk '{print $1}' | xargs oc delete crd
oc get mutatingwebhookconfigurations  | grep istio | awk '{print $1}' | xargs oc delete mutatingwebhookconfigurations
oc get validatingwebhookconfiguration  | grep istio | awk '{print $1}' | xargs oc delete validatingwebhookconfiguration
oc get clusterroles  | grep istio | awk '{print $1}' | xargs oc delete clusterroles
oc get clusterrolebindings  | grep istio | awk '{print $1}' | xargs oc delete clusterrolebindings