$ oc adm must-gather --image=registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel7
Maistra Service Mesh is a platform that provides behavioral insight and operational control over the service mesh, providing a uniform way to connect, secure, and monitor microservice applications.
The term service mesh describes the network of microservices that make up applications in a distributed microservice architecture and the interactions between those microservices. As a service mesh grows in size and complexity, it can become harder to understand and manage.
Based on the open source Istio project, Maistra Service Mesh adds a transparent layer on existing distributed applications without requiring any changes to the service code. You add Maistra Service Mesh support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between microservices. You configure and manage the service mesh using the control plane features.
Maistra Service Mesh provides an easy way to create a network of deployed services that provides discovery, load balancing, service-to-service authentication, failure recovery, metrics, and monitoring. A service mesh also provides more complex operational functionality, including A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.
If you experience difficulty with a procedure described in this documentation, or with OpenShift Container Platform in general, please open an issue.
If you have a suggestion for improving this documentation or have found an error, please see the contributors guide or open an issue.
When opening a support case, it is helpful to provide debugging information about your cluster to Red Hat Support.
must-gather tool enables you to collect diagnostic information about your
OpenShift Container Platform cluster, including virtual machines and other data related to
Maistra Service Mesh.
For prompt support, supply diagnostic information for both OpenShift Container Platform and Maistra Service Mesh.
oc adm must-gather CLI command collects the information from your cluster that is most likely needed for debugging issues, such as:
You can specify one or more images when you run the command by including the
--image argument. When you specify an image, the tool collects data related to that feature or product.
When you run
oc adm must-gather, a new pod is created on the cluster. The data is collected on that pod and saved in a new directory that starts with
must-gather.local. This directory is created in the current working directory.
Access to the cluster as a user with the
The OpenShift Container Platform CLI (
You can use the
oc adm must-gather CLI command to collect information about your cluster, including features and objects associated with Maistra Service Mesh.
To collect Maistra Service Mesh data with
must-gather, you must specify the Maistra Service Mesh image:
$ oc adm must-gather --image=registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel7
The following are the only supported configurations for the Maistra Service Mesh:
Red Hat OpenShift Container Platform version 4.x.
OpenShift Online and OpenShift Dedicated are not supported for Maistra Service Mesh.
The deployment must be contained to a single OpenShift Container Platform cluster that is not federated.
This release of Maistra Service Mesh is only available on OpenShift Container Platform x86_64.
This release only supports configurations where all Maistra components are contained in the OpenShift cluster in which it operates. It does not support management of microservices that reside outside of the cluster, or in a multi-cluster scenario.
This release only supports configurations that do not integrate external services such as virtual machines.
For additional information about Maistra Service Mesh lifecycle and supported configurations, refer to the Support Policy.
The Kiali observability console is only supported on the two most recent releases of the Chrome, Edge, Firefox, or Safari browsers.
This release only supports the following Mixer adapter:
3scale Istio Adapter
Maistra Service Mesh provides a number of key capabilities uniformly across a network of services:
Traffic Management - Control the flow of traffic and API calls between services, make calls more reliable, and make the network more robust in the face of adverse conditions.
Service Identity and Security - Provide services in the mesh with a verifiable identity and provide the ability to protect service traffic as it flows over networks of varying degrees of trustworthiness.
Policy Enforcement - Apply organizational policy to the interaction between services, ensure access policies are enforced and resources are fairly distributed among consumers. Policy changes are made by configuring the mesh, not by changing application code.
Telemetry - Gain understanding of the dependencies between services and the nature and flow of traffic between them, providing the ability to quickly identify issues.
3scale Istio Adapter
This release of Maistra Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Maistra Service Mesh adds support for Istio 1.6.5, Jaeger 1.20.0, Kiali 1.24.2, and the 3scale Istio Adapter 2.0 and OpenShift Container Platform 4.6.
In addition, this release has the following new features:
Introduces a re-architected control plane. The Mixer component has been deprecated and will be removed in a future release. The other control plane components, Pilot, Galley, Citadel, have been combined into a single binary known as Istiod. The "d" stands for daemon.
Simplifies installation, upgrades, and management of the control plane.
Reduces the control plane’s resource usage and startup time.
Improves performance by reducing inter-control plane communication over networking.
Adds support for Envoy’s Secret Discovery Service (SDS). SDS is a more secure and efficient mechanism for delivering secrets to Envoy side car proxies.
Removes the need to use Kubernetes Secrets, which have well known security risks.
Improves performance during certificate rotation, as proxies no longer require a restart to recognize new certificates.
Adds support for Istio’s Telemetry v2 architecture, which is built using WebAssembly extensions. This new architecture brings significant performance improvements.
Updates the ServiceMeshControlPlane resource to v2 with a streamlined configuration to make it easier to manage the Control Plane.
Introduces WebAssembly extensions as a Technology Preview feature.
Some features available in previous releases have been deprecated or removed.
Deprecated functionality is still included in OpenShift Container Platform and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
The Mixer component is deprecated and will be removed in a future release. While using Mixer for implementing extensions is still supported in release 2.0, you should be migrating your extensions to the new WebAssembly mechanism.
The following resource types are no longer supported in Maistra Service Mesh 2.0:
Policy (authentication.istio.io/v1alpha1) is no longer supported. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.
ServiceMeshPolicy (maistra.io/v1) is no longer supported.
PeerAuthentication, as mentioned above, but place in the control plane namespace.
RbacConfig (rbac.istio.io/v1alpha1) is no longer supported.
AuthorizationPolicy (security.istio.io/v1beta1), which encompasses behavior of
ServiceMeshRbacConfig (maistra.io/v1) is no longer supported.
AuthorizationPolicy as above, but place in control plane namespace.
ServiceRole (rbac.istio.io/v1alpha1) is no longer supported.
ServiceRoleBinding (rbac.istio.io/v1alpha1) is no longer supported.
In Kiali, the
LDAP strategies are deprecated. A future version will introduce authentication using OpenID providers.
These limitations exist in Maistra Service Mesh:
Maistra Service Mesh does not support IPv6, as it is not supported by the upstream Istio project, nor fully supported by OpenShift.
Graph layout - The layout for the Kiali graph can render differently, depending on your application architecture and the data to display (number of graph nodes and their interactions). Because it is difficult if not impossible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts. To choose a different layout, you can choose a different Layout Schema from the Graph Settings menu.
The first time you access related services such as Jaeger and Grafana, from the Kiali console, you must accept the certificate and re-authenticate using your OpenShift Container Platform login credentials. This happens due to an issue with how the framework displays embedded pages in the console.
These are the known issues in Maistra Service Mesh:
Gateways created in a non-control plane namespace will not be automatically deleted. Users will need to manually delete these resources after removing the gateway definition from the SMCP spec.
Prometheus scraping (
spec.addons.prometheus.scrape set to
true) does not work when mTLS is enabled. Additionally, Kiali displays extraneous graph data when mTLS is disabled.
Both problems can be addressed by excluding port 15020 from proxy configuration, for example,
spec: proxy: networking: trafficControl: inbound: excludedPorts: - 15020
OSSM-296 When adding health configuration to the Kiali custom resource (CR) is it not being replicated to the Kiali configmap.
OSSM-291 In the Kiali console, on the Applications, Services, and Workloads pages, the "Remove Label from Filters" function is not working.
OSSM-289 In the Kiali console, on the Service Details pages for the 'istio-ingressgateway' and 'jaeger-query' services there are no Traces being displayed. The traces exist in Jaeger.
OSSM-287 In the Kiali console there are no traces being displayed on the Graph Service.
OSSM-285 When trying to access the Kiali console, receive the following error message "Error trying to get OAuth Metadata". The workaround is to restart the Kiali pod.
Istio-14743 Due to limitations in the version of Istio that this release of Maistra Service Mesh is based on, there are several applications that are currently incompatible with Maistra. See the linked community issue for details.
MAISTRA-1947 Technology Preview Updates to ServiceMeshExtensions are not applied. The workaround is to remove and recreate the ServiceMeshExtensions.
MAISTRA-858 The following Envoy log messages describing deprecated options and configurations associated with Istio 1.1.x are expected:
[2019-06-03 07:03:28.943][warning][misc] [external/envoy/source/common/protobuf/utility.cc:129] Using deprecated option 'envoy.api.v2.listener.Filter.config'. This configuration will be removed from Envoy soon.
[2019-08-12 22:12:59.001][warning][misc] [external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file lds.proto. This configuration will be removed from Envoy soon.
MAISTRA-806 Evicted Istio Operator Pod causes mesh and CNI not to deploy.
istio-operator pod is evicted while deploying the control pane, delete the evicted
MAISTRA-681 When the control plane has many namespaces, it can lead to performance issues.
MAISTRA-465 The Maistra Operator fails to create a service for operator metrics.
MAISTRA-453 If you create a new project and deploy pods immediately, sidecar injection does not occur. The operator fails to add the
maistra.io/member-of before the pods are created, therefore the pods must be deleted and recreated for sidecar injection to occur.
MAISTRA-193 Unexpected console info messages are visible when health checking is enabled for citadel.
MAISTRA-158 Applying multiple gateways referencing the same hostname will cause all gateways to stop functioning.
These are the known issues in Kiali:
KIALI-2206 When you are accessing the Kiali console for the first time, and there is no cached browser data for Kiali, the “View in Grafana” link on the Metrics tab of the Kiali Service Details page redirects to the wrong location. The only way you would encounter this issue is if you are accessing Kiali for the first time.
KIALI-507 Kiali does not support Internet Explorer 11. This is because the underlying frameworks do not support Internet Explorer. To access the Kiali console, use one of the two most recent versions of the Chrome, Edge, Firefox or Safari browser.
These limitations exist in Jaeger:
Apache Spark is not supported.
These are the known issues in Jaeger:
TRACING-1166 It is not currently possible to use the Jaeger streaming strategy within a disconnected environment. When a Kafka cluster is being provisioned, it results in a error:
Failed to pull image registry.redhat.io/amq7/amq-streams-kafka-24-rhel7@sha256:f9ceca004f1b7dccb3b82d9a8027961f9fe4104e0ed69752c0bdd8078b4a1076.
TRACING-809 Jaeger Ingester is incompatible with Kafka 2.3. When there are two or more instances of the Jaeger Ingester and enough traffic it will continuously generate rebalancing messages in the logs. This is due to a regression in Kafka 2.3 that was fixed in Kafka 2.3.1. For more information, see Jaegertracing-1819.
The following issues been resolved in the current release:
MAISTRA-2010 AuthorizationPolicy does not support
request.regex.headers field. The
validatingwebhook rejects any AuthorizationPolicy with the field, and even if you disable that, Pilot tries to validate it using the same code, and it does not work.
MAISTRA-1979 Migration to 2.0 The conversion webhook drops the following important fields when converting
SMCP.status from v2 to v1:
Upgrading the operator to 2.0 might break client tools that read the SMCP status using the maistra.io/v1 version of the resource.
This also causes the READY and STATUS columns to be empty when you run
oc get servicemeshcontrolplanes.v1.maistra.io.
MAISTRA-1983 Upgrading to 2.0.0 with an existing invalid
ServiceMeshControlPlane cannot easily be repaired. The invalid items in the
ServiceMeshControlPlane resource caused an unrecoverable error. The fix makes the errors recoverable. You can delete the invalid resource and replace it with a new one or edit the resource to fix the errors. For more information about editing your resource, see [Configuring the Red Hat OpenShift Service Mesh installation].
Maistra-1502 As a result of CVEs fixes in version 1.0.10, the Istio dashboards are not available from the Home Dashboard menu in Grafana. The Istio dashboards still exist. To access them, click the Dashboard menu in the navigation panel and select the Manage tab.
Bug 1821432 Toggle controls in OpenShift Container Platform Control Resource details page do not update the CR correctly. UI Toggle controls in the Service Mesh Control Plane (SMCP) Overview page in the OpenShift Container Platform web console sometimes update the wrong field in the resource. To update a SMCP, edit the YAML content directly or update the resource from the command line instead of clicking the toggle controls.
TRACING-1631 Multiple Jaeger production instances, using same name but within different namespaces, causing Elasticsearch certificate issue. When multiple service meshes were installed, all of the Jaeger Elasticsearch instances had the same Elasticsearch secret instead of individual secrets, which prevented the Elasticsearch Operator from communicating with all of the Elasticsearch clusters.