Prerequisite to getting into Flow

Short feedback loops sets Software Engineering apart from every other profession. Just take a second and think about it. A Cellular Bioengineer applies a substance to a set of cells and has to wait for weeks to know the effect! Meanwhile, we get bored restarting Spring applications that take up to 2 minutes and we built the Quarkus framework (I’m loving it!!) to solve this.

The short feedback loop is a prerequisite to getting into Flow. Let’s take a mini step back and explore what we understand with flow.

Everyone knows what Flow is!

Nonetheless, a concise summary: Flow is the state you get in when solving a task that approximately matches your skill level, that keeps it challenging, but not to the point of frustrating you. And still not that simple as to make it frivolous and get you bored. For the people in this state (let’s call them Flowees) time passes quickly and they are full of love (forealdo), and it’s the time in which the Flowee experiences the most personal development.

The two properties that get a Flowee in Flow are:

1. How challenging a task is and
2. How skilled the Flowee is

Mihaly Csikszentmihalyi the creator of the Flow concept, charted these properties in axis Y and X respectively in the well known “Flow zone chart”

According to the chart inorder to have fun at work and improve yourself continuously you want to stay in the “Loving Life” zone  (i.e. the Flow zone) and let me tell you Flowee, staying there is not an easy feat.

Principles of staying in Flow

As always, the principles are simple (but their implementation is difficult):

1. If you are too skilled and you’re getting bored, then you pick up a more challenging task and
2. If a task challenges you to the point of self-doubt, then you read about the technology or ask a friend to help you grasp the concept.

The principles are visually represented in the image below:

Basically, you need to be aware when a task is too difficult and you need new knowledge, and when your tasks are too easy and you need to increase difficulty (even if you need to change your company).

The technique of Parallel-Flowing

Considering the short feedback loop of our tasks and that you are a serious Flowee (and you shelter yourself from distractions), you can optimise the benefit of “Flow” by using it in multiple areas:

• Learning the latest programming language which is “Hip”
• Expanding your skill set in front-end
• Learning cloud engineering

As a result your personal Flow diagram will look as follows: 

This is a beautiful Parallel-Flowing chart, it is the perfect depiction of a healthy career. If you maintain this, your skills will sky rocket! And let me tell you one tiny secret fellow Flowee, the improvement in skill in one area carries to other areas, directly or indirectly (explaining how this happens indirectly warrants another article).

Caution: Parallel-Flowing on itself does NOT depict a healthy career. Below we have an example of a terrible Parallel-Flowing chart, can you figure out why?

Here Parallel-Flowing is used as an escape mechanism! To clarify, as soon as the tasks get too challenging for the Flowee in area ‘a‘ he switches over to area ‘b‘, where after a period the tasks get difficult as well and then he switches over to area ‘c‘. Where if we look closely we notice another mistake! The tasks are kept at the same difficulty level, so there is no skill development at all.

In general, Flowees that use Parallel-Flowing as an escape mechanism are not aware of their mistake. Moreover they have some logical reasoning “My team needs my help in area ‘c‘” and “Sadly, I am scattered in different areas”, or my favourite “I wish I could get back to ‘b‘ as I was almost cracking it” and then does everything possible to just not get back to that area.

A note from me: I am writing a book, and I definitely am using my blog as an escape mechanism!

Tip: To prevent using Parallel-Flowing as an escape mechanism you need to only follow one rule: “Never, ever leave a task while it is too challenging“. Push through until you get into the Flow zone and then you may switch to another area.

The shrinking of the ‘Flow’ zone

We got back to where we started. My dear Flowee, I am sad to tell you that the area of the ‘Flow’ zone is not static. As you mature in your area of expertise, the region of “Flow” shrinks and proportionally the time spent in it dwindles. As a result every new task is either immediately ‘Boring‘  or if it’s ‘Challenging‘ you invest a short period of time to master the challenge, and it jumps right back into being boring again. Your Flow chart on the best-case scenario will get very Zig-zag-y, as shown below:

In the chart above, we can see that the “Flow” zone has shrunk and mathematically, keeping yourself in the “Zone” is more difficult as it has a smaller surface area. And practically, for most Flowees the chart above is not feasible, as only rare companies will have continuously tough challenges which require getting out of your comfort zone to accomplish the task. In most cases, you’ll end up doing a boring task that needs to be done, and you just take it one boring day at a time.

Tip: A solution to working on some ultra boring tasks is to invite colleagues who don’t know how to, and then to do it together. The act of explaining the intricacies of the subject will represent a challenge on its own and having a person to talk about something you enjoy is always fun.

Conclusion

To summarize make use of Parallel-Flowing. Pay attention when you are using it as an escape mechanism! And enjoy the “Zone” while it still didn’t collapse.

Another note from me: I’ll refine this article sometime in the future. Nonetheless, I hope you liked it in the current crude form as well and would love to hear your thoughts! Best!

The post Parallel-Flowing and the shrinking Flow zone appeared first on Rinor Maloku.

Clone the repository and checkout to the envoy branch by executing the command below:

git clone https://github.com/rinormaloku/k8s-mastery.git \
    && cd k8s-mastery/ \
    && git checkout envoy

Now build the images and run the containers by executing the command below from the k8s-mastery directory:

docker-compose --compatibility up --build

Check that the application is up and running in http://localhost/. It is! That was quick and simple. Let’s get a quick overview of how this works

Envoy Edge Proxy an Overview

The Sentiment Analysis application is composed of three external facing services:

• SA-Frontend is routed to on three cases:
  1. Base path: /
  2. Static files: /static
  3. Images Files: .png, .ico etc.
• SA-WebApp is routed to when the request path is /sentiment.
• SA-Feedback is routed to when the request path is /feedback.

To be able to target the clusters the envoy needs an address, this is dependent on the environment, and in our case, we are using Docker Compose which creates an entry for every service, in other words with the following config in docker-compose.yaml:

  sa-frontend:
    build: 
      context: ./sa-frontend
    image: rinormaloku/sentiment-analysis-frontend:feedback
    networks:
      - envoymesh
    expose:
      - "80"

This way the other services in the network can access sa-frontend using its name sa-frontend. Which is used in the envoy configuration in the next section.

Envoy Static Configuration

In Envoy nomenclature services are called clusters (i.e. there can be more than one service in the cluster), and below we display the configuration for sa-frontend cluster to forward requests to the address sa-frontend, defined in the file external-envoy.yaml:

  clusters:
  - name: sa-frontend
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    http_protocol_options: {}
    hosts:
    - socket_address:
        address: sa-frontend
        port_value: 80

With the cluster defined we need to route to them, which is specified in the filters section for the listener on port 0.0.0.0:80, as shown below:

- match:
    path: "/"
  route:
    cluster: sa-frontend
- match:
    prefix: "/static"
  route:
    cluster: sa-frontend
- match:
    regex: '^.*\.(ico|png|jpg)$'
  route:
    cluster: sa-frontend

And this is it! With this simple config we are:
1. Listening for requests from the downstream (can be any client in our case it’s the browser) in port 80.
2. In case of matching any of the options, we are forwarding the requests to the upstream, i.e instances of sa-frontend cluster.
3. Docker compose has added a DNS entry that forwards requests of sa-frontend to its container.

Summary

In this article, in under 10 minutes, we got the microservice application up and running and we got introduced to the most important Envoy Proxy concepts:

• Downstream,
• Upstream,
• Listeners,
• Clusters,
• Static Configuration

If you like quick and to the point articles like this one, let me know!

The post Routing with the Envoy proxy in under 10 minutes appeared first on Rinor Maloku.

Because, Istio takes these responsibilities from our services and offloads them to the Envoy Proxies, which means that by the time when requests reach our services they are already authenticated and authorized, and we just write the code that provides business value.

Let’s just jump into it!

Authentication with Auth0

As an Identity and Access Management server, we are going to use Auth0, which has a trial option, is intuitive to use, and I just love it! That said the same principles can be used for any OpenID Connect implementation like KeyCloak, IdentityServer and many more.

To get started, navigate to Auth0 Portal login with your preferred account, create a tenant navigate under Applications > Default App and pick up the Domain, as seen in the image below:

Update the file resource-manifests/istio/security/auth-policy.yaml to use your domain:

With this resource, the pilot configures the envoys to authenticate requests before forwarding them to the services: sa-web-app and sa-feedback. At the same time, this is not applied to the envoys of the service sa-frontend enabling us to get the frontend unauthenticated. To apply the Policy, execute the command:

$ kubectl apply -f resource-manifests/istio/security/auth-policy.yaml
policy.authentication.istio.io "auth-policy" created

Go back to the page and make a request, you will see that it will end in 401 Unauthorized, now let’s forward users from the frontend to authenticate with Auth0.

Authenticating Requests with Auth0

To Authenticate requests of an End User we need to create an API in Auth0 that represents the authenticated services namely: reviews, details, and ratings. To create an API, navigate to Auth0 Portal > APIs > Create API, as seen in figure 2.

The important information here is the Identifier later used in the script as:

• Audience: {YOUR_AUDIENCE}

And the rest of the needed details are under Applications in the Auth0 Portal and then select the Test Application created automatically with the same name as the API.

Here note down:

• Domain: {YOUR_DOMAIN}
• Client Id: {YOUR_CLIENT_ID}

Scroll down in the Test Application to the Allowed Callback URLs text field, where we specify the URL where the call should be forwarded after the Authentication is completed, in our case it is:

http://{EXTERNAL_IP}/callback

Add for the Allowed Logout URLs add the following URL:

http://{EXTERNAL_IP}/logout

Let’s move over to the frontend.

Updating the Frontend

Switch to the branch auth0 of the repository. In this branch the frontend contains code changes to forward users to Auth0 for authentication and uses the JWT Token in requests to the other services as shown below:

To update the frontend to use your tenant’s details navigate to the file sa-frontend/src/services/Auth.js and replace the following values, with the ones we noted down earlier:

The application is ready, specify your docker user id in the command below and then build and deploy the changes:

$ docker build -f sa-frontend/Dockerfile \ 
    -t {DOCKER_USER_ID}/sentiment-analysis-frontend:istio-auth0 sa-frontend

$ docker push {DOCKER_USER_ID}/sentiment-analysis-frontend:istio-auth0

$ kubectl set image deployment/sa-frontend \ 
    sa-frontend={DOCKER_USER_ID}/sentiment-analysis-frontend:istio-auth0

Give the app a try! You will be forwarded to Auth0 where you have to log in (or register) and are forwarded back to the page and can make Authenticated Requests. Meanwhile, if you try the earlier curl commands you will get a 401 Status Code indicating that the request was Unauthorized.

Let’s go one step further and authorize requests.

The post Authentication in Istio appeared first on Rinor Maloku.

Authentication enables us to know who a user is, but we need the authorization to know what they can access. Istio provides the tools for this as well!

As an example, we’ll create two groups of users (shown in figure 24):

• Users: with access to only the SA-WebApp and SA-Frontend service.
• Moderators: who can access all three services.

To create the user groups, we will use the Auth0 Authorization extension, and then using Istio we will provide them with different levels of access.

Install and Configure Auth0 Authorization

In the Auth0 portal navigate to Extensions and install the ‘Auth0 Authorization’ extension. After the installation, navigate to the Authorization Extension and configure by clicking your tenant in the top right corner and selecting the menu option “Configuration”. Enable Groups and click the button Publish rule.

Create Groups

In the Authorization Extension navigate to Groups and create the Moderators group. Meanwhile, we will treat all Authenticated Users as Regular Users and as such, there is no need to create an additional group.

Select the Moderators group and click Add Members, add your main account. Leave some users without any Group to verify that for them access is forbidden. (You can register new Users manually in Auth0 Portal > Users > Create User)

Add Group Claim to the Access Token

Users are added to the groups, but this information won’t be reflected to the access token. To stay OpenID Connect conformant and at the same time to return the groups we need to add custom namespaced claims to the token. This can be done using Auth0 rules.

To create a rule in the Auth0 Portal navigate to Rules, click “Create Rule” and pick an empty rule from the templates.

Paste the code below and save the new rule named as “Add Group Claim”.

function (user, context, callback) {
    context.accessToken['https://sa.io/group'] = user.groups[0];
    return callback(null, user, context);
}

Note: This code picks the first group of the user as defined in the Authorization Extension and adds it to the access token as a custom namespaced claim.

Go back to the Rules page and verify that you got two roles in this order:

• auth0-authorization-extension
• Add Group Claim

The order matters because the groups field is retrieved asynchronously by the auth0-authorization-extension rule and then it is added as a namespaced claim by the second rule, resulting in the following access token:

{
  "https://sa.io/group": "Moderators",
  "iss": "https://bookinfo.eu.auth0.com/",
  "sub": "google-oauth2|196405271625531691872"
// [shortened for brevity]
}

Now we must configure the Envoy proxies to verify user access by extracting the group from the claim https://sa.io/group in the returned access token. That is the topic of the next section, let’s move over there.

Configuring Authorization in Istio

For Authorization to kick in we need to Enable RBAC for Istio. To do so apply to the Mesh the following configuration:

1. Enables RBAC only for the services and or namespaces specified in the Inclusion field.
2. Include the list of services specified.

Apply the configuration by executing the command below:

$ kubectl apply -f resource-manifests\istio\security\enable-rbac.yaml
rbacconfig.rbac.istio.io "default" created

Now all services require Role-Based Access Control, in other words access to all services is denied and will result in the response “RBAC: access denied”. Enabling access to authorized users will be the topic of the next sections.

Configuring Regular User access

All users should be able to access the SA-Frontend and SA-WebApp services, this is achieved with these Istio’s resources:

• ServiceRole: specifies the permissions that a user has and
• ServiceRoleBinding: specifies to whom a ServiceRole applies.

For regular users we will allow access to the specified services:

And using the regular-user-binding we will apply the ServiceRole to all visitors of our page:

Oh! All users this means that unauthenticated users can SA WebApp? No, the policy will still check the validity of the JWT token.

Apply the configurations:

$ kubectl apply -f resource-manifests/istio/security/user-role.yaml
servicerole.rbac.istio.io/regular-user created
servicerolebinding.rbac.istio.io/regular-user-binding created

Configuring Moderator User access

For our moderators we want to enable access to all the services:

But we want to bind it only to users whose Access Token has the claim https://sa.io/group equal to the value Moderators.

To apply the configurations, execute:

$ kubectl apply -f resource-manifests/istio/security/mod-role.yaml
servicerole.rbac.istio.io/mod-user unchanged
servicerolebinding.rbac.istio.io/mod-user-binding unchanged

Due to caching in the envoys it could take a couple of minutes for the Authorization rules to go in effect, but after that, you will be able to verify that Users and Moderators have different levels of access.

Seriously have you ever seen a simpler process, zero-effort scalable authentication and authorization concept? Neither have I. Let’s move on to the conclusion.

The post Authorization in Istio appeared first on Rinor Maloku.

Istio? Let’s call it Beast-io!! Because it’s such a BEAST! It provided us with:

• Observability over our services by answering what services are running, how are they performing and how are they related, using Kiali.
• Metric collection and visualization, with Prometheus and Grafana, out of the box!
• Request tracing with Jaeger (german for Hunter).
• Full and fine-grained control over the network traffic, enabling Canary Deployments, A/B Testing, Shadowing.
• Easy implementation of Retries, Timeouts, CircuitBreakers.
• All of these reduces the overhead of introducing new services to the cluster.
• Added Authentication and Authorization to all microservices in different programming languages, without code changes on server side.
• Reduced the complexity of Authorization to two configuration files.

Beastio (yes cheezy but I roll with it) enables your team, once again to provide business value and focus on the domain, without the overhead of services, which is taken care by Istio.

I want to thank you for joining me on this voyage. Since you read this far I know that you loved this article and would be interested in more. I write articles that go to this depth of detail for new technologies every 3 months. You can always expect an example application, hands-on practice and a guide that provides you with the right tools and knowledge to tackle any real-world project.

To stay in touch and not miss any of those subscribe to my newsletter and follow me on Twitter.

The post Istio series Summary appeared first on Rinor Maloku.

For demonstration purposes, we will continue to use the buggy version of sa-logic, where the random failures simulate the unreliability of the network.

The buggy service has a one-third chance of taking too long to respond, one-third chance of ending in an Internal Server Error and the rest complete successfully.

To alleviate these issues and improve the user experience we could:

1. Timeout if the service takes longer than 8 seconds and
2. Retry on failed requests.

This is achieved with the following resource definition:

1. The request has a timeout of 8 seconds.
2. We attempt 3 times
3. An attempt is marked as failed if it takes longer than 3 seconds.

This is an optimization; the user won’t be waiting for more than 8 seconds and we retry three times in case of failures, increasing the chance of resulting in a successful response.

Apply the updated configuration with the command below:

$ kubectl apply -f resource-manifests/istio/retries/sa-logic-retries-timeouts-vs.yaml
virtualservice.networking.istio.io/sa-logic configured

And check out the Grafana graphs to view the improvement in success rate, as shown in figure 1.

Before moving into the next section delete sa-logic-buggy and the VirtualService by executing the command below:

$ kubectl delete deployment sa-logic-buggy
deployment.extensions "sa-logic-buggy" deleted
$ kubectl delete virtualservice sa-logic
virtualservice.networking.istio.io "sa-logic" deleted

Circuit Breaker and Bulkhead patterns

Two important patterns in Microservice Architectures that enable self-healing of the services.

The Circuit Breaker is used to stop requests going to an instance of a service deemed as unhealthy and enable it to recover, and in the meantime client’s requests are forwarded to the healthy instances of that service (increasing success rate).

The Bulkhead pattern isolates failures from taking the whole system down, to take an example, Service B is in a corrupt state and another service (a client of Service B) makes requests to Service B this will result that the client will uses up its own thread pool and won’t be able to serve other requests (even if those are not related to Service B).

I will skip implementations of these patterns because I’m way too excited to showcase Authentication and Authorization, though for implementations of those patterns you can check the official docs.

The post Timeouts, Retries and CircuitBreakers with Istio appeared first on Rinor Maloku.

Let’s continue with the same buggy subset of sa-logic to demonstrate canary deployments, by boldly sending 20% of the users to the buggy version (this represents the canary deployment) and 80% to the healthy service by applying the VirtualService below:

1. Weight specifies the percentage of requests to be forwarded to the destination or subset of the destination.

Update the previous sa-logic virtual service configuration using the following commands:

$ kubectl apply -f resource-manifests/istio/canary/sa-logic-subsets-canary-vs.yaml
virtualservice.networking.istio.io/sa-logic configured

We immediately see that some of our requests are failing:

$ while true; do \
    curl -i http://$EXTERNAL_IP/sentiment -H "Content-type: application/json" \
    -d '{"sentence": "I love yogobella"}' \
    --silent -w "Time: %{time_total}s \t Status: %{http_code}\n" -o /dev/null; \
    sleep .1; done
Time: 0.153075s          Status: 200
Time: 0.137581s          Status: 200
Time: 0.139345s          Status: 200
Time: 30.291806s         Status: 500

VirtualServices enable Canary Deployments and with this method, we reduced potential damages to 20% of our user base. Beautiful! Now we can use Shadowing and Canary Deployments every time we are insecure about our code, basically always .

The post Canary Deployments with Istio appeared first on Rinor Maloku.

To test out this feature lets create a second instance of SA-Logic that is buggy by executing the command below:

$ kubectl apply -f resource-manifests/kube/shadowing/sa-logic-service.buggy.yaml

Execute the following command and verify that all instances are labeled with the respective versions and additionally with app=sa-logic:

$ kubectl get pods -l app=sa-logic --show-labels
NAME                              READY   LABELS
sa-logic-568498cb4d-2sjwj         2/2     app=sa-logic,version=v1
sa-logic-568498cb4d-p4f8c         2/2     app=sa-logic,version=v1
sa-logic-buggy-76dff55847-2fl66   2/2     app=sa-logic,version=v2
sa-logic-buggy-76dff55847-kx8zz   2/2     app=sa-logic,version=v2

Because the service sa-logic targets pods labeled with app=sa-logic, any incoming requests will be load-balanced between all instances, as shown in figure 1.

But we want requests to be routed to the instances with version v1 and mirrored to the instances with version v2, as shown in figure 2.

This is achieved using a VirtualService in combination with a DestinationRule, where the destination rule specifies the subsets and VirtualService routes to the desired subset.

Specifying Subsets with Destination Rules

We define the subsets with the following configuration:

1. Host defines that this rule applies only when routing has occurred towards sa-logic service
2. Subset name used when routing to instances of a subset.
3. Label defines the key-value pairs that need to match for an instance to be part of the subset.

Apply the configuration executing the command below:

$ kubectl apply -f resource-manifests/istio/shadowing/sa-logic-subsets-destinationrule.yaml
destinationrule.networking.istio.io/sa-logic created

With the subsets defined we can move on and configure a VirtualService to apply to requests towards sa-logic where the requests are:

1. Routed to the subset named v1 and,
2. Mirrored to the subset named v2.

And this is achieved with the manifest below:

As everything is self-explanatory let’s just see it in action:

$ kubectl apply -f resource-manifests/istio/shadowing/sa-logic-subsets-shadowing-vs.yaml
virtualservice.networking.istio.io/sa-logic created

Add some load by executing the following command:

$ while true; do curl -v http://$EXTERNAL_IP/sentiment \ 
    -H "Content-type: application/json" \ 
    -d '{"sentence": "I love yogobella"}'; \
    sleep .8; done

Check the results in Grafana, where we can see that the buggy version is failing about 60% of the requests, but none of the failures affected the end-users as they were responded by the currently active service.

In this section we saw for the first time a VirtualService that was applied to the envoys of our services, to be clearer, when the sa-web-app makes a request towards sa-logic this goes through sa-web-apps envoy, which via the VirtualService is configured to route to the subset v1 and mirror to the subset v2 of the sa-logic service.

I can see you thinking “Darn man Virtual Services are amazing!”, and they are going to get even better!

The post Shadowing – VirtualServices in Practice appeared first on Rinor Maloku.

Using the Envoy’s Istio provides a host of new capabilities to your cluster enabling:

• Dynamic request routing: Canary deployments, A/B testing,
• Load balancing: Simple and Consistent Hash balancing
• Failure Recovery: timeouts, retries, circuit breakers.
• Fault Injection: delays, abort requests etc.

In this series, we will showcase these capabilities in our application and get introduced to new concepts along the way. The first concept we will delve into is DestinationRules and using those to enable A/B Testing.

A/B Testing – Destination Rules in Practice

A/B Testing is used when we have two versions of an application (usually those differ visually) that we are not 100% sure which will increase user interaction and so we try both versions at the same time and collect metrics.

To demo this let’s deploy a second version of the frontend (a green button instead of the white one).  By executing the command below:

$ kubectl apply -f resource-manifests/kube/ab-testing/sa-frontend-green-deployment.yaml
deployment.extensions/sa-frontend-green created

The deployment manifest for the green version differs in two points:

1. The image is based on a different tag: <image>:istio-green and
2. Pods are labeled with version: green.

And as both deployments have the label app: sa-frontend requests routed by the virtual service sa-external-services to the service sa-frontend will be forwarded to all of them and will be load balanced using the round robin algorithm, which causes the issue presented in figure 1.

The files are not found because they are named differently in the different versions of the app. Let’s verify that:

$ curl --silent http://$EXTERNAL_IP/ | tr '"' '\n' | grep main
/static/css/main.c7071b22.css
/static/js/main.059f8e9c.js

$ curl --silent http://$EXTERNAL_IP/ | tr '"' '\n' | grep main
/static/css/main.f87cd8c9.css
/static/js/main.f7659dbb.js

This means that for our app to work we need to introduce the restriction that “the version of the app that served the index.html, must serve subsequent files”.

We’ll achieve this using Consistent Hash Loadbalancing, which is the process that forwards requests from the same client to the same backend instance, using a predefined property, like an HTTP header.

The next Istio resource makes Consistent Hash Loadbalacing possible. Let’s dive into DestionatioRules.

DestinationRules

After a request gets routed by the VirtualService to the correct service, then using DestinationRules we can specify policies that apply to the traffic intended for the instances of this Service, as presented in figure 2.

Note: Figure 2 visualizes how Istio resources are affecting the network traffic, in an easily understandable way. To be precise the decision to which pod to forward the request is done directly from the Ingress Gateway’s Envoy. And all the resources are configuring it.

Back to the main topic, using Destination Rules we can configure load balancing to have session affinity and ensure that the same user is responded by the same instance of the service. This is achievable with the following configuration:

1. Generate a consistent hash according to the contents of the “version” header.

Apply the configuration by executing the command below and give it a try!

$ kubectl apply -f resource-manifests/istio/ab-testing/destinationrule-sa-frontend.yaml
destinationrule.networking.istio.io/sa-frontend created

Execute the command below and verify that you get the same files when specifying the version header:

$ curl --silent -H "version: yogo" http://$EXTERNAL_IP/ | tr '"' '\n' | grep main

Note: To test from the browser you can use this extension to add different values to the version header.

DestinationRules have even more LoadBalancing capabilities for all the details check out the official docs, and besides load balancing destination rules enable us to define service subsets, something very useful in upcomming sections.

Before moving on to explore VirtualService in more detail, remove the green version of the app and the destination rule by executing the commands below:

$ kubectl apply -f resource-manifests/kube/ab-testing/sa-frontend-green-deployment.yaml
deployment.extensions "sa-frontend-green" deleted
$ kubectl delete -f resource-manifests/istio/ab-testing/destinationrule-sa-frontend.yaml
destinationrule.networking.istio.io "sa-frontend" deleted

The post A/B Testing – DestinationRules in Practice appeared first on Rinor Maloku.

By intercepting all network communication Istio is fed with metrics and data that can be used to gain observability of the whole application. Kiali, an open source project uses this data to provide the answer to the question: What microservices are part of my Istio service mesh and how are they connected?

Kiali – Observability

Before installing Istio in our cluster we created a secret for kiali (and another for grafana) where we specified that both user and password are admin. To access Kiali’s Admin UI execute the command below:

$ kubectl port-forward \
    $(kubectl get pod -n istio-system -l app=kiali \
    -o jsonpath='{.items[0].metadata.name}') \
    -n istio-system 20001

And open http://localhost:20001/ login using “admin” (without quotes) for user and password. There is a ton of useful features, like checking the configurations of Istio Components, visualizing services according to the information collected by intercepting network requests. (i.e. it answers who is calling who, which version of a service has failures etc.). Take some time to checkout Kiali before moving on to the next goodie, visualizing the metrics collected by Grafana!

Grafana – Metrics Visualization

The metrics collected by Istio are scraped into Prometheus and visualized using Grafana. To access the Admin UI of Grafana execute the command below and open http://localhost:3000.

$ kubectl -n istio-system port-forward \
    $(kubectl -n istio-system get pod -l app=grafana \
    -o jsonpath='{.items[0].metadata.name}') 3000

On the top left click the menu Home and select Istio Service Dashboard and on the top left corner select the service starting with sa-web-app, you will be presented with the collected metrics, as seen on the image below:

Holly molly that’s an empty and totally non-exciting view. Let’s cause some load by executing the command below:

$ while true; do \
    curl -i http://$EXTERNAL_IP/sentiment -H "Content-type: application/json" \
    -d '{"sentence": "I love yogobella"}' \
    --silent -w "Time: %{time_total}s \t Status: %{http_code}\n" -o /dev/null; \
    sleep .8; done

Now we have prettier graphs to show to the management! And additionally (though less importantly) we have the amazing tools of Prometheus for monitoring and Grafana for visualizing the metrics that enables us to know the performance, health and the improvement or degradation of our services throughout time!

Note: Leave the above command running as it will be important for upcoming sections.

Lastly, we will investigate Tracing requests throughout services.

Tracing

The more services we have the harder it gets to pinpoint the cause of failure. Let’s take the simple case in the image below:

The request goes in, failure goes out, what was the cause? The first service? Or the second? Exceptions are in both, Let’s get to the logs of each. How many times do you find yourself doing this? Our job feels more like Software Detectives than Developers.

That said this is a prevalent problem in Microservices and it’s solved using Distributed Tracing Systems where the services pass a unique header to each other and then this information is forwarded to the Distributed Tracing System where the request trace is put together. An example is presented in figure 4.

In Istio tracing is supported by the Jaeger Tracer that implements the OpenTracing API, a vendor-neutral framework. To get access the Jaegers UI execute the command below:

$ kubectl port-forward -n istio-system \
    $(kubectl get pod -n istio-system -l app=jaeger \
    -o jsonpath='{.items[0].metadata.name}') 16686

Then open the UI in http://localhost:16686, select the sa-web-app service, if the service is not shown on the dropdown generate some activity on the page and hit refresh. Afterward click the button Find Traces, which displays the most recent traces, select any and a detailed breakdown of all the traces will be shown, as presented in figure 5.

The trace shows:

1. The request comes to the istio-ingressgateway (it’s the first contact with one of the services so the Trace ID is generated) then the gateway forwards the request to the sa-web-app
2. In the sa-web-app service the request is picked up by the Envoy container and a span child is created (that’s why we see It in the traces) and forwarded to the sa-web-app
3. Here the method sentimentAnalysis handles the request. These traces are generated by the application, meaning that code changes were required).
4. From where a POST request is started to sa-logic. Trace ID needs to be propagated by sa-web-app.
5. …

Note: At the 4^th point our application needs to pick up the headers generated by Istio and pass them down on the next requests. This is shown in figure 6. where our application was responsible to propagate the headers under points B.

Istio does the main heavy lifting by generating the headers on incoming requests, creating new spans on every sidecar, propagating them, but without our services propagating the headers as well, the chain will be broken and the full trace will be lost.

The headers that we need to propagate are:

x-request-id
x-b3-traceid
x-b3-spanid
x-b3-parentspanid
x-b3-sampled
x-b3-flags
x-ot-span-context

Despite it being a simple task, there are already many libraries that simplify the process, for example in the sa-web-app the RestTemplate client is instrumented to propagate the headers by simply adding the Jaeger and OpenTracing libraries in the , and for getting additional application traces configuring the Environment Variables for the Jaeger Host in the Kubernetes Deployment.

Note: The Sentiment Analysis app showcases implementations for Flask, Spring and ASP.NET Core.  

Now after investigating what we get out of the box (and partially out of the box ) let’s get to the main topic here, fine-grained routing, managing network traffic, security and more!

The post Istio out of the box: Kiali, Grafana & Jaeger appeared first on Rinor Maloku.

The VirtualService instructs the Ingress Gateway how to route the requests that were allowed into the cluster.

For our application requests coming through the http-gateway must be routed to the sa-frontend, sa-web-app and sa-feedback services (shown in figure 1).

Let’s break down the requests that should be routed to SA-Frontend:

• Exact path / should be routed to SA-Frontend to get the Index.html
• Prefix path /static/* should be routed to SA-Frontend to get any static files needed by the frontend, like Cascading Style Sheets and JavaScript files.
• Paths matching the regex ^.*\.(ico|png|jpg)$ should be routed to SA-Frontend as it is an image, that the page needs to show.

This is achieved by the following configuration:

Additionally, important here are the following points:

1. This VirtualService applies to requests coming through the http-gateway.
2. Destination defines the service where the requests are routed.

Note: The configuration above is in the file sa-virtualservice-external.yaml, it contains the configuration to route to SA-WebApp and SA-Feedback but was shortened here for brevity.

Apply the VirtualService with the command below:

$ kubectl apply -f resource-manifests/istio/sa-virtualservice-external.yaml
virtualservice.networking.istio.io "sa-external-services" created

Note: When we apply this resource (and actually all Istio CRD resources) the Kubernetes API Server creates an event received by Istio’s Control Plane which then applies the new configuration to the envoys (istio proxies, sidecar proxies) of every pod. And the Ingress Gateway controller is another Envoy which is configured by the Control Plane, visually presented in figure 2.

The Sentiment Analysis app is accessible on http:/{{EXTERNAL-IP}}/. If you get a Not Found status, do not worry sometimes it takes a couple of minutes for the configuration to go in effect and update the envoy caches.

Before moving into the next section generate some traffic needed to demonstrate what we get out of the box from Istio. It’s insane! Meet me in the next article!

The post Istio in Practice – Routing with VirtualService appeared first on Rinor Maloku.

A best practice for allowing traffic into your cluster is through Istio’s Ingress Gateway which positions itself at the edge of the cluster and on incoming traffic enables Istio’s features like routing, security, monitoring.

During Istio’s installation, the Ingress Gateway component and a service that exposes it externally were installed into the cluster, to get its External IP execute the command below:

$ kubectl get svc -n istio-system -l istio=ingressgateway
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)             
istio-ingressgateway   LoadBalancer   10.0.132.127   13.93.30.120   80:31380/TCP,443[...]

In the continuation of this article we will access the application on this IP (referred to as the EXTERNAL-IP), for convenience, save it in a variable by executing the command below:

$ EXTERNAL_IP=$(kubectl get svc -n istio-system \
   -l app=istio-ingressgateway \
   -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')

Try to reach the IP in your browser and you will get a Service Unavailable error, as by default Istio doesn’t allow any incoming traffic until we define a Gateway.

The Gateway Resource

A Gateway is a Kubernetes CustomResourceDefinition defined upon Istio’s installation in our cluster that enables us to specify the Ports, Protocol and Hosts for which we want to allow incoming traffic.

In our scenario, we want to allow HTTP traffic on Port 80, for all hosts. Achieved with the following resource definition:

All the configuration is self-explanatory besides the selector istio: ingressgateway. Using this selector, we can specify to which Ingress Gateway to apply the configuration, and in our case, it is the default ingress gateway controller installed on Istio setup.

Apply the above configuration by executing the command below:

$ kubectl apply -f resource-manifests/istio/http-gateway.yaml 
gateway.networking.istio.io "http-gateway" created

The gateway now allows access in port 80 but it has no concept where to route the requests.

That is achieved using Virtual Services, which is the main topic of the next article, let’s get over there!

The post Istio in Practice – Ingress Gateway appeared first on Rinor Maloku.