Rootconf Hyderabad edition

On SRE, systems engineering and distributed systems

Tickets

Service mesh and the future of microservices at scale.

Submitted by Sudipta Biswas (@sudiwas) on Sunday, 5 May 2019

Section: Full talk Technical level: Intermediate Session type: Lecture Status: Rejected

Abstract

As monolithic applications are broken down into self functioning units in the form of microservices, the paradigm of deployments for these applications have become truly distributed. While it provides the flexibility of operating /updating individual modules for easy maintenance, loose coupling and faster deployments, it also posses the challenge of managing them at scale. This gives rise to some very interesting use cases that were not possible earlier. Some examples of these are fault injection, circuit breakers, distributed tracing etc.

At Avi Networks we are developing a Universal Service Mesh using technologies like Istio, Kubernetes and OpenShift that is truly built for managing multi-Cluster and multi-cloud use cases from a single control plane.

Through this talk, we will look into the future of micro services based applications running at scale. We will present the evolving service mesh paradigm and discuss projects like Istio, OpenTracing and Kubernetes. We will additionally help with understanding how they work together to implement a service mesh. This in turn would help in visualisation, greater control and fault tolerance of truly distributed applications managed from a single control plane across sites in the cloud. Finally, we will demonstrate our experience of working with these technologies and how they can truly help solve network traffic issues and provide rich analytics of data for finer control in a distributed application environment.

Outline

  • Evolving software architecture from monolith to Microservices.
  • Challenges posed in managing microservices at scale.
  • Importance of traffic visualization, analytics and the role of load balancers.
  • Introduction to Service mesh.
  • Introduction to Istio, OpenTracing on top of Kubernetes for understanding a service mesh in reality.
  • Evolving future of service mesh for multi-cloud / multi-cluster use cases.
  • A look at how the industry is trying to solve the service mesh problem along with a look of AVI Network’s work in this domain.
  • Sharing some practical experience of working with a service mesh and problems at scale.

Speaker bio

Sudipta Biswas works at AVI Networks as a software designer leading the effort of building AVI’s Universal Service Mesh built on top of Istio and Kubernetes. Prior to this, Sudipta has worked extensively on product offerings around docker and kubernetes as a tech lead. He has also served as the maintainer of OpenStack Zun (OpenStack’s container project) besides contributing to core components of OpenStack for around 3 years. He has a total of 12 years of work experience and has worked extensively on cloud and related technologies for over 7 years. Outside of work, he is the vocalist of the Rock Band from Bangalore called Dark Project.

Slides

https://www.slideshare.net/secret/NKwsGnUb1r8GA

Preview video

https://www.youtube.com/watch?v=-m8CCvpEECQ&feature=youtu.be

Comments

  •   Zainab Bawa (@zainabbawa) Reviewer 10 months ago

    Can you help explain why service mesh? What problem(s) do(es) service mesh solve?

    •   Sudipta Biswas (@sudiwas) Proposer 10 months ago (edited 10 months ago)

      If we agree on the point on adoption of microservices growing over traditional monolithic applications then we would realize that there’s essentially a requirement to be able to control different parts of the applications from the infrastructure itself as opposed to something that the application owners need to care about. Examples of one such simple use case is, having the ability to trace an API call that hops from one service to another and publish telemetry results for the same. By deploying a Service Mesh (like Istio), one can embedd a ‘sidecar’ container proxy to an existing application POD (kubernetes term for a self functioning unit of a microservice) without affecting any business logic of the application itself. The ‘sidecar’ can work as a proxy for the POD and in turn have the ability to get programmed via a centralized control plane. Now imagine, all PODs in an application have this side car fitted. The developer of these applications - didn’t have to care about them but the infra owner just inserted them against all the application pods. Now there are multiple possibilities that can happen as a result of this:

      • A fully policy driven way of controlling microservice behavior at scale.
      • Report application telemetry to a centralized control plane leading to better visualization and observability of applications.
      • External programming of rate-limiters, circuit breakers, application call retries without modifying anything inside the application logic.

      Besides this, as the multi-cloud/multi-cluster use cases become more and more popular - there’s a need to essentially have policy driven ways of communication between services that are placed across clusters/across clouds. A service mesh’s future lies in making such use cases a reality.

      For more on service mesh you could refer to the following links:
      https://avinetworks.com/what-are-microservices-and-containers/
      https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh
      https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/
      https://istio.io/docs/concepts/what-is-istio/
      https://aws.amazon.com/app-mesh/

  •   Zainab Bawa (@zainabbawa) Reviewer 10 months ago

    What are the alternative approaches, and why choose service mesh only?

    •   Sudipta Biswas (@sudiwas) Proposer 10 months ago (edited 10 months ago)

      Service mesh is essential for enterprises for making their distributed applications behave reliably in production. With micro-services - the service to service communication becomes the fundamental determining factor behind the application behavior in production. Service mesh gives you essentially a way to program the distributed applications without affecting their business logic.

      If one has the need to have observability , understanding service dependencies, managing security or managing application fragility - a service mesh is a must. We don’t imagine the world to run only on microservices, there can be pieces of software that still run inside legacy Virtual Machines. In such scenarios the service mesh acts as a fabric that connects these two worlds (containers and VMs) using policies.

  •   Zainab Bawa (@zainabbawa) Reviewer 9 months ago

    Thanks for the clarifications, Sudipta. One challenge is that the proposal should not become a pitch for AVI Networks’ solution, leading to the danger that this will become an advertisement. Instead, the focus should be on the problem statement, different approaches to solving the problem of which AVI Networks’ approach is one of them, and comparisons.

    Also, the takeaways slides don’t really seem like takeaways. They appear more like summary points from the overall discussion. A takeaway is an insight, something which was not obvious to participants, but which you are helping make evident. Some thinking is required on this front too.

  •   Zainab Bawa (@zainabbawa) Reviewer 9 months ago

    Thanks for the clarifications, Sudipta. One challenge is that the proposal should not become a pitch for AVI Networks’ solution, leading to the danger that this will become an advertisement. Instead, the focus should be on the problem statement, different approaches to solving the problem of which AVI Networks’ approach is one of them, and comparisons.

    Also, the takeaways slides don’t really seem like takeaways. They appear more like summary points from the overall discussion. A takeaway is an insight, something which was not obvious to participants, but which you are helping make evident. Some thinking is required on this front too.

  •   Aditya Patawari (@adityapatawari) 9 months ago

    Hi folks.
    Thanks for the proposal. What you have described here is more or less what Istio does out of the box. Do you have anything over and above that? For example, you could present a case study about the improvements that you saw after deploying Istio, backed with actual numbers. If Avi networks is doing something different that what standard Istio does, then we would be interested in that. Please note that we’d like to ensure that the tech you are presenting is open-source. Therefore, ensure that whatever you are doing on top of Istio is either a clever configuration or open source code.

  •   Sudipta Biswas (@sudiwas) Proposer 9 months ago (edited 9 months ago)

    Thanks guys. The tech AVI networks is building above istio is pretty much open source and you can find the project details here: https://github.com/avinetworks/servicemesh

    The use cases AVI can solve are complimentary to istio or beyond Istio since AVI is a enterprise grade software load balancer and a multi cloud solution.

    However, with that said i dont think the intention of this talk was ever to make it a company pitch or delve into AVI’s offerings in specific. I think it is more intended towards talking about service mesh as a paradigm, why is it necessary, looking at use cases it can solve, debate on some of the upcoming approaches and so forth.

    I can relook at the takeaways slide mentioned the comment above.

    •   Zainab Bawa (@zainabbawa) Reviewer 9 months ago

      The structure of the talk comes across as a pitch. Also, if the intent is to get people to understand the sevice mesh paradigm, then it has to be done through a case study rather than going over the theory. An audience of practitioners at Rootconf are aware about service mesh. What is it that you are building on top of their understanding already?

Login with Twitter or Google to leave a comment