Driving Google Cloud Platform via Python : Scaling and Design considerations, and how to extract the best performance
Google Cloud Platform (GCP) is a cloud computing service offered by Google, on the same infrastructure it uses for its services such as Google Search and Youtube. GCP provides a Python SDK which can be used to automate and drive the various features and services, allowing developers to deploy and manage their application and infrastructure.
As we started adding support to run our Application Delivery Controller (ADC) product, Avi Vantage, on GCP via the Python SDK, we faced some interesting challenges :
- Python’s threading mechanism, “twisted”, had some scale limitations that limited the solution performance
- GCP API’s inherent serialization requirement introduced a significant performance delay, especially in scale scenarios typically required for ADC provisioning on GCP..
In this talk, we share how we refactored our design and improved performance, using :
- Async Python constructs for better multi threading performance.
- GCP’s batching mechanism for their APIs
We will share some benchmark results between the various design choices, and the resulting performance gains.
- GCP overview / constructs that we are interested in (Eg Network, Route provisioning)
- Challenges and considerations on Scaling GCP
- Solutions and Best Practices incorporated to improve performance
Ramkumar Sankar (Ramku) is the Head, Avi Networks India. He focuses on High Availability and Software Defined Controller Architecture.