Call for round the year submissions for Rootconf in 2020

Submit a proposal at any time in the year on DevOps, infrastructure security, cloud, and distributed systems. We will find you a suitable opportunity to share your work.

The curious case of slow/fast grequests code

Submitted by saurabh hirani (@saurabh-hirani) on Oct 1, 2019

Section: Full talk Technical level: Intermediate Session type: Demo Status: Submitted

Abstract

If you have made a single HTTP request in Python, you have probably used the requests module.

If you have made concurrent HTTP requests in Python, you have probably used the grequests module.

I ran into an issue in production which made a grequests based code execute HTTPS GET requests serially without utilising the concurrency goodness promised by grequests. What was more interesting is that depending on the Python modules installed on my system, the same codebase ran fast or slow.

Tumbling down that rabbit hole made me understand Python profiling, function tracing, understanding how gevent works and opening a pull request to fix the issue with a gevent based Python module.

Outline

In this talk, I want to share my learnings with the audience through a working dockerized demo which showcases the following stages for the same codebase while explaining why each stage behaves the way it does:

  • Stage-0 - Trigger which led to the exploration - May run slow/fast for Python 2.7 and slow/fast for Python 3.7 depending on the modules installed on your system.
  • Stage-1 - Baseline with dockerized environment - Runs fast both for Python 2.7 and Python 3.7
  • Stage-2 - Detection of slowness and verification - Runs slow both for Python 2.7 and Python 3.7
  • Stage-3 - Python 2.7 fix - Runs fast for Python 2.7 but slow for Python 3.7
  • Stage-4 - Python 3.7 fix - Patching a gevent based Python module which fixes the issue and makes Python 3.7 fast again.

Please check out the preview video to get a sneak peek into the issue and watch the same codebase run slow/fast depending on the Python version chosen.

Requirements

Curiosity + programming basics.

Speaker bio

  • Saurabh Hirani normally doesn’t talk about himself in third person, but when he does, he is writing a speaker bio.
  • He enjoys programming and infrastructure automation.
  • He is currently working as a Site Reliability Engineer at Autodesk and has spoken at Python meetups, Rootconf conferences and firmly believes in sharing hard earned knowledge that everyone can benefit from.

Slides

http://saurabh-hirani.github.io/slides/decks/grequests-https-talk/main.html#/

Preview video

https://www.youtube.com/watch?v=6bMCMfyZmdg

Comments

{{ gettext('Login to leave a comment') }}

{{ gettext('You need to be a participant to comment.') }}

{{ formTitle }}
{{ gettext('Post a comment...') }}
{{ gettext('New comment') }}

{{ errorMsg }}