Call for round the year submissions for Rootconf in 2019

Submit a proposal at any time in the year on DevOps, infrastructure security, cloud, and distributed systems. We will find you a suitable opportunity to share your work.

Propose a session

The curious case of slow/fast grequests code

Submitted by saurabh hirani (@saurabh-hirani) on Tuesday, 1 October 2019


Preview video

Section: Full talk Technical level: Intermediate Session type: Demo

Abstract

If you have made a single HTTP request in Python, you have probably used the requests module.

If you have made concurrent HTTP requests in Python, you have probably used the grequests module.

I ran into an issue in production which made a grequests based code execute HTTPS GET requests serially without utilising the concurrency goodness promised by grequests. What was more interesting is that depending on the Python modules installed on my system, the same codebase ran fast or slow.

Tumbling down that rabbit hole made me understand Python profiling, function tracing, understanding how gevent works and opening a pull request to fix the issue with a gevent based Python module.

Outline

In this talk, I want to share my learnings with the audience through a working dockerized demo which showcases the following stages for the same codebase while explaining why each stage behaves the way it does:

  • Stage-0 - Trigger which led to the exploration - May run slow/fast for Python 2.7 and slow/fast for Python 3.7 depending on the modules installed on your system.
  • Stage-1 - Baseline with dockerized environment - Runs fast both for Python 2.7 and Python 3.7
  • Stage-2 - Detection of slowness and verification - Runs slow both for Python 2.7 and Python 3.7
  • Stage-3 - Python 2.7 fix - Runs fast for Python 2.7 but slow for Python 3.7
  • Stage-4 - Python 3.7 fix - Patching a gevent based Python module which fixes the issue and makes Python 3.7 fast again.

Please check out the preview video to get a sneak peek into the issue and watch the same codebase run slow/fast depending on the Python version chosen.

Requirements

Curiosity + programming basics.

Speaker bio

  • Saurabh Hirani normally doesn’t talk about himself in third person, but when he does, he is writing a speaker bio.
  • He enjoys programming and infrastructure automation.
  • He is currently working as a Site Reliability Engineer at Autodesk and has spoken at Python meetups, Rootconf conferences and firmly believes in sharing hard earned knowledge that everyone can benefit from.

Slides

http://saurabh-hirani.github.io/slides/decks/grequests-https-talk/main.html#/

Preview video

https://www.youtube.com/embed/6bMCMfyZmdg?version=3&vq=hd720

Comments

  • Zainab Bawa (@zainabbawa) Reviewer a month ago (edited a month ago)
    1. Started at 7:04 PM (IST). Ended at 7:45 PM (IST)
    2. You could show a lot of code execution. Can you show the containers running in parallel? Otherwise, there is too much back and forth. Have four parallel windows. This will be a cleaner visualization.
    3. Visualization of Python Tables is better than plain tables. You may want to try for visualization: https://github.com/nvdv/vprof
    4. The initial scenarios can be explained concurrently.
    5. Highlighting text for emphasis (on the real stage) could be distracting for the audience. You could be using a clicker for the actual talk.
    6. The talk was fairly deep. 25-30 mins in the talk there was a question about what is it that I am taking away from the talk. Don’t keep the takeaway slide towards the end. Intersperse your takeaways between the different stages. Call out the takeaways in the beginning.
    7. The tone was a lecture, rather than a connect with the audience. Try modulating your voice to create intrigue, empathy or surprise. You have to think of your talk as a performance. You switch between topics quite fast.
    8. The flow is interesting from the POV of how an issue was detected and how it was resolved. The details are good.
    9. Increase font size because text is not visible.
    10. Too much text on the slides. Reduce text.
    11. Limit the demo time (and number of demo times). Replace demo by putting this into slides. Don’t switch too often between demo and context. When you switch, give deliberate pauses.
    12. Use different colour coding for different versions of Python. This will give visual cues for participants to stay connected.
    13. Use gender neutral language throughout the presentation. Avoid using the word guys.
  • saurabh hirani (@saurabh-hirani) Proposer a month ago

    Thanks for collecting and updating the feedback Zainab. I have updated the slides - and added more diagrams to show the flow and inlined some of the demo output so as to save time.

    Updated slides - same url - http://saurabh-hirani.github.io/slides/decks/grequests-https-talk/main.html#/

Login with Twitter or Google to leave a comment