Rootconf 2018

On scaling infrastructure and operations

Job scheduling at Helpshift with Jenkins

Submitted by Vineet Naik (@naiquevin) on Saturday, 3 March 2018

videocam
Preview video

Technical level

Intermediate

Section

Crisp Talk

Status

Confirmed & Scheduled

View proposal in schedule

Vote on this proposal

Login to vote

Total votes:  +10

Abstract

This talk will be about our experience of building a distributed job scheduling platform on top of Jenkins at Helpshift. It’ll mainly focus on:

  1. Our expectations and requirements from a generic job scheduler that we could standardize across the organization
  2. How we could solve the problems with our earlier approach by using the jenkins automation platform and it’s plugin ecosystem

Outline

  • The use cases for batch jobs at Helpshift
  • Our old approach and it’s problems
  • Problem statement
    • Need for a standard job scheduling platform
  • Why jenkins?
  • New jenkins based approach
    • Architecture and implementation
    • Benefits
      • How it addresses the problems with the earlier approach
    • Integration with our release workflow
      • Automated and seamless onboarding of new projects and new jobs
    • Production Readiness
      • High availability
      • Monitoring, alerting
    • Known limitations and future plans
  • Questions

Requirements

A general understanding of batch jobs, client-server architecture and domain specific languages (DSL). Familiarity with Jenkins is not necessary but will definitely help.

Speaker bio

I work at Helpshift as a Software Architect and am actively involved in the implementation of this job scheduling platform. As a result of using Jenkins previously while automating our CI and release workflows, I’ve a good understanding of it and it’s ecosystem.

Links

Slides

https://speakerdeck.com/naiquevin/job-scheduling-at-helpshift-with-jenkins

Preview video

https://drive.google.com/file/d/1sBUcJdoDCFRjS5ZIasMmYOYSlETlpgoJ/view?usp=sharing

Comments

  • 1
    Zainab Bawa (@zainabbawa) Reviewer 8 months ago

    @Vineet, I cannot access your slides. Check the link you have shared.

  • 1
    Vineet Naik (@naiquevin) Proposer 8 months ago

    Didn’t notice it was marked “private” on speakerdeck. Published it now https://speakerdeck.com/naiquevin/jenkins-as-a-distributed-job-scheduling-platform

Login with Twitter or Google to leave a comment