Job scheduling at Helpshift with Jenkins
Submitted by Vineet Naik (@naiquevin) on Saturday, 3 March 2018
Section: Crisp Talk Technical level: Intermediate
This talk will be about our experience of building a distributed job scheduling platform on top of Jenkins at Helpshift. It’ll mainly focus on:
- Our expectations and requirements from a generic job scheduler that we could standardize across the organization
- How we could solve the problems with our earlier approach by using the jenkins automation platform and it’s plugin ecosystem
- The use cases for batch jobs at Helpshift
- Our old approach and it’s problems
- Problem statement
- Need for a standard job scheduling platform
- Why jenkins?
- New jenkins based approach
- Architecture and implementation
- How it addresses the problems with the earlier approach
- Integration with our release workflow
- Automated and seamless onboarding of new projects and new jobs
- Production Readiness
- High availability
- Monitoring, alerting
- Known limitations and future plans
A general understanding of batch jobs, client-server architecture and domain specific languages (DSL). Familiarity with Jenkins is not necessary but will definitely help.
I work at Helpshift as a Software Architect and am actively involved in the implementation of this job scheduling platform. As a result of using Jenkins previously while automating our CI and release workflows, I’ve a good understanding of it and it’s ecosystem.