Rootconf Mini 2024

Background jobs, more commonly referred to as asynchronous tasks or jobs, are a technique in software development for managing tasks that can be executed independently of the primary user interaction or request-response cycle. Background jobs are utilized to enhance system responsiveness, manage time-consuming tasks, and offload resource-intensive operations from the main application thread or process.

In Freshdesk, the bulk of in-app background processing — jobs enqueued and processed by the Rails application — happens using Sidekiq. Sidekiq is a super-fast Redis-based queueing system that uses threads to handle many jobs at the same time and in the same process. For a product as big as Freshdesk, we have numerous background queues for various types of functionalities.

However, such jobs, when executed with high concurrency, can exhaust system resources, such as CPU, memory, or network bandwidth. These operations can also strain downstream systems, including RDS, Redis, Elasticsearch, and others. This can impact the performance of other genuine jobs that share the same resources, resulting in performance degradation or system failures, thereby becoming a “noisy neighbor.” An effective job throttling mechanism can help in easing the load on the resources.

Throttling gives you granular control in such critical situations, especially in a production environment for jobs where it’s not important to finish as quickly as possible, and at the same time you don’t want to pile up your queues with lots of jobs that are heavy - which can potentially impact your resources, leading to an outage.

This talk will focus on the primary motivations, architecture, benefits, and challenges companies should prepare for - if they decide to use a high concurreny Redis-based job framework for their application.

Key Takeaways

• Get insights into the many of the unexpected challenges a redis based queuing system poses at scale.
• Explore how a highly configurable throttling system helps ease the load on system resources and mitigates potential production outages.
• Empower the audience to make informed decisions when designing queuing systems for their applications.

Target Audience

• SRE, Systems, and DevOps Engineers.
• Software Developers and Architects.
• System Design Enthusiasts.
• Technically Curious Individuals.
  `