February 2023 Rustacean Meetup

Submission Draft:

Rust is a fast-growing systems programming language. It is being used in place of C++ because of its better performance to solve many systems programming problems. One such problem that was solved using Rust is classifying high volume DNS traffic, with near real-time processing of 15 million records per second. While this data mining of DNS queries can be of immense value to business, the process of realizing this right from data collection, enrichment, ingestion and subsequent analysis poses significant challenges.

The complete backend of this application was written in Rust. However, implementation in Rust comes with its own challenges.

One such challenge is updating global variables in Rust. After trying known approaches of mutex, RWLock and RefCell, which either did not work or gave inadequate performance, we implemented an innovative solution to update these global variables that gave us better performance compared to the previous solutions.

Another challenge was load-balancing and horizontal scaling with long lived connections (like TCP). This challenge was addressed by implementing multi connections-based solution in Rust. The connections can intelligently switch, while ensuring data transfer is reliable and no data is lost in the process.

Further, the performance was optimized by limiting the number of concurrent writes to storage, as well as influx of data from the client by limiting number of connections and packet rate limiter. eBPF tools were used to derive optimal values for these limits.

Attending this session will give you an insight into these classic problems, and how new and innovative solutions were developed in Rust to tackle them.

Presenters: Ruchi Saluja, Deepakala Bharath.