Harnessing the power of the Erlang VM at Housing
Abhijit Pratap Singh
RoR and Django has ensured we remain productive in the face of rapidly changing product requirements at Housing. However we ran into issues of memory and speed when we had to scale throughput and interface with other services in our SOA. This talk describes how we rewrote some core parts of our infrastructure to ride on the coattails of the awesome Erlang VM.
I would briefly introduce the audience about the Erlang/Elixir, the actor model and its benefits.
We will discuss the following three use cases as examples of where we used the Erlang VM to overcome limitations of speed and memory in our core infrastructure
Ruby APIs, implementing search with Elasticsearch, have been the backbone of our realtime search infrastructure. After having experienced a few server side scalability bottlenecks due to inefficient Unicorn workers, we decided to port some of the feasible and critical code to Elixir which exploits the soft real-time capabilities of the Erlang VM. In a Service Oriented Architecture centric environment, various Ruby APIs doing IO on network fitted a great use case for the Erlang VM.
CouchDB is another software written in Erlang. It plays a significant role in storage of custom JSON data and enable real-time streaming search through CouchDB Views. We injected a Elixir evaluation runtime in CouchDB to allow us to write views in Elixir, which results in performant View generation times while also assuring developer productivity.
RabbitMQ has been a critical component for our backend infrastructure . The use of topic exchanges can enable the source logic and the consumer logic to be decoupled enabling us to write generic publishers which could just be embedded as a library in other services.
Finally I would also like to shed some light upon production issues that can occur in Erlang VM and the ways to fix them.