The Fifth Elephant 2015
A conference on data, machine learning, and distributed and parallel computing
AS
Abhijit Pratap Singh
@sabhi
Harnessing the power of the Erlang VM at Housing
Submitted Jun 15, 2015
Section:
Crisp Talk
Technical level:
Intermediate
RoR and Django has ensured we remain productive in the face of rapidly changing product requirements at Housing. However we ran into issues of memory and speed when we had to scale throughput and interface with other services in our SOA. This talk describes how we rewrote some core parts of our infrastructure to ride on the coattails of the awesome Erlang VM.
Outline
I would briefly introduce the audience about the Erlang/Elixir, the actor model and its benefits.
We will discuss the following three use cases as examples of where we used the Erlang VM to overcome limitations of speed and memory in our core infrastructure
-
Ruby APIs, implementing search with Elasticsearch, have been the backbone of our realtime search infrastructure. After having experienced a few server side scalability bottlenecks due to inefficient Unicorn workers, we decided to port some of the feasible and critical code to Elixir which exploits the soft real-time capabilities of the Erlang VM. In a Service Oriented Architecture centric environment, various Ruby APIs doing IO on network fitted a great use case for the Erlang VM.
-
CouchDB is another software written in Erlang. It plays a significant role in storage of custom JSON data and enable real-time streaming search through CouchDB Views. We injected a Elixir evaluation runtime in CouchDB to allow us to write views in Elixir, which results in performant View generation times while also assuring developer productivity.
-
RabbitMQ has been a critical component for our backend infrastructure . The use of topic exchanges can enable the source logic and the consumer logic to be decoupled enabling us to write generic publishers which could just be embedded as a library in other services.
Finally I would also like to shed some light upon production issues that can occur in Erlang VM and the ways to fix them.
Requirements
No requirements
Speaker bio
Abhijit Pratap Singh is a Software Developer at Housing.com. He has been working at Housing.com for the past two years. He has played an active part in developing the product backend for Housing.com.
Pranav Rao is a Software Developer at Housing.com. He was the primary developer behind the geo-services backend at Housing.com working on search and storage of other geo-entities.
{{ gettext('Login to leave a comment') }}
{{ gettext('Post a comment…') }}{{ errorMsg }}
{{ gettext('No comments posted yet') }}