In 2014, infrastructure components such as Hadoop, Berkeley Data Stack and other commercial tools have stabilized and are thriving. The challenges have moved higher up the stack from data collection and storage to data analysis and its presentation to users. The focus for this year’s conference on analytics – the infrastructure that powers analytics and how analytics is done.
Talks will cover various forms of analytics including real-time and opportunity analytics, and technologies and models used for analyzing data.
Proposals will be reviewed using 5 criteria:
Domain diversity – proposals will be selected from different domains – medical, insurance, banking, online transactions, retail. If there is more than one proposal from a domain, the one which meets the editorial criteria will be chosen.
Novelty – what has been done beyond the obvious. Insights – what insights does the proposal share with the audience that they did not know earlier. Practical versus theoretical – we are looking for applied knowledge. If the proposal covers material that can be looked up online, it will not be considered.
Conceptual versus tools-centric – tell us why, not how. Tell the audience what was the philosophy underlying your use of an application, not how an application was used. Presentation skills – proposer’s presentation skills will be reviewed carefully and assistance provided to ensure that the material is communicated in the most precise and effective manner to the audience.
For queries about proposals / submissions, write to email@example.com
Data Collection and Transport – for e.g, Opendatatoolkit, Scribe, Kafka, RabbitMQ, etc.
Data Storage, Caching and Management – Distributed storage (such as Gluster, HDFS) or hardware-specific (such as SSD or memory) or databases (Postgresql, MySQL, Infobright) or caching/storage (Memcache, Cassandra, Redis, etc).
Data Processing, Querying and Analysis – Oozie, Azkaban, scikit-learn, Mahout, Impala, Hive, Tez, etc.
Big data and security
Big data and internet of things
Data Usage and BI (Business Intelligence) in different sectors.
Please note: the technology stacks mentioned above indicate latest technologies that will be of interest to the community. Talks should not be on the technologies per se, but how these have been used and implemented in various sectors, enterprises and contexts.
Visualizing large data sets
I’m going to showcase how to visualize large data sets, i.e. that have thousands to millions of data points.
This goes beyond standard techniques like bar plot etc, and requires using tools like d3, Processing, ggplot2, circos and more.
I will demostrate working samples, that I have created, using open source tools.
Folks will gain an understanding of concepts, techniques and tools to create large data visualizations.
One of the best ways to explore and understand a dataset is with a visual.
We basically depict the data into a visual space and let our brain explore the patterns.
Later, when we want to present the findings to others, visualizations are essential for efficiently communicating our findings.
However, this is challenging with large data sets. Whatever it is, a good data visualization, big or small, aesthetically appealing or not, helps us see what the data has to say. And if a visual cannot give that insight, then it’s not that useful.
Rather than going through a exhaustive list of visualization tools, that may be searched on the Internet, I’m going to show you some working demos that I have created on large data sets, on products that I have worked on in Yahoo and outside.
So the ideas I’m going to share in this tech talk are focused on both usefulness & aesthetics. And for someone who’s fond of both art & science, this is fantastic area, this is a great area as we are limited by only what we can imagine.
The only prereq for this tech talk is your level of curiosity, you dont need to be a programmer to take advantage of these visuals I’m going to show. This tech talk, thus, is less academic and more targeted on creating some good visualizations that can simplify analyzing complex data.
More than 15 years in the software industry on web, mobile, analytics, video, edge and visualizations.
Lived and worked in US, Spain, UK, India.
BE from BIT Ranchi India, MS from NEU Boston MA USA, EPBM from IIM Calcutta.
Product Head of performance & analytics at Yahoo.