JSFoo 2016

With the rapid growth in data, it turns very important to visualise them to get the inferences. Web, being the most commonly used platform, this talk is on how to work with large data in browsers. We’ll talk about the common experiences we encounter on attempting to do so. Firstly we’ll try to manage the data points effectively using data aggregation or over-plotting reduction.We will see what are the problems those are yet to be addressed even after the above operations. How to render the entire large data without a script error along with smooth interaction? Is SVG an option for it or does canvas stands better at it? What are the limit for canvas, and can we strech it? The talk explains what is batch rendering and how it helps in enhanching the performance with a live illustration.But using canvas comes at a cost of interaction support. We’ll see how to mock browser events algorithimically. And thereby, introducing kd-tree, explaining the benifits and discussing its performance limitations. It continues on how improving the tree-building implementaion provided that extra-edge. Discussing the boost recieved and trade-offs considered while designing the modified kd-tree version. Lastly, the 9-grid pre-rendering algortihim will be discussed to attain an effective zooming and panning user interface with live illustrations.

Outline

• Working with large data
  • Growing size of the data
  • Need for visualising it.
  • Web - the ubiquitous platform
  • Common experiences
    • Crashing of browsers
    • Script time-out error
    • Taking hours to render
    • Over-plotting of data
    • Issues with interactions
  • Managing data points effectively
    • Data aggregation
      • What is data aggregation?
      • How it solves the issue?
    • Over-plotting reduction
      • Culling the coincident data plots
      • How it solves the issue?
• A part of the problem is yet to be addressed
  • Data aggregation vs. actual data
  • At times. coincident rendering is essential!
• Rendering the entire large data
  • Without getting a script error / time lag
  • With smooth interaction post rendering
• Is SVG an option for it?
  • Live demo
  • Comparison of the rendering time vs DOM element counts
  • Analysis of the performance
• Is canvas an option for it?
  • Live demo
  • Comparison of the rendering time vs. number of data points rendered
  • Analysis of the performance
  • Performance comparison between SVG and canvas
• How far can you go with canvas?
  • Threshold of performance in canvas rendering
  • Factors behind the observed limitation
  • Can we stretch the limit?
• Batch rendering
  • What is batch rendering?
    • Divide the entire data in smaller chunks.
    • Render them individually instead of the whole at a single go
  • Live Demo
  • How it helped in pushing the limit?
• Interactive issues with canvas
  • No interactivity support
  • Mocking browser events algorithmically
• kd-tree
  • What is kd-tree?
  • Benefits of using kd-tree
  • Performance limitations
• Modified kd-tree
  • Modifications in the kd-tree
  • Benefits of this modification
  • Trade-off
• Progressive rendering
  • Why progressive rendering is required wrt zooming and panning ?
  • 9-grid algorithm
  • Live Demo
• Final Thoughts
  • A peek into FusionCharts zoom scatter chart
  • Say YES to large data visualisation in browsers.

Requirements

Basic understanding of Javascript.

Speaker bio

I am a Javascript developer, currently working at FusionCharts. I have deep love for writing JS codes, especially in VanillaJS. I am always keen to learn and improve myself. Apart from them I love robotics and trying out small stuffs using my Arduino.

StackOverflow: http://stackoverflow.com/users/5102631/ayan
GitHub: https://github.com/AyanGhatak
LinkedIn: https://in.linkedin.com/in/ayanghatak

Slides

https://docs.google.com/presentation/d/1YAFPUWvGoOjPEd9X6jtAni6MgviqTj18qOcA4i9OnpM