Real-Time DataQuality on Flink
Session type: Full talk of 40 mins
My use case is to provide monitoring, and improving the overall search data quality, also to find the unusual patterns of user’s search behavior, and notifying the intent on-site back to the respective business stakeholders. To achieve the same, I explored various big data processing engines, which can process the huge data with complex business logic in real time. Eventually, I used Flink Stream processing. This talk will showcase how I used Flink to accomplish my goal.
What is Real Time Aggregation ?
Flink vs Spark
Flink Cluster setup
Flink on Yarn
100% data completeness
Batch vs Realtime
I am a Staff Software Engineer in Walmart and Apache Oozie Committer. I am currently trying to solve some of the search problems. I am in Big Data space since last 10 years.
Building a Location Intelligence Platform for audience segmentation
The ROI of OOH (Out of Home Advertisement) depends on precise and intelligent targeting of advertisements. The media buyers therefore require detailed understanding and visibility of the audiences across various attributes so that they can then plan their OOH media buy to specifically target a selected set of audiences. Location information of the user, device level audience data, enriched with r… more