Advanced NLP and Deep Learning for document classification - A case study in civil aviation safety prognosis
In this presentation, I apply a set of data-mining and sequential deep learning techniques to accident reports published by the National Transportation Safety Board (NTSB), in order to support real-time prognosis of adverse events. The focus here is on learning with text data that describes sequences of events. NTSB creates post-hoc investigation reports which contain raw text narratives of their investigation and their corresponding concise event sequences. Classification models are developed for Class A passenger air carriers, that take either an observed sequence of events or the corresponding raw text narrative as input and make predictions regarding whether an accident or an incident is the likely outcome, whether the aircraft would be damaged or not and whether any fatalities are likely or not.
Sequential models for NLP are gaining popularity and this presentation talks about a case-study of applying these techniques to solve real problems for the Civil Aviation in the US. The classification models are developed using Word Embedding and the Long Short-term Memory (LSTM) algorithm. The proposed methodology is implemented in two steps: (i) transform the NTSB data extracts into labeled datasets for building supervised machine learning models; and (ii) develop DL models for doing prognosis of adverse events like accidents, aircraft damage or fatalities. We also develop a prototype for an interactive query interface for end-users to test various scenarios including complete or partial event sequences or narratives and get predictions regarding the adverse events. The presentation is accompanied by a demo component and the resulting F1-score metrics are used to evaluate the effectiveness of the technique. Audience will gain in-depth insight into the technology stack used for this deep learning application and the ways to troubleshoot the usual problems of noise in natural language.
Prabhakar Srinivasan as obtained a Masters in Computer Science from DePaul university, Chicago and has over 13 years industry experience working for companies like Apple, Yahoo!, and Cisco and start-ups like Coffeemeetsbagel. With a breath of experience in developing Enterprise-scale applications like Recommendation Engines and Deep-Learning applications for Forecasting Sales and Demand Prediction in Supply Chain, the author has in-depth knowledge of the tools and technologies used for developing pragmatic machine learning applications.