Robust Painpoint Detection in Conversational Videos
We (Talka.ai) are developing models for the better understanding of conversational text. One of our products is a painpoint prediction system capable finding the chunks of the video where clients express their needs and issues regarding a given product, which tend to lead to failed costumer acquisition when these are not address. As these videos tend to be very long (40-60 minutes) and most painpoints are expressed in short spans of speech (1-2 minutes), being able to predict where these are located can drastically reduce the time sales managers have to spend analysing their data. The goal of this project is find features in order to improve a painpoint detector in conversational videos. Currently, the features are based on BERT embeddings from the text (from the ASR transcription). We wish to extend this list of features to include both textual, audio and multimodal features. Some examples include using sentiment analysis, since most painpoints occur in negatively entailed sentences, or use emotional features from the video or audio (e.g. the costumer is angry). The intern is allowed to explore features that are related to his interests and work or brainstorm with the rest of the team in order to establish the feature set to implement.
Contact: Wang Lin ( wanglin1122@gmail.com )
Active Learning for multimodal rare event detection
The data used in our models is created by annotating videos with the desired events such as interruptions, laughters, smiles and painpoints. Our interface currently supports active learning for painpoints by providing the model based probabilities for painpoints at the sentence level, so that annotators can more quickly traverse the video by skipping low probability sentences. This improves the annotation rate since most non painpoint sentences are easy to detect and can be skipped. The goal of this project is to implement the same active learning approach for other events, such as smiles, nodding, interruption etc... and evaluate whether the annotation rate can be improved.
Contact: Wang Lin ( wanglin1122@gmail.com )