Projects

Huawei-HIT Joint Laboratory - Multimodal Voice Assistant Rejection Project

Developed models to distinguish interference (like non-targeted voice assistant speech or background noise) during continuous multi-round dialogue listening by voice assistants.

Responsible for building a dual-tower rejection model of voice (Wav2vec2) + text (BERT), and adopting various approaches (like data augmentation, MoE) to improve model generalization (OOD) across domains.

• Audio Processing
• Natural Language Processing

October 2021 - November 2022

Undergraduate Thesis - Contextual Spoken Language Understanding based on Heterogeneous Graph Networks

Improved the contextual spoken language understanding system by using Heterogeneous Attention Networks (HAN) to jointly model the semantic information and domain transitions of dialogue, and disambiguate semantic frame parsing by explicitly modeling domain transition phenomena.

Experiments on the SGD dataset showed that the proposed method achieved SOTA results, leading previous best models by 2.18% in semantic frame accuracy and 1.1% in intent accuracy.

Explored the performance of pre-trained models in the system, achieving an improvement of 1.02% in semantic frame accuracy on top of the proposed model.

Supervised by Prof. Libo Qin.

• Task-oriented Dialogue System
• Graph Neural Networks
• Natural Language Processing

June 2021

Tencent Technology (Shenzhen) Co., Ltd. - Software Defect Prediction Plugin

Predict the vulnerability risk on the code repository based on information like code hotspots and developer count using machine learning methods.

• DevOps
• Machine Learning
• Python

June 2020 - August 2020