Latest Work
Learning a Transparent Test-time Augmentation Policy via Gradient-Free Optimization
Traditional test-time augmentation(TTA) like five-crop or ten-crop aims to reduce the variance during the test-phase. We proposed a novel white-box tta method that will enhance the test images with explainable transformations in order to get an optimized performance for a black-box deep learning model.
In our work, we model the TTA process as a simple optimization problem over a set of normalized continuous parameters that controls the intensity of the transformations, with identity transformation explicity included by initializing the parameters to be 0.5.
Inspired by the work of guided evolutionary strategies(GES) and other gradient estimation method, we propose a novel TTA optimization algorithm based on surrogate gradient estimation method.
[preprint] [paper in mandarin]
(Working Project) Self-supervised facial expression recognition with multi-task learning
Recently, constrastive learning method has proved to be successful in many computer vision tasks. We propose a novel self-supervised facial expression recognition (FER) framework with a facial landmark auxiliary task to facilitate the performance of FER model.
We also propose a FER-specific augmentation by warping the images with mapping the corresponding Delauney triangles to create high-dimension embeddings with rich semantic information with regards to the facial units.
I’m still working on this project.
Hypergraph neural network for emotionmal analysis
In this project, we devised a novel modality-attentional block to enhance the hypergraph neural network (HGNN)’s performance on the task of affective computing, which is supposed to predict a valence and arousal level from multi-modal physiological signals. By blending the intermediate hyperedge features, we report an augmented performance compared to the SOTA results.
We also introduce a hand-crafted feature to help the model focus on the individual difference.