Publications

2023

1. 

   A Density-Guided Temporal Attention Transformer for Indiscernible Object Counting in Underwater Videos

   Cheng-Yen Yang, Hsiang-Wei Huang, Zhongyu Jiang, Hao Wang, Farron Wallce, and Jenq-Neng Hwang

   In Submission
2. 

   Multi-object Tracking by Iteratively Associating Detections with Uniform Appearance for Trawl-based Fishing Bycatch Monitoring

   Cheng-Yen Yang, Alan Yu Shyang Tan, Melanie J. Underwood, Charlotte Bodie, Zhongyu Jiang, Steve George, Karl Warr, Jenq-Neng Hwang, and Emma Jones

   In 2023 IEEE International Conference on Image Processing (ICIP)
3. 

   Iterative Scale-Up ExpansionIoU and Deep Features Association for Multi-Object Tracking in Sports

   Hsiang-Wei Huang, Cheng-Yen Yang, Jenq-Neng Hwang, and Chung-I Huang

   In Submission
4. 

   Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation

   Zhongyu Jiang, Zhuoran Zhou, Lei Li, Wenhao Chai, Cheng-Yen Yang, and Jenq-Neng Hwang

   In Submission
5. 

   Enhancing Multi-Camera People Tracking with Anchor-Based Clustering and Spatio-Temporal Consistency Calibration

   1st place in 7th AI City Challenge Track 1: Multi-Camera People Tracking

   Cheng-Yen Yang*, Hsiang-Wei Huang*, Zhongyu Jiang, and Jenq-Neng Hwang

   In 2023 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops
6. 

   Learning Temporal Attention based Keypoint-guided Embedding for Gait Recognition

   Hung-Min Hsu, Yizhou Wang, Cheng-Yen Yang, Jenq-Neng Hwang, Hoang Le Uyen Thuc, and Kwang-Ju Kim

   In IEEE Journal of Selected Topics in Signal Processing
7. 

   Observation Centric and Central Distance Recovery on Athlete Tracking

   3rd Place in SportsMOT Challenge on Multi-actor Tracking at DeeperAction Workshop (ECCV 2022)

   Hsiang-Wei Huang, Cheng-Yen Yang, Jenq-Neng Hwang, Pyong-Kun Kim, Kyoungoh Lee, and Kwang-Ju Kim

   In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)

   Abs

   Multi-Object Tracking over humans has improved rapidly with the development of object detection and re-identification. However, multi-actor tracking over humans with similar appearance and non-linear movement can still be very challenging even for the state-of-the-art tracking algorithm. Current motion-based tracking algorithms often use Kalman Filter to predict the motion of an object, however, its linear movement assumption can cause failure in tracking when the target is not moving linearly. And for multi-players tracking over the sports field, because the players in the same team are usually wearing the same color of jersey, making re-identification even harder both in the short term and long term in the tracking process. In this work, we proposed a motion-based tracking algorithm and three post-processing pipelines for three sports including basketball, football, and volleyball, we successfully handle the tracking of the non-linear movement of players on the sports fields. Experimental results, with achieved HOTA of 73.968 on the testing set of ECCV DeeperAction Challenge SportsMOT Dataset, show the effectiveness of the proposed framework and its robustness in different sports scenes.

8. 

   CameraPose: Weakly-Supervised In-the-wild 3D Human Pose Estimation from a Single Image

   Cheng-Yen Yang, Jiajia Luo, Lu Xia, Yuyin Sun, Nan Qiao, Ke Zhang, Zhongyu Jiang, Jenq-Neng Hwang, and Cheng-Hao Kuo

   In 2023 IEEE Winter Conference on Applications on Computer Vision (WACV)

   Abs PDF

   To improve the generalization of 3D human pose estimators, many existing deep learning based models focus on adding different augmentations to training poses. However, data augmentation techniques are limited to the ”seen” pose combinations and hard to infer poses with rare ”unseen” joint positions. To address this problem, we present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image, which can not only be applied on 2D-3D pose pairs but also on 2D alone annotations. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity and the 3D poses can be implicitly learned by reprojecting back to 2D. Moreover, CameraPose introduces a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators. Experimental results demonstrate that the CameraPose brings in clear improvements on cross-scenario datasets. Notably, it outperforms the baseline method by 3mm on the most challenging dataset 3DPW. In addition, by combining our proposed refinement network module with existing 3D pose estimators, their performance can be improved in cross-scenario evaluation.

2022

1. 

   GAITTAKE: Gait Recognition by Temporal Attention and Keypoint-Guided Embedding

   Hung-Min Hsu, Yizhou Wang, Cheng-Yen Yang, Jenq-Neng Hwang, Hoang Le Uyen Thuc, and Kwang-Ju Kim

   In 2022 IEEE International Conference on Image Processing (ICIP)

   Abs PDF

   Gait recognition, which refers to the recognition or identification of a person based on their body shape and walking styles, derived from video data captured from a distance, is widely used in crime prevention, forensic identification, and social security. However, to the best of our knowledge, most of the existing methods use appearance, posture and temporal feautures without considering a learned temporal attention mechanism for global and local information fusion. In this paper, we propose a novel gait recognition framework, called Temporal Attention and Keypoint-guided Embedding (GaitTAKE), which effectively fuses temporal-attention-based global and local appearance feature and temporal aggregated human pose feature. Experimental results show that our proposed method achieves a new SOTA in gait recognition with rank-1 accuracy of 98.0% (normal), 97.5% (bag) and 92.2% (coat) on the CASIA-B gait dataset; 90.4% accuracy on the OU-MVLP gait dataset.

2. 

   Unsupervised Domain Adaptation Learning for Hierarchical Infant Pose Recognition with Synthetic Data

   Cheng-Yen Yang, Zhongyu Jiang, Shih-Yu Gu, Jenq-Neng Hwang, and Jang-Hee Yoo

   In 2022 IEEE International Conference on Multimedia and Expo (ICME)

   Abs PDF

   The Alberta Infant Motor Scale (AIMS) is a well-known assessment scheme that evaluates the gross motor development of infants by recording the number of specific poses achieved. With the aid of the image-based pose recognition model, the AIMS evaluation procedure can be shortened and automated, providing early diagnosis or indicator of potential developmental disorder. Due to limited public infant-related datasets, many works use the SMIL-based method to generate synthetic infant images for training. However, this domain mismatch between real and synthetic training samples often leads to performance degradation during inference. In this paper, we present a CNN-based model which takes any infant image as input and predicts the coarse and fine-level pose labels. The model consists of an image branch and a pose branch, which respectively generates the coarse-level logits facilitated by the unsupervised domain adaptation and the 3D keypoints using the HRNet with SMPLify optimization. Then the outputs of these branches will be sent into the hierarchical pose recognition module to estimate the fine-level pose labels. We also collect and label a new AIMS dataset, which co—tains 750 real and 4000 synthetic infants images with AIMS pose labels. Our experimental results show that the proposed method can significantly align the distribution of synthetic and real-world datasets, thus achieving accurate performance on fine-grained infant pose recognition.

2021

1. 

   U3D-MOLTS: Unified 3D Monocular Object Localization, Tracking and Segmentation

   1st Place on KITTI-STEP Challenge

   Haotian Zhang, Yizhou Wang, Zhongyu Jiang, Cheng-Yen Yang, Jie Mei, Jiarui Cai, and Jenq-Neng Hwang

   In 2021 6th Workshop on Benchmarking Multi-Target Tracking (ICCV Workshop)

   Abs PDF

   Sensing and perception systems for autonomous driving vehicles in road scenes are composed of four crucial components: object detection, tracking, panoptic segmentation, and 3D localization. While all these components are inherently intertwined, most relevant papers tend to only focus on a subset of these components. We separate the unified video panoptic segmentation tasks into two subtasks: 1) multiobject tracking and segmentation, and 2) semantic segmentation. For the first stage, we propose a unified monocular 3D based framework that effectively tracks detected moving objects over time and estimates their 3D localization information as well as instance segmentation masks from a sequence of 2D images captured from a dash camera on a moving vehicle. Our system contains an RCNN-based Localization for Tracking Network (Loc4Trk-Net). The object association leverages deep pairwise contrastive learning to identify objects in various poses and viewpoints with appearance cues. A straightforward combination of a 3D Kalman filter and the Hungarian algorithm is further utilized for robust instance association via both feature similarity and 3D localization information. For the second stage, we adopt the existing DeepLabV3+ for semantic segmentation and further enhanced the performance with data augmentation using label propagation. Our proposed pipeline achieves an STQ score of 67.55 on the KITTI-STEP dataset as well as the state-of-the-art performance on the KITTI-MOTS leaderboard.

2. 

   Long-Tailed Recognition of SAR Aerial View Objects by Cascading and Paralleling Experts

   Honorable Mention Award on SAR Track of the MAVOC Challenge

   Cheng-Yen Yang, Hung-Min Hsu, Jiarui Cai, and Jenq-Neng Hwang

   In 2021 New Trends in Image Restoration and Enhancement Workshop (CVPR Workshop)

   Abs PDF

   Aerial View Object Classification (AVOC) has started to adopt deep learning approaches with significant success in recent years but limited to optical data. On the other hand, Synthetic Aperture Radar (SAR) has wild aerial view related applications in the remote sensing field. However, SAR has received far less attention due to the special characteristics of the SAR data, which is the long-tailed distribution of the aerial view objects that increase the difficulty of classification. In this paper, we present a two-branch framework, including the cascading expert branch and paralleling expert branch, to tackle the long-tailed distribution of the dataset. Our proposed multi-expert architecture achieves 24.675% and 26.029% in the development phase and testing phase, respectively, in the NTIRE 2021 Multi-modal Aerial View Object Classification Challenge Track 1. The proposed method is proved to possess the effectiveness (top-tier performance among 157 participants) and efficiency (i.e., a lightweight architecture) for the AVOC task.

2019

1. 

   Weakly-Supervised Learning for Attention-Guided Skull Fracture Classification In Computed Tomography Imaging

   Cheng-Yen Yang, Chi-Hsin Lo, Huan-Chih Wang, Jen-Hai Chou, and Yu-Chiang Frank Wang

   In 2019 IEEE International Conference on Image Processing (ICIP)

   Abs PDF

   We propose a novel attention-guided deep learning framework for image classification, with the goal of predicting both image and pixel-level labels for input images. While the training images are with either positive or negative labels, we do not assume that each training image is with annotated pixel-level ground truth information, and thus our method can be realized in such a weakly supervised setting. Our proposed module can be easily combined with standard CNN architectures with no extra parameter needed. With the above advantages, we evaluate our model on a skull CT dataset, and the experimental results confirm the effectiveness and robustness of our approach over recent popular CNN architectures.