Shenzhen Research Institute of Big Data, Shenzhen, China
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
天气预报怎么说 你好, 很高兴认识你
我明天过来接你 为我们的合作成功干杯
Introduction
This MCCS dataset is the first large-scale Mandarin Chinese Cued Speech dataset. This dataset covers 23 major categories of scenarios (e.g, communication, transportation and shoping) and 72 subcategories of scenarios (e.g, meeting, dating and introduction). It is recorded by four skilled native Mandarn Chinese Cued Speech cuers with portable cameras on the mobile phones. The Cued Speech videos are recorded with 30fps and 1280x720 format. We provide the raw Cued Speech videos, text file (with 1000 sentences) and corresponding annotations which contains two kind of data annotation.
One is continuious video annotation with ELAN,
the other is discrete audio annotations with Praat.
DownLoad
This MCCSD contains 1000 Mandarin Chinese Cued Speech sentences. We currently only provide it to universities and research institutions for research purposes. Please complete the following steps to obtain the dataset:
Complete the agreement appropriately after reading it carefully. Currently the agreement is only accepted to be signed by a full-time staff member (students are not accepted)
Please provide the signed agreement and send it to Prof. Li Liu (avrillliu@hkust-gz.edu.cn)
CS generation
If you are interested in our CS generation work, check out the following links for more details: GitHub
References
If you use this MCCS dataset for your research, please consider citing the following papers:
Objective Hand Complexity Comparison between Two Mandarin Chinese Cued Speech Systems.
Li Liu, Gang Feng, Xiaoxi Ren, Xianping Ma International Symposium on Chinese Spoken Language Processing (ISCSLP) (2022)
LINK
Re-synchronization using the Hand Preceding Model for Multi-modal Fusion in Automatic Continuous Cued Speech Recognition.
Li Liu, Gang Feng, Denis Beautemps, Xiao-Ping Zhang.
IEEE Transactions on Multimedia (2020)
LINK
A Pilot Study on Mandarin Chinese Cued Speech. Li Liu, Gang Feng. American Annals of the Deaf (2019)
LINK
Cross-modal mutual learning for cued speech recognition. Liu, L., & Liu, L. (2023, June). 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023) .
LINK
A Novel Interpretable and Generalizable Re-synchronization Model for Cued Speech based on a Multi-Cuer Corpus. Lufei Gao, Shan Huang, & Li Liu (2023). arXiv preprint arXiv:2306.02596. Accepted by Interspeech 2023 LINK
Contact
If you have any questions about the dataset and our research works, please feel free to contact us: