pretty_name: EmoMosaic
language:
- en
tags: - multimodal
- sentiment-analysis
- missing-modalities
- cmu-mosei
- research
configs: - config_name: file_inventory
default: true
data_files:- split: files
path: manifest.csv
- split: files
EmoMosaic
EmoMosaic contains the four original data attachments used by the project Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness: video clips, transcripts, labels, and aligned/unaligned features from a supplied CMU-MOSEI-derived package. This repository publishes the project's source data files, with their file bytes and internal directory structure preserved under data/. It contains 244 data files, totaling 4,077,821,811 bytes (about 3.80 GiB), plus documentation and file manifests. Experimental predictions, metrics, explanations, model weights, and training caches are not included.
中文简介： 本仓库发布 EmoMosaic 项目使用的四类原始数据附件，包括视频、转写、标签以及对齐/未对齐特征，共 244 个数据文件，约 3.80 GiB。这些文件来自项目持有并整理的 CMU-MOSEI 衍生数据包，本次上传保留文件内容和目录层级；不能视为与官方完整 CMU-MOSEI 或官方标准划分完全相同。仓库不包含实验预测、评价结果或模型。网页预览展示的是文件清单，不是逐条语料样本。
Contents
| Attachment | Samples and purpose | Files |
|---|---|---|
attachment1_raw_multimodal |
100 raw clips from 37 video IDs, with transcripts and sentiment labels; used for the separate word-alignment pipeline | 100 MP4 + 1 XLSX |
attachment2_features |
4,850 labeled utterances, with aligned and unaligned representations and a label workbook | 2 PKL + 1 XLSX |
attachment3_missing_modalities |
30 unlabeled utterances, each with aligned and unaligned inputs for missing-modality inference | 60 PKL |
attachment4_interpretability |
20 unlabeled utterances, each with aligned and unaligned features and a corresponding clip in both version directories | 40 PKL + 40 MP4 |
There are 102 PKL, 140 MP4, and 2 XLSX files. The two feature versions of attachment 2 represent the same 4,850 utterances, not 9,700 independent examples. Attachment 4's 40 video files represent 20 clips duplicated across the aligned and unaligned directories. Counts across attachments should not be added to infer a deduplicated corpus size.
The label split supplied with attachment 2 is:
| Split | Utterances | Negative | Neutral | Positive |
|---|---|---|---|---|
train |
3,395 | 967 | 758 | 1,670 |
valid |
728 | 206 | 184 | 338 |
test |
727 | 207 | 158 | 362 |
| Total | 4,850 | 1,380 | 1,100 | 2,370 |
Sentiment intensity is in [-3, 3]. Classes are Negative for y < 0, Neutral for y == 0, and Positive for y > 0; stored class IDs are 0, 1, and 2. The project's test split was inspected during model development, so its reported results are not a fresh blind evaluation.
Download and inspect
Install the Hugging Face CLI, then download the repository:
python -m pip install -U huggingface_hub
hf download config-h/EmoMosaic --repo-type dataset --local-dir EmoMosaic
To download only the aligned labeled feature file:
hf download config-h/EmoMosaic data/attachment2_features/aligned_50.pkl --repo-type dataset --local-dir EmoMosaic
The default Dataset Viewer configuration, file_inventory, reads manifest.csv: one row describes one file. It does not deserialize PKL files, decode video, or expose the 4,850 utterances as a Hugging Face Dataset. Calling load_dataset on this configuration loads the file index. Download the actual files and use the formats described in DATA_GUIDE.md. The manifests record file sizes and SHA-256 checksums.
Provenance and limitations
This is the project's supplied, modified/reorganized CMU-MOSEI-related package, not an official CMU-MOSEI mirror or a claim to reproduce the official corpus and canonical split exactly. The detailed history of changes before this packaging has not been independently reconstructed. The supplied clips have not been verified against their original online sources, and the original toolchain for the supplied 74-dimensional acoustic and 35-dimensional visual features has not been verified. The current release preserves the local source files; it does not regenerate features or repair annotations.
The predictive experiments use attachment 2's aligned features. The raw-clip alignment experiment uses attachment 1 separately; its extracted outputs do not feed those predictive experiments. Aligned sequences have 50 text WordPiece positions, including special tokens, rather than 50 words. Zero vectors may indicate missingness, padding, or natural absence; these causes cannot be identified from zero values alone.
The release contains source videos and transcript text, which may contain offensive language or other objectionable material. No systematic offensive-content audit was performed. Sentiment labels describe an annotation task and should not be treated as reliable evidence of an individual's emotional state.
Terms and associated work
See TERMS.md for the documented licensing boundary. No standard dataset license is asserted by this card. The MIT license on the associated code does not relicense third-party videos, transcripts, labels, or features.
Associated code: EmoMosaic on GitHub. The associated manuscript is titled Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness; this card does not assert publication or conference acceptance.
For the source CMU-MOSEI work, cite the published ACL paper; the author list below follows its published PDF:
@inproceedings{zadeh-etal-2018-mosei,
title = {Multimodal Language Analysis in the Wild: {CMU}-{MOSEI} Dataset and Interpretable Dynamic Fusion Graph},
author = {Zadeh, Amir and Liang, Paul Pu and Vanbriesen, Jonathan and
Poria, Soujanya and Tong, Edmund and Cambria, Erik and
Chen, Minghai and Morency, Louis-Philippe},
booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2018},
pages = {2236--2246},
publisher = {Association for Computational Linguistics},
doi = {10.18653/v1/P18-1208},
url = {https://aclanthology.org/P18-1208/}
}
When referring to this derivative package, also identify config-h/EmoMosaic and the downloaded repository revision so that the exact files are traceable.