A Benchmark for Detecting Social Media Content in TV News Archives
Social2TV is a manually curated multimodal gold-standard dataset for detecting social media references in U.S. cable television news. It covers 36 hours of prime-time programming from CNN, MSNBC, and Fox News across five dates (one per year, 2020–2024), with labels for:
- Social media logos and post screenshots visible on TV screen
- Verbal mentions of social media in closed captions
The underlying video and caption text remain in the Internet Archive TV News Archive. This repository releases annotations, timestamps, and episode identifiers that link back to those broadcasts.
- Quick links to dataset files
- Overview
- Repository structure
- Corpus
- Visual annotations (logos & post screenshots)
- Closed-caption annotations
- Accessing source broadcasts
- Data availability and usage restrictions
- Acknowledgments
- License
Positive visual annotations (frames that contain a social media logo and/or a post screenshot):
| Channel | File | Direct download |
|---|---|---|
| CNN | gold_standard_images_cnn.csv |
Download |
| Fox News | gold_standard_images_foxnews.csv |
Download |
| MSNBC | gold_standard_images_msnbc.csv |
Download |
Folder: LogoAndPostScreenshots/
Gold-standard labels for every caption segment (without redistributing caption text):
| Channel | File | Direct download |
|---|---|---|
| CNN | gold_standard_cc_cnn.csv |
Download |
| Fox News | gold_standard_cc_foxnews.csv |
Download |
| MSNBC | gold_standard_cc_msnbc.csv |
Download |
Folder: ClosedCaptions/
Social media content frequently appears in television news through platform logos, post screenshots, and verbal references, extending its reach beyond the original platform. These moments typically occupy only a small fraction of airtime, yet they matter because they show when posts, platforms, and online activity are selected, contextualized, and amplified for TV audiences.
Social2TV addresses the lack of a shared evaluation resource for this sparse, multimodal phenomenon. Beyond the benchmark itself, it demonstrates a reusable workflow for enriching television archives with searchable, research-ready metadata.
Contributions
- A multimodal gold standard covering 36 hours of CNN, MSNBC, and Fox News from five selected dates (2020–2024).
- An extensible annotation workflow adaptable to new dates, channels, platforms, countries, and research domains.
- Open evaluation resources for detecting visual and caption-based social media references (see Related code).
Social2TV/
├── LogoAndPostScreenshots/ # Visual gold standard (logos + post screenshots)
│ ├── gold_standard_images_cnn.csv
│ ├── gold_standard_images_foxnews.csv
│ └── gold_standard_images_msnbc.csv
├── ClosedCaptions/ # Closed-caption gold standard (no caption text)
│ ├── gold_standard_cc_cnn.csv
│ ├── gold_standard_cc_foxnews.csv
│ └── gold_standard_cc_msnbc.csv
└── LICENSE
| Property | Value |
|---|---|
| Networks | CNN, Fox News, MSNBC |
| Window | U.S. prime time, 8:00–11:00 p.m. Eastern Time |
| Selected dates (ET) | 2020-03-13, 2021-01-09, 2022-04-24, 2023-01-21, 2024-01-31 |
| Episodes | 36 one-hour news programs (15 Fox News, 12 MSNBC, 9 CNN) |
| Frames reviewed | ~130,000 one-second frames |
| Visual positives released | 1,445 annotated frames with logo and/or screenshot |
| Caption segments | 59,982 labeled segments |
Dates were chosen via GDELT Television Explorer peak-mention searches to enrich for social media references. Social2TV is a positive-enriched benchmark for developing and evaluating detectors—not a probability sample for estimating overall prevalence across TV news.
Archive episode IDs use UTC timestamps (prime-time Eastern often falls on the following UTC calendar day). Example: March 13, 2020, 8–9 p.m. ET → CNNW_20200314_030000_....
Fox News (15)
| Date (ET) | 8–9 PM | 9–10 PM | 10–11 PM |
|---|---|---|---|
| 2020-03-13 | Fox News at Night With Shannon Bream | Tucker Carlson Tonight | Hannity |
| 2021-01-09 | Watters World | Justice With Judge Jeanine | The Greg Gutfeld Show |
| 2022-04-24 | Life, Liberty & Levin | The Next Revolution With Steve Hilton | Sunday Night in America With Trey Gowdy |
| 2023-01-21 | One Nation With Brian Kilmeade | Unfiltered With Dan Bongino | Lawrence Jones Cross Country |
| 2024-01-31 | Fox News at Night | The Five | Jesse Watters Primetime |
MSNBC (12)
| Date (ET) | 8–9 PM | 9–10 PM | 10–11 PM |
|---|---|---|---|
| 2020-03-13 | The 11th Hour With Brian Williams | The Rachel Maddow Show | The Last Word With Lawrence O'Donnell |
| 2021-01-09 | The Week With Joshua Johnson | The Week With Joshua Johnson | — (Dateline Extra excluded) |
| 2022-04-24 | The Mehdi Hasan Show | Ayman | — (Dateline excluded) |
| 2023-01-21 | American Voices With Alicia Menendez | Ayman | — (Dateline excluded) |
| 2024-01-31 | The 11th Hour With Stephanie Ruhle | Alex Wagner Tonight | The Last Word With Lawrence O'Donnell |
CNN (9)
| Date (ET) | 8–9 PM | 9–10 PM | 10–11 PM |
|---|---|---|---|
| 2020-03-13 | CNN Tonight With Don Lemon | Anderson Cooper 360 | Cuomo Prime Time |
| 2021-01-09 | — (documentary excluded) | CNN Newsroom Live | CNN Newsroom Live |
| 2022-04-24 | — (documentary excluded) | — (documentary excluded) | CNN Newsroom Live |
| 2023-01-21 | — (special excluded) | — (special excluded) | — (special excluded) |
| 2024-01-31 | Laura Coates Live | Anderson Cooper 360 | The Source With Kaitlan Collins |
Nine documentaries/specials outside regular prime-time news were excluded from the final 36-episode corpus.
Path: LogoAndPostScreenshots/
Each CSV lists frames that contain at least one visual social media signal (logo and/or post screenshot). Frame files follow:
{episodeID}-{second}.jpg
Example: CNNW_20200314_030000_CNN_Tonight_With_Don_Lemon-000136.jpg is the frame at second 136 of that episode and (in the gold standard) shows both a Twitter logo and a Twitter post screenshot.
| Column | Description |
|---|---|
filename |
One-second frame filename (episodeID-second.jpg) |
Social Media Logo |
Yes / No — whether a social media platform logo is visible |
Social Media Logo Type |
Platform(s) for the logo (e.g., Twitter, TikTok, Twitter (X), Instagram) |
Social Media Screenshot |
Yes / No — whether a social media post screenshot is visible |
Social Media Screenshot Type |
Platform(s) for the screenshot (e.g., Twitter, Truth Social, No Platform) |
| Channel | Annotated frames | Logo = Yes | Screenshot = Yes | File |
|---|---|---|---|---|
| CNN | 229 | 154 | 221 | csv |
| Fox News | 768 | 728 | 188 | csv |
| MSNBC | 448 | 448 | 282 | csv |
| Total | 1,445 | 1,330 | 691 |
Frames without logos or screenshots are not listed; for full-episode evaluation, treat unlisted seconds in the 36 episodes as negatives.
Path: ClosedCaptions/
Each row is one closed-caption segment. Caption text is not redistributed. Labels and timing metadata are provided, plus a clip_url pointing into the Internet Archive so you can inspect the corresponding audiovisual context.
| Column | Description |
|---|---|
date |
Broadcast date in the Archive identifier (UTC calendar date) |
channel |
CNN, FOXNEWS, or MSNBC |
episodeid |
TV News Archive episode identifier |
clip_url |
60-second Archive clip containing the caption midpoint |
starttime / endtime |
Caption segment times (HH:MM:SS,mmm) |
starttime_seconds / endtime_seconds |
Same bounds in seconds |
Social Media Mention |
Yes (social media reference), No, or Ad (commercial break) |
Social Media Platform |
Platform(s) mentioned when Social Media Mention is Yes (e.g., Twitter, Social Media, TikTok) |
Archive.org serves programs as contiguous 60-second windows (0–60, 60–120, …). For each caption segment with midpoint (m = (t_{\mathrm{start}} + t_{\mathrm{end}}) / 2):
https://archive.org/details/{episodeid}/start/{T}/end/{T+60}
where (T = 60 \lfloor m / 60 \rfloor). Boundary ties prefer the window with greater overlap (earlier window on a remaining tie). Caption clocks can be slightly offset from video; treat clip_url as a retrieval pointer, not an exact cut.
Example clip:
https://archive.org/details/CNNW_20200314_030000_CNN_Tonight_With_Don_Lemon/start/420/end/480
| Channel | Segments | Mention = Yes | Mention = Ad | File |
|---|---|---|---|---|
| CNN | 14,764 | 83 | 2,028 | csv |
| Fox News | 22,561 | 121 | 3,257 | csv |
| MSNBC | 22,657 | 81 | 4,775 | csv |
| Total | 59,982 | 285 | 10,060 |
Social2TV does not ship video frames or closed-caption text.
| Need | How |
|---|---|
| Watch a full episode | https://archive.org/details/{episodeid} |
| Watch a time range | https://archive.org/details/{episodeid}/start/{S}/end/{E} |
| One-second frames / bulk captions | Contact the Internet Archive (see below) |
| Explore caption mention volume | GDELT Television Explorer |
For bulk research access beyond the public interface, contact roger@archive.org and info@archive.org with a description of the proposed research and intended use.
- Visual detection / multimodal LLM evaluation: internetarchive/tvnews_socialmedia_mentions (GSoC 2025)
- Annotations in this repository are released for research use under the MIT License.
- Television media and closed-caption text are owned by rights holders / the Internet Archive and are not redistributed here.
- Use episode IDs, timestamps, and
clip_urlvalues to retrieve corresponding material from the TV News Archive. - Do not treat this positive-enriched set as a prevalence sample of platforms or networks.
We thank Roger Macdonald (founder, Internet Archive TV News Archive), Mark Graham (Director, Internet Archive), and Kalev Leetaru (GDELT Project) for supporting this research and facilitating access to Television News Archive data.
Annotation files and documentation in this repository are released under the MIT License. Source television media and closed captions are not included in this release and remain subject to the copyright and usage terms of the Internet Archive.