Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives

Metadata

Publisher: SMPTE
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Motion Imaging Jour.
Volume: 135, No. 5, pp. 28–37
Abstract: Large-scale video archives, such as the NHK Archives with over ten million assets, face critical retrieval challenges: manual metadata is costly and incomplete. At the same time, cloud-based AI solutions introduce unacceptable data security risks. This paper presents AXIS (Archives Cross-modal Intelligent Search), a metadata-free retrieval system built on three integrated components. First, keyframe-centric processing reduces the analysis corpus by 100-1,000× versus full-frame processing–making billion-scale analysis computationally tractable. Second, offline deployment of a Japanese-language Contrastive Language-Image Pretraining (CLIP) model enables on-premise semantic feature extraction, ensuring data sovereignty. Third, Approximate Nearest Neighbor (ANN) indexing via Facebook AI Smiliary Search (FAISS) enables sub-second retrieval at scale. Evaluated on a 30-million-keyframe subset of the NHK Archives, the system achieves a mean query latency of 0.48 sec (text) and 0.64 sec (image), processing 8.4 million keyframes per day on commodity CPU hardware. The hybrid on-premise/cloud architecture supports intuitive multimodal search while preserving institutional data security.
Publication Date: 2026-09-01
DOI: 10.5594/JMI.2026/CJWA9953
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2026/CJWA9953
Author(s): Yo Narita
Keyword(s): Nhk Archives, Content Retrieval, Keyframe-Centric Processing, CLIP
Copyright: © 2026 SMPTE

Bibliographic Reference(s)

A. Radford, “Learning Transferable Visual Models From Natural Language Supervision,” Proc. 38th Int. Conf. Machine Learning (ICML), 139: 8748–8763, 2021. EXTERNAL
J. Johnson, M. Douze, and H. Jégou, “Billion-Scale Similarity Search With GPUs,” IEEE Trans. Big Data, 7 (3): 535–547, Jul. 2021. EXTERNAL
M. Flickner, “Query by Image and Video Content The QBIC System,” IEEE Computer, 28 (9): 23–32, Sep. 1995. EXTERNAL
A. F. Smeaton, P. Over, and A. R. Doherty, “Video Shot Boundary Detection: Seven Years of TRECVid Activity,” Comput. Vis. Image Understand., 114 (4): 411–418, Apr. 2010. EXTERNAL
J. Xu, “MSR-VTT: A Large Video Description Dataset for Bridging Video and Language,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR): 5288–5296, 2016. EXTERNAL
F. Caba Heilbron, “ActivityNet A Large-Scale Video Benchmark for Human Activity Understanding,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR): 961–970, 2015. EXTERNAL
H. Xu, “VideoCLIP: Contrastive Pre-Training for Zero-Shot Video-Text Understanding,” Proc. Conf. Empirical Methods Natural Language Processing (EMNLP): 6787–6800, 2021. EXTERNAL
M. Bain, “Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval,” Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV): 1728–1738, 2021. EXTERNAL
LY Corporation, “clip-japanese-base,” Hugging Face, 2024. [Online]. Available: https://huggingface.co/line-corporation/clip-japanese-base EXTERNAL
Y. Fang, “EVA-02: A Visual Representation for Neon Genesis,” Image Vis. Comput., 149: 105171, Sep. 2024. EXTERNAL
FFmpeg Developers, “FFmpeg.” [Online]. Available: https://ffmpeg.org/ EXTERNAL
Amazon Web Services, “Amazon OpenSearch Service,” AWS Documentation, 2024. [Online]. Available: https://aws.amazon.com/opensearch-service/ EXTERNAL
Y. A. Malkov and D. A. Yashunin, “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs,” IEEE Trans. Pattern Anal. Mach. Intell., 42 (4): 824–836, Apr. 2020. EXTERNAL
R. Guo, “Accelerating Large-Scale Inference With Anisotropic Vector Quantization,” Proc. 37th Int. Conf. Machine Learning (ICML): 3887–3896, 2020. EXTERNAL
S. Ramirez, “FastAPI,” 2025. [Online]. Available: https://fastapi.tiangolo.com/ EXTERNAL
A. Paszke, “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” Adv. Neural Inf. Process. Syst., 32, 2019. EXTERNAL
A. Mahendran and A. Vedaldi, “Understanding Deep Image Representations by Inverting Them,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR): 5188–5196, 2015. EXTERNAL
American Archive of Public Broadcasting (AAPB), “About the AAPB,” GBH and Library of Congress, 2024. [Online]. Available: https://americanarchive.org/about-the-american-archive EXTERNAL
International Federation of Film Archives (FIAF), The FIAF Moving Image Cataloguing Manual. FIAF Press: Brussels, Belgium, 2016. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2026-CJWA9953.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Yo Narita; Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives, MIJ 2026, Volume 135, Number 5 (pp. 28 to 37); SMPTE, 2026. Available at https://doi.org/10.5594/JMI.2026/CJWA9953

Snippet:

Yo Narita; Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives, MIJ 2026, Volume 135, Number 5 (pp. 28 to 37); SMPTE, 2026. Available at https://doi.org/10.5594/JMI.2026/CJWA9953

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Yo Narita; <cite>Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives</cite>, MIJ 2026, Volume 135, Number 5 (pp. 28 to 37); SMPTE, 2026. Available at <a href="https://doi.org/10.5594/JMI.2026/CJWA9953" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2026/CJWA9953</a></span>

SMPTE's HTML Pub

Preview:

Yo Narita; Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives, MIJ 2026, Volume 135, Number 5 (pp. 28 to 37); SMPTE, 2026
doi: 10.5594/JMI.2026/CJWA9953
url: https://doi.org/10.5594/JMI.2026/CJWA9953

Snippet:

<li>
Yo Narita; <cite id="bib-10-5594-jmi-2026-cjwa9953">Keyframe-Based Multimodal Video Archive Retrieval Using CLIP Embeddings: A Scalable and Secure Framework for NHK Archives</cite>, MIJ 2026, Volume 135, Number 5 (pp. 28 to 37); SMPTE, 2026
<span class="doi">10.5594/JMI.2026/CJWA9953</span>
</li>