Abstract/Summary

Purpose: This research aims to extract emotional features from New York Times news images (2018–2023) using few-shot learning approaches. Leveraging machine learning, it offers a systematic investigation into how image-driven emotions affect investor behavior in the U.S. equity market and contribute to the prediction of market movements. Design: This study employs the DeepEMD model to extract emotional features from 181,233 news images, constructing a daily sentiment index based on visual media. By defining sentiment thresholds, the study develops differentiated strategies for positive and negative emotional signals. In addition, it integrates four machine learning models—AdaBoost, Support Vector Machine (SVM), ExtraTrees, and Random Forest (RF)—alongside a traditional linear regression model to forecast the prices of various U.S. stock market indices. Findings: This study finds that news image sentiment has a significant impact on financial markets. Positive sentiment strategies applied to serious news topics are associated with higher returns, whereas negative sentiment in entertainment-related content signals potential opportunities for contrarian investment. Moreover, the influence of image-based sentiment on the market exhibits a delayed effect of approximately 2–3 days, with particularly strong predictive power for small-cap stocks. Compared to traditional linear models, machine learning approaches demonstrate superior performance in capturing the nonlinear dynamics between sentiment and market behavior, offering novel analytical tools for behavioral finance research and sentiment-driven anomaly-based investment strategies. Value: This study integrates visual data analysis into the domain of behavioral finance, highlighting the distinctive role of image-based sentiment in uncovering market anomalies and informing investment strategies.