Assessing the Aptitude of Language Models in Comprehending Advertisements

[ongoing] Advertisement media are fundamentally different from typical videos and images. They are more than just their content, persuade users to take certain actions, and often use creative atypicalities to deliver their message.
Advertisement images from the Kovashka Ads Dataset were textually verbalized. These were presented to text‐based language models like GPT‐3.5, GPT‐4, and FLAN‐T5, which were evaluated on Action‐Reason pair and Atypicality understanding tasks.
As a comparison between text and vision, Vision‐Language models like BLIP2 were also evaluated on the advertisement images.
Supervised by: Adriana Kovashka - University of Pittsburgh