Articles

In an era where the value of journalism is increasingly scrutinised, understanding its impact presents a complex puzzle. From traditional metrics to innovative methodologies, the quest to measure impact evolves, with new tools emerging.

Both journalists and donors recognise that the media operates within a larger ecosystem, making it difficult to isolate and measure the precise impact of journalism. Furthermore, impact can vary significantly depending on the goals and priorities of different journalists and newsrooms. For instance, while one news organisation might prioritise educating its audience and measure impact by assessing changes in public opinion on contentious issues, other outlets may focus on different indicators.

Still, journalists and media organisations worldwide are engaged in assessing the impact of their work, driven by a dual motivation of self-evaluation and the need to demonstrate value to donors, investors, and the public. Increasingly, media outlets recognise that communicating the positive outcomes of their work not only boosts trust and loyalty among audiences but also holds potential for revenue growth.

This shift reflects a common belief among funders that investing in journalism requires tangible evidence of impact, moving beyond viewing it solely as a public good. Such recognition underscores the need for donors to evaluate their investments in media and journalism programmes thoroughly.

Insights into measuring impact, such as those outlined by the International Journalists’ Network, highlight its multifaceted nature, encompassing not only the dissemination of information but also its broader societal effects. From shaping public opinion and encouraging public discourse to driving policy change, the impact of journalism spans various phases, often defying simple cost-benefit analyses. Moreover, methodologies for measuring impact extend beyond traditional metrics like online engagement, with a broad range of tools developed by academics. Add to that cross-border collaborations, or even negative consequences, such as potential backlash against journalists.

To analyse impact, there are a lot of different studies across various fields like economics and political science, focusing on topics from government spending and corruption to voter behaviour, argues Anya Schiffrin, director of the media, technology, and communications specialisation at Columbia University’s School of International and Public Affairs (SIPA), adding that these “measurement tools are becoming more and more sophisticated.”

The Impact Dashboard, developed by Pluralis in collaboration with the Media and Journalism Research Center (MJRC), is one of such tools. It evaluates supported media organisations across three key dimensions: long-term sustainability, plurality, and accessibility to information, analysing impact on three levels. On the micro level, it tracks how supported organisations change in terms of revenues and audience reach over the course of working with the funder. On the meso level, the Dashboard collects evidence to measure the level of media plurality in the country. Finally, on the macro level, it examines the impact on society, searching for evidence such as potential policy changes resulting from coverage by the media receiving the grant.

Another approach is proposed by Schiffrin, who, with Andre Correa d’Almeida, Lindsay Green-Barber, Adelina Yankova, and Dylan W. Groves developed a multi-faceted metric system to analyse media impact. As they argue, three primary research strands contribute to understanding media impact: social scientists focus on identifying causal effects, often related to citizen knowledge, attitudes, and government responses; media researchers offer accounts of causal processes and diverse media effects; and media practitioners provide an insider’s view, highlighting the impact on journalists and media organisations. These approaches can complement each other, hence their proposed taxonomy that can unify measures of media impact, and inform decisions by practitioners and donors.

The taxonomy comprises three levels of impact. At the individual level, media reports influence beliefs, attitudes, and behaviours. The network/organisation level involves the collective impact on social networks, norms, and actions, including effects within media organisations and journalist communities. The institution level pertains to long-term effects on institutions and culture. The taxonomy also distinguishes between external impact on society and internal impact within the journalism community, providing a framework that can be adapted to different objectives, acknowledging the diverse goals of donors, activists, journalists, and media outlets. “One can look at the table and then measure those metrics their organisation cares about,” Schiffrin said.

In evaluating the impact of journalism, it is important to recognise that some news organisations have smaller audiences rendering metrics like pageviews or listenership alone inadequate. Furthermore, while one traditional measure of journalism’s impact involves influencing government policy or prompting officials to address issues, achieving such outcomes often demands sustained reporting over an extended period.

Furthermore, in countries with more autocratic rulers and intensified attacks on journalism, government responses to policy concerns or official misconduct exposed by investigative reporting cannot be expected. In fact, in such landscapes if donor funding contributes to the survival of a news organisation, it can already be perceived as having a significant impact. “It is important to keep the flame alive,” Schiffrin said. “You don’t want these outlets to die.” She added that, when measuring impact, journalism donors should avoid burdening grantees with overly demanding reporting requirements, focusing instead on listening to their feedback.