In an era where the value of journalism is increasingly scrutinised, understanding its impact presents a complex puzzle. From traditional metrics to innovative methodologies, the quest to measure impact evolves, with new tools emerging.
Both journalists and donors recognise that the media operates within a larger ecosystem, making it difficult to isolate and measure the precise impact of journalism. Furthermore, impact can vary significantly depending on the goals and priorities of different journalists and newsrooms. For instance, while one news organisation might prioritise educating its audience and measure impact by assessing changes in public opinion on contentious issues, other outlets may focus on different indicators.
Still, journalists and media organisations worldwide are engaged in assessing the impact of their work, driven by a dual motivation of self-evaluation and the need to demonstrate value to donors, investors, and the public. Increasingly, media outlets recognise that communicating the positive outcomes of their work not only boosts trust and loyalty among audiences but also holds potential for revenue growth.
This shift reflects a common belief among funders that investing in journalism requires tangible evidence of impact, moving beyond viewing it solely as a public good. Such recognition underscores the need for donors to evaluate their investments in media and journalism programmes thoroughly.
Insights into measuring impact, such as those outlined by the International Journalists’ Network, highlight its multifaceted nature, encompassing not only the dissemination of information but also its broader societal effects. From shaping public opinion and encouraging public discourse to driving policy change, the impact of journalism spans various phases, often defying simple cost-benefit analyses. Moreover, methodologies for measuring impact extend beyond traditional metrics like online engagement, with a broad range of tools developed by academics. Add to that cross-border collaborations, or even negative consequences, such as potential backlash against journalists.
To analyse impact, there are a lot of different studies across various fields like economics and political science, focusing on topics from government spending and corruption to voter behaviour, argues Anya Schiffrin, director of the media, technology, and communications specialisation at Columbia University’s School of International and Public Affairs (SIPA), adding that these “measurement tools are becoming more and more sophisticated.”
The Impact Dashboard, developed by Pluralis in collaboration with the Media and Journalism Research Center (MJRC), is one of such tools. It evaluates supported media organisations across three key dimensions: long-term sustainability, plurality, and accessibility to information, analysing impact on three levels. On the micro level, it tracks how supported organisations change in terms of revenues and audience reach over the course of working with the funder. On the meso level, the Dashboard collects evidence to measure the level of media plurality in the country. Finally, on the macro level, it examines the impact on society, searching for evidence such as potential policy changes resulting from coverage by the media receiving the grant.
Another approach is proposed by Schiffrin, who, with Andre Correa d’Almeida, Lindsay Green-Barber, Adelina Yankova, and Dylan W. Groves developed a multi-faceted metric system to analyse media impact. As they argue, three primary research strands contribute to understanding media impact: social scientists focus on identifying causal effects, often related to citizen knowledge, attitudes, and government responses; media researchers offer accounts of causal processes and diverse media effects; and media practitioners provide an insider’s view, highlighting the impact on journalists and media organisations. These approaches can complement each other, hence their proposed taxonomy that can unify measures of media impact, and inform decisions by practitioners and donors.
The taxonomy comprises three levels of impact. At the individual level, media reports influence beliefs, attitudes, and behaviours. The network/organisation level involves the collective impact on social networks, norms, and actions, including effects within media organisations and journalist communities. The institution level pertains to long-term effects on institutions and culture. The taxonomy also distinguishes between external impact on society and internal impact within the journalism community, providing a framework that can be adapted to different objectives, acknowledging the diverse goals of donors, activists, journalists, and media outlets. “One can look at the table and then measure those metrics their organisation cares about,” Schiffrin said.
In evaluating the impact of journalism, it is important to recognise that some news organisations have smaller audiences rendering metrics like pageviews or listenership alone inadequate. Furthermore, while one traditional measure of journalism’s impact involves influencing government policy or prompting officials to address issues, achieving such outcomes often demands sustained reporting over an extended period.
Furthermore, in countries with more autocratic rulers and intensified attacks on journalism, government responses to policy concerns or official misconduct exposed by investigative reporting cannot be expected. In fact, in such landscapes if donor funding contributes to the survival of a news organisation, it can already be perceived as having a significant impact. “It is important to keep the flame alive,” Schiffrin said. “You don’t want these outlets to die.” She added that, when measuring impact, journalism donors should avoid burdening grantees with overly demanding reporting requirements, focusing instead on listening to their feedback.
The Ethical Media Alliance (EMA) in Romania aims to tackle quantitative metrics: the main flaw of the digital advertising ecosystem that undermines public interest journalism. The initiative aims to allocate funds based on ethical principles to support trustworthy media in achieving a positive social impact and financial sustainability, while also securing brand safety for advertisers.
In the age of web 2.0, the advertising market disproportionally incentivises clickbait content. The quest for the higher reach at the lowest cost has led to an emphasis on viral content, often at the expense of responsible, public interest journalism. However, amidst this challenging landscape, initiatives like the Ethical Media Alliance (EMA) in Romania are emerging as beacons of hope for independent media organisations, especially those dedicated to public interest journalism.
The initiative was born as a “result of frustration over how much money is funnelled to irresponsible content producers,” says Dragos Stanca, EMA’s initiator. He thinks that there is a critical flaw in the digital advertising ecosystem, where the focus on quantitative metrics such as clicks and impressions undermines the value of public interest journalism. This has led even serious publishers to embrace clickbait content to survive in the era of programmatic advertising.
In the first phase of the project, EMA positions itself as a not-for-profit sales house guided by ethical principles, involving journalistic startups and projects often excluded from commercial funding due to their relatively modest audience numbers. Stanca acknowledges the necessity of speaking the language of the advertisers, thereby integrating metrics and key performance indicators (KPIs) into their approach. Additionally, EMA aims to foster a positive social impact by supporting content essential for democracy.
The network currently includes 15 journalistic projects employing over 120 journalists, with a joint monthly reach of 1.2 million users and 550,000 video views on average. Advertisers are required to commit to a minimum one-month campaign that spans across all portals. In other words, the same ad is on display on all the sites thus increasing its reach to a level that, as Stanca puts it, “makes sense for a media buyer.” EMA ensures brand safety by allowing only organisations producing public interest content to join. This guarantees advertisers that their ads will be associated with responsible content.
Furthermore, EMA also reforms the distribution of ad revenue in the network. Half of it is distributed among partners based on the number of employed journalists, while the other half is based on quantitative metrics (35% based on the number of ad impressions and 15% on social media reach). This follows the usual ‘cost per mille’ (CPM) approach, which is the cost an advertiser pays for one thousand views or impressions of an advertisement.
The initiative sets an ambitious target: diverting 1% of the total ad spending in Romania to public interest journalism. In Romania, where the total ad market is €700m a year, with only €30-35m spent online by local companies (out of a total digital ad spending of €255m), the EMA initiative has the potential to double the funds allocated to digital journalism. Currently, no more than €3.5-5 million a year is allocated to digital journalism, according to the initiators of the Ethical Media Alliance.
Stanca believes that shifting even relatively smaller amounts to trustworthy media could significantly enhance their financial sustainability. Particularly for emerging journalistic startups, even a few thousand euros per month can make a significant difference.
Early successes are evident, with the initiative launching last fall and the first campaigns commencing in October. The two largest banks in Romania have joined to date, contributing €35,000 for the first two months as a test campaign.
“Drawing from our experience in the commercial digital brokerage market, I can confidently say that anything that is new takes up to one year to become adopted by the market,” says Stanca, adding that he aims for an ad spending of €100,000 per month by the end of the year.
Recently, he outlined the operating principles of the alliance and presented what his team considers to be an initial format for ethical advertising in a dialogue hosted by independent journalist Petrisor Obae, who operates the media-focused portal Pagina de Media. The principles the EMA operates on aim to create an “ethical algorithm” to be used for the allocation of funds from the ad space. The principles the EMA operates on aim to create an “ethical algorithm” to be used for the allocation of funds from the ad space. The primary goal is to provide enough resources to motivate especially young people to choose a career in journalism, according to Stanca. Moreover, the EMA wants to motivate them to produce content in and for the public interest, “not just to focus on gaining the programmatic advertising revenues or, more seriously, to exclusively serve the interests of media owners with questionable agendas or to write solely for and about brands that are essentially seeking disguised advertising,” Stanca added. “The Ethical Media sales house part is just the first phase of the project; we plan to propose additional initiatives that support an ecosystem which, in our opinion, is essential for the survival of democracy,” he said.
While the initiative is currently confined to Romania, Stanca is open to expansion. As EMA pioneers ethical advertising to support public interest journalism, it could have an impact far beyond national boundaries, ushering in a new era for responsible media funding.
A new study by Maria Latos, Frank Lobigs, and Holger Wormer, TU Dortmund, attempts to systematically transfer established funding models in research to journalism, where the state is involved in funding, but peer review models reduce funding bias. Using the example of the German Research Foundation (GRF), the authors developed a concept for a German Journalism Foundation, which awards funding to journalists and cooperative projects based on a peer review process.
The peer-based journalism funding model aims to address the challenges faced by media organizations by promoting excellence in journalistic projects through a competitive peer review process. It suggests awarding funding to projects in areas that are often neglected due to time and cost constraints, such as investigative journalism and fact-checking. The proposed funding areas also include innovations, infrastructure, technology, training, and in-depth journalism.
The concept involves creating an association under private law, similar to GRF, that would allow journalists, editorial teams, and cooperative projects to organize independently. The review process for funding proposals involves a combination of methods, including written statements for individual funding and group decision-making for coordinated programs. It is suggested to have interdisciplinary review groups, which include non-journalists from foundations, NGOs, and communication scientists, to ensure a comprehensive evaluation. Emphasis is placed on clear criteria for reviewer selection and application evaluation.
The study highlights the need for alternative measures to determine eligibility for journalism funding, suggesting certification or registration procedures that are aligned with journalistic principles. Funding sources could include a combination of reallocating funds from public media, direct financing from the government, and contributions from foundations and private donors.
The proposed funding amount is conceptualized with a gradual approach, initially starting with smaller amounts and eventually reaching a maximum estimate of approximately €700 million per year in Germany, which falls within the same range as the annual VAT reduction for newspapers in the country. This funding model aims to compensate for the economic challenges faced by journalism, ensuring a more targeted and sustainable approach to supporting the industry.
Latos, M., Lobigs, F., & Wormer, H. (2023). Peer-based research funding as a model for journalism funding. Journalism, 0(0). https://doi.org/10.1177/14648849231215662