24 Nov 2021 GMMP 2020: Methodology

A key characteristic of longitudinal research is the assessment of change over time on the observed indicators.   In the case of the 2020 Global Media Monitoring Project (GMMP), the methodology, indicators, approach to data collection and analysis are consistent with past editions of the research in order to allow for the usual historical comparisons.

The global monitoring day scheduled initially for the first quarter of 2020 was postponed to later in the year due to the upheavals caused by the first coronavirus (Covid-19) wave worldwide. As the April monitoring day approached, it became quickly clear that proceeding as planned would result in a news sample that would be almost entirely focused on coronavirus stories. A new need emerged to address the practicalities of monitoring during the lockdowns and curfews imposed to contain the spread of the virus, as the regular sit-down communal coding sessions were now out of the question for most teams.

The GMMP technical advisory group and the database development team Code for Africa¹ worked to systematically address the issues. A new monitoring date was set for September, the coding tools were tweaked to capture Covid-19 stories without compromising on the ability to compare results across time based on story topics, exhaustive audio-visual training resources on how to code in a pandemic were put in place, electronic coding instruments were developed and the teams were re-trained in numerous webinars.

As with previous editions of the GMMP, the initial data capture was conducted offline by volunteer teams across the 116 participating countries. For the 2020 GMMP, a spreadsheet version of the coding sheets was provided, to allow for electronic recording of the observations.

In the period leading up to the monitoring day, a series of regional and national training sessions were organised to build a uniform understanding of the teams on the methodology and approach to coding. The teams received training on media selection, newscast and article selection, and the number of media to code.

For the 2020 GMMP, teams could choose from two possible options for the monitoring:

• Full monitoring, whose results provide a comprehensive picture of the status of gender equality dimensions in news media.
• Short monitoring, a shorter version which focuses on the key GMMP indicators, for teams who wished to participate but for various reasons could not implement the full monitoring.

To ensure accuracy in the coding process, radio and television bulletin were recorded, and copies of digital and print media items were collected. Across the different media types- both for the full and short monitoring-monitors captured information about the story, its main themes and the people in the story, as journalists, as story subjects and sources.

Additionally, three optional special questions, unique to each country, allowed individual countries to analyse issues of national interest. For standardisation purposes, as well as the multilingual nature of this study, all responses were numerically coded from fixed lists.

To enable comparability of data gathered from a pandemic-heavy news agenda with the historical results, an additional question was included which asked whether the story was related to Covid-19. For such stories, monitors were requested to select the most relevant secondary topic. While global news stories had diversified to pre-pandemic levels by the global monitoring day in September 2020, the regional analysis demonstrated the significance of this question, particularly for North America and the Middle East, which recorded 37% and 36% of Covid-19-related stories respectively.

Media bands

The media bands system was introduced in 2005 to ensure a more even spread of data and also serve as each country’s reference point on the minimum number of media to monitor. This system was retained for the 2020 GMMP and was updated with the input of country coordinators.

Weighting

While the GMMP seeks to understand how gender is represented in media across the world, differences in media access and impact across the participating countries mean that a simple aggregation of the data would lead to biased results. For example, if a country like France submitted data from 100 media, the entries from a smaller country like Fiji would have little, if any, impact on the results.

Additionally, while two countries may have similar numbers of newspapers, their impact, in terms of the number of people who read them, may be significantly different. To address these challenges, GMMP 2020 updated, re-tested and applied the weighting system first developed for the 2005 edition.

Accuracy

The GMMP involved several thousand people across 116 countries from diverse gender and media stakeholder groups, with different research abilities and working in a wide range of languages. For a study of this scale, it was crucial that accuracy was considered at each stage, to maintain the high levels achieved in previous years.

Data entry and processing errors can have severe biasing effects on the data analysis, resulting in misrepresentation of the observed variables. To minimise this risk, we leveraged on a variety of automated processes, as well as the extensive media monitoring experience of the country coordinators.

Limitations

As with any study, great effort was made to ensure accuracy of the data. As observed in previous GMMPs, an exact error of measurement cannot be determined due to the study’s magnitude. Conventional error measurement would involve different researchers coding the same story and then calculating a level of error from the differences between the results. Although this was not possible for GMMP, we followed best practice to make sure that there were minimal errors in the data capture and analysis generation process.

A fuller discussion of the methodology used can be found in the GMMP 2020 Final Report.

Note

1. Code for Africa (CfA) is the continent’s largest network of indigenous African civic technology and investigative data journalism laboratories, with over 70 staff in 19 countries, who build digital democracy solutions that are intended to give citizens unfettered access to actionable information that empowers them to make informed decisions and that strengthen civic engagement for improved public governance and accountability.

Photo: A group of women journalists in Malaga, Spain, carry out a monitoring exercise using the GMMP 2020 methodology. Photo: AMP.