Skip to Content View Previous Reports

Methodologies

The data for this study were collected in two parts. The first part consists of data originally conducted by other people or organizations that PEJ then collected and aggregated. The second part, particularly the content analysis, is original work conducted specifically for this report.

For the data aggregated from other researchers, the Project took several steps. First, we tried to determine what data had been collected and by whom for the eight media sectors studied. In many cases this included securing rights to data through license fees or other means. We organized the data into the seven primary areas of interest we wanted to examine: content, audience, economics, ownership, newsroom investment, alternative news outlets and digital trends.

Next, the Project studied the data closely to determine where elements reinforced each other and where there were apparent contradictions or gaps. In doing so, the Project endeavored to determine the value and validity of each data set. That in many cases involved going back to the sources that collected the research in the first place. Where data conflicted, we have included all relevant sources and tried to explain their differences, either in footnotes or in the narratives.

In analyzing the data for each media sector, we sought insight from experts by having at least three outside readers for each sector chapter. Those readers raised questions, offered arguments and questioned data where they saw fit.

All sources are cited in footnotes or within the narrative, and listed alphabetically in a source bibliography. The data used in the report are also available in more complete tabular form online, where users can view the raw material, sort it on their own and make their own charts and graphs. Our goal was not only to organize the available material into a clear narrative, but to also collect all the public data on journalism in one usable place. In many cases, the Project paid for the use of the data.

In addition, PEJ conducted original research in a number of special reports and features. The methodologies for each can be found below. You can scroll through them all or click to go directly to the report of interest.

Mobile Devices and News Consumption
What Facebook and Twitter Mean for News
A Year in the News Content Analysis

Mobile Devices and News Consumption/
What Facebook and Twitter Mean for News

Two special reports, Mobile Devices and News Consumption, and What Facebook and Twitter Mean for News, are based on aggregated data from three telephone surveys conducted in January 2012 (Jan. 12-15, Jan. 19-22 and Jan. 26-29) with national samples of adults 18 years of age or older living in the continental United States. Interviews were conducted with a total of 3,016 adults (1,809 respondents were interviewed on a landline telephone, and 1,207 were interviewed on a cellphone, including 605 who had no landline telephone). The survey was conducted by interviewers at Princeton Data Source under the direction of Princeton Survey Research Associates International. A combination of landline and cellphone random digital dial samples was used; both samples were provided by Survey Sampling International. Interviews were conducted in English. Respondents in the landline sample were selected by randomly asking for the youngest adult who was at home. Interviews in the cellphone sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older.

The combined landline and cellphone sample are weighted using an iterative technique that matches gender, age, education, race, Hispanic origin and region to parameters from the March 2011 Census Bureau’s Current Population Survey and population density to parameters from the Decennial Census. The sample also is weighted to match current patterns of telephone status, based on extrapolations from the 2011 National Health Interview Survey. The weighting procedure also accounts for the fact that respondents with both landline and cellphones have a greater probability of being included in the combined sample and adjusts for household size within the landline sample. Sampling errors and statistical tests of significance take into account the effect of weighting. The following table shows the sample sizes and the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the survey:

Sample sizes and sampling errors for other subgroups are available upon request.

In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.

A Year in the News

A number of people at the Project for Excellence in Journalism worked on this report. Associate Director Mark Jurkowitz and Director Tom Rosenstiel wrote the report. The data was compiled by senior researcher Paul Hitlin, senior methodologist Hong Ji and content & training coordinator Mahvish Shahid Khan. Charts were created by manager of the Weekly News Index Tricia Sartor and researcher/coder Steve Adams. Communications and Creative Design Manager Dana Page handled the web and communications. Researcher Nancy Vogt copy-edited the text. Coding of news content was conducted by the people above plus the following researchers: Steve Adams, Monica Anderson, Jeff Beattie, Heather Brown, Kevin Caldwell, Emily Guskin, Jesse Holcomb, Katerine Matsa, Kenny Olmstead, Dana Page, Angela Sanson, Laura Houston Santhanam and Sovini Tan.

The study, The Year in the News 2011, is based primarily on the real-time content analysis data derived from the Project for Excellence in Journalism’s in-house news coding operation.

The data regarding media coverage from traditional news outlets come from a summative analysis of the weekly reports known as PEJ’s News Coverage Index . (Click here for a detailed methodology on how this real-time weekly coding is conducted.) PEJ began the NCI in January 2007, and it is the largest effort in the United States that measures and analyzes the agenda of the American news media on a continuing basis. The Index examines 52 news outlets in real time to determine what is being covered and what is not, who are the leading newsmakers and other elements of the news each week. A team of coders works on the project. Each member of the coding team has been given extensive training in PEJ’s methods and met high standards of reliability. The work is measured regularly for inter-coder reliability and agreement. Coders read, listen-to or watch each piece of news content from newspapers, online news sites, television broadcasts and radio programming.

The data regarding social media, specifically Twitter and blogs, come from the weekly reports known as PEJ’s New Media Index. (Click here for a detailed methodology.) The goal of the NMI is to measure the leading topics of conversation on various social media outlets. PEJ relies on a combination of human coding and tracking services for social media to derive this analysis. The tracking services measure the links present on blog posts or tweets to determine the most discussed issues. When a social media user links to a page online, it suggests that the user believes the page is important, even if they do not agree with the contents. PEJ relies on four social media tracking sites (Tweetmeme, Technorati, Icerocket and Twitteruly) to tabulate the most linked-to pages.

PEJ began the New Media Index in January 2009. In August 2011 we made some changes to the methodology to improve the system. Those adjustments are described in detail here. There were two major changes. First, PEJ expanded the number of tracking sites used from two to four. Second, PEJ expanded the number of sources from which content could originate. Previously, the NMI was focused on “news” stories as determined by tracking sites which were mostly from traditional news outlets such as CNN.com and Washingtonpost.com. However, the current methodology places no such restrictions on where content can originate.

The report also uses survey data from the Pew Research Center for the People & the Press. In particular, the survey results regarding public interest come from their weekly News Interest Index.