Case Study

Beginning in March last year, a   year-long media-based campaign was aired on radio and television and in print   in British Columbia. Aimed at smokers in the age group of twenty to thirty   years, it is intended to influence them in ways that will lead them to stop   smoking.

The logic model for the smoking   cessation campaign is quite simple and reflects the prevailing views about   program theory that are relevant to this kind of campaign. Given below is a   part of that logic model, focusing on outcomes. The components,   implementation activities, and outputs are not shown, but they focus on the   amounts of media coverage on the radio and television and in newspapers. The   campaign is intended to last a year and is intended to be implemented in   segments. The first segment begins in March and ends in April, the second one   is being implemented in July and August, and the third in October and   November.

As you can see from the logic   model, the campaign is intended to initially reach a group of smokers in   British Columbia (awareness of the campaign) and, once they are aware of the   campaign, to increase their knowledge of smoking-related issues that might   appeal to persons in that age group.

One television advertisement, for   example, shows a man standing outside a bar smoking a cigarette. As he   inhales, he ages. As he exhales, he becomes younger again. The message is   that if you quit smoking, most of the effects on your body are reversible. A   website is provided so that viewers can get information to help them quit.   Exposure to these messages is intended to affect attitudes toward smoking (to   make them develop a view that smoking is not good for them and that they want   to change their behaviors). Attitude change is expected to lead to changes in   behaviors: attempts to quit, cutting back on smoking, and actually quitting.

The Evaluation

The evaluation of this   province-wide program is intended to determine whether the program logic   works as intended, i.e., whether people are aware of the campaign, whether   the campaign increases their knowledge, whether their attitudes change, and,   finally, whether they try or succeed in quitting smoking.

The evaluation of the planned   smoking cessation campaign in British Columbia is focused on the   effectiveness of the campaign. More specifically, the evaluation is intended   to address these questions:

What   impact, if any, did the mass media campaign have on:

o Awareness   of the campaign itself.

o Knowledge   of the consequences of smoking.

o Attitudes   toward smoking.

o Smoking-related   behaviors.

The key issue for the evaluation   design is to determine how the campaign affected the aforementioned   attributes of persons who have self-identified as smokers.

Given that the campaign is   province-wide, it will be difficult to find a control group that has not been   exposed to the campaign. Given the costs of the campaign, the evaluation will   need to focus on British Columbia and not try to find a comparison group in   another province.

Population surveys of British   Columbia residents suggest that approximately 15%–20% of the people in the   age group of twenty to thirty years smoke. These smoking rates vary from one region   to another and with sociodemographic characteristics of British Columbia   residents.

Tasks:

Design an evaluation report that   will be able to assess whether the smoking cessation campaign was successful.   An important part of any proposed design is to assess whether the program   theory actually worked the way it was intended, i.e., whether persons became   aware of the campaign, whether their knowledge increased, whether their   attitudes then changed, and whether attitude changes led to smoking cessation   behaviors.

To ascertain whether the campaign   impacts smokers in the way the program logic is intended, it is important to   design an evaluation that actually tracks the way the campaign influences   smokers over time. Ideally, the evaluation team wants to be able to see   whether the program was visible; if it was, whether knowledge increased; if   that happened, whether attitudes changed; and if attitudes changed, whether   changes in behavior have been reported.

In a 3-page paper, explain the   following:

What are   the main features of your evaluation design?

How would   it be implemented?

How does   the design deal (if it does) with possible rival hypotheses (internal   validity threats)?

How does   the design address statistical conclusions, construct, and external validity?