Research Design paper

CPP 524 (Spring 2020 – Session A)
Faculty Dr. Jesse Lecy
Arizona State University
Master of Science in Program Evaluation and Data Analytics

PART I- Research Questions

  1. Problem: Public littering is a well-recognized global problem with adverse health, environmental and socio-economic effects. The amount of litter, not just plastic, is increasing exponentially on roadside, countryside, oceanfront, public spaces and tourist places. Litter, besides being an eyesore, also causes negative impacts on people’s feelings of welling-being and safety . Littered streets feel abandoned, consequently their inhabitants do tooi. It poses health risks and harms the environment and the wildlife. Cleaning litter also puts pressure on the city’s financial budget because of increased staff cost in cleaning.

  2. The Program: The Street Can Program will place street trash cans on 15 major street crossings in Chandler, AZ within a three months’ period in a phased manner. The program aims to find the effect of placing street cans in an attempt to reduce public littering in the city area. The study also investigates and collects data related to pride in the neighborhood, socio-economic aspects of the litterers, public attitude towards cleanliness, and places and item types people litter.

  3. Program Change: Recent studies have shown changes to trash cans’ placement and using trash cans with lids could be an important tool in litter control . Reduction in litter in a neighborhood promotes a sense of ownership, well-being and responsibility. People are usually reluctant to litter in a clean place. The program aims to reduce ugliness of neighborhood litter which causes increased crime rates and reduced property values. Furthermore, by changing the littering habits of a neighborhood the program is going to promote long-term litter prevention and control towards a cleaner city.

  4. Dependent Variable in the Study: The study will examine the changes in litter due to addition of public trash cans around 15 major street corners for a period of 5 months.

  5. Motivation of the study: Chandler city’s mayor is focused in developing innovation and evidence-based practices to enhance the well-being of the city. The city has proposed a budget of $1,000,000 to place new trash cans around the city as an initiative towards litter reduction. Strong evidence is needed that placing additional trash cans will produce positive results before the program can be extended to the whole city.

  6. Program Enrollment Criteria: The city has identified 15 major street crossings out of its almost 75 such crossings. Within the identified street crossings there are commercial establishments in the corners and residential neighborhoods. The trash cans are placed on the identified street crossings. The participation in the program by residents and passersby is automatic. There are enough foot traffic or people on the streets as these are major street corners in Chandler. The study also consists of a pre and post treatment survey questionnaire for collection of qualitative data, participation in this survey, however, is voluntary.

PART II: Evaluation Design

  1. Program Design: The program uses pre-post reflexive design. The design calculates the program impact by taking the differences between the post and pre-treatment results. The program also accounts for any trends that might exist or come into play because of some public awareness program or education in the area to alter littering habits in public.
    The City used the map prepared by Chandler Fire, Health and Medical Department to identify 15 major cross streets within the city, which are also called “the mile street” on the map. These streets corners are similar in terms of population, demographics, ratio of commercial and residential spaces.
    The street corners are then randomly assigned to three different groups with five streets in each group. The first group of the street corners are surveyed for litter one month and one week before leading up to the placement of trash cans. The street corners are again measured for litters one month after the treatment. The same process is repeated for the second group of the streets. After a month it is repeated for the third group as well. Even though the treatment period is three months, the data collection period starts a month before and ends a month after the treatment period.

  2. Design Instrument: The study, apart from placing trash cans on the street corners in a phased manner also collects a survey questionnaire at the beginning and the end of the program. The questionnaire is divided in two parts. The first part contains questions related to socio-economic background of the respondents. For example: race, age, gender, marital status, education level, and residential status in the neighborhood under study.
    The second part contains nine questions related to littering in public places. First five questions are related to what and why respondents litter in public places. The next three questions are directed to assess the respondents’ attitude towards littering in general and how far are they are willing to carry their trash before they dispose of it . The ninth question is related to public impression of the neighborhood.
    City maintenance officials assign a litter score based on the litter index matrix developed by Philadelphia Parks & Recreation . The scores are maintained for all the groups at one month and one week prior to the treatment and one month after the treatment. The study is conducted with the city’s existing Park and Recreation staff members. The concerned staff members are provided training on litter index and the scoring via a training video developed by City of Philedelphia to train its own staff members.

  3. Study Group: The Study group consists of reflexive treatment groups. The first group acts as a treatment group. The data is collected on this group pre and post treatment. Groups two and three, even though they will receive treatment eventually, can be studied as the control groups before they receive the treatment. This helps to study any changes in trends of littering behaviors.
  4. Data collection: The survey questionnaire is pretested by a set of analysts for its accuracy and consistency. The results are measured as a mean of percentage for consistency. People from the selected street corners for the program are offered to fill out the questionnaire face to face. In addition, physical copies are sent to the resident’s mailboxes and digital copies are made available to them via Google forms to increase participation. This is done a month before the treatment begins and one month after the treatment in each group.
    The city officials assign a litter score based on litter index developed by the City of Philedelphia one month and one week before the actual placement of trash cans. The litter scores are re-assigned to these street corners one month after the trash cans are placed. The same is done for rest of the two streets’ groups as well. The results are recorded and presented to observe the differences in the litter scores of the street corners before and after the treatment.
    The litter index for the street corners are also used to measure staff hours spent picking up the litter. Further, the litter index for an area can be related to the resources an area has to encourage residents to do their part to keep their neighborhoods clean.

  5. Method: Measurement of the litter score is a good comparison to measure any changes in littering habits pre-post treatment. The five months of the study period is a good amount of time to reflect upon the changes in littering behavior because of the introduction of trash cans in that particular street corner.
    Although, the participation in the survey questionnaire to capture the sentiments of the people is voluntary, the participation due to the placement of the trash cans in the area is automatic. The study is more concerned about changes in litter score due to the program treatment. The survey questionnaire data serve as an additional resource to understand peoples’ littering behavior. The whole neighborhood, bicyclists, and pedestrians are part of the sample for the study. There are some changes in population of the street corners during the study period but that only affects survey questionnaire related data collection. The litter score on a street corner is affected by the number of people on the street. Warmer weather, fairs and holiday season put more people on the street which increases the amount trash generated in the area as well. The study has comparison data from all of the three groups which can be averaged to measure the changes in the litter scores.

  6. Results and conclusion: The study examines the impact of additional trash cans accessible to the public in an area. It also measures the litter score and staff hours spent collecting the litter because of the changes due the program. The program collects qualitative data related to socio-economic background and the littering habits of the voluntary the respondents.
    The study documents a significant decrease in litter score for the street corners where additional trash cans are placed. At the same time, overall staff cost for picking the litter also decreased. Even though, littering behaviors are hard to change in short term, the access to convenient litter disposal can significantly decrease littering.
    This impact can be further verified with the data collected by the pre and post treatment survey questionnaire from the area. The data indicates that most respondents indicate that they litter because there is a lack of enough number of littering boxes, and they litter if the place is already dirty. There is no significant indicator in socio- economic data. However, there is a connection between age, education, and littering. Older and/or educated population seem to be more conscious about keeping their surroundings clean. Reduced litter accounts for positive changes in the public sentiments about the area, this is validated by the data collected for pre and post treatment questions about the public impression of the neighborhood.
    The study itself is well-done and offers persuasive enough evidence that increasing the presence of public trash cans would be an effective litter reduction strategy for the city. The topline findings are that when there are fewer trash cans, less trash is generally collected, but a smaller number of the public trash cans “dramatically” increase the amount of trash collected as litter, and increase staff spending cleaning up litter . With all the details and data available for the Street Can Program, the program can be successfully replicated in the rest of the Chandler city with success.

PART III: Competing Hypothesis

Omitted Variable bias

  1. Selection: +1
    The selection of the groups of the streets are random. They represent similar demographics and population. The sample sizes are large enough to make unbiased comparison. There is no selection bias in the study.
    Taking part in the pre-post survey questionnaire is voluntary and people who choose to participate are different than those who choose not to. The collection of qualitative data resulting from the survey questionnaire does not affect the actual study. The data is collected for further research in littering behavior.

  2. Non- random Attrition: +1
    A change in population in the assigned treatment area is inevitable. This is also reflected in difference in the number of participants for the pre-post treatment questionnaire survey. Again, this does not affect the study as the main goal is to measure the changes in litter score due to the program treatment.
    The area might be going through gentrification or construction during the study period. That will change the litter score of the area. The number of foot traffic on the streets are large enough to make a good sample size. Any changes in population or seasonal changes in number of people on the street, which can change litter index for the street, is reflected in pre-post treatment results and can be averaged out to avoid fluctuations.

Study Calibration and Measurement

  1. Time frame of Study: +1
    The time frame of the study is five months. This time period is sufficient to understand the short-term impact on littering due to addition of public trash cans in an area. However, to understand the long-term changes in littering behavior the program may need to run a little longer. This program is not looking to analyze the behavioral changes at this point. If the city decides to do further research on the subject, the researchers have the qualitative data available to them which is collected by the pre- post survey questionnaire.

  2. Measurement error in Dependent Variable: 0
    The dependent variable for the study is amount of the litter generated. The study aims to measure the changes in litter due to placement of public trash cans. The litter is measured in terms of litter score assigned on the basis of litter index.
    There may be errors in measuring the litter scores, different city officials may assign a different score to the same litter.

  3. Intervening Effect: +1
    No intervening effects such as public awareness program, natural disaster or one-time big event was noted during the time period of the study.

Campbell Score for the study: 9/10

Reference

  1. R. Coward, “Telling litterers to change is a waste of time. Here’s another solution”, Theguardian.com, 3 March 2018, https://www.theguardian.com/commentisfree/2018/mar/29/litterers-wont-change-litter-zero-waste-national-strategy, (accessed 28 February, 2020)
  2. Published by Office of the Mayor, “GovLabPHL Releases Behavioral Science Studies on Litter Reduction, City of Philedelphia, 26 September 2018, https://www.phila.gov/2018-09-26-govlabphl-releases-behavioral-science-studies-on-litter-reduction/, (accessed 29 February, 2020)
  3. COC Enterprise GIS, “Chandler block numbers for addresses and ranges mile and half-mile street alignments”, [map], Chandler Fire, Health & Medical Department , January 2017, https://www.chandleraz.gov/sites/default/files/documents/imported/MapGallery_COCBlockNumbers.pdf, (accessed 3 March, 2020)
  4. A.M.Freije, H.A.Nassar, “Attitudes and opinions towards public littering in the Kingdom of Bahrain”, Arab Journal of Basic and Applied Science, Volume 26 2019, Issue 1, 27 June 2019, https://www.tandfonline.com/doi/full/10.1080/25765299.2019.1628688, (accessed 29 February, 2020)
  5. City of Philadelphia ,“Philadelphia Parks and Recreations Detailed Litter Index Matrix”, 2019 Litter Index , July 16 2019, https://cleanphl.org/wp-content/uploads/2019/07/2019LitterIndexReport_7.16.19.pdf, Pg.7, (accessed 3 March, 2020)
  6. The Mayor Office of Philadelphia, “Litter Index Explained”, Clean Philly, n.d., https://cleanphl.org/what-is-the-litter-index/, (accessed 28 February, 2020)
  7. John Geeting, “Water is Wet. Street Cans Reduces Litter, City Behavioral Study Finds” Philedelphia 3.0., 27 September 2018, http://www.phila3-0.org/water_is_wet_street_trash_cans_reduce_litter_city_behavioral_study_finds, (accessed 28 February, 2020)