AAPS Achievement Plan: It is rocket science

Ann Arbor Public Schools Board of Education study session (Nov. 10, 2010): Last Wednesday evening saw the unveiling of the Ann Arbor Public School’s “achievement gap elimination plan,” a document outlining the comprehensive set of strategies being used by the district to close the gaps in academic achievement between different groups of students. Preschool achievement data, positive behavior support programs at the middle school level, and a newly-created fifth grade social studies unit on African civilizations were highlighted as examples of the plan’s initiatives.

The meeting also included a review of secondary discipline data that showed disproportionate numbers of male students and African-American students receiving suspensions.

In response, the board made some suggestions on data collection processes and possible cross-references that could add depth to the analysis of student assessment and discipline data.

Achievement Gap Elimination Plan

The impetus for enumerating all AAPS gap-closing initiatives in a single document came from two public meetings held last spring – the College and Career-Ready Review, and Beyond the Talk. At each of those AAPS-sponsored events, participants argued that though achievement gap elimination efforts have long been part of the district’s ongoing work, these efforts are not being well-communicated.

Achievement Gap Elimination Plan: Overview

At Wednesday’s meeting, Lee Ann Dickinson-Kelley – AAPS interim deputy superintendent of instruction – explained how the achievement gap elimination plan forms the core of the district’s school improvement plan. “There is nothing more important,” she said. “The achievement gap elimination plan is foundational to the district school improvement work.”

Speaking to the format of the plan as presented to the board, which was included in the meeting’s board packet, Dickinson-Kelley noted that the plan was written in the state-mandated school improvement plan (SIP) layout, since it is embedded within the district’s SIP. Noting that the format was “not necessarily pretty,” Dickinson-Kelley explained that after receiving board input, the district could create a more user-friendly document to present to the community.

The plan presented to the board included sections on achievement-raising initiatives targeting each of four core subjects – math, English language arts, science, and social studies – as well as district efforts to engage parents and the greater community.

Board members questioned the wording of the “gap statements” at the beginning of each section, as well as whether the plan should include more of the general social context within which the district is situated. Dickinson-Kelley agreed to amend the gap statements to be more clearly worded, and acknowledged that the plan is a “living document” designed to incorporate ongoing feedback from the community.

Achievement Gap Elimination Plan: Data Collection

The bulk of the administration’s presentation focused on the assessment measures being used to determine effectiveness of the plan’s initiatives. Dickinson-Kelley explained that combating disparities in achievement is complex work with multiple aspects. “It is rocket science,” she argued. “It should change as we assess what we do, and the impact of what we do. If it’s not working, we should adapt.”

Dickinson-Kelley explained the difference between the formative and summative assessments used to monitor student academic achievement and growth, noting that the board usually only sees summative assessments, such as Michigan Education Assessment Program (MEAP) data. Jane Landefeld, AAPS director of student accounting and administrative support, explained that on the other hand, monitoring the ongoing success of a program requires the collection of formative data. Dickinson-Kelley pointed out that the frequency with which formative data is collected is important, as it directly informs instruction and allows intervention before students fail.

Joyce Hunter, assistant superintendent of secondary education, reviewed a chart with board members that listed the assessment schedule used by principals and counselors throughout the year. The chart included state assessments such as the MEAP and the Michigan Merit Exam (MME), as well as recent district report card grades, reading inventories, and other local assessment measures of individual student achievement.

Trustees Susan Baskett and Andy Thomas asked about the viability of making such a comprehensive set of student achievement data available to parents. Landefeld suggested that the current data format might not be understandable to parents, but suggested that creating accessible student profiles could be considered. Board president Deb Mexicotte noted that there would be privacy issues in developing “e-files” on student achievement, but that the board should consider it in the future.

Trustee Irene Patalan commented that by having principals and counselors review this data in a systematic and formalized way, as laid out in the chart, “if there is a pattern where students are not successful, it would inform instruction the next day.” Dickinson-Kelley agreed that systemic assessment data review allows for more responsiveness to students’ academic needs.

Trustee Glenn Nelson suggested that any complete profile on students should contain building-level data such as the percentage of parents who attend parent-teacher conferences there, as well as individual student data on absences, suspensions, and expulsions. Trustee Simone Lightfoot added that other variables could be identified as relevant indicators that are not test scores. Dickinson-Kelley agreed that it is important to collect information “beyond quantifiable test data.”

Achievement Gap Elimination Plan: Parent Engagement, Community Involvement

Dickinson-Kelley then highlighted the parent engagement and community involvement sections of the achievement gap elimination plan. Saying that much was gleaned from the minutes of the community meetings last spring, she reiterated that the plan is meant to evolve with continued input from parents and community members. She noted that one focus of the newly developing lab school at the Mitchell/Scarlett campus is to engage parents, and equity advisory committees.

Dickinson-Kelley acknowledged that AAPS could do better in its efforts to reach out to community partners. She noted that the achievement gap elimination plan suggests creating a community pamphlet or repository to catalog partnerships that AAPS has with local organizations such as the Family Learning Institute, 826 Michigan, the Minority Students Achievement Network (MSAN), and the Pacific Education Group (PEG). She also pointed out that these partnerships “complement each other extraordinarily well.”

Achievement Gap Elimination Plan: Board Response

Thomas stated that it was “very important that the board put our money where our mouth is” in addressing the achievement gap. He asked for an assessment of district resources devoted to research, data analysis, and ability to prepare ad hoc reports, and noted that he would support an increase in allotted resources if necessary. Dickinson-Kelley noted that she was planning to bring some suggestions regarding needed resources to an upcoming meeting she has scheduled with the board’s performance committee.

Acting on a suggestion from Baskett, Mexicotte assigned the board’s planning committee to work with Allen and Liz Margolis, AAPS director of communications, to create a more user-friendly achievement gap elimination plan document that could be used to reach out to the community, with an online component that would link to deeper sections of the plan.

Patalan complimented Dickinson-Kelley on the presentation, and Dickinson-Kelley, gesturing to her colleagues at the table, acknowledged that the creation of the achievement gap elimination plan was a cooperative effort. “The achievement plan is commonly owned, and behind all of us are legions of people working really hard as well.”

Preschool Achievement Data

Dickinson-Kelley, along with Ann Arbor Preschool and Family Center principal Michelle Pogliano and vice principal Kecia Rorie, reviewed achievement data from preschool cohorts as they moved into higher grades. The data show that gains made by students in AAPS preschool remain past the third grade, when most effects of the federally-sponsored early intervention program Head Start have worn off.

Part of the explanation, according to Dickinson-Kelley, is that AAPS preschool combines students enrolled in Head Start and those enrolled in the Great Start Readiness Program (GSRP) into blended classrooms, while meeting all curricular requirements of both programs. Head Start, she explained, was initially formed to mitigate poverty, and thus requires teaching about nutrition, hygiene, and other topics that can supercede academic learning goals. When AAPS noticed an achievement gap between Head Start and GSRP students, they decided to combine the classes and have seen longer-term gains for all students.

Thomas congratulated the preschool staff, noting that improvement was seen across all cohorts, and that the gap between African-American students and other groups is narrowing.

Nelson questioned how these data compare to the achievement of students who did not attend AAPS preschool, noting that AAPS “continually import[s] the proficiency levels of the rest of the world into our schools.” Landefeld did not have that data available.

Dickinson-Kelley stressed that the district is in full compliance with all goals of Head Start, while preschool staff provide more of an academic foundation as well. She also noted that U.S. Secretary of Education Arne Duncan is looking to re-frame Head Start into a “wraparound school” model with stronger emphasis on foundational numeracy and literacy. Mexicotte suggested that the district find ways to trumpet the success of its preschool to the community, as well as to the state legislature, since it exemplifies the kinds of programs that Duncan is interested in supporting.

Positive Behavior Support

As part of the budget process for the 2010-11 school year, the board approved the elimination of student planning centers at the middle schools, which were a key element in the student discipline process. Planning centers were replaced with the institution of positive behavior support (PBS) programs in all the middle schools, and this was the board’s first review of the implementation.

Positive Behavior Support: Overview

Hunter introduced the idea of PBS as “a system-wide approach to encouraging positive behavior” that is structured around three tiers of intervention. With PBS, she said, “you’re teaching and modeling positive behavior, and everyone is involved in reinforcement.” Oftentimes, Hunter explained, rules are set up for 20% of the students, when 80% of the students are making good choices. A big element of PBS, she pointed out, is to recognize and reward positive behavior.

As an example of how desired behavior is taught, Hunter showed a short video clip from Clague Middle School. Clague, she said, chose to identify positive behavior choices as those that are “responsible, respectful, safe, and caring.” The video featured Clague students, and contrasted unsafe and safe hallway behavior in a brief and humorous manner. Hunter explained that it was used as a teaching tool during student advisory periods.

Hunter then passed out a summary of the three tiers in PBS programs, and board members asked for clarification on some of the language. She then introduced principals Janet Schwamb of Forsythe Middle School and Chris Curtis of Slauson Middle School to discuss how PBS had been incorporated into the culture of their buildings.

Positive Behavior Support: Programs at Slauson and Forsythe

Curtis began by describing how the first step in setting up a PBS program is establishing a shared set of expectations, first among staff and then extending to student council review. Slauson came up with nine basic expectations, such as coming to class focused and prepared, not running in the halls, and being respectful of property. Next, Curtis said, the expectations need to be taught. At Slauson, advisory time was used for this teaching, parents were introduced to the new system, and the expectations were listed in the student handbook.

Finally, Curtis explained, incentives were developed to encourage positive behavior. At Slauson, incentives have included: class trips to a water park or ice skating, passes that allow students to leave class early and go to the front of the lunch line, quarterly raffles for iPod Nanos, and occasional raffles of school spirit items such as baseball caps. Curtis described the database he crafted with the help of a former Slauson parent, which is used to meticulously track students’ behavior, and assign them to an “honor level” to help qualify them for certain incentives. A key element of the system, added Curtis, is that any infractions a student receives expire after 14 days, so students are constantly able to improve their standing.

All that, Curtis said, is Tier 1. Any students who are past Tier 1 go through the achievement team process, which often leads to the creation of a behavior contract to formally identify the behaviors being addressed. Curtis summarized by saying that PBS helps to move away from the suspension model where the administrator is “serving as the judge and jury all day long … [With PBS,] the behaviors in the building and the building culture come around quickly.”

Schwamb then described the implementation of Forsythe’s PBS program, which she said she “loves.” First, students and staff collaborated to set the four main behavior expectations as being “prompt, prepared, polite, and productive.” Then, teachers developed lessons to teach those behaviors, which were delivered during advisory time. Students also took mini-field trips to different parts of the building during the first three weeks of school to model what positive behavior would look like in those areas. Posters were made by students to reinforce the behavioral expectations, and were posted throughout the school.

Forsythe also uses a tiered tracking system, Schwamb explained, which allows students to start over every 14 days and at the beginning of each quarter. PBS has been well-received by parents, she said, who like that it “levels the playing field,” and requires all teachers and administrators to deal with infractions the same way. Schwamb noted that Forsythe provides similar rewards as Slauson, including passes to the front of the line, raffles for school clothing or iPods, and gift cards for school supplies.

Schwamb expressed excitement about PBS, saying that the “paradigm shift” allows her to enjoy more her relationships with students, has considerably reduced suspensions, rewards students who have correct behavior every day, keeps kids in class, and gets plans in place sooner for kids who need more intervention.

Positive Behavior Support: Board Questions and Response

Thomas asked how the principals felt that PBS worked, compared to the student planning centers (SPC) that have now been closed.

Curtis answered that overall PBS is a significant improvement, but that losing the SPC staff has caused a lack of ability to check up constantly on “frequent flyers” – students whose behavior would be much improved with more regular contact. Without the SPC teachers, he said, it’s harder to be proactive. However, Curtis concluded, he would not want to return to having SPCs. “It’s like trading in a jalopy for a Cadillac,” he quipped. “This new model is much better. I don’t want a pull-out model [where students are removed from classrooms], but still want it to be push-in.”

Patalan asked how students like the PBS programs. Curtis and Schwamb both reported that the programs have been well-received by students, especially 6th graders. Curtis noted feedback he received from student council about incentives they liked, and a wish list of new incentives. Schwamb said her students clamor to read the list of student names posted each Thursday, which reflects their current honor level.

Baskett asked about staff preparation for and training in PBS. Curtis remarked that PBS empowers teachers to enforce behavior, while keeping student-staff relationships intact: “The teacher doesn’t reject students from the room. Consequences occur in the room. Everyone knows what the deal is, and what happens. [PBS] keeps the authority within staff.” Schwamb noted that there was “enormous prep time” before the beginning of the school year; Hunter added that training is continuous.

Baskett asked how PBS defines insubordination, when some would call behavior insubordinate while others would say, “It’s just adolescence.” Schwamb answered that administration is trained to administer infractions consistently, and that an infraction would have to be “over the top” to warrant a suspension. Curtis added, “With the authority in the right place, frustration does not build up. It would have to be off the chart to suspend for insubordination.”

Schwamb explained the process for issuing a violation: “The first warning is that the teacher puts the [violation] sheet on the student’s desk. The second warning means the student puts his or her name on the paper, and the third time, the teacher collects it.” PBS, she said, provides an opportunity for students to redirect and self-regulate.

Dickinson-Kelley asked Elaine Brown, assistant superintendent of student intervention and support services, and Ruth Williams, interim assistant superintendent for elementary schools, to describe efforts to bring PBS to the elementary level. Williams responded that all the elementary principals are excited about PBS, and will receive training from Wayne RESA. Brown added that implementing PBS is a state mandate, and that the district is looking at how to provide schools with as much support as possible to ensure that PBS is successful. She argued that AAPS needs to be able to pinpoint what makes the difference when transitioning students toward positive behavior. Lastly, Brown noted that more social work support may be needed.

Baskett asked how the programs are being monitored, and Dickinson-Kelley said that all administrators were retrained this year in Power School – a web-based student information system – to be sure that all time away from instruction was accurately represented.

Baskett also asked whether PBS would be extended to the high schools. Hunter noted that the honor level system was already in place at Pioneer, but that other high schools have not implemented it yet. She likened it to how Read 180 started in the middle schools before expanding to other levels in the district.

Patalan expressed appreciation for the “14-day slate-clear restart” element of the programs. “I think of the life of a child through the school year, [and how it is good for them] not to be labeled,” she said. Curtis responded that for kids on tiers 2 or 3 of PBS interventions, one strategy that can be used to help them meet expectations is to shorten that reset window to 8 or 10 days, and then wean them back up to a 14-day cycle over time.

Lightfoot asked what capacity PBS puts in place to evaluate staff. Schwamb said she has invited teachers to meet and discuss the issuing of violations. Also, she said, staff can now identify which expectations are consistently hardest for kids to meet, and develop additional lesson plans to teach those behaviors. Lightfoot reiterated, “As the data continue to mount, I don’t want it to take years and years for teachers to get it together.” Nelson agreed, and argued that “a good school system should have summative and formative assessments of teachers and administration.”

Mexicotte commended Curtis and Schwamb, saying that they “took the step many of us have wanted to take for many years, and implemented what was a state mandate … The reason that PBS systems fail is that the staff does not buy in hook, line, and sinker, and that is something you clearly tacked around this.”

Inclusive Curriculum Development

Chuck Hatt, AAPS coordinator for literacy and social studies instruction, described a new unit the district had developed for inclusion in 5th grade social studies classes. He began by explaining that the district’s ongoing equity work led to a review of curricula by a set of “non-dominant-culture eyes.” One thing that was noticed, Hatt said, was that in the study of colonial U.S. history, students’ introduction to African-Americans is through slavery. This, he suggested, “is neither kind, nor appropriate,” given that students of European descent are exposed to numerous positive cultural touchstones to their ancestry before learning about America’s colonial period.

By working with a consultant who taught AAPS curriculum staff about the history of ancient African kingdoms, critiqued texts, and offered practical pedagogy about working with African-American children, the district was able to craft a unit on African civilizations that presents a more inclusive look at the origins of African-Americans. Hatt asserted, “This is something we should feel good about as a district – we have a lot of internal capacity to write good curricula.”

Board members expressed excitement about the new curriculum, and were given a copy of the unit to peruse. They confirmed that teachers had been trained in it, and that it would be implemented this year across the district. Lightfoot suggested selling the curriculum, and Baskett suggested submitting it for a Michigan Association of School Boards (MASB) award. Mexicotte added that if the district does pursue selling it, “we should set a reasonable price, because the important thing is to spread the word.”

Review of Student Discipline Data

Hunter passed out a summary of suspensions issued so far this year at each of the middle schools and high schools, which showed higher rates of suspension among male students, as well as among African-American students. Thomas noted that nearly 50% of the suspensions were given to African-American students, and that “insubordination” was the most common reason for suspensions. Mexicotte said those elements of this suspension data have been typical, and that PBS programs (described above) should help to better codify “insubordination.”

Board members clarified that all suspension data referred to out-of-school suspensions, and that in-school suspensions do not remove students from instructional time. Brown explained that principals have been trained to “do in-school suspension totally differently,” especially with the removal of the student planning centers from the middle schools. Dickinson-Kelley also recognized both Hunter and Williams for their efforts to “keep the conversations in front of principals, asking ‘How often are your kids away from instruction?’”

Trustees also questioned whether the overall number of suspensions was higher than other years. Hunter acknowledged that the number at Pioneer “seems high.” Mexicotte noted that “whenever we ratchet up our focus on discipline, it might skew us into thinking we are doing worse when we are just getting more consistent about reporting.”

Board members suggested tracking additional information alongside suspension data to allow for more robust analysis of the suspension numbers in the future. Baskett asked if the district keep data on any mitigating circumstances in the students’ lives, such as divorce or trauma, happening near the time of the suspension. Nelson asked if there was a narrative block on the standard form or input screen used to record a suspension. Mexicotte suggested cross-referencing suspension data with data on other plans students may have, such as Individualized Education Plans (IEPs) or 504 plans, which function to support students during temporary conditions such as having a broken arm, or being depressed.

Landefeld noted that some of the data suggested is entered in Power School or would be collected as part of the achievement team process. She also acknowledged that administrators had been trained to label infractions consistently.

Thomas thanked the administrative team for their time and effort in addressing the problems discussed during the study session. Nelson also noted his appreciation at getting this information organized and systematized. He stated that tracking suspensions would be a new part of teacher evaluations, and Mexicotte added that the data would be stratified by students versus incidents, since certain students may earn repeated suspensions regardless of their teacher.

Agenda Planning

Board members requested that Dickinson-Kelley and her team present updates on both achievement data and suspension data quarterly, as the initiatives and programs that are part of the achievement gap elimination plan are being put in place. They also requested that Brown provide them with a comprehensive update on special education services before the special education millage that will come to voters in the spring. Administration was supportive of their requests.

Present: President Deb Mexicotte, vice president Susan Baskett, secretary Andy Thomas, treasurer Irene Patalan, and trustees Glenn Nelson and Simone Lightfoot. Absent: trustee Christine Stead. Also present was Robert Allen, interim superintendent of AAPS.

Next regular meeting: Nov. 17, 2010, 7 p.m., at the downtown branch of the Ann Arbor District Library, with an AAPS Education Foundation donor reception held immediately before the meeting at 6:30 p.m. [confirm date]