Data is not information, information is not knowledge, knowledge is not understanding, understanding is not wisdom.
~Clifford Stoll
Showing posts with label assessment. Show all posts
Showing posts with label assessment. Show all posts

Monday, August 1, 2011

Unacceptable

Today I'd like to introduce you to John Kuhn, superintendent of the Perrin-Whitt Consolidated Independent School District in North Texas. Kuhn is a life-long advocate of public education and public accountability. This weekend, at the National Save Our Schools Rally in Washington DC, he spoke passionately about the vital need for great public schools, the critical importance of a no-excuses public attitude toward educating America's kids, and what it will really take to create the equitable education system our great country deserves. For now, as we work toward those goals, Kuhn says, "I stand before you today bearing proudly the label of unacceptable because I educate the children [private schools] will not educate.... I am unacceptable and proud of it!"




Make no mistake: unacceptable as he is according to current educator grading methods, Kuhn is not an apologist for bad teaching; he is only asking for appropriate systems of evaluation. In April, Kuhn appealed to his state legislature for a two-year moratorium on high-stakes testing. When one representative asked Kuhn why he thought teachers shouldn't be graded, he wasn't prepared with his answer. However, he recognized that this is a question that keeps coming up for educators, and being unprepared to answer makes us look, in his words, "like [we] are just whiners who don't want to be held accountable at all." So he thought about it and developed an articulation, not of why teachers shouldn't be graded, but of how the state has become a bad teacher and, thus, an irresponsible grader.

His emphatic and empathetic response to this question provides a useful starting point for all of us who believe teachers should be graded, but fairly. It's only a starting point, as some folks pointed out in the comments. It is up to the rest of us to develop our own articulations of how to create better forms of teacher accountability, how to fairly evaluate teachers and push them toward achieving greatness. Until we put meaningful teacher evaluation, support, and development systems in place, we must join Kuhn in proudly bearing the label of unacceptable.

Friday, July 22, 2011

Motivation in the 21st Century

The testing debate is really heating up this week, with cheating investigations continuing in Atlanta and New Jersey, and teachers in Ogden, Utah, choosing whether to resign or sign a new merit-pay contract. Meanwhile, New York City's mayor Bloomberg has canceled the city's merit-pay pilot program, yet reinforced his support for performance pay in general.

The only good news, recently, on the issue of merit-pay comes from Ohio, where Republican Governor Kasich has made the entirely reasonable decision to hand teachers the task of creating a fair merit-pay process. To comply with Race to the Top requirements, the final plan must include teacher evaluations that weight student academic achievement for at least 50% of teacher ratings. However, if done right, the decision to let teachers design the details could result in a merit-pay system that has the support of teachers because it supports teachers in making the efforts they understand to be essential to their jobs, like collaborating with colleagues and developing relationships with students' families.

I'm encouraged by efforts of politicians to respect and defer to the knowledge and skills of teachers when designing education policy, but I'm having trouble getting truly excited about even a good merit-pay program. Focusing efforts on merit-pay program details begs the question of whether merit-pay is an effective incentive, and if so, what it actually incents. Early studies have shown that offering performance bonuses of 20% of salary might slightly motivate top-performing college students to enter teaching, but any motivational effect on current teachers to change their teaching is unproven at best. A McKinsey study revealed that bonus pay isn't even in the top five motivators for current teachers to consider teaching in a high-need school, and a RAND study revealed that many teachers in NYC's merit-pay pilot program "reported viewing the bonus as a reward for their usual efforts, not as an incentive for changing their behavior."

For another perspective on how to truly motivate creative and thoughtful teachers, check out Vicki Davis's excellent article on "The Freedom to Teach" in today's Washington Post. Davis's thinking is exactly what 21st century students need. As Levar Burton always said on Reading Rainbow, "You don't have to take my word for it!" because a few members of the 'Net Generation would like you to hear it from them:


More Links:

Wednesday, July 20, 2011

FOUR, FOUR, FOUR improvements in ONE

The big news out of the Big Apple this week is that the city is ending its merit pay pilot program after a study showed that the merit pay had no effect on student achievement. Analysts at RAND, the independent research company that designed the study of the program, suggest that the small bonuses were not motivating enough to change teacher behavior in the current context of one-test-fits-all carrot-and-stick "accountability."

The NYC merit pay program focused on motivating existing teachers to work hard to boost student scores, but what if we shift the focus to motivating existing hard workers to lend their efforts to same goal? This would more closely resemble the national strategies of top-performing Finland, Singapore, and South Korea. McKinsey&Company, another research company, recently released the report of their comprehensive study of what factors might motivate current teachers and top college grads to teach in high-needs schools.

Much has been made of this study, and the reports and commentaries I had read focused heavily on radically increasing (essentially doubling) teacher pay in order to draw top students into teaching, especially in high-needs schools. However, the study itself does not suggest that increasing teacher salary is the only, or even necessarily the most effective strategy to provide the highest-need students with top-tier teachers. In fact, the results of the study imply that the most effective progress could be made by combining a few different strategies targeted to different populations.

Salary increases were especially motivating for top grads in Science, Technology, Engineering, and Math (STEM) who expect to take their pick from a range of well-paid career options. Salary also motivated top college grads, but the study found that simply marketing teaching careers to college students and providing drastically better training for entering teachers would go a long way toward motivating top grads to pursue teaching careers.

The one group not very motivated by salary increases was current teachers. These folks, who had already spent a few years working in the system, placed more value on good working conditions, professional development, and excellent school leadership. The fabulous news here is that all three of these indicators also independently raise student performance, and all three can be achieved by simply attracting excellent school leadership to work in the highest need schools. The McKinsey report proposed several different "scenarios" to attract top-tier teachers, but by focusing on independent variables instead of combined effects they missed this simplest one.


Links:

Monday, July 18, 2011

Read my lips: reform requires taxes

In today's news, Illinois Schools Superintendent Christopher Koch pointed out that funding cuts are bad for education reform. Now, I wasn't a huge fan of the Illinois reforms to begin with (the big focus was on preventing teacher strikes and paying teachers for high test scores), but Koch is right that education reform is going to take some serious funding.

Real reform, like the suggestions from Iowa's roundtables and the initiatives AFT president Randi Weingarten highlighted last week at the AFT Teach conference, will require real funding. Selling this reality to parents, pundits, politicians, and private sector CEOs will be the biggest challenge we'll face in achieving the long-term goal of improving the US education system to rival those of Finland, Singapore, and South Korea.

While we work on the mid-term goal of selling folks on the idea of higher taxes to fund real school reform, we could act in the short-term to divert funding away from unproven and disproved reform efforts, such as bonus pay for high test scores and developing standardized tests for preschoolers.

More links:

Friday, July 8, 2011

Watch where you put that yardstick!

On the theme of What Gets Measured Gets Improved, here is some very exciting news from the great state of Utah: State lawmakers are considering replacing grade-specific high-stakes tests with adaptive tests that allow students to demonstrate what they actually know and have learned, even if they are performing well-below or well-above grade level. This is encouraging news for all students and teachers, most especially those involved in education of exceptional students in special education and gifted and talented programs.

Two major problems of traditional grade-level tests limit their usefulness as measures of individual student progress. First, tests vary, often greatly, in content and rigor from one grade to the next, making them meaningless for tracking learning from year to year. Second, the tests fail to measure the progress of exceptional students. In New York, if a student is reading on a kindergarten level in third grade and improves to a first-grade reading level by fourth grade, that student's year of progress is obscured and diminished by a Level 1 score ("Not Meeting the Standard") on the state test. Meanwhile, a third grader capable of seventh-grade math could score a Level 4 ("Meeting the Standard with Distinction") for three years in a row without making any individual progress at all.

Computer adaptive tests solve these problems by adapting to students' abilities regardless of grade level. Basing questions on student performance during the test, the software allows more-advanced students access to more-advanced questions and less-advanced students access to questions at their current level of academic function. This targeted testing provides students and educators with meaningful data about how much students actually know and can do. Paired with software to analyze test results and provide students and educators with individualized "next steps," computer adaptive testing could give students and teachers an incredibly useful tool for growth and development: the feedback loop.

How big of a deal is this? The Deseret News quotes State Board of Education member Dave Thomas's statement to the Education Interim Committee: "This is the biggest change we've seen in public education in the last decade.... It really puts Utah at the forefront.... And I mean right at the forefront." I agree. For now, Utah is the only state with an adaptive test pilot program because it's the only state with a NCLB waiver. State legislators will consider adopting the program statewide during the 2012 legislative session.

Keep an eye on this one; if computer adaptive testing is allowed to develop and improve in Utah, the innovation could spread to New York State to support the development (and boost the morale!) of under-recognized students and teachers right here at home.


Links
Thomas Goetz's excellent Wired Magazine article

Molly Farmer's Deseret News report

Wednesday, July 6, 2011

Raise up your staff, part 2

In part 1, we looked at new principal training programs preparing school leaders to manage increasingly businessified education settings. Now we turn our attention to the preparation of the teachers -- the professional subjects of educational management and the front-line leaders of classrooms. New York City currently claims a three-year new teacher attrition rate of 40% or more. Clearly new teachers are under-supported and under-prepared for their professional roles.

Doug Lemov, once a successful teacher and now a managing director with Uncommon Schools, recognized this and set out to do something about it. As the national rhetoric has shifted toward "getting rid of bad teachers," Lemov realized that many good people were failing as teachers simply because they had never actually learned how to teach. Replacing them with equally ill-prepared peers would not solve the problem. He set out to discover and codify the specific practices effective teachers employ. Last year Lemov published his book of "49 Techniques that Put Students on the Path toward College." He uses the book as a guide to train new teachers in the Uncommon Schools network.

Last spring, I had the opportunity to read an advanced draft of the book and see the techniques in practice in several classrooms at Uncommon's Leadership Prep Bedford Stuyvesant Charter School. The effect was impressive. Students as young as five were consistently orderly, engaged, and on-task. It was all very controlled. Admittedly, this is a huge step up from the disorganization and lack of focus I have witnessed in over a dozen New York City schools.

It disturbs me, though, especially in light of recent research on problem solving and executive functioning, that Lemov devotes only one chapter out of 12 -- less than 15 pages of his 310 page book -- to "Challenging Students to Think Critically." As long as we're businessifying our educational system, let us not forget that What Gets Measured Gets Improved. Current high-stakes tests prioritize measuring content knowledge over critical thinking, and Lemov's Taxonomy aims to improve those measurements.

I agree with Lemov (and Deborah Lowenberg Ball, and so many others) that we need to commit to training teachers effectively if we expect them to be effective in the classroom. Lemov's techniques have been critical to the excellent test scores achieved by Uncommon's students, but let's not trick ourselves into thinking that excellence in test scores equals excellent education. Lemov's Taxonomy is a wonderful and necessary start to preparing effective educators, but for the sake of excellence let it not be an end.

Monday, July 4, 2011

Celebrating independence

We hold these truths to be self-evident, that all men (sic) are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.
~ Declaration of Independence of the United States of America

Sharona Coutts and Jennifer LeFleur at ProPublica recently published the results of their incredibly thorough analysis of recently released survey data from the Department of Education's Office of Civil Rights. Unsurprisingly, the analysis revealed a national correlation between race and access, and a more striking correlation between wealth and opportunity. Disparity differs by state, however, and some states (Florida is the major dark horse here, with Texas close behind) seem to be leveling the field in terms of providing equal access to higher-level courses across richer and poorer districts.

It is intriguing that Florida and Texas are the leaders in equality of educational access, especially because they are far from the top according to another measure: the states are ranked 32 and 35, respectively, out of 52 (50 states plus D.C. and overseas military bases) in terms of student performance on national standardized tests. Meanwhile Kansas (ranked 16) and Maryland (8) are among the states providing the least equality of opportunity. Are poorer kids in Maryland scoring higher than their economic peers in Texas, or, more likely, are the scores the result of a higher prevalence of poverty in Texas relative to Maryland?

The available data do not yet show whether [more] equal access to higher level coursework at the high school level in Florida and Texas is translating into higher rates of high school graduation, college enrollment, and success at the college level. ProPublica promises future articles on the subject as more information becomes available. I would also be interested to know just how wide the economic gaps are in each of these states. Does equity of access correlate with economic parity, or are the most accessible states making gains in spite of, or in response to, great economic disparity? How will recent gains in educational access affect the national standings of states like Florida and Texas in future rankings based on national standardized tests. Finally, to what extent is the success of Florida and Texas replicable?


One final, unrelated note on the theme of Life, Liberty and the pursuit of Happiness: check out this story of a kid who, with the support of those around him, is taking his own "walk down freedom's trail."


On Wednesday Ed Nerd returns to the topic of leadership; check back for part 2 of Raise up your staff.