(Mis)measurements: The Economy & Standardized Tests

The GNP [GDP] measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country.  It measures everything, in short, except that which makes life worthwhile.” — Robert F. Kennedy, 1968

In the book, Child Honouring* (a book I highly recommend), Ronald Colman questions what is actually measured in the economy and tells us that:

“all of us have been hooked on the illusion that economic growth equals well-being and prosperity. Indeed, there is probably no more pervasive and dangerous myth in our society than the materialist assumption that more is better.”

Colman goes on to describe the many things that can drive progress in the economy that actually degrade our quality of life (adapted from his chapter “What Matters Most”):

  • crime and imprisonment – one of the ‘fastest growing sectors’ in the American economy
  • production, sale and use of materials that are harmful to our world/environment
  • addictive gambling
  • depression and the sale of medications and services to help with depression
  • war and the production/sale of artillery/ammunition
  • divorce
  • over-consumption of food and resulting diet/weight-loss products/services.
  • consumer debt (my addition)

He also discusses the booming child-care industry in Canada and questions how this could possibly be a good thing as this means parents are spending less time with their children (thus the need for full day kindergarten in BC – my addition).  The chapter takes a hard look at the consumerist desire for (and perceived value of) economic growth; he brings to light the many costs of a growing economy that are detrimental to our children and society that go unnoticed and unmeasured.  Society believes that we are doing well when the economy is growing but fails to consider many other aspects of societal worth – time, natural resources, care, education, health – and other parts that “make life worthwhile”.   He questions the value of economic measures that do not take these other more important aspects into consideration.

Relate this to the use of standardized tests to measure learning in our classrooms and schools (tests in which every student in a particular province/state in a certain grade takes the same, often multiple choice, test at a mandated time of the year… and then results are often published and used to rank/compare schools).  People attempt to measure how our children are doing using these tests but, in fact, they often measure things like:

  • how motivated the child is to do the test
  • how much the class has been focusing on the test
  • how much the teacher has been teaching TO the test
  • if the parent has allowed the child to write the test (and how many children write the test)
  • how the child is feeling the day of the test
  • how much anxiety was the child feeling (also, add the pressure in the US that the resulting scores may impact the teacher’s employment)
  • how much the teacher(s) helped the students on the test
  • who marked the test
  • the family income and education background
  • (also) the assumption that every child progresses at the same rate

The following is a list of skills that we often say we value as a society but are not measured by standardized tests (those often measured with standardized tests are in brackets):

  • critical thinking (vs memorization)
  • creativity (vs fact telling)
  • collaboration (vs independent test-taking)
  • leadership (vs compliance)
  • care, empathy for others
  • understanding of other cultures (vs a euro-central model to learning)
  • care for our planet (vs just memorizing facts about environment)
  • respect
  • awareness of students’ strengths and challenges (vs ability to take tests)
  • communication skills (vs information telling)

If we have high test scores , does this mean our children are truly learning how to lead a worthwhile life? How many of us have been “hooked on the illusion” that good standardized test scores equal effective learning and a quality education system?

When we focus on the economy, we realize there are many ways that can drive growth that can unfortunately be harmful to society and, in particular, our children.  When we focus on standardized test scores, we realize that there many ways to increase these scores that can unfortunately be harmful to learning and, in particular, our children.

I am not saying we should ignore the economy nor should we ignore the results of standardized tests but we need to reflect on the actual meaning of these measurements. With regards to education, I understand we want to know how our children and schools are doing but we need to ask ourselves: how much time do we want to spend analyzing these (mis)measurements of learning?  How much time do we want our teachers and children spending on these tests? Can we place a numerical value on learning the skills needed to lead a flourishing life?

*Cavoukian, R. and Olfman, S., eds.  Child Honouring: How To Turn This World Around. Salt Spring Island, BC. Homeland Press, 2006


Standardized Testing: A Parent Voice


I am pleased to have Sheila Stewart as a guest writer.  I have been introduced to Sheila’s passion around education  through Twitter and although Sheila is from Ontario, her thoughts on technology, standardized testing, and learning reach far beyond that province and her words tie in well with BC’s discussion around the local standardized test, the Foundation Skills Assessment (FSA).

Sheila is a very engaged parent in the world of education.  Her roles include:

  • a parent of 2 teens in Thunder Bay, Ontario
  • School Council Co-Chair at the High School
  • Coordinator, School Council Chairs Network
  • Parent member, Parent Involvement Committee
  • Network member & online community moderator, People for Education, Ontario
  • Follow her on Twitter at @sheilaspeaking

by Sheila Stewart

I shouldn’t let newspaper editorials get to me so…..but…..when they sound so definitive on education like this, Measuring Literacy

….other voices are needed…….so here was my response:

Learning and Measurement

A recent editorial noted the variety of literacy activities highlighted locally during Ontario’s Literacy Week.  The editorial, “Measuring Literacy” (Sun., Jan. 27) touched on technology, literacy, standardized testing, and associated outcomes.  I also believe there are many ways to foster literacy development and demonstrate that learning.  Everything is very connected, and impacts can be both negative and positive.  The influence of technology can be overwhelming, but I have remained willing to consider the positive opportunities for education.  I have continued to see examples of how technology as one tool in the learning environment can provide many opportunities to further support literacy, communication, collaboration, and engaged learning.  As with any new technology, the use of it as a teaching tool can only be as effective as the training and experience of the teacher.  Digital literacy doesn’t necessarily promote the use of sub-standard language skills per se.  As with many things, there are always many variables that can impact skills and outcomes.

I am aware of many educators in Ontario and in other provinces who care deeply about creating authentic learning opportunities to help their students’ strengths, passions, and individual growth as citizens.  There is much more to supporting learners that may not be reflected on a standardized test.  And much more feedback on learning is important for students, and for their parents as well.  If the individuals who spend time in classrooms are expressing concerns about the impact of standardized testing on the learning environment, then I say we should at least have a listen and not conclude that it is only about a fear of being evaluated themselves.  It is my understanding that current standardized testing in Ontario is more of a “system” measure and much less to do with information about individual students and teachers.

I would also be cautious in saying that “parents”, in an all-inclusive way, value the information from standardized tests.  Parents are diverse, as is our province.  I know a number of parents who have become unsupportive of standardized testing and have concern about the impact of so much focus on these outcomes.  Can we feel satisfied that all is well based on reports of one set of data?

There must be something valid to the concerns about standardized testing if so many provinces are involved in a debate about their usefulness in supporting student learning.  I agree with the writer of the editorial, “Good education is essential to Ontario’s future.”  For us in the north, I hope we continue to dialogue with all partners in education and keep asking the questions of why….what does that mean….what else is needed….is that enough….how can we help….as well as, what is possible with technology?


A deeper look into school rankings

As a teacher I never paid much attention to the annual Fraser Institute Rankings; when our school did well, people applauded and when our school fared poorly, people raised questions.  The interesting part for me was that we had the same staff and same curriculum, yet our rankings changed year over year.

When I became a principal, parents began to ask me about our FSA results and Fraser Institute Ranking.  I cannot say that I am now actually interested in this harmful process but I do feel I need to comment.

The latest rankings state that our school is ranked 761/876 schools in the Province of BC.  Lets look a bit deeper into the “data” the FI has used to determine this ranking.  The primary piece of data that is used for these rankings are the results of our FSA tests that our grade 4’s wrote in February of 2009.  Last year, parents had the option of requesting their children being exempted from writing this test; a letter was sent home from the Teachers’ Association explaining this.  Many of our parents were concerned about the educational value of this test so only 27 out of 58 students wrote the test; LESS THAN HALF of our students were included in the data used for the FI rankings.  On the FSA reports website, it states that only 29% of our students were meeting /exceeding expectations; in actuality, 17/27 students that wrote were meeting/exceeding.  I am no statistician but I do know that 17/27 is much higher than 29%!  In addition, a large number of the students that did not write were students who, on their report cards, were in the C+ or higher range – meaning that they were meeting expectations – so if they had written this test, it would have helped our results and our ranking, although it still would not change my view of the rankings.

Each year, the principal works with parents to develop the school goals.  Our main goal is to help each student to ‘develop his/her unique talents and interest and leave our school as a confident learner’.   Spending weeks on a test does not really align with our goal and we could not even use the resulting data from 2009 because we know that with that few students writing the tests, the data had little use.  ( I will avoid sharing my views on the test itself but if you would like to discuss this with me, please contact me at any time!).  Using this data to represent our school makes it seem like we have taught all our students for a number of years.   We had 4 students register at the school in January and February of 2009 and they wrote the FSA for our school.  How can we use the data that tests students who we have barely had the chance to work with?  A more valid and reliable form of data to assess literacy would be to test the students who have actually been at our school for at least 60% of the education.  If we looked at these students, they would have had the opportunity to obtain support to increase learning through a variety of teaching methods.  (Having said this, if we tested our schools/students this way, we would also be testing the impact of remaining in the same school for a number of years.)

I am not opposed to using data/evidence to help determine school goals but this data must be valid, reliable and not used to rank schools.  Michael Fullan, a respected author and educational researcher has been working with the Ministry of Education in Ontario to develop valid and reliable assessments; he has an agreement that any data from schools is NOT to be used to rank schools due to the harm that it creates in the system.

Looking at the rankings on a broader scale, schools are expected to maintain/improve their test results year over year; this becomes a challenge when, at our school, the school counseling position, learning assistance teaching support time, administrative time (principal and vice principal’s opportunity to work with students), special education assistant time, library teaching time, the lunch program, field trips, and learning resources have all been cut to an all-time low.  At our school, we continue to do more with less and I am very proud of what all our staff and students achieve.  Our successful art, physical education, science, music, culture, and extra-curricular programs are not included in the rankings and these are some parts of education, in addition to numeracy and literacy, that we truly value.

Someone once said that “ranking schools based on a test score is no different than ranking dentists based on the number of cavities”.  What they were commenting on is the fact that so many other factors come in to play when assessing children and their schools: socio-economic status, access to resources, funding, student nutrition/health, urban vs suburban vs rural schools, student transition rate (how often students move to and from schools), parent education, home situations, etc.  How can schools accurately be ranked when there are so many variables?  I teach my grade 5 science students to always ensure their experiments are a “fair test”; even they would tell you that there are far too many variables to consider the assessment of schools (based on one test) a “fair test”.  With this, comparing schools throughout the province is not helpful at all; can our school be effectively compared to a “choice” school in Abbotsford, a rural school in Fort St. James, a school in the British Properties, or a private school with a tuition of tens of thousand dollars a year?  We always look at ways to improve our school but we do not look at schools that are that dissimilar and say ‘we need to do what they are doing’.

We have a school, like most others, that has a number of unique challenges; many of these all help to make our school so great!  As the saying goes, “the greater the challenge, the greater the triumph!”.  I, along with the staff, look forward to coming to school every day to learn alongside with the students and work with other staff, parents, and community members to continue to increase student learning.

There is no ranking for student happiness nor is there a ranking for true education (one that leads to a healthy, worthwhile life); what I can tell you is that Kent School is a great school and if you ever want to rank us, spend a week, month, or year with our staff and students – you will never rank us that low again.  Better yet, if the Fraser Institute actually spent time in a school, they would soon realize that it is better to not rank at all.