What can recommendation systems in museums teach us about underachievement in school?

It is worth thinking about the detail in Anne Salmond’s opinion piece in this morning’s NZ Herald Anne Salmond Open Entry for Maori a near miss Monday July 6 2009.

In addressing Pita Sharples’ suggestion of open entry for Maori students to universities, Salmond uses research findings from the Starpath Project to make three disturbing claims about education in New Zealand.

Claim #1.  The management of educational data in New Zealand has more to do with the distribution of resources rather than with tracking the long-term success or failure of students.

As a result – “schools are often unaware when bright students begin to fail; or when groups of students (say, Maori boys) begin to follow pathways that lead to failure and early exit.”

Claim #2.  New Zealand educational investment in initiatives aimed at enhancing student achievement are “largely working blind”.  We not only do not know if the investment is making a positive difference we also do not know if the investment is targeting the real problem.  

A result – “leading to many uncoordinated, short-term initiatives (80 in one school that Starpath studied) and a failure to identify those approaches that really work, so that they can be adopted across the education system.”

Salmond also calls for a more careful monitoring on government funded initiatives that claim to enhance learning outcomes for Maori, Pacific and low-income students.

Those initiatives that don't have a positive impact on student outcomes should be dropped, while those that are highly successful should be adopted across the education system.

I find it somewhat disturbing that Salmond feels the need to recommend “dropping initiatives that don’t have a positive impact on student outcomes" – it suggests that this is not currently the case. 

If the Starpath Project has evidence of initiatives that do and initiatives that don’t – I hope they offered this evidence to Anne Tolley before the recent budget  - and that Anne Tolley respected their findings - so that we can be confident that we are no longer funding initiatives that don’t have a positive impact on student outcomes.

Claim #3 – Our qualification system NCEA is so complex that families cannot make wise decisions about which courses to study, which educational pathways to pursue.

As a result, “while most Maori, Pacific and low-income students aspire to gain university entrance (78 per cent in one study), it is too easy for them to find themselves on NCEA pathways that foreclose this option.”

I will admit to being attracted to chaos theory, fuzzy logic and ambiguity in what I read but it unnerves me just a little to realise that the Starpath Project research suggests the New Zealand Government is funding educational initiatives where uncertainty of focus and indeterminate outcomes rule.  In truth it is easier to understand Pita Sharples call for open entry for Maori to universities if you accept that  the current framing/ educational design and funding of the MoE initiatives designed to raise Maori and Pacific achievement is closer to whimsy than anything professionally responsible.

Claim#3 captures my interest this evening.  Our latest budget set aside “$8 million to ensure NCEA assessment tools are of a high-standard and well understood by teachers.” Presumably on the basis that teachers understanding of NCEA assessment leaves something to be desired.  Salmond’s remarks suggest that parents and students similarly lack the understanding needed to make good decisions about NCEA assessment.

It all makes me wonder how we could re-design the NCEA course options available at secondary schools so that students and their parents would find it easier to make wise choices

There is more to it than this of course - Any New Zealand student studying for NCEA can relate instances where they or their friends have been excluded from courses or dissuaded from certain option lines on the grounds that the institutions deems the chance of student success unlikely. 

In New Zealand secondary schools the right to try (and fail) is seldom available.  Like Etruscans divining the future from the entrails of sacrified animals, secondary teachers continue to confidently (and perhaps patronisingly) practice haruspicy on the NCEA course selections of their students.

So parents and students not only need to understand NCEA well enough to make wise choices they also need to understand it well enough to fight the institution for their right to access courses based upon these choices.

Nina Simon in the Musem2.0 Blog has been looking at ways museums can design recommendation systems for their visitors in Designing recommendation systems that go beyond “You’ll like this”  . 

Much of her thinking about customised museum tours can be usefully transferred to educational contexts – where museum exhibits become educational options and courses.  For example Simon’s thinking helps me think in new ways about the design of solutions to help students and their families make sense of the courses and learning pathways available to them.

When it comes to museums, recommendation systems are a natural solution for the problem of the customized tour. How can a museum offer each visitor suggestions for exhibits and experiences that will uniquely serve their interests? There are many lovely example of museums providing quirky tours based on particular interests. For example, The Tate Modern offers a set of pamphlets featuring different tours of the museum based on emotional mood. You can pick up the "I've just split up" tour and wallow in depression, or the "I'm an animal freak" tour and explore your wilder side. And the site I Like Museums lets you find whole institutions of interest based on your preference for trails like "making things," "nice cup of tea," or simply "pigs."

Salmond's call for more careful monitoring of student learning outcomes .... 

Above all, the compulsory education system needs re-engineering. Information systems in schools should be tracking the educational journeys of students, identifying the strengths and potentials of individual students (so that they and their parents get optimal advice), and patterns of success and failure across the student body (so that initiatives are accurately targeted).

... sounds like it could be answered in part by the design principles in the recommendation system Simon suggests for museums – one based upon collaborative filtering   (“like the one used to recommend new songs to you on Pandora or new movies on Netflix?”) –

Perhaps we can design a platform for the monitoring of individual educational journeys in New Zealand – one that could aggregate content about the strengths and potentials of individual students and build it into a Pandora/Netflix like recommendation system – a system alerting students and their families of the educational targets to be met and the course options available.

Then the patterns of success data available might allow students and their families to thoughtfully design learning pathways - pathways that not only meet their aspirations but also extend them to create alternative educational reach.

Comments

[Cherrie]

Though I guess this isn't a "system" with "records" and all that - but good schools and teachers are able to pick up on changes in student achievement, are they not?

But then I'm probably speaking from the ignorance of having only been to good schools...

[nix]

As we approach NCEA with child number six (only one of the previous five did NCEA as the rest went through the old exam system), the complete lack of pathway information from the school's official channels is scary.

Over the next few months we'll be starting to make some choices about the directions that the boy takes. Given the familial history of late educational maturation I am wary of him being silo-ed too early. I'm determined to grow a well rounded young man with sufficient open options to make choices when he is ready.

[Artichoke]

RE: but good schools and teachers are able to pick up on changes in student achievement, are they not?

Try Graham Nuttall's interview with Kim Hill http://tinyurl.com/lsudv6

[Steu Mann]

I enjoy reading your blog. It has given me some insight about teaching and I think other teachers will benefit from what you are writing.

I have added it to my education portal at http://www.educationreporting.com/#blogs

Let me know if you have any questions and thanks for your ideas and thoughts.

Steu Mann, M. Ed.
http://twitter.com/cathriving

[Cherrie]

ooho ok i just read the first couple of paragraphs and it made me think that we should replace "teachers" with "doctors" and "students" with "patients" and we'd probably get the same outcome from that sort of research. Maybe the only type of behaviour out there that is truly well documented and understood... and easily influenced/managed is consumerism.!!!

*shrug*

[Nina Simon]

It would be great if the paths of students were available in a (safe) way to each other. Think about how encouraging it would be to hear, "that test clearly isn't right for you. You're just like Johnny in Auckland, who learned to read in a whole other way. Let's go in that direction." Students could become a distributed community of learners, with each's path helping the others.

[Artichoke]

RE: Maybe the only type of behaviour out there that is truly well documented and understood... and easily influenced/managed is consumerism.!!!

And we can get insight on consumerism most every place we look ...

According to McKibben, our consumption finds its most complete expression in the SkyMall catalogue, available in the seat back pocket of every aircraft in North America. He concludes that we now have solutions to problems we never knew we had, as exemplified by the trouser rack that keeps twenty pairs neatly hung; the giant capacity mailbox that holds up to two weeks of mail (catalogues, presumably); the ultra-violet toothbrush cleaner, and my hands down favourite – the automatic watch winder. This device is for those who have a lot of watches, and it mimics the actions of the human wrist, with intermittent timers and directional controls, to keep all of them fully wound at all times.
In - Museums in a troubled world – Janes 2009 Uncontrollable consumption P33

... unlike education and health science where the insights available are mostly of the walled garden variety - and any less than reverential comment/response from outside causes consternation

[Cherrie]

*wrings hands*

and what greater wall than that built by the tertiary institution!! I still haven't found my way out...

...at least I haven't found David Bowie yet...

[Artichoke]

Hi Nina,thanks for making time to comment here.

Imagining how we could make the paths of students available in a(safe)way to each other - made me think about developing an intuitive/responsive/intelligent version of The Art Institute of Chicago PATHFINDER

Except this one would be constantly aggregating learning pathway information from the distributed community of learners.

My guess is that given the complexity of the possible learning pathways we would need to accommodate - we would have to build in/allow for the development of desire paths

[Artichoke]

Thanks Steu, networking with others in the comment threads of blogs and twitter threads is certainly a catalyst for many of my new imaginings. I will enjoy adding another educator to my twitter stream.

[Artichoke]

Making choices that influence other choices is frightening for students and their parents - especially when the options lie outside the experience and expertise of families

Had a similar experience recently with accessing tertiary degree options and child number three - that insider stuff that ensures the children of privilege have a head start - Gladwell exposes it well in his "Outliers" book

Got a follow up email from Scott Summers regarding the sorts of information provided for college level students in US - see below

• Hi Pam,

  I came across your webpage < http://artichoke.typepad.com/artichoke/ > and I work with college directory and career education websites managed in Kansas that I thought would make great resources for this page. I've added our websites information below for your review and consideration. If there are any questions please don’t hesitate to e-mail me. Thanks for your time and have a good day!

  Career Explorer: <http://www.careerexplorer.net/>

  Description: Career Explorer is designed to help future college students find which career is best for them. The site has aptitude tests, career videos for specific career fields and a number of articles to help students make good decisions about college.

  
  U.S. College Search: <http://www.uscollegesearch.org/>

  Description: U.S College Search is your official guide to the nation's colleges, universities and online degrees. We offer one the most comprehensive guides to regional colleges on the Internet with a database of 9734 schools and universities.

  Technical Schools Guide: <http://www.technical-schools-guide.com/>

  Description: Technical Schools Guide provides links and information to technical schools and colleges throughout the United States.

  
  Scott Summers

Perhaps this is a project you and the kids could consider - creating an online pathway information site for NCEA but built by kids for kids - would probably be a more powerful kid built info source than Rate My Teachers