Sunday, December 22, 2013

Observing out loud

Imagine for a moment that you're testing out a new website for local startup company.  The designer is sitting behind you and observing as you are given tasks to complete (such as "register for an upcoming event" or "find the address of our nearest location").  Since the design is a bit unusual, you struggle through the tasks.  The designer sees your struggle, but she has a hard time knowing why.  In design classes at Olin, we learned how to narrate our thoughts as we worked through these tasks, allowing the observing designer the ability to quickly see our assumptions as we interacted with the product.  The goal was not to pass judgment or opinions forward, but just to give the observer a deeper insight into your brain, allowing her (the expert) to go back and improve the design to better accommodate the users' assumptions.

Thinking back on this, I have two ideas:

  1. What if we started observing our students in a formal, rigorous way as they interact with our product (the course / curriculum)?  Every so often, I have a moment in class where nobody has a question and everyone is engaged in what they are working on (it's rare, but peaceful when it happens).  My favorite thing to do is just watch a couple kids closely.  Some are really in the zone and working incredibly hard.  Others will fidget, turn to their iPads / phones, flip between tabs on the computer, stare off into space, or find other ineffective ways to cope with boredom.  When I create new iterations of my courses, these are the kids I really want to reach and design for.  The problem is that I'm not completely sure what is going through their heads.
  2. What if we taught all kids to vocalize their thoughts like a good product tester?  This might make it a lot easier to peer into the minds of the disengaged kids if they can put words to the things they are doing as they start to do their work.  However, it might also provide me with a ton of insight into the mindset and assumptions of the engaged kids.  Perhaps there is a way of thinking or broader philosophy that I can explicitly introduce to my struggling kids.
I don't know what the best environment is for these kinds of observations.  Perhaps a flipped classroom or project-based classroom with lots of kids simultaneously engaged in different tasks could work.  Maybe a 1-on-1 or 2-on-1 after school session would be more private and thus make it easier for a student to speak his or her mind.  Either way, I think it would be a lot easier for teachers to design a better user experience for their students if they could collect such rich user observations.


  1. When I first met Sanjoy (back when he was still at MIT, and agreed to supervise my AHS capstone) he showed me what he'd done with the circuits textbook he was writing. He presented each chapter to his students as it was written, asking them for feedback -- and he was very deliberate in framing it in terms of their expertise.

    They were experts at being novices, experts at being students and checking out the inadequacies of his book to explain things. That was what he needed, and he could not do it on his own. They were helping him find and fix the bugs in his book.

    A slight change in wording, a big shift in power.

  2. As you alluded to in earlier posts, this quest for quality feedback from students is contingent upon them trusting that you won't think their way of thinking is dumb or somehow inferior to others' ways...