Sunday, November 13, 2016

Grand Challenge Design: Q1 is in the Books

The first quarter of Grand Challenge Design flew by.  Though it didn't go like I planned, it was a very positive start to this year-long adventure.  The key relational, cultural, procedural, and skills groundwork are in place, our game is going to launch tomorrow, and we have a productive space for learning.

To start the year, every student wrote a letter about themselves, their future, and the course.  This letter was based on Ben Zander's concept of transforming relationships and giving every student an A.  In the letter, each student gave me the insight I needed to start a successful working and mentoring relationship with them.  As first drafts, nearly every letter came in looking awful.  To address this, I started with comments and suggestions over Google Docs.  For some students, this was all that was needed to lead to a more productive draft.  For others, I found myself having one or two 1:1 meetings at lunch, during advisory time, or after school to verbally probe into some of the questions I wanted them to explore.  I also reached out to my English teacher colleagues who were amazing in helping me and my students through the writing process.

Once a letter was accepted as complete, I used it to match the student with the best-fitting mentor I could think of.  Every student is paired with their own mentor, a professional engineer or designer chosen from my college and local friends who were willing to jump in.  I don't know if my students realize how top-notch these people are yet, but they will in time.  One mentor, after receiving his mentee's introductory letter, even wrote back with his own letter, written as if he was in high school but then continuing to the present day.  As I see some of the email exchanges that I am CC'd on and poke my head into some of the Skype chats during class, I can't help but get excited for all the things students will learn from NOT me this year.  As I get texts and emails from mentors seeking additional advice and counsel on how best to help their student, I am amazed at how much these people care about a group of random kids in Byron, MN.

Our Grand Challenge for the first quarter was not the individual work, but the team effort that was required to build up the course.  After getting a sense of what students were interested in doing, I divided the class into teams of 2-4 with different areas of focus.  From there, I presented the core of the vision to the class: this quarter, we were going to get our game built and ready to play and improve our classroom to be a fun and effective learning space.  I structured this into "OKRs", or "Objectives and Key Results", a goal-setting and measuring format created by employees at Google so that everyone could have autonomy while moving in the same direction, have clarity of what they and others were focusing on, and have a system to measure their own progress.  I asked each student team to do the same.  Here are a few example OKR docs from the teams in class: CNC team, NodeJS team, Rules team.

At this stage, my role involved checking in with each team each day to offer ideas, direction, feedback, or whatever was needed.  I also made sure that the different groups were communicating with each other.  At first, this process seemed relatively successful.  After a month, I found that teams were splitting two different directions: the teams that had long-term projects did okay managing their day-to-day work productively, but the teams with many small tasks had a hard time figuring out where to go next without heavy support.  To address this, I introduced three elements of the scrum framework: a product owner, a Kan Ban board, and a daily stand-up

Scrum is used heavily in the software world as a way to help teams self-organize and complete tasks around user needs.  The role of a product owner is to decide which tasks need to get done, and what "done" looks like, to achieve the objectives of the company.  As the lead person pulling everything together, I was a product owner.  However, I also invited one student from each team to take on that role and work with me to generate and refine their team's task list.  From here, each team put the tasks on a large kanban board divided into four sections: backlog, in-progress, ready-to-check, and done.  The backlog includes all of the tasks that need to be done but had not been started.  In-progress is what you would expect.  Ready-to-check includes all tasks that the doer thinks is done but needs a product owner to review to make sure it truly meets the definition of done.  "Done" is truly done.

We used the board as part of the daily stand-up meeting.  Within the first five minutes of class, the entire class crowded around the board and said (1) what they accomplished yesterday, (2) what they were planning to do today, and (3) if anything was blocking them.  (1) was for public accountability -- I only realized after we stopped doing these daily meetings how well that was actually working (a few students slipped into multiple unproductive days without this).  (2) was for me -- if they said they were doing something, but it didn't sound very useful or sound like they knew what they were talking about, I would either correct on the spot or follow-up with that person after the meeting.  (3) was for everyone else to listen to -- if two teams were going to be doing something in the same space or with the same resource, they would have a heads up and be able to work it out ahead of time.  This didn't come up a lot during the time where I used the daily stand-up with the entire class, but now that we are closer to fully integrating everything, it seems more likely to happen.  I plan to return to using this whole-class meeting daily in Q2 on team days.

Back in August, I pictured the game starting up in a couple weeks after school started.  It was only after a couple weeks passed that I realized that my sense of reality was heavily warped.  The most obvious challenge we faced was finished the game board surface.  Doing so required the use of a machine that we bought in isolated components and assembled ourselves.  A former student and older brother of a GCDer led the design, assembly, and software setup for the machine.  Unfortunately, the challenges involved in making everything work were more than even this mentor could quickly master.  The two students working on the machine, with some weekend help from their mentor, found faulty limit switches, a bad stepper motor, wiring issues, and software configuration problems.  Then they went through a time-consuming process of calibrating the machine so that a software inch would be a true inch.  To actually cut material, they first needed to design a part in CAD (something they knew well), but then needed to run it through CAM (computer-aided manufacturing) software to tell the machine how to make the part.  They needed to figure out which bits to use and how fast to spin them.  They needed to find a consistent way to mount the stock wood in the machine so it would remain steady.  They needed to find a way to mount a shop vac hose to the machine with a dust-trapper so that the excessive amounts of sawdust would stay contained.  And this is not a complete list!  Though this team dealt with the most extreme set of challenges for what sounded like a very simple task, nearly all teams found themselves experiencing dozens of complex problems nested within something that sounded straightforward at the beginning.

I love watching students work through this kind of challenge.  Given that I am used to students giving up on tasks with lots of direction, resources, and a knowledgeable guide, seeing students persevere with none of these is refreshing.  Here are a few action shots from the semester as students persevere through this (thank you, Kris Nelson!):


Finally, I asked students to end the quarter with a video summary that they would share with me and their mentor.  This video had to be at least two minutes long and discuss what they did and what they learned.  Watching these has been a ton of fun for me since I sometimes forget where students started the quarter.  Thank you, Jen Hegna, for recommending that I do this!  A few examples are below:

Despite the number of tasks that are not completed yet, I am really excited about launching the game tomorrow and building the airplane at 10,000 feet.  We will alternate game days and team days so there is time to continue getting key tasks completed while allowing us to experience what things are actually needed for game play.  And hopefully, this time, I will blog more than once per quarter and keep each post under 1500 words.

Sunday, October 16, 2016

Reflecting on Design Thinking

As part of my Winona State University cohort, we all engaged in a condensed experience in Design Thinking, a process that facilitates user-centered design.  Back at Olin, Design Thinking was the heart of the engineering curriculum, pulling together our technical work with entrepreneurship and the humanities in a process that puts people at the core, so this was not a new experience.  However, it was the first time I used Stanford-based materials, the first time I did a design project alone, and the first time I went through a formal application of design since I started teaching.

To focus my design work, I decided not to look at all of my students.  Instead, the problem I wanted to better understand was why my Grand Challenge Design course was so male-dominated (20 young men, 3 young women), so I studied current 10th and 11th-grade female students, the ones who could register for GC Design starting in January.  I did not enter with an expected outcome, but rather let the process reveal insights as they came up.  The point was to truly understand and empathize with this group that I was clearly having a hard time appealing to.  Starting with this understanding, I could improve the design of the course and more effectively communicate what was already good to these students.

Students are only one of many stakeholders in public education.  The federal, state, and local governments all have an important stake, as to community members, parents, administrators, teachers, and support staff.  However, given that students are the end customers, it is amazing how little we seek to understand them.  An empathy-centered process stops asking what students want (more time to do their existing work, better resources, etc.) and observes what students do, say, and think.  From there, I made inferences and connected the commonalities to create a picture of who these students are.  One of the more powerful insights I gained from this process is that the students I interviewed were open to creating and the engineering process.  Food and art were common places to make and invent.  However, content and peer groups mattered.  Nearly all of the young women I talked to had clear passions and career interests that developed in middle school, directing their decisions on courses and extra-curricular activities.  They were also aware of which courses were likely to be mostly guys, and most of them avoided these.  A surprising theme was the belief that they would fall behind in a technical course.  This is especially curious because the majority of them said that they did well in their 8th grade STEM course and never fell behind (where is this belief coming from?!).

After this research, I designed my first prototype solution: a two-hour course called "LED Art" on our school's exploratory lesson day (twice per year).  I described it as a chance to design your own art project on foam board and bring it to life with custom-programmed LED lights.  The description worked!  I recruited a class of 13 gals and 12 guys.  Though I was working with a lighter supply of materials than I was hoping for, I had enough to run the class.  On game day, it was a total disaster.  Students managed to mostly have fun and learn a thing or two, but the logistics of helping everyone get a basic circuit running proved just a hair too much for only two hours with my planning.  After running the course, I have a few dozen specific changes that I plan to make that will allow everyone to be up and running significantly faster, allowing more time for explorations, programming, and actual art design.

The most fascinating observation of the whole course was what happened when I asked students if they wanted to buy a $10 kit of parts to take home: 8/12 guys signed up, 1/13 of the ladies did.  While working, there were only a few people that appeared to know what they were doing, and yet the guys were the ones who seemed either confident enough or interested enough to want to take it home and keep learning.  I'm curious is this is an inherent fact about this group, or if there would be different results on taking a kit home if I provided more structured handouts and guides from the start.  I wonder if the outcome would have changed if I took the time to learn more names during the session and encouraged each person as they worked.

In the end, the whole experience was incredibly worthwhile.  More than any specific insights, of which there were many, I am fully re-convinced that the Design Thinking process needs to guide any important decisions that I am making.  I can guess what students want or need based on my past experience, and I can read a lot of relevant research that gives me insight, but direct, targeted observation and iterative design with student feedback leads to solutions that really nail the important details.  Beyond my role as a teacher, if I want to be an innovative instructional leader in my school, I need to be able to facilitate a team working through this process together.  With a partner or team, the process involves a lot of discussion and skimming of insights at every stage of engaging with users.  By leading the design process, I can expand the number of people who are empathizing effectively with students at our school and be directly supported in improving the design work I do for my own classroom.

Thursday, September 8, 2016

Giving every student an A on the first day

I spent a large chunk of last spring and the summer designing my new Grand Challenge Design course, and one of the sizable thought-sinks was figuring out grading.  I had zillions of rubrics I could design from, but after watching this video by Benjamin Zander, I just couldn't get it out of my head.  Skip the first 30 seconds of fluff.  If you don't have time for the whole thing, watch at least the first 5 minutes.

My college partner-in-crime, Marco, recommended it to me along with other course feedback.  Zander is a well-known conductor and music teacher out in Boston who speaks about possibilities.  His TED talk looks at the change in worldview between the approaches of:

  • "Only 3% of people like classical music.  If only we can increase that to 4%."  vs.
  • "Everybody likes classical music!  They just don't know it yet!"

The energy with that kind of thinking is contagious.  That's why his idea to give every student an A, then teach the student to become an A student, had a fascinating logic to it.  By getting the grade out of the way immediately, students could stop operating in the extrinsic / comparator mode and have nothing left but learning to seek.  The expectations do not go down in this environment -- they actually go up for everyone.  They have to if everyone is actually an A student.

As I kept thinking about it, this approach is not that uncommon.  I am given a paycheck every two weeks with the same amount regardless of the quality of my work.  When I do good work, I am encouraged to keep doing it.  When I do bad work, my peers and admin help me clean things up and make it good.  The expectation is that I do great work, and so when things falter, I get help, not a pay decrease.  If things are working correctly, except for occasional bumps in the road, I will be earning the money I have already been promised.  Even 2000 years ago Jesus started grading with this kind of system, handing out unquestioned grace first and then fixing people up from that point forward, so the idea has been around for a long time.  Despite this, it is just SO different than school has ever operated.

I have never been afraid of change or trying out crazy things, and I went into this plan feeling pretty good.  When I told the students, they were a bit skeptical, but I convinced them that all they had to do was write me a letter (that I would of course have to consider acceptable), and they would have an A for the quarter that I would absolutely not revert.  I did make it clear that, if needed, I would ask for extra time, call parents, etc. as extrinsic tools to support learning, and that if a student completely gave up I would ask him or her to drop the course.  Despite these minor caveats, after laying out the plan for them, I was feeling pretty vulnerable.

It wasn't until the second night after school started that I had a hard time falling asleep.  WHAT HAD I DONE?!?  What would other teachers and administrators say?  What would parents say?  What if students took advantage and didn't push themselves to learn and grow?  Is this the kind of thing that gets people banished to the no-friends corner for being too far out there?  I value my peers and work with an amazing group of teachers, and yet I never felt confident enough to discuss my plan with them before I just publically committed to it in front of my students.  What was I thinking?  Was I thinking?

Since the plan is now in motion, and there isn't much I can do to stop it for the next 2 months until the quarter ends, I'm going to do my best to capture the ups and down on the blog.  Today, I introduced the letter-writing assignment, the one where students date the letter at the end of the year and start with "Mr. Pethan, I deserve an A because...".  I asked them to write as much as they needed to in order to help me understand who they are and who they want to become.  I will use the letters to figure out how to best teach each person so they grow and learn and develop into that amazing person they write about.  I will also use the letters right away to find every student their own mentor for the year, a mix of awesome people that I know locally and around the country (possibly world) from college and other experiences.  Even though students are self-selecting into teams that will develop different parts of the course right now, I will use the letters to redirect and shuffle those groups to better help them reach their long-term goals.

My plan is to make sure that every student in my class DESERVES the A they received.  The fact that they knew they got it on day 1 is irrelevant.  If students all learn many meaningful skills and can tell that powerful story of transformation, nobody will question their grade.  Deep down, I know I went forward with this plan because it aligns with everything I know about human motivation, creativity, autonomy, and mentorship.  I really believe that grading in this class would have killed passion and brave new ideas in favor of checking the boxes that get an A.  This plan is the best thing I know how to do right now, and as a one-quarter pilot, I have a long-term out if the concept is truly flawed.  I'm excited, optimistic, and above-all, terrified.

Tuesday, September 6, 2016

Real World Learning

I am now entering the 4th of 5 courses in my Winona State Innovative Instructional Leadership Certificate program: "Real World Learning Design".  It comes at a good time for me as I am up to my eyeballs in the creation of Grand Challenge Design, a course intended to create a very meaningful and real-world learning experience for students.  Prof Jen's first task is to answer, and defend, the five questions below about the meaning and purpose of real-world learning.

What is your definition of “real world learning”?  

I liked the definitions from EdGlossary and the Schools We Need Project as starting points:

"Connecting what students are taught in school to real-world issues, problems, and applications...learning mirrors real-life contexts, equips them with practical and useful skills, and addresses topics that are relevant and applicable to their lives outside of school" -- EdGlossary

It includes the following key attributes:
"Having a real audience for work.
Contextualizing locally, but connecting globally.
Projects and problems are based on themes of social significance and personal interest"
 -- Schools We Need Project

I would extend these by saying that it is the kind of learning we would encourage others to pursue in a world without schools.  In many cases, the work would tie to industry, but it would also involve going deep into areas of passion that may not be especially useful in a profession.  Students should get out of the classroom to interact with people who are likely more passionate and experienced than the teacher in the given area.  The teacher's role in this environment is to act as the central hub of many relationships that students engage in, not the content expert in each area.

What are the specific elements that can make learning  “real world”?

The Real World Learning Network's five-finger model ( offers a helpful starting point of key elements: understanding, transferability, experience, empowerment, and values.

  • Understanding -- identify the concepts key to understanding a topic.
  • Transferability -- ensure that the topic fits in to many areas of life (this enables connections across the brain and increased relevance).
  • Experience -- touch, see, hear, smell, taste, and emotionally feel the situation (simply reading or watching is not enough).
  • Empowerment -- ability to take action around topic for positive change (understand problems AND work on solutions).
  • Values -- show empathy and care for other people, future generations, and the Earth.

What does the “real world” look like specifically (for Grand Challenge Design students in 2016)?

The Grand Challenges for Engineering include problems like "provide universal access to clean water", "advance health informatics", and "restore urban infrastructure".  They are the problems that a wide variety of future engineers will need to work on in order to maintain and improve life on Earth.  These real-world engineers come are all problem-solvers, but they may have received their training from very diverse fields.  The social sciences offer a power lens for understanding people and their experiences.  The physical sciences give humanity a deeper understanding of our environment and how it works.  Engineers are experienced in setting design constraints and designing and testing solutions to a problem.  Entrepreneurs use a value-centric mindset to identify which aspects of an idea matter to people and find a way to sustain the idea through production.

The "real world" in the GCD topic areas is not about a single field, but a team-based approach to try to better understand key challenges and work together to develop and spread effective solutions.  They need to develop the common skills that employers already value (percentages based on NACE):

  • Leadership 80.1% 
  • Ability to work in a team 78.9% 
  • Communication skills (written) 70.2% 
  • Problem-solving skills 70.2% 
  • Communication skills (verbal) 68.9% 
  • Strong work ethic 68.9%

What are the opportunities and challenges when providing K-12 students real world learning experiences?   Do all students benefit?

Often, real-world learning leaves students in charge of many aspects of their learning.  The problem with this?  A lack of control.  As a teacher, I like to keep 30 spreadsheets that track every micro-detail of where students are at in a known progression of learning, making it easier for me to provide feedback and next steps.  Planned progressions with detailed feedback have their place as efficient and effective ways to pick up a new set of wanted skills.  When I want to learn something new, nothing beats a well designed course, especially if it is self-paced and available from home on-demand.  However, in many cases students are not interested in the skills that school schedules drop in front of them, so they disengage.

When letting go, students have the opportunity to do fascinating things that you could have never planned for them, especially when they engage with expert mentors who offer guidance in their learning.  On the opposite end of the spectrum, students have more opportunities to get lost, give up, or coast without pushing themselves.  Unlike a good factory, there is a ton of variation and fewer tools to address the low-end of the achievement spectrum.

Another challenge in developing real-world learning experiences is the time required for teachers to set it up.  There is no textbook that you can purchase for meaningful opportunities in your subject area in your local community.  There is not a yellow pages for supportive mentors in every topic.  Teachers need to create and modify projects, meet many people in their community, and establish a lot of goodwill with others as they start asking for constant favors on behalf of their students.  I could not create the class I'm building right now without a minimum of 3-5 years of relationship-building in my district and community -- there are simply too many pieces that need to come together that rely on the incredible support of a village, not just a willing individual.

The upside, as a teacher, is that my work is incredibly fulfilling when I connect with experts in the community and form my own set of meaningful relationships.  At conferences, I get to talk to parents about the cool things their child is doing, not the deficiencies in their skills based on my last unit test.  I spend time with former students and local volunteers having fun while making new things on the weekend.  If I was spending all of my time focused on making sure students achieve in only a close-ended set of tasks without outside connections or creativity, I would fallen away from the profession in my first 5 years.  I've never been as excited to teach as I am this year, despite the very real possibility that it will be my hardest year as a teacher.

Monday, August 8, 2016

Reflecting on a summer of self-directed learning

The transition of Grand Challenge Design to a game/scenario based course was the focus on my "genius hour" project for my Winona State course on Mindset, Motivation, and Self-Directed Learning taught by @jenhegna.  The two parts of this class that I focused on were the book study and genius hour project.

As a class, we studied George Couros's book "The Innovator's Mindset" and discussed it on our Google+ site.  This laid a powerful foundation for thinking about a project that would impact student motivation and learning.  The reflections on problem-based learning, supporting risk taking, community-based design, and empowerment led me to change the course from "real world projects" to "simulated world".  The new design puts problems before technology, creates a safe space for failure, puts students fully in charge of many key aspects, but still invites the wider community to engage with the class and students.  I didn't fully realize all of this until reflecting in the post today, but there is a clear hand-off from the book study to the "why" behind the course redesign.

Six weeks ago, the game concept was rough, but the overall vision was in place.  Though the initial blog post wasn't perfect, it gave me a public medium for putting the concept in front of friends and colleagues with different areas of expertise.  I got over 20 responses that both encouraged me and pushed me to adapt the idea so that students would truly develop useful skills within the course (THANK YOU!!).

Four weeks ago, I wrote an update post that got more specific with the purpose of the change and what the student experience might look like.  I recently recorded this, along with a few updates, into my final course presentation.

Since then, details have been the name of the game.  I began compiling all of the rules and key details in a Google Doc.  I left it open for public commenting, and it will be a year-long work-in-progress, so please jump in.  The doc goes through the physical and digital game board, the objectives (individually maximize health, productivity, and happiness, and collectively maintain 90%+ citizens in good health), governance and law, how virtual citizens and their housing work, how the economy operates, transportation logistics, trade, goods, and electrical power.  It also details the application interface (API) so players know how to interact with pieces of the digital infrastructure.  The doc ends with goals for students and the course to make sure that future rules and adaptations stay focused on the point of the game.

The design and learning process that led to the development around each industry usually looked like this:
  1. research big problem area such as access to clean water
  2. find lots of news articles that highlight aspects of this problem locally or globally
  3. make up a scenario that sounded like some of the news articles but for the game
  4. imagine what in-game solutions might look like
  5. figure out what components would be needed on hand to deal with that problem
  6. search eBay for the cheapest solution possible
  7. perform zillions of Google searches to better understand the components so I can differentiate the $2.30 and $2.80 components, and realize that neither will do the job well, and then find a $1.50 component that works perfectly
  8. click buy, print receipt, put in pile
  9. repeat
More than anything else, the past few weeks is when I have been joined by two former students, Paul Klompenhower and Evan Richardson, who have been insanely helpful in turning the game from a rulebook to reality.  Evan was a programmer since his robotics days in high school, but thanks to more practice at school (MSOE) and his summer internship at Mayo, he has become an awesome web developer, especially with NodeJS.  He has been developing the digital game board, translating my API docs into something that actually works, and developing a way to observe all of the ditially-tracked game stats with a sane and logical (and awesome-looking) interface.  In the process, he is also teaching me how to do this so I can pick-up where he leaves off and continue to update the game software as bugs surface and new needs pop-up during the year.  This learning is especially important for me since I need to help students setup their own NodeJS-based servers!

Paul is a master of mechanical design and fabrication.  He was the one that first nudged me in late 2012 to start a robotics team in Byron.  Since then, he too has been developing his skills through his insane curiosity (with answer-side help from Google) and projects with friends.  He created the CAD model for the physical game table and has been helping me get all of the parts and processes together to have a working CNC (computer controlled) router in class (think laser cutter, but with a drill instead of a laser etching and cutting things out).  Besides making it possible for the class to make their own creations from CAD, this will also allow us to cut out the 632 hexagons that make-up the game board.  It will be *quite* the project to get all of this set-up and working, but will put the class in a great position to start building new things right away.

Moving forward, I am focused on getting the students and classroom ready.  I met with my first group of 7 students last week to present the idea, take questions, and solicit feedback, and will meet with most of the rest of the class tomorrow.  Students were a bit overwhelmed at first, but after giving them some time for the ideas to sink in, they started firing off all kinds of questions.  I am really excited to see what kinds of conflicts arise in the virtual world, as many of the students were thinking more about self-centered ends than world-improvements.  After reading John Hunter's book "World Peace and Other 4th Grade Accomplishments" about the way he manages his class during the World Peace Game (a class-wide game/simulation that he invented), I might be a bit heart-broken by this attitude, but know that it is necessary to let it play out if students are going to learn anything valuable.  Cooperation and competition will have a fascinating balance given the interdependencies between players and the lack of any forced structure.  Worst case, I will maintain some "vigil ante" power to knock out key infrastructure in the middle of the night from students who are not playing nice.  You know you're being too mean when a bat symbol lies across a series of snipped wires.

The classroom itself will also be quite the challenge, but will need to be a work in progress.  My initial vision was doing a bunch of electrical work, buying and building furniture, and painting before school starts.  With less funding, less of my time, and less time with the classroom emptied out than I anticipated, much of this will not be happening.  Instead, one parent has been finding tons of ideal furniture on Craigslist that we are buying for a steal, and painting projects will come from student proposals that I will approve during the year.  Though a mid-year painting project is a pain, it means that there will be time for students to really own the process.

We're only four weeks away from students coming everyday to class!  Back to work.

Friday, July 15, 2016

Ultimate Frisbee

The concept of Fantasy Football + computer simulation + statistical analysis + a game accessible to all students = the Ultimate Frisbee Draft.

Back in my student teaching year, I went a bit off the rails with project design in my Stats class.  After a rough first semester of Stats that turned around with a successful Minute to Win It project, I decided to tie every content area into some project.  My favorite, and the longest-lasting one over seven semesters, is the Ultimate Frisbee Draft.

We lead off the unit by watching one of my favorite movies, Moneyball, as a class.  Even students who have already seen it get a chance to watch it from the perspective of better understanding sports analytics while newbies get introduced to the power of data analysis in the sports world.  This intro event sets up a class discussion around which statistics matter and how someone could use those to predict overall value in any industry.

From there, I introduce students to our sport, Ultimate Frisbee.  I very intentionally didn't choose baseball, football, or another common sport -- these have tons of existing analytical approaches already covering the web that students could simply adopt without deep thought.  In addition, I have a number of students who are not motivated by sports.  Ultimate Frisbee, however, is one of those rare activities that nerds (my robotics students LOVE Ultimate), chatters, and intense athletes can all find really engaging.  To get an intuitive sense of the game, we spend a day outside or in the gym playing short games during class.  I use the time after games to reflect on the last match and encourage students to track specific stats of their own during the next game.

Finally, we get to the data.  I wrote a computer simulation that plays games of Ultimate between virtual players and tracks all throws, catches, and drops, per player, during the game.  After each virtual player plays 30 games (a number large enough for analysis but small enough for lots of noise to creep into the data), I dump the data of all 98 players into a giant spreadsheet.  From here, teams of students start to dig into the overwhelming pile of numbers to figure out who they want to draft for the ultimate Ultimate team.

Once given the hook and a chance to explore, I introduce the statistical tools that will become their friends.  Depending on when I teach it, I like to start with a search for the stats that actually matter, using a few rounds of multiple regression with the most important team stat, winning, against nearly everything else.  Since there are only 30 games, picking a player based on their number of wins has far too much noise to be reliable.  However, if you find that something like short catch percentage or the number of successful long throws are highly correlated across all players, you can use those much larger numbers and sort by categories with less noise.

We also talk about tools for sorting individuals.  You could use the raw stat, a rank order, a percentile, or a z-score.  Each has their own benefits.  When students want to combine two stats into one "power stat", I usually recommend the use of the z-score since it makes both values unitless while retaining a the magnitude relative to the mean.  Since there is no obvious right way to do all of this, it leads to fantastic student questions and discussions.

Some groups of students take my recommended approach of finding key stats, getting z-scores, adding the totals, and sorting.  Others modify this to break players into categories / positions, choosing a strong thrower or two with a supporting cast of receivers.  Others put their money on defense, looking for players who effectively deny the Frisbee on the logic that it ends drives quickly and gives them a short field for offense.

On the practice draft day, student teams are assigned a random draft order.  From there, they have 30 seconds to pick an unchosen player when they are up.  I only do 2-4 rounds in the practice draft and then assign the remainder of the slots on their teams with equivalent players.  Once teams are selected, I run their teams back through the original simulator to have them square off against one another.  The simulator will actually play through a full game, simulating one throw at a time based on probabilities from the throwing player, their defender, the match-ups of the receivers, the selected receiver to get the pass, and whether or not the pass was caught based on ability, defense, and other factors.  Fortunately, computers are crazy fast, so this takes fractions of a second.  The simulator, looking old-school fancy with its command-line spewing of text, prints out the records of each team as they play every other team 21 separate times, each up to 15 touchdowns.  It then enters into a "tournament mode" where teams go into a bracket (based on their "regular season" rank) and face off in a single match.  The 21 games per competitor series gives useful feedback on which teams are statistically solid, while the playoffs is a chance for even the worst of underdogs to eek out an occasional win.  This is analogous to many real sports, and it leads to more interesting discussions that a lesson on the "Law of Large Numbers" could ever do.

All of this is just a practice round -- I do this to help students become familiar with the format.  However, it also has the nice side effect of motivating teams with awful strategies and analysis to pull things together.  I have seen a number of teams make a strong turn-around based on lessons learned in the practice round.  When students share out their strategies, I have them focus on process, not their "secret sauce", since they tend to get pretty competitive.

By the final game day, teams are ready with spreadsheets, have a process to cross out the selected players as they get drafted, and get pretty audibly upset when someone takes their star player right in front of them.  Coming in much more organized, the draft usually takes only 5-10 minutes for all 7 rounds to complete.  After running through the regular season, I hand out awards/candy to the top teams, but then save the ultimate treat for the playoffs.  I click through just one game at a time here, encouraging teams to cheer and show their pride before revealing their fate.  In the end, we all have a lot of fun, including many students who usually don't get into sports or anything that smells like fantasy football.

Altogether, I spend 6-7 days in a block class with this unit.  It is long, but it is always memorable for students, creates fantastic student questions, and often inspires interesting projects when I provide an open-ended opportunity later in the course.

See videos and spreadsheet links for students to reference / download.

How to get up and running with the simulation:

  • Download and install Python 2.7 for Windows or Mac.  This is the environment that runs that code.  It is a simple "click next" type of installer.  You may want to reboot at the end if things are not running properly in later steps.
  • After installation, download this zip file and unzip it somewhere on your computer.
  • Double click "" to run the drafting program -- this is where you will setup a class, name the teams, and input the results of the 7-round draft.  Whatever name you give your "period" will be the same name you use later when running the simulator.
  • Double click "" to run the season + tournament simulator.  Whenever the screen pauses, just press enter to make the computer do the next thing.  This can be run as many times as you like using the same draft data.
  • Tweet at me, email me, or comment below if things don't work the way they should.  I would love to help you get this running!  Thanks to @stoodle for encouraging me to finally write this up after years of procrastinating!

Friday, July 8, 2016

GCD Update: Feedback and Research

Since my last post, I have been heavily researching and designing the new Grand Challenge Design course with the help of a huge support network. Here is my most succinct summary of the need and solution that make up the course:

Students are entering a world with variety of Grand Challenges, problems that are interdisciplinary, interconnected, and extremely complex. They span topic areas like the environment, healthcare, security, and urban infrastructure. A growing technology trend, developing "smart" devices that are being connected to the "internet of things", and then developing software platforms that make sense of all of the data that is generated, offers a powerful starting point for rethinking many of the most intractable problems. Grand Challenge Design was proposed as a course that gives students the tools and skills to create smart devices and exposure to the Grand Challenges where they can be applied for the greatest benefit to humanity.
To facilitate students immersing into the problem areas, the course will run as a year-long simulated world. Picture a 5'x13' wooden table with 4" squares marking out territories and waterways. Students will own territories and manage a growing society of virtual citizens. They will start with simple farms (fast-growing plants in solo cups), harvesting enough to feed their people and selling off the excess to grow their cash supplies. From there, they will advance to create powered city plots, running actual data lines from their Raspberry Pi and Arduino under the table to LEDs on their plots. As the societies advance, students will improve their farms with automated irrigation and moisture sensors. They will create factories with working actuators and engage in commerce with overseas markets to sell their products. While they build out these basic game elements, I will introduce new challenges such as polluted water supplies, disease outbreaks, unstable market prices, rising populations, and other twists that can only be addressed through effective student cooperation and the design of continuously smarter cities. As students all advance in skills, the challenges will become increasingly realistic with the help of community experts coming to class to pose the next round of problems and test out potential solutions. Even for simple things, like getting a loan to build an automated factory, I will require students to create a plan and pitch it to real investors before handing out a dime from my virtual bank.
Students will live through dozens of problems in healthcare, security, infrastructure, and the environment. They will experience building water treatment devices, factory assembly lines, disease tracking apps, automated irrigation and lighting systems for farms, and other smart systems that take real-world actions based on processed sensor data. They will interact with adults working on similar problems to see how their new skills can provide real value immediately, and through consulting challenges mid-game, actually build devices for real-world use. With the help of this course and the network of amazing people supporting it, we will build the next wave of Grand Challenge Designers.

The most exciting part of the design process thus far has been engaging with the feedback of those willing to reach out, particularly those who were critical of various concepts proposed in the last post. Discussing the challenges and refining the ideas is an ongoing process. This process ramps up next week with a few more in-person planning sessions. Things will start to get finalized in early August when I have a couple design sessions with students enrolled in the course. More than anyone else, I want students to be on-board with the plan before it gets etched in stone.

I also wanted to document some of the research I have been pouring through recently. Pouring through my search history over the past week reveals two major themes: world problems and sensors.

While researching the problem areas, I was honestly concerned I would get added to the FBI's watchlist: what teacher looks up details around urban infrastructure vulnerabilities? I read about bridge collapses, problems in the aging electrical distribution system, threats from terrorism, threats from hackers, water treatment systems, and other similarly riveting reads. I broadened the problem space by looking at the 14 Engineering Grand Challenges (where the course got its name) and past topics from the Future Problem Solving Program (topics ranging from "nutrition" to "space junk" to "virtual corporations"). I also looked at related solution spaces such as the smart grid, targeted irrigation (and how to build an in-home version), and simulating disease vectors through the cooperative game "Pandemic". As I tried to imagine what it would look like in a game setting, I watched an hour of Civilization 5 game play on YouTube.

With the sensors, I spend most of my time with my jaw on the floor at the insanely low cost of anything made and shipped from China via eBay. Arduinos are known for being that dirt cheap electronics prototyping platform at under $30/board. Given the open-source hardware design, they are also open to knock-offs. Somehow, it is possible to make them, list them on eBay, and ship them to the US from Hong Kong for $3.60/board. WHAT?! Given that it takes over a month for them to get here, I went on a shopping spree for these, moisture and current sensors, water pumps, relays, and other key components for our projects at insanely low prices. I found sample code with a number of the sensors that interfaces simply with the Arduino. Based on this research, most of our devices will be a two-step solution connecting a Raspberry Pi 3 to an Arduino to the sensor / actuator. This allows us to have a full computer gathering data and running multiple processes while also having a dedicated devices running simple, continuous loops with existing software.

There are a lot of logistics to work through as we turn this concept into a working, playable simulation with its huge physical footprint and its digital infrastructure. Even if it is rocky, the learning that I have along the way will better prepare me with the tech and problem area knowledge that will be essential to be an awesome facilitator of this course. And it will make for a fun summer!