Teaching at Scale with Clickers

Computer science is wildly popular at Universities right now, owing in no small part to the robust job market for CS graduates. This market is driven by the voracious appetite of businesses and the public for “tech.” A consequence of increasing CS popularity is the substantial growth of CS course sizes.  Such growth presents a significant challenge to instructors of those courses, as the number of CS instructional staff have generally not scaled linearly with the number of students. How can we teach many more students with roughly the same number of instructors but without a (substantial) reduction in overall quality? One part of my answer to this question is: “clicker” quizzes.

I recently blogged about CMSC 330, the undergraduate programming languages course I (co-)teach at the University of Maryland. That post focused on the content of the course. This post considers the more general question of how to deliver a computer science course at scale, focusing on the lecture component and a key piece of my approach: the use of clicker quizzes during class.  I’ll consider other aspects of large course management — student Q&A and grading — in a future post.

CS in College: The Landscape

Interest and enrollment in computer science are surging. The Computing Research Association (CRA) has documented the situation quite well in a recent report. UMD CS’s enrollment has tripled in the last 8 years, from 1100+ majors in 2010 to 3600+ in 2017. 1 This makes it one of the largest programs in the US.

UMD Computer Science Major Trends

In such a large department, delivering a course required of every CS major presents a challenge of scale. During Spring’18, CMSC 330 had 450+ students in four separate lectures delivered by Anwar Mamat (200ish students), Niki Vazou and Tom Gilray (100ish students), and myself (140ish students). Supporting us were 31 teaching assistants, all but two of whom were undergraduates who had previously excelled in the course. These TAs taught discussion sections, helped put together and grade projects and exams, held office hours, and answered on-line questions. There is no textbook for 330, so the lecture notes serve as the de-facto text, supplemented with some on-line materials.

Our efforts aimed to address scale can be broken down into those that deal with lecturestudent-staff Q&A, project and exam grading. In this post I’ll focus on lectures, and in a later one I’ll consider the latter two.

Dealing with Large Lectures

Large lectures (probably anything with more than 40 or so students) can be daunting. Being one uncertain face in a large crowd can discourage a student from asking a question, but questions not asked and answered can leave that student feeling lost and discouraged.

Even when one can follow a lecture’s content in the best of circumstances, it can be hard to just listen and take notes for 75 minutes. I used to make lectures interactive by posing questions to the room, to get students thinking actively about the technical material being presented. But in a large class I find that it is always the same small group of students who respond. Nevertheless I persisted, hoping that if students don’t answer my questions directly they are at least working out the answers in their heads. But colleagues have said to me that studies find this often doesn’t happen. Many students are happy to stay passive and not mentally engage.

Flipped Classrooms?

One frequently proposed remedy to the problem that lectures are ineffective is to employ active learning. This means getting students actively engaged in the learning process, rather than just passively listening. One approach is to have students break into groups to work out problems during lecture. Taking this idea to the extreme leads to flipped classrooms, a teaching style in which students read or watch videos in advance of class, and then work on problems during class.

I have not tried a flipped classroom, but colleagues who have report good experiences. Getting students to do the prep work in advance can be a challenge, so some colleagues incentivize it by having a quiz at the start of every class. Another challenge is getting all students to participate equally; those more passively inclined might be fine to let the others in their group do the work. This can also probably be addressed with the right incentives.

Rather than go the route of a flipped classroom, I have adopted a different style of active learning: in-class clicker quizzes.

Graded, In-class “Clicker” Quizzes

My lecture in 330 is structured around a “traditional” slide presentation in Keynote or Powerpoint. I try very hard to make the slides well-formatted, pedagogically sound, and self-contained, with all of the key ideas present on the slides. That way, students can come back to them later for easy reference and refreshing. This is in contrast to the “Presentation Zen” style in which slides have lots of pictures, requiring the instructor soundtrack to make sense. I record my lectures using Panopto, which is integrated with UMD’s course management system. Panopto is really easy to use and syncs slides and audio, indexing the recording by slide, making it easy to locate a particular topic later.

To make the lecture more “active,” I intersperse multiple-choice quiz questions throughout the lecture, typically around five per 75-minute lecture, each placed after a relevant sub-topic. When we get to one of these questions, I tell the students to come up with what they think is the right answer. They are free (indeed, encouraged) to confer with their neighbors. Once they’ve done so, they each individually submit an answer using an App on their phone, or a physical device called a clicker. Software running on my laptop receives each answer and tallies the results. After a minute or so I close the quiz, with the result that a histogram is displayed on-screen. Then I display the correct answer to the quiz, and everyone can compare it to their own answer, and the histogram. A good example of the spacing and style of clicker questions I use can be found in this lecture on finite automata and regular expressions (but see the other examples, too, in our lecture schedule).

The clicker quiz results of the day are associated with the student’s ID and they are graded for correctness. Ultimately, 5% of a student’s final grade owes to clicker quiz outcomes. To take the pressure of performance off and to allow for questions that I expect students should get wrong (for pedagogical reasons), I drop the lower 20% of quiz outcomes. As such, if we had 100 clicker questions throughout the semester, a student could get 20 of them wrong and still receive full credit for the 5% clicker portion of their final grade. (Last semester, the average student got about 4.25 of the 5 percentage points on offer for clickers.)

picture of turning point clickers

Turning Point Clickers

At UMD we use Turning Point Clicker software. UMD pays for each student’s license to use it, so it’s no extra cost to them. Turning has a way to embed clicker quizzes inside of Powerpoint slides, but I don’t use it. Rather, I start “on demand” polls using the Turning Point dashboard during class when we reach a quiz, which is formatted just as usual Powerpoint. Doing this avoids needing to install a Powerpoint extension and avoids the negative impact on old slides of Powerpoint software upgrades.


I have found that clicker quizzes have several benefits. Most importantly, using them gets all students actively engaged, trying to apply the concepts they have just been taught. Everyone has their own clicker and can anonymously submit an answer, so there is no social pressure against participation.

Unlike flipped classrooms, there is no risk of a lack of preparedness getting in the way of participation, nor is there a problem that shy students let more outgoing ones in a small group do the work. It’s also easy to adapt a traditional lecture-based class to use clickers, whereas a flipped classroom is a significant investment.

Like flipped classrooms, clicker quizzes break up the monotony of long periods of just me talking, creating a mental break before moving on. They are also social events. While people are working out their answer there is a buzz in the class that is really exciting. The benefits of the group-based active approach come out here, as those not sure of the answer first try on their own, but can get help from a neighbor if they are stuck. It’s a lot of fun when the histogram pops up and there in audible “ahh!” or “aww!” as a result. Several students commented in their teaching evaluations this year that they really enjoyed the social aspect of clicker quiz answering.

To get these benefits, I have found that it is important to incentivize participation by grading clicker quiz results. If they are not graded, students will not come to class, and even if they do, they will not participate. This is what a colleague of mine found out when making clicker quiz questions ungraded in 330 last Fall. Grading for correctness, and not just “participation,” is also important, or else students will not be incentivized to actually try to come up with the right answer. They’ll just click “A” every time. My colleague saw this behavior too. Dropping the lower 20% of scores reduces the impact of failure on the student’s grade.


Large classes in computer science present a significant challenge to the goal of high-quality education. Active learning techniques are helpful overall, but are challenging in large classes, and can require a lot of work to get right. I have found that in-class clicker quizzes strike a nice balance. They bring the benefits of active learning without the large investment of flipping a classroom, and without some other downsides too. I have used clickers for two semesters and I find my lectures now are more fun and engaging, even with large classes (or perhaps because of them), than they were before. Indeed, my teaching evaluation scores are the highest they have ever been (and they were good before).

Of course, other elements of a course beyond the lecture are challenged by large enrollments. I will discuss these  — notably student Q&A and grading — in a future post.


  1. I obtained these statistics from UMD’s public reports site.

The post Teaching at Scale with Clickers appeared first on The Programming Languages Enthusiast.

find the cost of your paper

Sep 13, Grand Remembrances

Today is Grandparents Day in the United States. Being a Grand is a special honor. I feel very blessed that my wife and I have two grandchildren. We were able to visit them today. Yes, we are still being cautious with the coronavirus, but we also find it very difficult to not see them when they live so close. So today we did drop by to visit Jacob (age 10) and Sophia (age 7) along with their parents. We brought donuts and caught up with them. Our grandchildren are still pretty young and this is a precious time in their lives – and ours!

I wish I had known my grandparents better. We never lived in the same place. Dad was a career Air Force pilot, so we moved around a lot. But we did get to see them once in a while when they would visit us, or we them.

A Plague of Giants

There are five known magical ‘kennings’ or types: air, water, fire, earth, and plants. Each nation specializes in of these kennings, and the magic influences the society. There’s a big pitfall with this diversity of ability and locale–not everyone gets along.

Enter the Hathrim giants, or ‘lavaborn’ whose kenning is fire. Where they live the trees that fuel their fire are long gone, but the giants are definitely not welcome anywhere else. They’re big, they’re violent, and they’re ruthless. When a volcano erupts and they are forced to evacuate, they take the opportunity to relocate. They don’t care that it’s in a place where they aren’t wanted.

I first read Kevin Hearne’s Iron Druid books and loved them (also the quirky The Tales of Pell), so was curious about this new venture, starting with A PLAGUE OF GIANTS. Think Avatar: The Last Airbender meets Jim Butcher’s Codex Alera series. Elemental magic, a variety of races, different lands. And it’s all thrown at you from page one.

But this story is told a little differently. It starts at the end of the war, after a difficult victory, and a bard with earth kenning uses his magic to re-tell the story of the war to a city of refugees. And it’s this movement back and forth in time and between key players in this war that we get a singularly grand view of the war as a whole. Hearne uses this method to great effect.

There are so many interesting characters in this book that I can’t cover them all here. Often in books like this such a large cast of ‘main’ character can make the storytelling suffer, especially since they don’t have a lot of interaction with each other for the first 3/4 of the book–but it doesn’t suffer, thankfully. And the characterization is good enough, despite these short bursts, that by the end we understand these people and care about what happens to them.

If there were a main character it would be Dervan, a historian who is assigned to record (also spy on?) the bard’s stories. He finds himself caught up in machinations he feels unfit to survive. Fintan is the bard from another country, who at first is rather mysterious and his true personality is hidden by the stories he tells; it takes a while to understand him. Gorin Mogen is the leader of the Hathrim giants who decide to find a new land to settle. He’s hard to like, but as far as villains go, you understand his motivations and he can be even a little convincing. There’s Abhi, the son of hunters, who decides hunting isn’t the life for him–and unexpectedly finds himself on a quest for the sixth kenning. And Gondel Vedd, a scholar of linguistics who finds himself tasked with finding a way to communicate with a race of giants never seen before (definitely not Hathrim) and stumbles onto a mystery no one could have guessed: there may be a seventh kenning.

There are other characters, but what makes them all interesting is that they’re regular people (well, maybe not Gorin Mogen or the viceroy–he’s a piece of work) who become heroes in their own little ways, whether it’s the teenage girl who isn’t afraid to share vital information, to the scholars who suddenly find how crucial their minds are to the survival of a nation, to the humble public servants who find bravery when they need it most. This is a story of loss, love, redemption, courage, unity, and overcoming despair to not give up. All very human experiences by simple people who do extraordinary things.

Hearne’s worldbuilding is engaging. He doesn’t bottle feed you, at first it feels like drinking from a hydrant, but then you settle in and pick up things along the way. Then he shows you stuff with a punch to the gut. This is no fluffy world with simple magic without price. All the magic has a price, and more often than not it leads you straight to death’s door. For most people just the seeking of the magic will kill you. I particularly enjoyed the scenes with Ahbi and his discovery of the sixth kenning and everything associated with it. But giants? I mean, really? It isn’t bad enough fighting people who can control fire that you have to add that they’re twice the size of normal people? For Hearne if it’s war, the stakes are pretty high, and it gets ugly.

The benefit of the storytelling style is that the book, despite its length, moves along steadily (Hearne is no novice, here). The bits of story lead you along without annoying cliffhangers (mostly), and I never got bored with the switch between characters. It was easy to move between them, and they were recognizable enough that I got lost or confused. The end of the novel felt a little abrupt, but I guess that has more to do with I was ready for the story to continue, despite the exiting climax.

If you’re looking for epic fantasy with fun storytelling and clever worldbuilding, check out A PLAGUE OF GIANTS.

The post A Plague of Giants appeared first on Elitist Book Reviews.

The Artwork Of Gary Choo

Gary Choo is a concept artist/illustrator based in Singapore. I’ve know Gary for a good many years ( 17, actually ), working together in animation studios in Singapore like Silicon Illusions and Lucasfilm. Gary currently runs an art team at Mighty Bear Games, but when time allows he also draws covers for Marvel comics, and they’re amazing –

The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo

To see more of Gary’s work or to engage him for freelance work, head down to his ArtStation.

The post The Art Of Gary Choo appeared first on Halcyon Realms – Art Book Reviews – Anime, Manga, Film, Photography.