Select a region
The last thing you will probably be thinking when you are a grad student is that one of your teachers is a robot.
But what if we told you that students enrolled in a masters level course at Georgia Tech’s School of Interactive Computing were fooled into thinking one of their teaching assistants was a human – but actually wasn’t?
Before you begin to to ask: “How could a bunch of computer science students be duped so spectacularly?”, there are two things you need to know.
First, this is an online course and all interactions with virtual TA Jill Watson was through an online student forum and, second, Jill was able to answer many of their questions with 97 percent certainty.
So why would anyone think otherwise?
It was only after a few weeks later after Jill started posting answers on the forum, students were told they were basically interacting with a bot. According to a Georgia Tech release, “the student response was uniformly positive”.
Apparently one student said “her mind was blown” when she learned of the truth while another jokingly asked if Jill could “come out and play”.
There was one particular student who had suspicions as to whether Jill was a bot and posted his thoughts on Piazza – a student Q&A platform.
“We were taking an AI course, so I had to imagine that it was possible there might be an AI lurking around,” said Tyson Bailey.
“Then again, I asked Dr Goel (the course leader) if he was a computer in one of my first email interactions with him. I think it’s a great idea and hope that they continue to improve it.”
Given the high number of students taking the Knowledge Based Artificial Intelligence (KBAI) course – a core requirement of the programme – professor Ashok Goel thought getting Jill to help out would be a good idea.
Around 300 students take the course each year and they post close to 10,000 messages in the online forum – a number too large for the course’s eight (human) TAs to handle.
To cope with the workload, Goel and his graduate students created a virtual teaching assistant, which was built on IBM’s Watson platform. Yes, that’s the very same Watson who defeated the world’s greatest Jeopardy players four years ago.
Goel and his team gathered around 40,000 questions that had been asked in the class forums since the course was launched two years ago. They then programmed Jill to answer these questions.
“One of the secrets of online classes is that the number of questions increases if you have more students, but the number of different questions doesn’t really go up,” said Goel. “Students tend to ask the same questions over and over again.”
Jill wasn’t very good at answering the questions during the testing stages and would often respond to questions with irrelevant and strange answers.
“Initially her answers weren’t good enough because she would get stuck on keywords,” said Lalith Polepeddi, one of the graduate students working on the project.
“For example, a student asked about organising a meet-up to go over video lessons with others, and Jill gave an answer referencing a textbook that could supplement the video lessons – same keywords – but different context. So we learned from mistakes like this one, and gradually made Jill smarter.”
But Jill got pretty good with her responses and the researchers started posting her answers to the KBAI student forum.
Sure, it can be pretty depressing to find out that robots can replace humans in yet another job, but Goel thinks there are benefits to having an AI teaching assistant helping the students.
“One of the main reasons many students drop out is because they don’t receive enough teaching support,” said Goel. “We created Jill as a way to provide faster answers and feedback.”
Goel would like to replicate the project again next semester but with a different name. “The goal is to have the virtual teaching assistant answer 40 percent of all questions by the end of year,” the university’s website says.
