Forum: support

RE: Experiences with R/exams for distance learning [ Reply ] By: Achim Zeileis on 2020-09-22 15:53	[forum:48159]
Thanks again to all of you who contributed to this thread! The resulting blog post is now online: http://www.R-exams.org/general/distancelearning/

RE: Experiences with R/exams for distance learning [ Reply ]
By: Achim Zeileis on 2020-09-09 01:12

[forum:48129]

Ulrike, thanks for this! I just realized that I never replied to this. I agree with your assessments.

An additional concern that was a deal breaker for me was the following: If one of the students has problems with the camera during the supervision (picture freezes, browser fails, internet connection problems, etc.), then you would have to be rather strict about, especially if you have a large number of participants. Thus, this would count as a discontinuation of the exam.

Hence, I decided to give the students the benefit of the doubt. I'm certain that we had far more students who were grateful for a fair shot at the exam and took it seriously - than we had students that cheated systematically. Of course, some of the statistics strongly suggest that there are individual students that almost certainly did not write their own exam but these were few exceptions (as far as I can tell).

So overall I'm happy with the approach we took and think it was a pragmatic solution. I hope, though, that we can return to pen-and-paper exams in the not-so-distant future.

RE: Experiences with R/exams for distance learning [ Reply ]
By: Ulrike Grömping on 2020-09-02 07:37

[forum:48091]

Achim,

I agree with you that it is probably infeasible to do a synchronous online exam with a video conference in large groups. The largest group I did it with hat 27 exam participants. I used Zoom (with students' consent), and I did identity checking in breakout rooms (the university rules see privacy problems doing it in the big room), and the identity check took almost an hour for the 27 people. I think that this would only work with a group of supervisors in larger groups (not sure whether/how the systems handle several moderators who walk the breakout rooms simultaneously).

I also did the exam itself in the breakout rooms, in order to minimize the effect of technical problems of individual students on others, and in order to be able to attend to individual questions without causing disruptions for other students. In fact, there were very few technical problems, but there seems to be always something. I did not do supervision in terms of sitting at the machine and watching everybody at all times. One severe disadvantage of the breakout rooms: if one wants to convey a message to everybody, like extending exam time, it is of course possible to send a message to the breakout rooms, but most students who work on the exam won't take notice. It luckily happened to me in a setting with only six students, and there I had to walk through the breakout rooms and tell everybody about the extension.

Many colleagues are very worried about cheating in online exams. Of course, there are increased possibilities for communication between students, but the time constraint is a limitation here for most students (and most students make honest attempts and are worried about technical difficulties but not about how to cheat). If students have individual exams - I did it with randomized versions of the same data set - even interpretation questions will be different enough that one has a good chance to find out if a student has copied an answer from someone else (I found a group of three students who worked together). After the first experience, I added the requirement that the student ID must be visible on every photo that is uploaded (students did not need to upload photos, but were free to decide they preferred photos over filling in the form).

Best, Ulrike

RE: Experiences with R/exams for distance learning [ Reply ] By: Achim Zeileis on 2020-09-02 00:44	[forum:48089]
P.S.: Thanks also to everyone else on this thread, this is really nice. Many things keep coming up in several setups but also many nice individual variations. Cool!

RE: Experiences with R/exams for distance learning [ Reply ]
By: Achim Zeileis on 2020-09-02 00:40

[forum:48088]

Ulrike, thanks for sharing this, very much appreciated!

Can you say something more about the video supervision? This is a question that keeps coming up these days. Hence, I would be interested in which tool you used, how many students and supervisors you had, and how you think it went overall.

My personal take is that online proctoring via video conference (Zoom, Big Blue Button, etc.) is ok for a small number of participants but not feasible for larger cohorts. The risk of technical problems or participants struggling with the technology increases too quickly. At least in our setup. hence, I didn't do it except for very small groups or 1:1 oral exams.

RE: Experiences with R/exams for distance learning [ Reply ]
By: Francisco Goerlich on 2020-09-01 19:14

[forum:48082]

Hi,
I use R/exams for Introductory Econometrics and Cross Section Forecasting courses. I mantain about 300 questions, most of them schoice and simple numerical exercises.

Even my University has Moodle as LMS (and I´m thinking in moving into it), I have been using this through Google forms. I collect responses there, and mark them automathically with an R script. Every student gets its own different test.

I´m also looking for tools to evaluate R code send by students.

Great pkg!

RE: Experiences with R/exams for distance learning [ Reply ]
By: John Tipton on 2020-09-01 15:06

[forum:48078]

I've been using R/exams for an R-based intro statistics course and I am also using R/exams for a new Unix/python/R programming course.

I've used pdf-based study guides using randomized questions and automatically generated solutions and assigning weekly quizzes based on these problem sets. This has worked well as the students can stress-test the questions during the problem set period and quizzes and because the questions are stochastic, it is easy to re-use them on exams.

The transition to distance learning has been more challenging but the exams package makes the transition to the online learning system much easier.

Thanks for the great package.

John

RE: Experiences with R/exams for distance learning [ Reply ] By: Ulrike Grömping on 2020-09-01 12:05	[forum:48076]
Of course cloze (not clone) exercises ... writing on the smartphone can be a pain.... Best, Ulrike

RE: Experiences with R/exams for distance learning [ Reply ] By: Rebecca Killick on 2020-09-01 12:00	[forum:48075]
Maths & Stats at Lancaster University use RExams for creating all our Moodle quizzes. We moved to using this exclusively for our undergraduate quizzes in the summer of 2019 and this served us well when COVID-19 hit. We are now expanding our online assessment using RExams. We mainly use cloze questions with randomization.

RE: Experiences with R/exams for distance learning [ Reply ]
By: Ulrike Grömping on 2020-09-01 11:45

[forum:48074]

Hi R/exams community,

I have used exams with Moodle for Bonus point exercises for quite a while (at home, mostly open for a week, best of three different attempts counts, automatic correction).

With Corona, I have also run online exams via Moodle in real time, supervised in a video conference; for these, I have allowed restarting the same attempt infinitely often, counting the last attempt. And I have not used auto correction, but have asked questions that also require free text answers from students (text fields with arbitrary correct answer of (e.g.) 72 characters in a clone exercise). Of course, the solutions support my manual correction process.

Achim and team, thank you very much for providing R/exams!

Best, Ulrike

RE: Experiences with R/exams for distance learning [ Reply ]
By: Niels Smits on 2020-09-01 11:12

[forum:48073]

Hi,

At the university of Amsterdam (dept. of Child Development and Education) we have been using R/exams in our statistics courses for a few years. Previously we used it for creating both formative (online at home) and summative (at university on a computer) testing.

But now, due to having to administer exams online (and therefore at home for students) due to corona, we needed extra randomization (both different draws from the same exercise and random order of questions) to frustrate cheating. We created tests for two statistics courses with a total of about 270 students. The tests were created using R/exams and administered in both Canvas and an online testing program (Testvision). For creating the tests, the functions exams2qti21 and exams2blackboard were used. For pretesting both exams2pdf and exams2html were used.

R/exams allows us create dynamic content, and to convert exams to many formats by which the durability of exercises is very large.

RE: Experiences with R/exams for distance learning [ Reply ] By: lisa lechner on 2020-09-01 09:28	[forum:48072]
R-exams was a life-saver during the difficult semester we faced. I used it for the statistics lecture with approx. 300 students (final online exam, but also arsnova questions) and for a smaller course on empirical methods in political science. But even before the pandemic, r-exams increased the fairness of my final exams and reduced the time spent on evaluating results. #bestRpackage2020

RE: Experiences with R/exams for distance learning [ Reply ] By: Achim Zeileis on 2020-08-10 14:08	[forum:48040]
Debbie, thanks for the feedback, that sounds like a cool application of R/exams. I look forward to hearing about your experiences when you go live!

RE: Experiences with R/exams for distance learning [ Reply ]
By: Debbie Yuster on 2020-08-10 14:04

[forum:48039]

Looking forward to getting some ideas from the upcoming blog post. I plan to use R/exams to maintain source files for "video quizzes" students will take after they watch lecture videos asynchronously in my Introduction to Data Science course. Instructors at different universities will be using these videos (created by the curriculum author), and I will distribute the quizzes to interested faculty,
each of whom may have a different preferred delivery mode. I'll be giving the quizzes through my LMS (Canvas), while others may administer the quizzes in learnr, or use a different LMS. With R/exams, I can maintain a single source, and instructors can convert the quizzes into their preferred format.

Experiences with R/exams for distance learning [ Reply ]
By: Achim Zeileis on 2020-08-10 13:54

[forum:48038]

Dear R/exams users,

after a semester of distance learning is before the next semester of distance learning. Hence, I wanted to put together a blog post that shows the diversity of different use-cases R/exams has for all of you.

So if you could reply here (or on Twitter) with a short info for which type of courses you used R/exams (or plan to use) and what kind of exercises and learning management system. That would be great. There are already a couple of replies on Twitter you can look at:

https://twitter.com/AchimZeileis/status/1292774253516001280

Thanks in advance!