Online robotics technology course design by balancing workload and affect
Abstract
This paper describes our course design approach that successfully transformed an undergraduate Robotics Technology course from in-person teaching to online guided by the TASKS model. Our course redesign process includes identifying conflicts, generating solutions, self-evaluation, and analyses of design solutions. We carefully balanced between Workload and students’ Affect, by designing comparable workloads as before, as well as maintaining students’ motivation similarly at the in-person level. Transforming this course consisting of lecture sessions and lab sessions with hardware and software elements yielded a set of course activities and teaching practices applicable to online teaching of other courses. When physical robots become unavailable, simulation projects were designed as alternatives and simulated versions of those physical projects used in face-to-face classroom. These simulation projects are in the areas of autonomous mobile robots, robotic manipulator, and advanced robotic control on MATLAB-ROS, respectively. Comparisons with past in-person results confirm that effective learning has been achieved remotely, having maintained student’s performance and motivation.
1Introduction
In response to the demand of transitioning courses from in-person teaching to online since spring 2020, educators in higher education acted promptly to start delivering lessons virtually. While online teaching has lasted several semesters, some educators still feel that they need a systematic course redesign approach or example to guide them through the process.
The context of this study is an undergraduate robotic course, CET 4952: Robotics Technology, offered as a technical elective to senior students in the department of Computer Engineering Technology (CET) at CUNY-New York City College of Technology. The course is structured to have a 3-hour lecture session and a 3-hour lab session each week. It is offered twice a year in both fall and spring semesters. Its course learning objective is to address fundamental subjects in both Autonomous Mobile Robot and Robotic Manipulator, as well as preparing students with necessary knowledge and skills in robotic programming, design, and system integration. Before the COVID-19 pandemic, the course was offered in-person. Lab sessions used physical experiments on VEX robots to reinforce knowledge discussed in lectures. This course had gone through several rounds of design and refinement when it was taught in-person (six semesters from spring 2017 to fall 2019). Students’ registration was usually good (21 or 22 out of the cap 22). Students teaching evaluation (STE) had been positive (an average of 4.7 out of 5.0) (Ma, 2021).
The COVID-19 pandemic has had enormous consequences for educational institutions. It raises safety concerns in laboratory facilities, where students typically work collaboratively with each other for experimental trials and data collection. Various robotics-related activities, such as summer camps, workshops (Gennert et al., 2020), robotic competitions (Gibeault et al., 2019), senior design projects, and undergraduate research (Maxwell & Meeden, 2000), were either suspended, reduced, or offered temporarily virtually.
Many institutions opt to deliver virtual distance education (online teaching, e-learning, remote/virtual learning), requiring professors to migrate their teaching environments to virtual meetings. The research question addressed in this paper is how to design and organize learning activities to sustain student learning performance under e-learning, a challenge faced by almost all instructors in higher education recently. In the light of our past teaching experiences (which were positive), our approach of transforming this robotic course online is to replicate past in-person teaching when possible, and if not, design course activities that are similar to their in-person counterparts.
Various frameworks and models were proposed in the literature to provide theoretical guideline on course design, including a student-choice model that allows students to choose how to engage in a course, i.e., in-person or online (McDowell, et al., 2019); a flipped classroom model that aims at nurturing active learning (Das et al., 2019); a model applying research-based learning principles (Prior Knowledge, Knowledge Organization, Mastery, Motivation, and Communication Principles) (Pantazidou & Kandris, 2020); the “Presence + Experience” framework for emphasizing social, cognitive, and teaching presences (Dunlap et al., 2016); the KMAP-O model used in health education that suggests behavioural interventions influence the outcome and performance (Wan et al., 2017; Wan et al., 2017; Wan, 2021); and the TASKS model that models the designer’s creativity (performance) as a combined result of Workload (Task to be conducted), Affect (emotional aspects such as motivation, stimulus, attitude), Skill, and Knowledge (Nguyen & Zeng, 2012; Yang et al., 2021).
Among these models, the TASKS model, which specifically states students’ Affect is one factor to consider in the design process, suits our design needs, since keeping students motivated and engaged for online learning becomes crucial during the pandemic. The TASKS model interprets course design from a point of view of implementation science as a type of design aiming to obtain desired behaviour change in the students (i.e., achieving better learning performance) (Soicher et al., 2020). Guided by the TASKS model, our course redesign carefully balanced between Workload (T) and students’ Affect (A) to yield reasonable and manageable workloads, keep students engaged and motivated in the course, and avoid causing extra stress on students.
Our course design objective is to achieve comparable student learning performance as those obtained in the traditional face-to-face setting, which is set as the benchmark performance for online teaching. Based on past experience, our approach is to create course activities that have similar workload as its in-person counterpart, as well as maintain student motivation and engagement similarly at the in-person level. In other words, we aim at creating an effective learning environment under e-learning that resembles its in-person counterpart as closely as possible.
Our approach is similar to several works in the literature in the sense of trying got maintain the pre-pandemic normalcy through similar pre-pandemic learning activities (Gillis & Krull, 2020; Akintewe, 2021). If lessons were previously delivered in an instructor-centred manner, lessons are still conducted this way except virtually (Wong, 2021). If lessons used to be given in a flipped classroom format, students are still encouraged to complete pre-class work before their subsequent meetings (previously in-person now online) (Garcia-Vedrenne et al., 2020; Hew, Jia, Gonda, & Bai, 2020). Other approaches to transform robotics courses online include using asynchronous teaching mode (Birk et al., 2021) or adopting the Fully Online Learning Community model that accentuate transformative learning, openness, and collaboration (Havenga, 2020). Evaluation of these different strategies using students’ grades and surveys show some evidence of benefits. Since each method has its own pros and cons, the key is to use the one that suits the course and students well, and help students achieve satisfactory learning performance.
For robotic courses involving physical experiments, educators have chosen to deliver kits and devices to students’ homes for at-home experiments (Benitez et al., 2020; Wu, et al., 2020; Sawatzki & Muraleedharan, 2021), demonstrate experiments lively on zoom or via recording, or use simulation projects (Havenga, 2020; Dey & S, 2020; Ruano et al., 2020). Delivering hardware kits to students’ homes can be suitable for lower-level courses, since these kits are usually affordable; the experiments are typically not challenging; and resultantly, the educators can still be helpful even remotely. While the demonstration method has the advantage of keeping hardware in the course, students, who only observe, may not feel as accomplished as actually doing the projects themselves. Using simulation projects instead allows all students to actively participate in the course, at the cost of temporarily removing hardware activities. We decided to develop computer-based simulations to provide students with hands-on practices. This is based on available resources, projects’ difficulties, and the percentage of hardware components in this course. Our approach of designing “almost-equivalent” simulation projects provides flexibility to switch between teaching modes promptly.
The scientific contribution of this work lies in the successful adoption of the TASKS philosophy into the course redesign process that transformed an undergraduate Robotics Technology course from in-person teaching to online in fall 2020, having achieved satisfactory student performance as compared with past in-person offerings. The action we took is to balance between Workload and Affect, by creating similar workload as in-person, maintaining similar student motivation, and by ensuring that all designed activities, along with how they are implemented, are reasonable and manageable for the students without causing extra anxiety. The complete course redesign process is described, including identifying conflicts, generating actions, self-evaluation, and analyses of the design solutions. So far, among the published work on transforming conventional courses to online in unpredictable times, very few have conducted evaluation of the efficacy of the transformed courses relative to their conventional counterparts (Hew et al., 2020). Ours is among the earlier work to report comparisons with past in-person offerings.
This study also reports a set of course activities and teaching practices that can be applied to online teaching of lecture-only courses, software-only courses, as well as courses that substitute hardware elements with software exercises (Boté-Vericad, 2021; Bote-Vericad, 2021; Tang et al., 2021). For instructors particularly teaching robotics relevant courses, the design solutions, or part of it, can be readily brought into their own class. Though higher education is bringing back more in-person instructions, online teaching may likely stay in certain ways due to the time and location flexibility it offers (Dhawan, 2020). The generated course activities will have an impact on teaching in other format(s) as well, e.g., some course activities can be used in face-to-face classrooms as supplements or alternatives.
Section 2 reviews the TASKS model and describes its adoption into our course design scenario. The course redesign process is then described, including identifying conflicts (in Sec. 2), generating itemized solutions (Secs. 3 and 4), performing self-evaluation (Sec. 5), and analysing design solutions (Sec. 6). Section 7 concludes the paper.
2Application of TASKS model in course design
The redesign of the Robotics Technology course was prompted by transforming the in-person course to an online version, guided by the TASKS philosophy. Firstly, the Yerkes-Dodson law characterizes the relationship between mental stress and performance (or creativity) by an inverse U-shaped curve (Yerkes & Dodson, 1908). It is generally believed the “best” or “optimal” performance is usually obtained when the stress level is around medium, i.e., not too low, not too high (Yerkes & Dodson, 1908). Secondly, the TASKS model qualitatively defined human mental stress as the ratio of workload over mental capability, as (Nguyen & Zeng, 2012; Yang et al., 2021):
(1)
• σ, the mental stress, denotes the students’ mental stress.
• T, the Task or Workload, represents the total workloads students need to conduct, i.e., all course activities such as lectures, homework assignments, quizzes, exams, classroom participation, labs, projects, presentations, and demonstrations.
• A, the Affect, represents students’ emotional aspects, such as stress, feeling, attitude, motivation, and stimulus. As shown in equation (1), Affect affects how much of Knowledge (K) and Skill (S) can be activated towards completing the workload (T).
• K, the Knowledge, denotes existing knowledge that students already obtained from other courses and past learning experiences.
• S, the Skill, denotes the new knowledge, skills, and logics that students have acquired in (or inside) the particular course in consideration.
Fig. 1
The TASKS model adopted to the course design context.
With students’ performance (P) related to their mental stress (σ), and mental stress (σ) is further modelled as in (1), students’ performance (P) is finally a combined effect of the factors T, A, S, and K:
(2)
Design started by identifying conflicts and needed actions. Notice that not every course needs to be significantly modified to fit into online teaching. For courses involving only programming, software, and simulations (such as programming courses and AutoCAD modelling courses), students may even benefit from the online teaching format since institutions now provide them licences and sometimes laptops. They have better access to the software than before. For courses requiring access to major facilities and equipment (such as some courses in dental hygiene, mechanical and manufacturing engineering), actions are usually taken at the institution level by giving them higher priority to return to campus. For courses having only lectures (such as mathematics and circuit courses), instructors may choose to use the same coursework.
What makes transitioning the Robotics Technology course challenging is (Fig. 2):
A. The Robotics Technology course is structured to have both lecture and lab sessions.
B. Its labs/projects include both hardware and software exercises.
C. Being a senior course, students need to be prepared with skills directly needed by future workforce.
Fig. 2
Composition of the Robotics Technology Course: Lectures, Software, and Hardware Exercises.
The first identified conflict is the unavailability of hardware devices (i.e., VEX robots) causing possible loss in hands-on skills. To maintain the higher education standards, the knowledge and skills gained by laboratory practices cannot be compromised. Actions are needed to seek alternative approaches that still provide students with meaningful learning experience remotely. Among the approach of delivering robotic kits to students’ homes for at-home experiments, doing live demonstrations with students observing remotely, and resorting to computer simulation that have been adopted for educational enhancement, we decided to develop computer-based simulations to provide students with hands-on practices, based on available resources, projects’ difficulties, and the percentage of hardware components in this course.
Firstly, there are only limited sets of VEX robots available. In terms of cost, the VEX robotic kits are not economical enough for the department to purchase and then give students to use at home. Secondly, the physical projects are quite challenging. Past teaching experience shows that students need help in hardware integration, sensor and actuator wiring and setup, and hardware-software wireless communication. These issues will be difficult to resolve remotely. Uncertainties such as un-functioning hardware/electronic devices, broken hardware-software communication, and missing components will add extra anxiety and frustration for both parties, cause higher failure rate of the lab/project activities, and may even degrade student’s performance in the lecture portion. Thirdly, hardware-related aspects are not the major components of this course. Practical hands-on practices in the past also focused on system integration, software design, and algorithm implementation, which can be properly addressed via simulations.
The action we took is to design the simulation projects as alternative exercises and simulated versions of those physical projects done at school. These simulation projects were designed with comparable complexity in similar steps as those physical experiments, trying to resemble their in-person counterparts as much as possible. Since aspects pertaining to the hardware were removed, focus was shifted to software design, programming, and algorithm implementation. Extra software-related workloads were created to maintain the overall project-related workloads.
The second conflict is that online teaching probably demands new ways of conducting exams. Our action is to divide each exam into two parts: a) an in-class Blackboard online test consisting of randomly-selected questions from pre-designed question pools/banks and b) the traditional exam is now taken at-home.
The third conflict is regarding students’ motivation in the course and their classroom participation, which may both degrade under e-learning. The actions we took include using frequent low-stakes quizzes during lectures to attract students’ attention. During lab sessions, students were asked to frequently and sometimes mandatorily share their computer screens to report progress.
The fourth conflict is the change of teaching format from in-person to online. Our action is to use synchronous virtual meetings to replicate, to the greatest extent possible, the past in-person teaching.
All these conflicts are summarized in Table 1. Corresponding actions are listed, details of which are elaborated in Secs. 3 and 4, for lecture-related and project-related workloads, respectively.
Table 1
Identified conflicts and proposed actions
| Conflicts | In lecture sessions | In lab sessions |
| Change of teaching format from in-person to online | Conducting lectures and labs via synchronous virtual meetings. | |
| Potential degradation in student motivation and classroom participation | Using frequent low-states quizzes to attract students’ attention. | Asking students to share their computer screens to report progress. |
| Difficulty in proctoring in-person exams under e-learning | Using Blackboard online tests that consist of randomly-chosen questions | N/A |
| Unavailability of hardware causing loss in hands-on skills | N/A | Designing simulation projects as simulated versions of physical experiments; increasing software-related workloads to compensate for reduction in hardware-related practices. |
3Online teaching practices
Course Content The instructional goals of this robotic course, CET 4952: Robotics Technology, are to introduce fundamental subjects in both Autonomous Mobile Robot (Siegwart et al., 2004) and Robotic Manipulator (Spong et al., 2020), as well as improving students’ hands-on skills in robotic design, system integration, and programming. Under e-learning, the same course content was strictly followed. Design actions are on developing 1) online-appropriate teaching practices that ensure active student participation in class and 2) online-appropriate assessment methods to evaluate student’s performance.
3.1Course content
A fifteen-week lesson plan is given in Table 2, showing synchronization between projects and lecture topics. Corresponding course learning outcomes are also listed below.
Table 2
Course content
| Week | Lecture (3 Hours per Week) | Project (3 Hours per Week) |
| 1 | Introduction | Project 1: |
| 2 | Map Building; Path Planning; Localization | •Construction of a mobile robot |
| 3 | Sensors | •Translation |
| 4 | Introduction to Image Processing | •Rotation |
| 5 | Camera Modelling | •Navigation |
| 6 | In-Class Exam #1 | •Map Building |
| 7 | Homogeneous Transformation | Project 2: |
| 8 | Parameterization of Rotation Matrix | •Construction of the robot with an arm |
| 9 | Forward and Inverse Kinematics (1) | •Modeling and Analysis |
| 10 | Forward and Inverse Kinematics (2) | •MATLAB Simulation |
| 11 | Trajectory Generation | •Implementation on the robot |
| 12 | Jacobian Matrix and Singularities | Project 3: More advanced robotic control tasks such as vision-based control, obstacle avoidance, or coordinated control of multiple robots. |
| 13 | Independent Joint Control | |
| 14 | Review | |
| 15 | In-Class Exam #2 |
Upon completion of the course, students are expected to be able to:
1. perform path planning for a mobile robot in a structured environment.
2. command the robot to follow a trajectory given by a list of waypoints.
3. build a simple map using the onboard range sensors on a mobile robot.
4. demonstrate knowledge and understanding of fundamental image processing functions and the camera’s perspective projection model.
5. demonstrate knowledge of the relationship between mechanical structures of industrial robots and their operational workspace characteristics.
6. demonstrate ability to apply a spatial transformation to obtain forward kinematics of robotic manipulators.
7. demonstrate an ability to solve inverse kinematics of simple robotic manipulators.
8. demonstrate an ability to generate a joint trajectory for motion planning.
9. demonstrate an ability to obtain the Jacobian matrix and use it to identify singularities.
3.2Teaching remotely
The following describes a set of online teaching practices that replicate traditional face-to-face teaching as much as possible, simply virtually.
• Synchronous Virtual Meetings (via Zoom): Teaching via synchronous virtual meetings was used for both lectures and labs since it is close to in-person teaching, creates predictability in weekly schedules, and encourages accountability (Conefrey & Smyth, 2021). Studies show that utilization of video conferencing in synchronous instruction impacts the learners’ feeling of association with their instructor (Malik et al., 2017). Therefore, during all virtual meetings, the instructor kept the video on to help students to overcome the feeling of being away from the instructor. All students had no problem starting and joining the zoom meetings. Features such as mute/unmute, screen sharing, chat, break room, participants, poll, and recording were used to ensure proper communication and discussion of course materials. Students regularly attended virtual meetings and responded actively to questions.
• Blackboard: Blackboard (Little-Wiles & Naimi, 2011; Eschenbach & Cashman, 2004) was used to post all course-related materials (announcements, weekly lesson plan, lecture materials, in-class notes, projects descriptions, homework assignments, solutions, and recordings). Students were asked to submit all their work to Blackboard. A weekly lesson plan (similar to Table 2) was maintained to reduce the cognitive load for the online user and to outline explicitly the connections between lecture topics and projects (Pantazidou & Kandris, 2020).
• Virtual Office Hours: Following our institution’s guideline of hosting office hours virtually, weekly office hours were scheduled, using the same zoom information as the virtual meetings. Hosting the virtual office hours allows students’ questions and concerns to be addressed promptly.
• Screen Sharing During Lab Sessions: Students frequently share their computer screens to show progress or ask for assistance. Each student was required to share their screens three times during the semester, one for each project, to ensure all students had the proper setup for each project. Screen sharing promoted classroom participation, student-instructor interaction, and student-student interaction.
3.3New teaching activities
Two newly-developed teaching activities are described below, including frequent low-stakes online quizzes (Little-Wiles & Naimi, 2011; Eschenbach & Cashman, 2004; Yadav et al., 2011; Prince et al., 2020) and Blackboard online tests of randomly-chosen questions (Khan et al., 2021). The routine online quizzes helped to maintain students’ engagement in classes. The online test of random questions can serve as an online-appropriate assessment tool replacing traditional ways of paper-based methods (Khan et al., 2021).
• Frequent/Routine Low-Stakes Online Quizzes:
One challenge that instructors face under e-learning is to keep students engaged during virtual classes. We used low-stakes online quizzes to keep students attentive (Prince et al., 2020; Teo, 2020). In fall 2020, nine online quizzes were conducted, with a total weight of 5% in students’ final grade. These quiz problems are the same for all students. Each student is given two attempts. Most quiz questions come from the current lecture, requiring students to pay attention during classes. We found that these quizzes provided a very good mechanism to encourage interactions between the instructor and students.
Overall speaking, both attendance and classroom participation were very satisfactory in the fall semester of 2020. Table 3 shows the number of students who did not participate in the online quizzes (out of 21 students, for totally 9 quizzes, in fall 2020). Notice that Quiz #1 was given in the first lecture when students were not aware of frequent/routine quizzes. The number of students who didn’t participate dropped from 4 to no more than two later on. It is thus safe to say these online quizzes help to bring students to classes. Students also said that the quizzes helped them to review course materials when preparing for tests.
• Online Tests Consisting of Random-Selected Questions
When designing online tests, one can choose to randomize the order of questions or to use multiple variations of the same question with different numeric values (Teo, 2020). Our approach is to randomly pick up problems from pre-designed question banks/pools. Both the midterm and final exams were structured to have two portions: an in-class Blackboard online test and a take-home (or at-home) exam.
Table 4 shows the design of Online Test 1, which has six modules. A pool is designed for each module. Inside each pool, multiple relevant questions are designed pertaining to the subject. For example, there are 21 questions in the pool of “Fundamental Mathematics” and nine questions in the pool of “Waypoint Navigation”. The online test was then designed to include nine randomly selected questions (out of 21 candidate questions) from the pool of “Fundamental Mathematics”, four random questions (out of 9 candidate questions) from the pool of “Waypoint Navigation”. The online test contains a total of 25 questions. The probability of two students getting exactly the same test would be very dim. Students’ scores confirm the test questions are challenging enough to differentiate students’ performance (Davis et al., 2021).
5 As seen in the 3rd column of Table 4, the number of questions selected from each pool varies. It is determined by the time spent on each module. For example, the mathematics background in linear algebra is very important for this course. A significant amount of time was spent reviewing relevant math subjects at the beginning. As a result, more questions are selected from “Fundamental Math” in Online Test 1. At the time of Online Test 2, math has been intensively practised. Its weight is made similar to weights of other topics. Clearly, it would be better to have more questions in each pool (the 4th column in Table 4). So far, due to time constraints and limited resources for question banks, we haven’t achieved a large pool for every module. For future improvement, more questions will be added, particularly to those modules that do not have a large pool yet. Ideally, we want to achieve a ratio of at least 3 : 1 (or even 4 : 1) between the total number and the selected number of questions in each pool.
Table 3
Number of students who did not participate in online quizzes (fall 2020, 21 students)
| Quiz # | #1 | #2 | #3 | #4 | #5 | #6 | #7 | #8 | #9 |
| # of students did not participate | 4 | 1 | 0 | 1 | 0 | 0 | 1 | 2 | 0 |
Table 4
Design of online tests with randomly-chosen questions (fall 2020)
| Modules (Question Pools/Banks) | # Of Questions Selected | # Of Questions in Each Pool | |
| Online Test 1 | Fundamental Mathematics | 9 | 21 |
| Waypoint Navigation | 4 | 9 | |
| Path Planning | 3 | 6 | |
| Map Building using Range Sensor | 3 | 5 | |
| Introduction to Image Processing | 3 | 6 | |
| Camera Perspective Projection Model | 3 | 6 | |
| Online Test 2 | Fundamental Mathematics | 4 | 8 |
| Simple Coordinate Frames | 2 | 7 | |
| Homogeneous Transformation and Rotation Matrices | 2 | 6 | |
| Composite Rules for Homogeneous and Rotation | 4 | 8 | |
| Forward and Inverse Kinematics | 4 | 15 | |
| Trajectory Generation | 4 | 6 | |
| Jacobian Matrix and Singularity | 2 | 4 | |
| Questions from Past Quizzes | 3 | 39 |
4Simulation projects
When labs were conducted in traditional face-to-face classrooms, students were asked to complete three projects over one semester (the 3rd column of Table 5). Project-based learning (Fini et al., 2018; Shekar, 2014) was used in the first two projects to guide students through fundamental algorithms. Inquiry-based learning (Altin & Pedaste, 2013; Edwards & Recktenwald, 2008; Beiley, 2018), a highly self-directed approach of learning and discovering through exploration or observation, was used in the final project, where students apply skills acquired so far to solve questions independently (with the least amount of help from the instructor). Computer simulations have been widely used to enhance learning (Koh, et al., 2010). Under e-learning, when experimentation requiring laboratory facilities becomes unavailable, we designed alternative simulation projects, as shown in the 4th column of Table 5. These simulation projects are alternatives and simulated versions of those physical projects done at school. They were designed with comparable complexity and quality in similar steps and procedures as those physical experiments. Both project-based learning and inquiry-based learning continued to be used in these simulation projects. Some hardware-related aspects, such as hardware assembly, sensors and actuators, and communication between two hardware devices, are illustrated by constructing simulated robots, obtaining data from simulated sensors, commanding simulated actuators, and communication between two software agents. The focus is shifted to software design and algorithm implementation.
Table 5
Projects used in face-to-face vs. online teaching (physical vs. simulation)
| # | Topic | Physical Robots and Projects | Simulation Projects |
| 1 | Autonomous mobile robot |  A mobile robot built using VEX |  Our own simulator |
| 2 | Robotic manipulator |  A VEX robotic manipulator |  Robotics Toolbox by Peter Corke (Corke, n.d.) |
| 3 | More advanced robotic control (Vision-based control) |  Vision-enhanced VEX robot |  MATLAB-ROS-Gazebo platform |
MATLAB has been widely used in research, industry, and higher education due to its high performance in scientific and technical computing, as well as its up-to-date Toolboxes (Khan et al., 2017). Pertaining to robotics, its Mapping and Navigation Toolboxes provide functionalities in mapping and obstacle avoidance of autonomous mobile robot; its Image Processing, Computer Vision, and Image Acquisition Toolboxes allow processing of images from both recorded videos and live image sequences captured by USB cameras; and its Reinforcement Learning and Deep Learning Toolboxes allow investigation of machine learning and deep learning algorithms. Further, MathWorks’ recent release of its ROS Toolbox provides easy access to ROS-compatible simulators (and even hardware) for the development of complex robotic control software systems. Using MATLAB, all simulation projects can be implemented in one unified software platform. Students thus do not need to learn different programming languages.
The required Toolboxes for each simulation project are:
∘ Project #1 on autonomous mobile robots: MATLAB Mapping Toolbox.
∘ Project #2 on robotic manipulators: MATLAB Robotics Toolbox by Peter Corke (Corke, n.d.).
∘ Project #3 on more advanced control tasks (such as vision-based control): MATLAB ROS, Image Processing, and Navigation Toolboxes.
4.1Path planning of a mobile robot using homemade simulator
The first project is to expose students to several important topics pertaining to autonomous mobile robots, including path planning, navigation, sensor detection, and basic motion control. Suppose a robot works in a known world with knowledge of locations of some fixed obstacles. We want the robot to go from one specified position (the initial position) to another specified position (the goal position), without collisions with any of these obstacles. Simultaneously, we would like the robot to build a map of its environment using some range sensors (for example, sonar sensors). The three figures on the top of Fig. 3 show the experimental setup at school using physical robots. A simulated version of this project was achieved via a homemade virtual environment, as shown at the bottom of Fig. 3.
Fig. 3
Path planning of an autonomous mobile robot: physical vs. simulation.
The physical project usually includes the following steps: 1) hardware assembly and software installation; 2) basic motion control, including translation (commanding the robot to go straight a specified distance) and rotation (commanding the robot to rotate a specified angle); 3) waypoint navigation (asking the robot to navigate through a given list of waypoints); 4) map building using sonar sensor (detect objects as the robot travels around); and 5) path planning (finding an obstacle-free path for the robot to go from a specified location to the goal location).
The simulation project includes 1) creation of a virtual environment; 2) detection of objects using range sensors; 3) clustering and recognition of clustered points as objects denoted by polygons; 4) waypoint navigation; and 5) path planning. Except for steps pertaining to controlling the physical robot, the simulation project includes all the key steps as the physical project. It also includes clustering and simple pattern recognition that were not addressed in the physical project.
4.2Trajectory generation of a robotic arm using MATLAB robotics toolbox
The second project is to expose students to several key components when controlling a robotic arm, including modelling, forward kinematics, inverse kinematics, and trajectory generation. Specifically, the task is to control a robotic arm’s end-effector to go through a given list of “via-points” in the 3D space. For in-person teaching, the VEX robotic arm (Fig. 4(a) and (b)) was analysed and used, which has a simple arm installed on its autonomous base. Under e-learning, the MATLAB Robotics Toolbox by Peter Corke (Corke, n.d.) was used to construct simulated robotic arms. Figure 4(c) and (d) show trajectory generation of a 4-DOF SCARA robot and a 6-DOF PUMA560 robot, respectively.
Fig. 4
Trajectory generation of Robotic arms: using physical robot (Ma & Ma, 2019) vs. simulated robot.
When using the physical VEX robot, students were guided through 1) modelling of the robot (i.e., measuring the length of links, angles, assigning coordinate frames); 2) obtaining the forward kinematics; 3) obtaining the inverse kinematics; 4) control of the robotic arm; and 5) trajectory generation that commands its end-effector to follow a given list of 3D “via-points”.
For the simulation project, students performed 1) construction of simulated robots; 2) obtaining forward kinematics; 3) derivation or implementation of the inverse kinematics; 4) implementation of trajectory generation in the 3D space; and 5) trajectory generation in the joint space. It can be seen that the key steps of forward and inverse kinematics and trajectory generation are performed in both projects.
4.3Vision-based control of autonomous robot using MATLAB-ROS
The third project is to introduce students to more advanced robotic control tasks. This has been a mission for this course since fall 2018. Simple vision-based control, i.e., visual servoing to a static object, was added as the third project. The control objective is to command a vision-enhanced VEX robot (Ma & Alborati, 2018; Ma et al., 2019) to face towards a visual landmark at a certain distance away. Figures on the top of Fig. 6 demonstrate the behaviour of the robot, together with the images captured by its onboard camera. The robot first orients itself towards the target and then moves to it. Correspondingly, the target on the image plane moves closer to the center of the image and then appears bigger.
For e-learning, we need a simulation platform that allows students to explore robotic applications of similar complexity. We resort to the Robot Operating System (ROS, https://www.ros.org/). Adoption of ROS in undergraduate curriculum is still limited, due to its demanding requirements of C++/Python programming skills and familiarity with Linux (Lehman & Hayder, 2015; Wilkerson et al., 2017; Khasim, 2017; Canas et al., 2020). MathWorks’ release of its ROS Toolbox allows much easier interaction with ROS-enabled physical robots or simulators (such as Gazebo). Students can also use many other MATLAB Toolboxes (such as Image Processing and Navigation Toolboxes) for fast data processing and algorithm prototyping (Avanzato, 2020; Avanzato & Wilcox, 2020).
ROS was introduced into the Robotics Technology course under the MATLAB-ROS-Gazebo package, which involves MATLAB, the ROS middleware, and the 3D Gazebo simulator. The whole package requires a fair amount of downloading, installation, and introduction of the entire system, including:
1) Installation of the ROS Toolbox on MATLAB.
2) Installation of the VMware Player Software (a virtual machine player).
3) Downloading of the virtual machine.
4) Opening the “Playground” virtual environment, Fig. 5(a).
5) Introduction of ROS and the Gazebo simulator.
6) Setting up IP addresses of ROS nodes (one on MATLAB and the other of the Gazebo simulator).
7) Receiving sensor data (images, odometry, laser scan) from the robot’s onboard sensors, Fig. 5(b).
8) Commanding the robot to move.
Fig. 5
MATLAB-ROS- Gazebo simulation environment.
Due to the complexity of the ROS project, sample programs were provided to the students. Students were asked to make extensions on top of these sample programs to achieve a control task that involves a) collection of sensor data, b) data processing, and c) control of the robot’s motion. With the preparation and introduction of the MATLAB-ROS package, students were able to perform visual servoing to a static object (i.e., the blue cube as shown in Fig. 6(b)), as a variation of the physical project shown in Fig. 6(a). Visual servoing is to control the robot’s orientation and/or position with respect to a visual landmark. An image-based visual servoing method was implemented, which adjusts the robot’s orientation until the target’s center is close to the image center horizontally. The MATLAB-ROS platform, once installed on the students’ own computers, allows them to continue exploring robotics-related tasks outside of the classroom, thus promoting activities at the level of undergraduate research.
Fig. 6
Vision-based control: using physical vs. simulated Robots.
In summary, this section presents the three simulation projects that were designed for e-learning, covering fundamental subjects in autonomous mobile robots, robotic manipulators, and advanced robotic control tasks. Since robotic software systems are complex, it is advisable to expose students gradually to such complexity (Canas et al., 2020). Thus, these simulation projects were designed with increasing complexity. Project 1 aims at preparing students to obtain basic understanding and programming experience. Students will then acquire more solid programming skills in Project 2. Project 3 eventually exposes students to real-world software design and algorithm implementation. Step by step, students will acquire the knowledge, skills, and experience of designing and implementing algorithms for robotic applications.
4.4Conduction of projects and deliverables
In fall 2020, the course was taught completely online. Students were allowed to work either individually or with another member. The maximum number of students allowed in each group is two. Out of 21 students, seven students chose to work individually. The rest worked in groups of two.
Since project-based learning was adopted for both physical and simulation projects, each student or group was allowed to take a slightly different pace. We aim at creating a friendly and encouraging learning environment where students will always be welcome. They will not be punished for just being a little behind. The majority of the students were able to proceed with the suggested pace, with few students faster or slower. Students also helped each other to identify issues and debug. When pre-recorded lessons are available, students could also review these recordings. Students who did not finish during the lab sessions would usually spend more time on their own (since they have access to the software all the time). They also came to the virtual office hours when they still had questions unanswered.
The purpose of Project 1 is to prepare students with basic programming skills for algorithm implementation. The instructor provided as much help as needed. Starting from Project 2, the instructor gradually became hands-off. For Project 3, where students were asked to define their own tasks, the instructor only provided guidance at the idea-level, e.g., discussing potential challenges and possibility of completing tasks in time. There was almost no coding help for Project 3. Under this teaching style, almost all students completed Projects 1 and 2. For Project 3, most students provided a solution to their self-defined tasks, though the difficulty of each task varies.
Since the simulation projects are designed as companions of those physical projects, students’ deliverables for simulation projects are the same as those for physical projects. That is, the deliverables are mainly determined by the nature of each project, instead of the teaching format. For Projects 1 and 2, the instructor checked each step during lab sessions; upon completion, students were asked to write a report documenting major steps. For Project 3, students’ deliverables are group presentations and live classroom demonstrations.
5Self-evaluation
We evaluated how closely the developed online workloads resemble their in-person counterparts. We divide the entire Workload (T) into four groups, denoted by T = {T 1, T 2, T 3, T 4}, where T 1, T 2 denote lecture-related workloads and T 3, T 4 are project-related workloads.
5.1Lecture-related workloads: online vs. in-person
For lecture-related workloads, Table 6 summarises our course redesign process, by listing each workload component, the identified conflict between in-person and online teaching (i.e., the need for redesign), the design solutions (i.e., actions we took), and our self-evaluation of the similarly to their in-person counterparts. The similarity is judged based on workload complexity, ability to reinforce intended concepts, and generated stress for students.
Table 6
Course redesign for lecture-related workloads
| Lecturing (T 1) | Assessment (T 2) | |
| Workload | –Course Content | |
| Component | –Learning Objectives | –Exam #1 |
| –Lecture Notes | –Exam #2 | |
| –Homework Assignments | ||
| Conflicts | Different teaching formats | Difficult to proctor in-person exams under e-learning. |
| Actions (Solutions) | –Keeping course content and learning outcomes the same. | –Dividing each exam into two parts: one is a Blackboard online test of randomly-chosen questions done in-class; the other is the traditional written exam now done at home. |
| –Delivering lessons virtually. | –Using online quizzes to prepare students to get familiar with taking tests online. | |
| Self-Evaluation of Similarity to In-Person Counterparts | Exactly the same (or at least extremely close) | Very close (with newly-added workloads) |
Some workloads (such as course content, learning objectives, lecture notes, and homework assignments) were used previously in the in-person teaching mode. These materials are carried on to the online teaching without any change. Under e-learning, the exams are heavier than before due to the addition of an online test. Moreover, a set of online quizzes are added in lectures to promote classroom participation, as well as preparing students to take tests online. In Fig. 7, workloads in the overlapped area represent the same workloads shared by both. Others are either conducted differently (exams) or added new (online quizzes).
We closely monitored students’ reactions about these newly-added workloads, making sure they actually served their good intention without overwhelming the students. The online quizzes are intended to be low-stakes, without giving students much pressure. In most cases, they serve as classwork rather than assessment tools. Should students have questions, the instructor will explain it to the whole class. Regarding online tests, after the first exam (Exam #1), we asked students if they liked the way Exam #1 was structured (i.e., consisting of an online test plus an at-home test). Students had no complaints about it. One student said he actually liked how Exam #1 was structured and conducted. Both the online quizzes and online tests were also used in two other courses (a junior-level electric circuit course and a senior-level control course) since e-learning started in spring 2020. So far, we have not received any complaints about them at all. We thus believe these added workloads do not cause extra stress for students.
Fig. 7
Lecture-related workloads conducted in-person vs. online.
In summary, we think our course redesign yielded similar workloads as those performed in-person, without causing extra stress for students.
5.2Project-related workloads: online vs. in-person
The course redesign process for project-related workloads is summarised in Table 7.
Table 7
Course redesign for project-related workloads
| Software-Related (T 3) | Hardware-Related (T 4) | |
| Workload | –Project 1 | –Hardware assembly |
| Component | –Project 2 | –Sensors and actuators |
| –Project 3 | –Communication between two hardware devices | |
| Conflicts | Previously implemented using C/C++on physical robots | Physical robots are unavailable under e-learning. Workloads in other aspects need to be increased to accommodate this loss. |
| Actions (Solutions) | Now implemented on simulated robots using MATLAB | –Conducting hardware-related workloads on simulated robots by construction of simulated robotic arms; obtaining sensor data from simulated robots; control of simulated robots; communication between two software agents (two ROS nodes). |
| –Creating new software-workloads to compensate for reduction in hardware-workloads, including clustering and data processing, trajectory generation in joint space, and obstacle avoidance, pick-and-place of PR2 robotic arm on MATLAB-ROS | ||
| Self-Evaluation of Similarity to In-Person Counterparts | Very close | Though concepts can be demonstrated via simulations, we still feel that the purely hardware-related aspects cannot be addressed adequately. |
Technically, software-related workloads in terms of software design and algorithm implementation can be adequately performed using simulations. Through significant course redesign, the achieved similarity to the in-person version can be very high. The only conflict hard to resolve is regarding those truly hardware-concentrated activities, including, for example, hardware assembly, wiring of sensors and actuators, and communication between two hardware devices. Though these topics can be demonstrated using simulated robots, we feel they cannot be adequately addressed via simulations.
To achieve similar overall project-related workloads, several new topics on algorithm implementation were added, including clustering and data processing, trajectory generation of robotic arms in joint space, obstacle avoidance, and control of the PR2 robotic arm on MATLAB-ROS. These topics were unable to be covered when using physical robots in the past, since handling the hardware took time. We are able to address them now under e-learning.
As illustrated in Fig. 8, our course redesign for project-related workloads obtained similar overall workloads as before, by keeping some workloads approximately the same, some performed slightly differently, some demonstrated as much as possible, and others added new.
Fig. 8
Project-related workloads conducted in-person vs. online.
In the next, we describe the arrangements we made, which ensured the execution of the simulation projects without causing too much stress on students.
Regarding the software, MATLAB is easy to install. Before the pandemic, our institution already provided a licence to students to install MATLAB on their own computers. This licence is updated annually. So, some students had MATLAB already. Moreover, MATLAB is commonly used in the STEM fields. Some students have already used it before in other courses (such as maths, circuit analysis, and feedback and control courses). Before classes began, we sent emails to students and posted announcements on Blackboard, asking students to install MATLAB in advance. During the first couple of weeks, should students ask questions about how to install MATLAB, we would spend time guiding them to complete the installation process (which is pretty standard). Other students who had similar issues can follow along. We also listed all the Toolboxes to be used for our projects, so that students can simply install them instead of all the Toolboxes to save computer space.
To make sure that students have the proper knowledge and programming skills using MATLAB, the first two lab sessions were used to cover the fundamentals of MATLAB programming and debugging, assuming basically zero prior background. These labs helped students to get familiar with the MATLAB programming environment. At the beginning of each project, students were required to share their computer screen. The instructor took a record of who had already shared. These records were inputted to Blackboard as part of classroom participation, reminding students such activities were to be completed. These mandatory screen sharing pushed the students to have the proper setup for each simulation project.
Once getting started, the instructor usually allocated 30 minutes in each lab session for students to share their computer screens and get assistance. Students were also encouraged to help each other to debug and fix errors in the codes. We made sure that a friendly and encouraging online learning environment was created, so that students were welcome to ask any questions. We also allowed students to choose to work either individually or with another member to get support.
All these arrangements help to ensure that the project-related workloads are reasonable and manageable, without over-stressing the students.
6Evaluation and analysis
When interpreting course design from the point of view of implementation science (Soicher et al., 2020), course design is also a type of design aiming to obtain desired behaviour change in the students (i.e., achieving better learning performance). Thus, when evaluating our course design solution, two performance indicators pertaining to students’ learning performance (exam scores and students’ creative thinking capabilities) are used. Since students’ motivation determines, directs, and sustains what they do to learn and is thus crucial for online learning, we also present two indicators reflecting students’ motivation (STE response and student satisfaction about the simulation projects). Evaluation and analyses using these four indicators are described in the next.
6.1Student exam grades
When the course was taught in-person from spring 2017 to fall 2019, the midterm exam and the final exam were used to evaluate course learning outcomes 1∼4 and 5∼9 (Sec. 3.1), respectively. The course learning outcomes (1∼4) are regarding students’ abilities on basic motion control of an autonomous mobile robot, while outcomes (5∼9) are on students’ abilities of conducting analyses and control of a robotic manipulator. These two exams were in-class, closed-book, closed-note, closed-computer, providing a summative assessment of students’ learning and knowledge (Guilford & Helmke, 2017).
Spring 2020 was the first time the course was taught online. The midterm exam was right on spot when transitioning from in-person to online. It was assigned as a take-home exam. All students got a score higher than 70 (out of 100). But we decided not to use these as assessment results since the exam was not performed during the class period. With an assessment in mind for online teaching, we prefer activities done in-class, which has the same time restriction as pre-pandemic in-person exams. To obtain an online-appropriate assessment, we resort to the Blackboard online test of randomly chosen questions.
In spring 2020, we did one online test as part of the final exam. In fall 2020, we did two online tests. Assessment using the online tests (in yellow), together with those collected in six past in-person offerings using in-class exams (blue), are shown in Fig. 9. Notice that assessment in Spring 2020 for Course Learning Outcomes (1∼4) was treated as “unperformed” and is left blank in Fig. 9(a). Setting 70% as the target for being “Proficient and/or Satisfactory”, course learning outcomes (1∼9) were successfully obtained under the considered semesters under online teaching.
Fig. 9
Assessment of course learning outcomes (1∼9): in-person vs. online.
To observe the effect of teaching modes (in-person vs. online) on students’ learning performance, students’ exam grades crossing eight semesters were analysed using One-Way analysis of variance (ANOVA) via MATLAB’s Statistics and Machine Learning Toolbox, particularly, the anova1() function. The One-Way ANOVA provides results to determine any statistically significant differences between the means of two or more independent samples, i.e., whether different groups of an independent variable have different effects on the response variable (Kumpass-Lenk et al., 2018; Boysen, 2015; Saleheen et al., 2015; Khan et al., 2018). In this analysis, the independent variable is the teaching mode having two levels: in-person and online. The response variable is students’ learning achievement reflected by their Exam#1 and Exam#2 grades.
Students’ Exam #1 grades (21 students in fall 2020) and Exam #2 grades (22 students in spring 2020 and 21 students in fall 2020) are compared with those obtained in six past in-person semesters (126 students in total). Table 8 shows the ANOVA results and their corresponding box plots. In both plots, Group #1 denotes the group of in-person students, and Group #2 is for the online students. Comparison between both groups of students using Exam#1 [Fig. (a)] shows a p-value of 0.0522. When using Exam#2, the ANOVA analysis yielded a p-value of 0.4878 [Fig. (b)]. Since both p-values are greater than 0.05, the results of the ANOVA tests indicate there is no sufficient evidence that these two groups of students under comparison perform differently. These results demonstrate the effectiveness of our course design and teaching practices in overcoming the challenges/barriers presented by online teaching.
Table 8
ANOVA test results of students’ exam grades
 |
In terms of difficulty and challenge, both Exam #1 and Exam #2 of each semester were designed to be comparable (approximately the same). The online tests decomposed a multi-step question into smaller steps, making them suitable to be tested online using multiple choice, fill-in-blank, and true/false questions.
To see if there were any initial differences in students’ prior knowledge in mathematics (which is very important for this course), students’ grades of two homework assignments (one in complex numbers and the other in linear algebra) were used. The homework on complex numbers (say hw#1) was assigned at the beginning of this robotic course. The homework on linear algebra (say hw#2) was assigned around week 6 in each semester. In this study, scores of the spring 2020 students were counted towards the in-person group, since their prior knowledge was obtained before online teaching started. Besides, both assignments were collected before the pandemic. For hw#1, the mean and standard deviation for the in-person and online students were 7.91 (SD = 2.97) and 6.63 (SD = 3.62), respectively. A further ANOVA test yielded a result of F (1, 167) =3.25, p = 0.0734. As for hw#2, the mean and standard deviation for the in-person and online students were 6.91 (SD = 3.50) and 6.96 (SD = 3.65). A further ANOVA test yielded a result of F (1, 167) =0.0035, p = 0.953. We feel it is safe to say that students in both in-person and online teaching demonstrated a similar level of readiness for the robotic course before entering the course.
6.2Student creative thinking capabilities
Creative thinking is an important skill needed for the modern workplace (Bielefeldt & Morse, 2019). Since robotics inherently demands creativity and requires investigation among several different methods to eventually come up with a solution (Eteokleous et al., 2020; Alimisis, 2013; Kuo et al., 2021), we have been assessing students’ creative thinking capabilities sing fall 2028 using Project 3. After preparing students with fundamental knowledge and skills using lectures and the first two projects, the third project is dedicated towards solving possibly open-ended questions, thus providing an opportunity for students to practice their creative thinking skills. The nature of being open-ended implies that multiple solutions might exist for a given problem. Students thus need to explore among different methods to find the most appropriate approach, or to come up with their own/new way. Further, our teaching methodology of providing only supervision, instead of detailed instructions, pushes students to take the risk factor into consideration. Five out of six performance criteria from the Creative Thinking Value Rubric were assessed (Ma et al., 2021), as given in Table 9. The criterion of “Connecting, Synthesising, and Transforming” was not utilized since it generally corresponds to the highest level of creative thinking and may not be adequately evaluated in this course that focuses on robotics subjects only. A numerical value was assigned by the instructor for each students’ performance under each criterion on a scale of 1 to 4, with Score-1 corresponding to “Unsatisfactory”, Score-2 for “Developing”, Score-3 for “Satisfactory”, and Score-4 for “Proficient”. The assessment results in three past in-person semesters and one online semester (fall 2020) are shown in Fig. 10.
Table 9
Performance criteria and assessment of creative thinking
| Performance Criterion | Evaluation Based on |
| Acquiring Competences | Each student’s cumulative performance in homework assignments, Project 1, and Project 2. |
| Risk-Taking | Complexity of the project, the existing knowledge and skills of the students, and whether or not the instructor was able to or willing to provide help. |
| Embracing Contradictions | If students attempted to consider an alternative design method, and if so, if they succeeded or not. |
| Innovative Thinking | If students have implemented any new ideas (or any of their own ideas) in the design. |
| Solving Problems | If the task was successfully completed. |
Fig. 10
Assessment of students’ creative thinking using project 3: In-person Vs. online.
Again using 70% as the target for being “Proficient and/or Satisfactory” (i.e., score of 3 and/or 4), it can be seen that students’ performance regarding the five indicators either stabilized around/above the target of 70% after some initial overshoot, or steadily improved/increased to reach and go beyond the target. When further compared with three past in-person offerings, students’ creative thinking performance in fall 2020 (under online teaching) is still comparable with those achieved in-person. This again demonstrates that our teaching approaches and practices are helpful to overcome the challenges and barriers presented by online teaching.
6.3STE response
We present students’ teaching evaluation (STE) responses to the question “The instructor held my interest and attention during class” as one indicator of students’ motivation and interest. Participants rated on a 5-point rating scale (5-strongly agree, 4-agree, 3-neural, 2-disagree, and 1-strongly disagree). Students’ ratings in six past in-person offerings and the online teaching in fall 2020 are shown in Table 10. Notice that STE was not conducted at our institution in spring 2020 as a college-wide decision.
Table 10
Students’ response to STE question “The Instructor Held My Interest and Attention During Class”
| 5-Strongly Agree (%) | 4-Agree (%) | 3-Neural (%) | 2-Disagree (%) | 1-Strongly Disagree (%) | # Of Participants (Total # of Students) | ||
| In-Person | 2017 Spring | 60 | 33 | 7 | 0 | 0 | 15 (20) |
| 2017 Fall | 67 | 17 | 11 | 6 | 0 | 18 (21) | |
| 2018 Spring | 53 | 37 | 11 | 0 | 0 | 19 (22) | |
| 2018 Fall | 71 | 24 | 5 | 0 | 0 | 21 (21) | |
| 2019 Spring | 53 | 42 | 0 | 5 | 0 | 19 (22) | |
| 2019 Fall | 79 | 21 | 0 | 0 | 0 | 19 (22) | |
| 2020 Spring | N/A (college-wide decision not to conduct STE) | ||||||
| (Half-Half) | |||||||
| 2020 Fall (Online) | 72 | 22 | 6 | 0 | 0 | 18 (22) | |
The sum of “Strongly Agree” and/or “Agree” scores to this STE question is plotted in Fig. 11, where scores of the six in-person offerings are plotted in blue, the score of the online teaching is in purple, and the average of all seven scores are in red. It can be seen that the STE score for online teaching (fall 2020) is slightly higher than the average.
Fig. 11
STE scores over seven semesters (six in -person vs. one online).
Table 11 shows the same data set prepared for the One-Way ANOVA analysis, which yielded a result of F (1, 127) =0.49, p = 0.484. This revealed there was not a statistically significant difference in students’ interests and motivation between the two groups of students (under in-person teaching and online). This further confirms students’ interest has been successfully maintained/sustained using our course design solution.
Table 11
Students’ STE Responses (On a 5-Point Likert-Type Scale)
| In-Person | Online | |||||
| 2017 | 2017 | 2018 | 2018 | 2019 | 2019 | 2020 |
| Spring | Fall | Spring | Fall | Spring | Fall | Fall |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| 4 | 5 | 5 | 5 | 5 | 5 | 5 |
| 4 | 5 | 4 | 5 | 4 | 5 | 5 |
| 4 | 5 | 4 | 5 | 4 | 5 | 5 |
| 4 | 4 | 4 | 5 | 4 | 5 | 5 |
| 4 | 4 | 4 | 5 | 4 | 5 | 4 |
| 3 | 4 | 4 | 5 | 4 | 5 | 4 |
| 3 | 4 | 4 | 4 | 4 | 4 | |
| 3 | 4 | 4 | 4 | 4 | 4 | |
| 1 | 3 | 4 | 4 | 4 | 3 | |
| 3 | 4 | 2 | 4 | |||
| 4 | ||||||
| 3 | ||||||
6.4Student satisfaction
For the three simulation projects, we collected students’ opinions, via zoom polls, on how each project helped them to understand and implement robotic control algorithms in the corresponding area (autonomous mobile robot, robotic manipulator, and more advanced robotic control). The survey question has four choices: “Absolutely help” (score 4), “Certainly help” (score 3), “Just help to some extent” (score 2), and “Does not help at all” (score 1). Figure 12 illustrates how the survey was conducted and how the results were collected on Zoom, using Project 2 as an example. Via zoom polling, the result is available instantly.
Fig. 12
Illustration of survey of student satisfaction of the simulation projects via zoom polling.
Out of 21 students, 21, 20, and 19 students participated in the survey of the first, second, and third project, respectively. Students’ responses are presented in Table 12.
Table 12
Survey results of students’ opinions about simulation projects (21 Students)
| Project # | # Of Students Participated | Score (4) | Score (3) | Score (2) | Score (1) | Satisfactory (Score 3 or 4) | Unsatisfactory (Score 1 or 2) |
| 1 | 21 | 29% | 52% | 19% | 0% | 81% | 19% |
| 2 | 20 | 30% | 55% | 15% | 0% | 85% | 15% |
| 3 | 19 | 26% | 42% | 27% | 5% | 68% | 32% |
Using 70% as the target for being “Satisfactory”, both Projects 1 and 2 meet students’ expectations very well. Specifically, more than 80% of the students thought these two projects were absolutely/certainly helpful. The MATLAB-ROS project (Project 3) falls slightly below the target of 70%. Considering the challenge of the MATLAB-ROS project, we think that the satisfaction rate of 68% is actually encouraging for bringing ROS into undergraduate robotic courses or curriculum.
The data presented in Secs. 6.3 and 6.4 are pertaining to students’ responses to one single question, i.e., if their interests and motivation were maintained, and if each of the simulation projects served its purpose. Though only one question was used in each case, the combined analyses of using different indicators agree with each other, thus supporting the conclusion of sustained students’ interests in a joint manner.
In the future, we will include assessment of students’ creative thinking capabilities (given in Sec. 6.2) and the satisfaction surveys (here in Sec. 6.4) as regular and routine activities to collect more data and evaluate our course design approach in the long term. We will also expand these satisfaction surveys to include more questions. For example, the question of asking “Does Project#1 help you understand path planning, navigation, sensor detection, map building, and basic motion control of autonomous mobile robots?” will be break down into several questions: 1) “Does Project#1 help you understand path planning?” 2) “help you understand navigation?”; 3) “help you understand sensor detection?”; 4) “help you understand map building?”; and 5) “help you understand basic motion control?”.
7Conclusion, discussion, and future investigation
This paper presents a course design example that balances between Workload and student’s Affect when transitioning an undergraduate robotic course from in-person teaching to online, as guided by the TASKS model. Workload denotes all course activities and assignments that students need to perform. Affect denotes students’ emotional aspects, such as motivation, stimulus, and attitude. The balancing between Workload and students’ Affect occurred in two stages. In the development stage, based on our past experience (that were positive), Workload was designed to resemble their in-person counterparts in both amount and difficulty. Making the amount of Workload reasonable avoids imposing extra stress on students. Maintaining the Workload to be similarly challenging helps to stimulate students’ interests. In the implementation stage, we closely monitored students’ responses. Adjustments were made whenever needed to ensure that the Workload was manageable to the students. To ensure students taking online classes perform as effectively as participants in conventional (pre-pandemic) classes, we utilized teaching practices to help achieve the goals of 1) sustaining student engagement using frequent online quizzes (during lectures) to draw students’ attention; 2) facilitating learning while minimizing anxiety by considering students’ Affect when designing online course activities; 3) encouraging faculty/student interactions by requiring mandatory screen-sharing to check students’ progress; and 4) maintaining consistency with pre-pandemic expectations by holding the same course standard as before.
When evaluating the course design and its implementation, we adopted mixed methods involving both quantitative and qualitative approaches. Comparisons of students’ learning performance (as measured by their grades of two exams and the assessment of their creative thinking capabilities) and their motivation and interests (as reflected by STE responses and satisfaction surveys) were conducted between (two) online and (six) past in-person semesters. The combined analyses using four indicators agree with each other, confirming that effective learning has been achieved remotely in maintaining students’ learning performance and motivation.
In our current solution, concepts of hardware-related activities (assembly, sensors and actuators, and integration between two hardware devices) were illustrated using simulated robots. Future research will investigate suitable hybrid/blended/mixed scheduling strategies to bring back some of these activities (Dziuban et al., 2018). In the future, we will also include assessment of students’ creative thinking capabilities and satisfaction surveys as routine activities to continue collecting data and evaluate our course design approaches in the long term. More questions will be added into the surveys to improve the rigourosity of the study.
References
1 | Akintewe, O. (2021). Remote Teaching Robotics Design Project. ASEE Annual Conference & Exposition. |
2 | Alimisis, D. ((2013) ) Educational Robotics: Open Questions and New Challenges, Themes in Science and Technology Education 6: (1), 63–71. |
3 | Altin, H. , Pedaste, M. ((2013) ) Learning Approaches to Applying Robotics in Science Education, Journal of Baltic Science Education 12: (3). |
4 | Avanzato, R. (2020). Development of a MATLAB/ROS Interface to a Low-Cost Robot Arm. AEEE Annual Conference and Exposition. |
5 | Avanzato, R. , Wilcox, C. (2020). Introductory Mobile Robotics and Computer Vision Laboratories using ROS and MATLAB. ASEE Annual Conference and Exposition. |
6 | Beiley, J. (2018). Exploring an Inquiry-Based Learning with Peer-teaching Pedagogy in a Phys-iological Signals Lab Course. ASEE Annual Conference & Exposition. UT. |
7 | Benitez, V. H. , Symonds, R. , Elguezabal, D. E. ((2020) ) Design of an Affordable IoT Open-Source Robot Arm for Online Teaching of Robotics Courses during the Pandemic Contingency, HardwareX 8: , e00158. |
8 | Bielefeldt, A. , Morse, A. (2019). Teaching and Assessment of Innovation and Creativity in Civil Engineering: Why? How? Now! Annual Conference and Exposition. |
9 | Birk, A. , Dineva, E. , Maurelli, F. , Nabor, A. ((2021) ) A Robotics Course during COVID-: Lessons Learned and Best Practices for Online Teaching beyond the Pandemic, Robotics 10: (5), 19. |
10 | Bote-Vericad, J.-J. ((2021) ) Challenges for theEducational System During Lockdowns: A Possible new Framework forTeaching and Learning for the Near Future, Education forInformation,, 31: , 149–153. |
11 | Bote-Vericad, J.-J. ((2021) ) Perceived Barriers for Distance Teachingin Higher Education during the COVID- Crisis: “I Never Did aVideo Before”, Education for Information 37: , 377–397. |
12 | Boysen, G. A. ((2015) ) Uses and Misuses of Student Evaluations of Teaching: The Interpretation of Differences in Teaching Evaluation Means Irrespective of Statistical Information, Teaching of Psychology 42: (2), 109–118. |
13 | Canas, J. M. , Perdices, E. , Garcia-Perez, L. , Fernandez-Conde, J. ((2020) ) A ROS-Based Open Tool for Intelligent Robotics Education, Applied Sciences 10: (21). |
14 | Conefrey, T. , Smyth, D. S. ((2021) ) High-Impact Practices for Transforming Online Learning, Journal of Higher Education Theory&Practice 21: (6), 79–91. |
15 | Corke, P. , (n.d.). Robotics Toolbox. Retrieved from https://petercorke.com/toolboxes/robotics-toolbox/. |
16 | Das, A. K. , Nguyen, Q. T. , Nguyen, A. T. , Nomikoudis, M. , Van, D. H. ((2019) ) Course Redesign to Incorporate Flipped Delivery: A Business Degree Case in Vietnam, Issues in Educational Research 29: (2). |
17 | Davis, J. L. , McDonald, T. , Kicklighter, B. L. (2021). Online Testing with Blackboard: Lessons Learned (Perspectives from Three Engineering Faculty). ASEE Annual Conference. |
18 | Dey, U. , S, K. C. (2020). AWeb-Based Integrated GUI for 3D Modeling, Kinematic Study, and Control of Robotic Manipulators. Comput Appl Eng Educ. doi:10.1002/cae.22282. |
19 | Dhawan, S. ((2020) ) Online Learning: A Panacea in the Time of COVID- Crisis, Journal of Educational Technology Systems 49: (1), 5–22. |
20 | Dunlap, J. C. , Verma, G. , Johnson, H. L. ((2016) ) Presence+Experience: A Framework for the Purposeful Design of Presence in Online Courses, Tech Trends 60: , 145–151. |
21 | Dziuban, C. , Graham, C. , Moskal, P. , Norberg, A. , Sicilia, N. ((2018) ) Blended Learning: the New Normal and Emerging Technologies, International Journal of Educational Technology in Higher Education 15: (3). |
22 | Edwards, R. , Recktenwald, G. (2008). Guided Inquiry in an Engineering Technology Classroom. ASEE Annual Conference & Exposition. |
23 | Eschenbach, E. , Cashman, E. (2004). Teaming In Freshman Design Using A Studio Teaching Approach And Blackboard®. ASEE Annual Conference & Exposition. |
24 | Eteokleous, N. , Nisiforou, E. , Christodoulou, C. (2020). Creativity Thinking Skills Promoted Through Educational Robotics. In Advances in Intelligent Systems and Computing. |
25 | Fini, E. H. , Awadallah, F. , Parast, M. M. , Abu-Lebdeh, T. ((2018) ) The Impact of Project-Based Learning on Improving Student Learning Outcomes of Sustainability Concepts in Transportation Engineering Courses, European Journal of Engineering Education 43: (3). |
26 | Garcia-Vedrenne, A. E. , Orland, C. , Ballare, K. M. , Shapiro, B. , Wayne, R. K. (2020). Ten Strategies for a Successful Transition to RemoteLearning: Lessons Learned with a Flipped Course. Academic Practice in Ecology and Evolution. doi:10.1002/ece3.6760. |
27 | Gennert, M. A. , Lotfi, N. , Mynderse, J. A. , Jethwani, M. , Kapila, V. (2020). Workshops for Building the Mechatronics and Robotics Engineering Education Community. ASEE Annual Conference & Exposition. |
28 | Gibeault, S. , Lorio, J. , Santillan, J. , Shen, H. , Tufenkjian, M. (2019). Practical Application of Robotics Competition for STEM Education. ASEE Annual Conference & Exposition. |
29 | Gillis, A. , Krull, L. M. ((2020) ) COVID- Remote Learning Transition in Spring: Class Structures, Student Perceptions, and Inequality in College Courses, Teaching Sociology 48: (4), 283–299. |
30 | Guilford, W. H. , Helmke, B. P. (2017). Summative versus Formative Assessments in Teaching Physiology to Biomedical Engineering Students: A Comparison of Outcomes. ASEE Annual Conference & Exposition. |
31 | Havenga, M. (2020). COVID-19: Transition to Online Problem-based Learning in Robotics – Challenges, Opportunities and Insights. International Conference on Active Learning in Engineering Education. Pathumthani, Thailand. |
32 | Hew, K. F. , Jia, C. , Gonda, D. E. , Bai, S. (2020). Transitioning to the “New Normal“ of Learning in Unpredictable Times: Pedagogical Practices and Learning Performance in Fully Online Flipped Classrooms. International Journal of Educational Technology in Higher Education. |
33 | Khan, K. (2017). ROS-Based Control of a Manipulator Arm for Balancing a Ball on a Plate. ASEE Annual Conferences and Exposition. |
34 | Khan, M. S. , Wu, N. , Ibrahim, M. (2018). A Study on Measuring Self-efficacy in Engineering Modeling and Design Courses. ASEE Annual Conference & Exposition. Salt Lake City, UT. |
35 | Khan, S. , Jaffery, M. H. , Hanif, A. , Asif, M. R. ((2017) ) Teaching Tool for a Control Systems Laboratory Using a Quadrotor as a Plant in MATLAB, IEEE Transactions on Education 60: (4), 249–256. |
36 | Khan, Z. R. , Sivasubramaniam, S. , Anand, P. , Hysaj, A. ((2021) ) E’-thinking Teaching and Assessment to Uphold Academic Integrity: Lessons Learned From Emergency Distance Learning, International Journal for Educational Integrity 17: (1), 1–27. |
37 | Koh, C. , Tan, H. S. , Tan, K. C. , Fang, L. , Fong, F. M. , Kan, D. , Wee M. L. (2010). Investigating the Effect of 3D Simulation-Based Learning on the Motivation and Performance of Engineering Studies. Journal of Engineering Education. |
38 | Kumpass-Lenk, K. , Eisenschmidt, E. , Veispak, A. ((2018) ) Does the Design of Learning Outcomes Matter from Students’ Perspective? Studies in Educational Evaluation 59: , 179–186. |
39 | Kuo, H.-C. , Yang, Y.-T. C. , Chen, J.-S. , Hou, T.-W. , Ho, M.-T. (2021). The Impact of Design Thinking PBL Robot Course on College Students’ Learning Motivation and Creative Thinking. IEEE Transactions on Education. doi: 10.1109/TE.2021.3098295. |
40 | Lehman, W. , Hayder, M. (2015). Introducing Kinematics with Robot Operating System (ROS). ASEE Annual Conference and Exposition. |
41 | Little-Wiles, J. M. , Naimi, L. L. (2011). An Examination of Faculty Perceptions and Use of Blackboard Learning Management System. ASEE Annual Conference & Exposition. |
42 | Ma, L. (2021). Teaching Undergraduate Robotic Course using VEX Robots. Journal of STEM Education, Research, and Innovation. |
43 | Ma, L. , Alborati, M. (2018). Enhancement of a VEX Robot with an Onboard Vision System. International Conference on Engineering Education. Malaysia. |
44 | Ma, L. , Ma, G. (2019). Modeling and Control of a VEX Robot. Hawaii International Conference on Education. |
45 | Ma, L. , Alamo, J. M. , Wang, Y. (2021). Assessment of Creative Thinking in an Introduction Robotic Course using Final Project. ASEE Annual Conference and Exposition. |
46 | Ma, L. , Alborati, M. , Song, J. (2019). Inclusion of a Visual Servoing Project into an Undergraduate Robotic Course. International Conference on Engineering Education. Japan. |
47 | Malik, M. , Fatima, G. , Ch, A. H. , Sarwar, A. ((2017) ) E-Learning: Students’ Perspectives about Asynchronous and Synchronous Resources at Higher Education Level, Bulletin of Education and Research 39: (2), 183–195. |
48 | Maxwell, B. A. , Meeden, L. A. ((2000) ) Integrating Robotics Research with Undergraduate Education, IEEE Intelligent Systems and their Applications 15: (6). |
49 | McDowell, T. R. , Schmittzehe, E. T. , Duerden, A. J. , Cernusca, D. , Collier, H. , Woelk, K. ((2019) ) A Student-Choice Model to Address Diverse Needs and Promote Active Learning, Journal of Science Education and Technology 28: , 321–328. |
50 | Nguyen, T. A. , Zeng, Y. ((2012) ) A Theoretical Model of Design Creativity: Nonlinear Design Dynamics and Mental Stress-Creativity Relation, Journal of Integrated Design and Process Science 16: (3), 65–88. |
51 | Pantazidou, M. , Kandris, K. ((2020) ) Examples of Applying Research-Based Learning Principles to the Redesign of an Environmental Geotechnics Course, International Journal of Engineering Pedagogy 10: (1), 31–50. |
52 | Prince, M. , Felder, R. , Prent, R. ((2020) ) Active Student Engagement in Online STEM Classes: Approaches and Recommendations, Advances in Engineering Education 8: (4). |
53 | Ruano, I. R. , Estevez E. E. , Garcia, A. S. , Garcia, J. G. , Ortega, J. G. (2020). Integration of Online Laboratories in Learning Platforms. XIV Technologies Applied to Electronics Teaching Conference (TAEE). |
54 | Saleheen, F. , Giorgi, S. , Smith, Z. T. , Picone, J. , Won, C.-H. (2015). Design and Evaluation of a Web-based Virtual Open Laboratory Teaching Assistant (VOLTA) for Circuits Laboratory. ASEE Annual Conference & Exposition. Seattle, WA. |
55 | Sawatzki, C. G. , Muraleedharan, R. (2021).Work in Progress: Using Cost-effective Educational Robotics Kits in Engineering Education. ASEE Annual Conference. |
56 | Shekar, A. (2014). Project Based Learning in Engineering Design Education: Sharing Best Practices. ASEE Annual Conference&Education. Indianapolis, IN.. |
57 | Siegwart, R. , Nourbakhsh, I. , Scaramuzza, D. (2004). Introduction to Autonomous Mobile Robot. MIT Press.. |
58 | Soicher, R. N. , Becker-Blease, K. A. , Bostwick, K. ((2020) ) Adapting Implementation Science for Higher Education Research: the Systematic Study of Implementing Evidence-Based Practices in College Classrooms, Cognitive Research: Principles and Implications 5: (54). |
59 | Spong, M. , Hutchinson, S. , Vidyasagar, M. (2020). Robot Modeling and Control. Wiley. |
60 | Tang, B. , Guo, S. , Yeboah, M. , Wang, Z. , Cheng, S. ((2021) ) Quality Evaluation of Online Courses During COVID- Pandemic Based on Integrated FCE-AHP Method, Journal of Intelligent&Fuzzy Systems 41: , 1487–1498. |
61 | Teo, O. E. ((2020) ) Challenges for Conducting the Online Assessment for A Large Class in Engineering Mechanics, Advances in Engineering Education 8: (4). |
62 | Wan, T. ((2021) ) Predictive Analytics for the KMAP-O Model in Design and Evaluation of Diabetes care Management Research, Health Services Research and Managerial Epidemiology 8: , 1–4. |
63 | Wan, T. , Terry, A. , McKee, B. , Kattan, W. ((2017) ) KMAP-O Framework for Care Management Research of Patients With Type Diabetes, World Journal of Diabetes 8: (4), 165–171. |
64 | Wilkerson, S. , Forsyth, J. , Sperbeck, C. , Jones, M. , Lynn, P. (2017). A Student Project using Robotic Operating System (ROS) for Undergraduate Research. ASEE Annual Conference and Exposition. |
65 | Wong, J. (2021). Transition of Instructional Methods from an In-Person to Online Course and the Lessons Learned. ASEE Annual Conference & Exposition.. |
66 | Wu, L. L. , Zhu, E. , Callaghan, C. , Irwin, D. , Reinsdorf, D. , Swanson, V. , Reinkensmeyer D. ((2020) ) Rapidly Converting a Project-Based Engineering Experience for Remote Learning: Successes and Limitations of Using Experimental Kits and a Multiplayer Online Game, Advances in Engineering Education 8: (4). |
67 | Yadav, A. , Subedi, D. , Lundeberg, M. A. , Bunting, C. F. ((2011) ) Problem-based Learning: Influence on Students’ Learning in an Electrical Engineering Course, Journal of Engineering Education 100: (2), 253–280. |
68 | Yang, J. , Yang, L. , Quan, H. , Zeng, Y. ((2021) ) Implementation Barriers: A TASKS Framework, Journal of Integrated Design and Process Science 25: . |
69 | Yerkes, R. M. , Dodson, J. D. ((1908) ) The Relation of Strength of Stimulus to Rapidity of Habit-Formation, Journal of Comparative Neurology and Psychology 18: , 459–482. |
Author Biographies
Lili Ma received her Ph.D. in Electrical Engineering from Utah State University focusing on autonomous ground vehicles in 2004. After that she did three-year post-doctoral training at Virginia Tech working with autonomous aerial vehicles. Prior to joining the department of Computer Engineering Technology at CUNY-New York City College of Technology in 2016, she taught at Wentworth Institute of Technology for many years. Her research areas include autonomous robots, vision-based control, visual servoing, visual tracking, coordinated control, sensing and perception techniques.
Yu Wang is an Associate Professor and the Program Coordinator in the department of Computer Engineering Technology at CUNY-New York City College of Technology. She received her Ph.D. in Electrical Engineering from the Graduate Center of City University of New York (CUNY). Her research areas of interest are in engineering education, biomedical sensors, optoelectronics, modelling real-time systems, embedded system design, deep neural network, and machine learning.
Chen Xu is an Associate Professor in the department of Computer Engineering Technology at CUNY-New York City College of Technology. She received her Ph.D. degree in Biomedical Engineering from University of Connecticut. Her research areas of interest are in biomedical sensors and instrumentation, image processing, signal processing, and non-invasive medical tests.
Xiaohai Li received his M.S. from Polytechnic Institute of NYU in 2004 and Ph.D. in Electrical Engineering from the Graduate Center of City University of New York (CUNY) in 2009. He worked as a Post-doc in the PRISM Research Center at City College of CUNY in 2010. He is currently an Associate Professor in the department of Computer Engineering Technology at CUNY-New York City College of Technology, where he founded the Robotics Research Lab. His current research interests include control systems, robotics, swarms, computer vision, perceptual computing, and IoT/IoRT.
