Blending a sweet pill to swallow with TRIZ and industry talks for enhanced learning during the COVID-19 pandemic
Abstract
BACKGROUND:
While studies have investigated relationships among learning motivation, social presence, and cognitive presence, there appear to be no studies on the inclusion of industry talks and the theory of inventive problem-solving (TRIZ) in strengthening engineering students’ learning motivation, social presence, and cognitive presence within a blended learning setting.
OBJECTIVE:
This study investigated the influence of industry talks and TRIZ on learning motivation, social presence, and cognitive presence in a blended learning environment.
METHODS:
Data samples were obtained from 98 engineering students in a blended learning course and analysed using Spearman’s correlation test, regression, ANOVA, and t-test.
RESULTS:
Findings suggested that TRIZ and industry talks strongly, positively, and significantly correlated with learning motivation, social presence, and cognitive presence. A well-rounded learning experience compounded of TRIZ and industry talks significantly affected learning motivation, social presence, and cognitive presence, thereby enhancing students’ programme outcome (PO) achievement.
CONCLUSIONS:
These findings can be attributed to the students’ independent learning capabilities with TRIZ and industry talks. Analogically, embracing TRIZ and industry talks helps turn blended learning into a “sweet instead of bitter pill to swallow” for engineering students in the face of the COVID-19 pandemic.
Poh Kiat Ng, PhD is an Associate Professor and Deputy Dean at the Faculty of Engineering and Technology, Multimedia University, Malacca, Malaysia. He has been working as an academician for over 13 years. Prior to academia, he was a manufacturing and process engineer in a few multinational manufacturing firms. His research interests include Ergonomics, TRIZ, Usability Testing, Biomedical Engineering, and Engineering Education.
Voon Chet Koo, PhD is a Professor at the Faculty of Engineering and Technology, Multimedia University, Malacca, Malaysia. He is also the Chairperson of the Centre of Remote Sensing and Surveillance Technologies with Multimedia University, the Chairperson of the IEEE Malaysia Section Geoscience and Remote Sensing Society Chapter, and a fellow of the ASEAN Academy of Engineering and Technology (AAET). His research interests include Radar Technology, Remote Sensing, Embedded Systems, and Engineering Education.
Yu Jin Ng, PhD is a Senior Lecturer at the Department of Social Science and Humanities, College of Energy Economics and Social Sciences, Universiti Tenaga Nasional (UNITEN), Putrajaya Campus, Malaysia. He majored in Electrical and Electronics Engineering before switching to education, and even social sciences to pursue his passion in academia. His research interests include Corpus-Based Studies, Vocabulary Pedagogy, Language Education, and Teaching and Learning.
Jian Ai Yeow, PhD is a Lecturer at the Faculty of Business, Multimedia University, Malacca, Malaysia. She has been teaching and supervising undergraduate students mainly in the area of International Business and Human Error (Operations Management). Her research areas include Human Error, Ergonomics, Safety and Health, Operation Management, International Business, and Teaching and Learning.
1Introduction
The outbreak of the COVID-19 pandemic severely impacted the emotional functioning and motivational quality of many students [1]. This pandemic has somewhat forced a change in the landscape of conventional teaching and learning methods in higher education. Many higher learning institutions around the world were forced to either fully or partially migrate their physical classrooms to virtual learning environments. Consequently, the use of blended learning and digital education tools slowly became a staple in teaching and learning practice that many educators around the globe had to embrace [2–6].
Researchers have reported challenges in implementing blended learning from the perspectives of both students and educators. Some of these challenges include technical challenges, organisational challenges, instructional design challenges, self-regulation challenges, challenges in using technology for teaching and learning, training support challenges, increased class preparation time, and fairness in online grading [7–10]. If educators were compelled to use blended learning for emergency remote teaching whilst confronting its challenges in the face of the COVID-19 pandemic, then blended learning could turn out to be a “bitter pill to swallow” for both learners and educators.
Studies have dived into the constructs of learning motivation, social presence, and cognitive presence, and their impacts on students’ learning performance in a blended learning environment [11–13]. Researchers have also investigated the success of a unique problem-solving method known as the theory of inventive problem solving (TRIZ) in aspects of education such as entrepreneurship education, industrial training, and engineering design [14–17]. Studies found that the educational experience of students also improves with more frequent industry talks and engagement [18, 19].
However, there is still a need to study how the inclusion of industry talks and TRIZ correlates with engineering students’ learning motivation, cognitive presence, and social presence during the COVID-19 pandemic. To understand this relationship, the following research question is proposed:
How are TRIZ and industry talks related to the learning motivation, social presence, and cognitive presence of engineering students in a blended learning setting?
Therefore, it is of interest for this study to investigate the impacts of TRIZ and industry talks on learning motivation, social presence, and cognitive presence in the blended learning environment of engineering students. The study begins with a review of literature on the relevant variables and development of the hypotheses. The research and data collection methods are then described, followed by the data analyses, results, and discussion. Finally, the conclusions are drawn together with the directions for future research.
2Literature and hypotheses development
2.1Learning motivation
According to Lin et al. [20], learning motivation is a mediator between stimulus and response, where learners have different knowledge acquisition needs due to their distinct and individual opinions. Learning motivation is a crucial enabler for academic success [21–23] and learning performance [11, 21].
Learning motivation can be classified as intrinsic and extrinsic. Studies on blended learning often suggest that intrinsic motivation is more effective in learning as students who are intrinsically motivated tend to perform better and complete assignments faster than extrinsically motivated students [11, 24, 25]. However, research has also shown that long video lectures for online learning during the COVID-19 pandemic increased the boredom in learning, and reduced students’ motivation to learn [26].
In engineering education, intrinsic factors such as individual attitudes and expectations can prove to have a more dominant motivating effect in contrast to extrinsic factors such as pulling forces, learning approach, and peer pressure [27]. Yacob and Saman [28] assert that both intrinsic and extrinsic factors contribute in motivating engineering students to learn better. They point out that the intrinsic factors include individual attitude, expectation, and challenging goals, while the extrinsic factors include clear direction, reward and recognition, punishment and social pressure, and competition.
2.2Social presence
Social presence refers to the ability of students in relating, communicating, and forming productive and personal relationships with their peers in the classroom [29]. It is a concept that can be used to examine the social interaction quality of a virtual learning setting [30]. According to Whiteside et al. [31], it is advantageous for institutions to support the integration of social presence into the learning environment as it enriches student learning experience. Several studies also concur that social presence improves student learning satisfaction and performance [30, 32–34].
In the context of students who are new to distance learning, So and Brush [35] suggested that the instructional scaffolding and modelling of social presence behaviours might be required. Beginners of online learning should be assisted not only in establishing and maintaining social presence, but also in reading and making sense of the social presence of others [36]. Instructors could also model strong social presence expectations via teacher immediacy skills when planning online course assessments and activities [32].
2.3Cognitive presence
Cognitive presence is defined as the degree of which online learners can construct and validate meaning according to continuous and critical thinking and communication [37–39]. Studies have found that cognitive presence reinforces critical thinking, evidence-based thinking, knowledge construction and learning experience [40–42]. Aside from teaching and social presences, cognitive presence was found to be the most powerful predictor for perceived learning and satisfaction in soft-pure, soft-applied, hard-pure and hard-applied academic disciplines [12]. Cognitive presence is positively affected by externally-induced motivation through grades and externally-facilitated regulation scaffolding techniques [43].
In the matter of cognitive presence development, Kovanović et al. [44] suggested that quality is more important than quantity when it comes to learning activities as highly metacognitively skilled learners can be just as successful as highly engaged learners. Hence, the lack of utilisation in learning management tools or systems would not necessarily allude to poor cognitive development. Other researchers pointed out that increasing cognitive presence among learners could automatically lead to the increase in their social presence to an extent [45]. In another perspective, online instructions could also emphasise on using social presence as a catalyst for enhanced cognitive presence [46].
2.4TRIZ
TRIZ is a Russian acronym which stands for the theory of inventive problem-solving [47, 48]. It is an analysis, forecasting and problem-solving approach based on logic as opposed to spontaneous creativity which emerged from the investigation of invention patterns in the global patent literature [49–51].
TRIZ improves self-efficacy which is a crucial component in the long-term development of both problem finding and problem-solving abilities [52, 53]. Belski et al. [54] found that engineering students developed significantly better problem-solving skills when learning TRIZ as contrasted with learning other engineering modules in their four-year programme. TRIZ not only positively affects learners’ problem analysis skills, but also improves their creativity and abilities in solution identification, selection, and implementation [16].
TRIZ can be used as a foundation for learners’ knowledge systematisation which is a didactic aspect involving concepts such as system, systemness, systems approach, and system development rules [14]. It is important for educators to harmonise current teaching arrangements with systematic thinking tools such as TRIZ, especially in new curriculums which aim to develop learners’ thinking abilities [55].
2.5Industry talks in higher learning institutions
In principle, industry talks organised in higher learning institutions are lectures or sharing sessions conducted by industry experts for the purpose of improving students’ exposure to the real-world practice. In an industry talk, the industry expert usually discusses important topics of influence over their business or ideas from their professional experience [56]. Students would also gain insights into their future workplace requirements by networking and interacting with the industry expert, and work towards developing applicable skills [57].
In Malaysian engineering programmes, the Engineering Accreditation Council [58] considers industry talks as key components of exposure to engineering practice that should be integrated throughout the engineering curriculum. The exposure to practical experiences via industry talks helps students appreciate their learning and go beyond academics through insights, practical knowledge, applications, and a critical mindset [59]. Industry talks also help students master the essential soft skills in graduate attributes such as responsibility and ethics, lifelong learning, and knowledge on the current issues [60].
2.6Hypotheses development
Numerous researchers have contended that the systematic TRIZ approach and exposure to industry talks can significantly enhance students’ learning, cognitive abilities, and communication skills [14, 16, 54, 55, 57, 59]. Nonetheless, the use of TRIZ and industry talks has never been observed or studied in a blended learning environment. Furthermore, while the relationships among learning motivation, cognitive presence, and social presence have been carefully studied [12, 61], the investigation of their relationships with TRIZ and industry talks have remained unresearched. Owing to the effectiveness of TRIZ in harnessing creativity and inventiveness in teaching and learning, and higher education [62–64], and success of industry engagement activities in engineering education programmes, it would be of interest to investigate how TRIZ and industry talks can be related with the learning motivation, social presence, and cognitive presence of engineering students in a blended learning environment [65–67]. However, as of now, there has yet to be a study that explored such a relationship. Hence, the following hypotheses are proposed:
H1. There is a positive and significant correlation between TRIZ and learning motivation.
H2. There is a positive and significant correlation between TRIZ and social presence.
H3. There is a positive and significant correlation between TRIZ and cognitive presence.
H4. There is a positive and significant correlation between industry talks and learning motivation.
H5. There is a positive and significant correlation between industry talks and social presence.
H6. There is a positive and significant correlation between industry talks and cognitive presence.
H7. Learning motivation, social presence, and cognitive presence are significantly influenced by TRIZ and industry talks.
3Methods
3.1Measurement instrument
The hypotheses were tested using a quantitative survey. Most of the survey items pertaining to learning motivation, cognitive presence, and social presence were built upon from the studies of Lim and Richardson [12], Law et al. [11], Kozan [46], and Kozan and Richardson [45]. Items related to industry talks were based on a study from Markom et al. [59] and a specific topic of the course known as intellectual property law. The items on TRIZ were based upon literature studies from Berdonosov [14], and Greenberg [55], and specific TRIZ tools known as function analysis and cause-and-effect-chain (CEC) analysis.
The full list of survey items can be found in the Appendix. The first section comprises the participants’ demographic profile, which includes age, gender, programme and year of study. The second section includes the items classified under learning motivation (LM), social presence (SP), cognitive presence (CP), TRIZ, and industry talks (IND).
A seven-point Likert scale was administered for this study as it is more accurate, more reliable, easier to use, and gives a better reflection of the respondent’s true evaluation in contrast to other Likert scales [68–70]. The internal consistency reliability of the survey instrument was evaluated through the Cronbach’s alpha coefficient. From Table 1, the alpha values suggested that the internal consistency of the survey instrument was high (α> 0.7) for all the variables [71, 72].
Table 1
No. | Variable | Cronbach’s Alpha (α) | No. of Items |
1 | Learning Motivation (LM) | 0.950 | 7 |
2 | Social Presence (SP) | 0.935 | 4 |
3 | Cognitive Presence (CP) | 0.953 | 6 |
4 | TRIZ | 0.962 | 4 |
5 | Industry Talks (IND) | 0.958 | 4 |
A factor analysis was done as well. From Table 2, the factor loading for every item of each variable exceeded 0.6, signifying that each factor extracts an acceptable amount of variance from the variable [73]. To avoid social desirability effects and common method bias, the survey was administered anonymously and online. To assure face validity, the survey was vetted by 2 education experts from different higher learning institutions who are familiar with engineering education, industry engagement activities, and TRIZ.
Table 2
Items | Factor Loadings | ||||
Learning | Social | Cognitive | TRIZ | Industry | |
Motivation (LM) | Presence (SP) | Presence (CP) | Talks (IND) | ||
LM1 | 0.818 | ||||
LM2 | 0.931 | ||||
LM3 | 0.913 | ||||
LM4 | 0.925 | ||||
LM5 | 0.904 | ||||
LM6 | 0.810 | ||||
LM7 | 0.891 | ||||
SP1 | 0.912 | ||||
SP2 | 0.892 | ||||
SP3 | 0.942 | ||||
SP4 | 0.924 | ||||
CP1 | 0.918 | ||||
CP2 | 0.885 | ||||
CP3 | 0.867 | ||||
CP4 | 0.910 | ||||
CP5 | 0.900 | ||||
CP6 | 0.921 | ||||
TRIZ1 | 0.946 | ||||
TRIZ2 | 0.945 | ||||
TRIZ3 | 0.968 | ||||
TRIZ4 | 0.931 | ||||
IND1 | 0.943 | ||||
IND2 | 0.915 | ||||
IND3 | 0.967 | ||||
IND4 | 0.942 |
3.2Participants and setting
The survey was shared with all the 98 students who enrolled in a blended learning course on engineering law. The response rate was 100%. The students came from 3 different engineering programmes, namely mechanical engineering, robotics and automation engineering, and telecommunications engineering.
The engineering law course consisted of topics such as introduction to law, contract law, tort, cyber law, intellectual property law, employment legislation, and laws pertaining to engineers. Due to the COVID-19 pandemic and movement control order in Malaysia, the lectures were facilitated online synchronously via Google Meet. Apart from the 40 hours of online synchronous lectures, there were 7 hours of asynchronous learning in the form of pre-recorded videos, infographics, articles, and notes for students to engage with before the lectures.
Digital education tools such as Padlet, Google Docs, and Quizziz were used during lectures to reinforce students’ understanding on the topics, and better prepare them for their group assignments. TRIZ tools such as function and CEC analyses were used alongside these digital tools to help students systematically break down and analyse cases and complex issues pertaining to engineering law. For the topic on intellectual property law, an industry expert from an intellectual property consulting firm in Malaysia was invited to a lecture session.
3.3TRIZ integration into teaching and learning
To spark the interest in learning topics on law among engineering students, a TRIZ technique known as function analysis was used. Students were asked to link the functions of every entity within the Malaysian legal system in a Padlet setting. This technique was used to improve the visualisation and memorising capability of the engineering students about the legal system. Figure 1 shows the Padlet activity. All students were required to participate in completing the function model with the guidance of the instructor.
Fig. 1
The students were also guided on how to formulate engineering contradictions for contract law case studies. One of the cases was adopted from Sobri [74]. The students were requested to type their contradiction statements about the case in a Padlet setting. The technique used in formulating the engineering contradiction was the If –Then –But technique (See Fig. 2). This technique was used so that the students are able to relate better with the details of the case study.
Fig. 2
Another TRIZ technique known as the cause-and-effect-chain (CEC) analysis was also taught to students under the topic of employment legislation. The case discussed in Padlet involved the reasoning behind an employee’s surprise to an employment termination. The students were required to ask the question “why, and why else?” until an acceptable root cause is ascertained (See Fig. 3). This technique was used to stimulate the critical thinking ability of students in employment law cases.
Fig. 3
The CEC analysis was also used in other case studies on the laws pertaining to engineers in Malaysia. In the example, the students were required to ascertain the root cause behind the inability of an engineering graduate, who studied a 3-year bachelor of engineering programme overseas, in registering as a professional engineer with a practising certificate in Malaysia (See Fig. 4). The TRIZ technique was used to help the engineering students better understand the requirements to register as a professional engineer and graduate engineer in Malaysia.
Fig. 4
3.4Industry talk integration into teaching and learning
The industry talk was integrated into a topic on intellectual property law. A CEO of an intellectual property consulting group in Malaysia was invited to deliver a 2-hour industry talk on “owning and monetising engineering ideas with intellectual property rights”. The CEO is a patent, trademark and industrial design attorney, a lawyer, and an angel investor. The talk covered how engineers could protect their innovation with the right type of IP, an overview of different categories of intellectual property rights, requirements of patents and patentability, and commercialisation of university intellectual property rights. An intellectual property assignment was created based on the industry talk. The assignment describes a company that is suing another company for a patent infringement. The sued company intends to invalidate the patent. For this intellectual property assignment, the students were required to give their expert opinions on the validity or invalidity of the patent.
3.5Survey analyses methods
For the hypotheses testing, Spearman’s correlation test was used to verify H1 to H6, while general regression and ANOVA were used to verify H7. These analyses methods were applied using IBM SPSS Statistics version 21.
A two-sample t-test was also done to compare the programme outcome (PO) achievement from the current year of study (2020) with the previous year (2019). The PO for this course was based on the one stipulated in Malaysia’s Engineering Accreditation Council standard, which was to “apply reasoning informed by contextual knowledge to assess societal, health, safety, legal and cultural issues, and the consequent responsibilities relevant to professional engineering practice, and solutions to complex engineering problems” [58]. In contrast to the method used in 2020, the teaching and learning method used in 2019 was the traditional brick-and-mortar method with little to no administration of blended learning or other creative approaches (such as TRIZ or industry talks) to potentially enhance learning performance.
4Results and discussion
4.1Participant demographics
According to the demographics reported in Table 3, the participants were mostly 20 to 23 years of age (about 90%). Males accounted for almost 89%, while females comprised only about 11% of the total participants. Most of the participants were from the mechanical engineering programme (about 58%). Approximately 23% of them came from the robotics and automation engineering programme, while around 18% originated from the telecommunications engineering programme. The majority of them (around 79%) were third year students.
Table 3
No. | Items | Levels (with scale) | No. of Participants | Mean Score | StDev |
1 | Age | 2:18-19 years old | 1 (1.020%) | 3.520 | 0.802 |
3:20-21 years old | 58 (59.184%) | ||||
4:22-23 years old | 30 (30.612%) | ||||
5:24-25 years old | 6 (6.122%) | ||||
6:26-27 years old | 2 (2.041%) | ||||
7: > 27 years old | 1 (1.020%) | ||||
2 | Gender | 1: Male | 87 (88.776%) | 1.112 | 0.317 |
2: Female | 11 (11.224%) | ||||
3 | Engineering Programme | 1: Mechanical | 57 (58.163%) | 1.602 | 0.783 |
2: Robotics and automation | 23 (23.469%) | ||||
3: Telecommunications | 18 (18.367%) | ||||
4 | Year of Study | 2: Second year | 16 (16.327%) | 2.888 | 0.451 |
3: Third year | 77 (78.571%) | ||||
4: Fourth year | 5 (5.102%) |
Notes: Total number of participants, N = 98.
4.2Spearman’s correlation
The data for the dependent and independent variables were not normally distributed. Therefore, Spearman’s correlation test was used to assess the strength and direction of association existing among the variables according to the hypotheses. Table 4 summarises the results of the Spearman’s correlation (r s) among the variables. The results showed that all variables positively and significantly correlated with each other based on the hypotheses (rs > 0, p < 0.001).
Table 4
Hypotheses | Variable Relationships | Correlations (rs) |
H1 | TRIZ and LM | 0.765*** |
H2 | TRIZ and SP | 0.794*** |
H3 | TRIZ and CP | 0.837*** |
H4 | IND and LM | 0.750*** |
H5 | IND and SP | 0.742*** |
H6 | IND and CP | 0.820*** |
Notes: N = 98; ***p < 0.001.
It was found that TRIZ correlated strongly with learning motivation (rs = 0.765, p < 0.001), social presence (rs = 0.794, p < 0.001), and cognitive presence (rs = 0.837, p < 0.001). TRIZ is known to be an approach that aids users to be independent learners, independent problem solvers, and creative thinkers [14, 15, 75, 76]. In the present study, the learners applied specific algorithms and rules derived from the TRIZ approach to work on their problems, cases, and assignments. The systematic approach may have resulted in students being more independent, thereby reducing their need to be more social or interactive when solving class or assignment problems. Hence, TRIZ correlated at a slightly higher magnitude with cognitive presence as contrasted with other variables due to the students’ self-directed and independent learning abilities. The correlations were still considered strong and significant. Hence, H1, H2, and H3 were not rejected.
While TRIZ encourages the ability for self-directed learning and strengthens motivation among learners [75], industry talks expose them to social responsibility and accountability in teams [59]. In the present study, industry talks correlated strongly with learning motivation (rs = 0.750, p < 0.001), social presence (rs = 0.742, p < 0.001), and cognitive presence (rs = 0.820, p < 0.001). Apart from creating exposure, industry talks create interactions between students and real-world experts in the classroom to achieve similar learning outcomes [57]. Furthermore, learning through industry talks nurtures the learners’ propensity for lifelong learning, which strengthens their cognitive ability in synthesising knowledge across diverse topics [60, 77]. The industry talks variable did not correlate as strongly with social presence perhaps because the assignment related with the talk was centred only on one topic, which was intellectual property law, hence limiting the versatility of the discussions among the student team members. Nonetheless, the correlation was still considered strong. Therefore, H4, H5, and H6 were not rejected.
The authors of the present study believe that the students were highly motivated due in part to the use of innovative tools and strategies such as TRIZ and industry talks in the course. More often than not, introducing a variety of innovative approaches in teaching and learning tends to improve students’ learning motivation [78–80].
4.3General regression and ANOVA
A general regression analysis and ANOVA were performed with a Box-Cox power transformation. The optimised lambda values for all the regression analyses were found to be around 1.1 to 1.3, and were hence rounded to a value of 1. The results of the general regression and ANOVA are summarised in Table 5. The variance inflation factors throughout the model were found to be less than 5 (VIF < 5), which is still within the acceptable range in regard to multicollinearity. The Durbin-Watson statistic was found to be within the normal range (1.5 to 2.5) with regard to positive autocorrelation.
Table 5
Output | General Regression Model Summary | ANOVA Summary | ||||||||
Variable | Model | t | Unstandardised Coefficients | Standardised Coefficients | VIF | R2 | Durbin-Watson Statistic | df (Residual) | F | |
B | SE | Beta (β) | ||||||||
LM | (Constant) | 2.094* | 0.690 | 0.329 | 0.733 | 2.146 | 95 | 130.113*** | ||
TRIZ | 5.413*** | 0.568 | 0.105 | 0.571 | 3.957 | |||||
IND | 2.961** | 0.301 | 0.102 | 0.312 | 3.957 | |||||
SP | (Constant) | 1.958 | 0.626 | 0.320 | 0.758 | 1.994 | 95 | 149.075*** | ||
TRIZ | 6.750*** | 0.687 | 0.102 | 0.677 | 3.957 | |||||
IND | 2.155* | 0.213 | 0.099 | 0.216 | 3.957 | |||||
CP | (Constant) | 1.542 | 0.436 | 0.283 | 0.804 | 2.066 | 95 | 195.353*** | ||
TRIZ | 6.451*** | 0.581 | 0.090 | 0.581 | 3.957 | |||||
IND | 3.816*** | 0.334 | 0.087 | 0.345 | 3.957 |
Notes: N = 98; *p < 0.05; **p < 0.01; ***p < 0.001.
On the whole, it was found that all the independent variables significantly predicted learning motivation, social presence, and cognitive presence. TRIZ was found to be a significant predictor for learning motivation [β= 0.571, t(95) = 5.413, p < 0.001], social presence [β= 0.687, t(95) = 6.750, p < 0.001], and cognitive presence [β= 0.581, t(95) = 6.451, p < 0.001]. Industry talk was also a significant predictor for learning motivation [β= 0.312, t(95) = 2.961, p < 0.01], social presence [β= 0.216, t(95) = 2.155, p < 0.05], and cognitive presence [β= 0.345, t(95) = 3.816, p < 0.001]. These predictors explained a significant proportion of variance in learning motivation [R2 = 0.733, F(2, 95) = 130.113, p < 0.001], social presence [R2 = 0.758, F(2, 95) = 149.075, p < 0.001] and, cognitive presence [R2 = 0.804, F(2, 95) = 195.353, p < 0.001]. The significant regression equations can be modelled as follows:
Y1 = 0.690 + 0.568 X1 + 0.301 X2
Y2 = 0.626 + 0.687 X1 + 0.213 X2
Y3 = 0.436 + 0.581 X1 + 0.334 X2
Where Y1 –Learning Motivation; Y2 –Social Presence; Y3 –Cognitive Presence; X1 –TRIZ; X3 –Industry Talks
The present research contends that the inclusion of TRIZ and industry talks in teaching and learning significantly affects learning motivation, social presence, and cognitive presence with an explained variability of around 70 to 80%. This finding could be attributed with the strong influence of TRIZ on students’ creativity and problem-solving abilities [16, 52, 53], and positive impacts of industry talks on students’ practical and lifelong learning [59, 60]. Thus, H7 is supported.
4.4T-Test
A two-sample t-test was done to compare the course’s PO achievements between year 2020 and 2019. According to the results in Table 6, the two groups differed significantly from each other [t(211) = 4.290, p < 0.001], with students from 2020 achieving higher PO scores than students from 2019 (Mean2020 = 71.5%, Mean2019 = 66.8%).
Table 6
Groups | Mean | StDev | SE Mean | df | t |
PO_2020 | 71.5% | 0.063 | 0.006 | 211 | 4.290*** |
PO_2019 | 66.8% | 0.101 | 0.009 |
Notes: N for 2020 = 98; N for 2019 = 125; *p < 0.05; **p < 0.01; ***p < 0.001.
This finding provides some general evidence that the integration of TRIZ and industry talks in a blended learning environment enhances the achievement of POs. The evidence is consistent with findings from several studies that combined the use of various innovative techniques and strategies to enhance programme and learning outcomes [16, 60, 81–83].
5Conclusions and directions for future research
In addressing the research question of this study, the researchers confirm that TRIZ and industry talks are positively and significantly correlated with the learning motivation, social presence, and cognitive presence of engineering students in a blended learning setting. The findings of this study support the concept of including TRIZ and industry talks along with other essential teaching and learning variables to reinforce learning performance in a blended learning environment. Such a concept could reduce some of the challenges in implementing blended learning. If imparted from an analogical viewpoint, the adoption of TRIZ and industry talks in teaching and learning could help turn blended learning into a “sweet instead of bitter pill to swallow” for both learners and educators in spite of the challenges faced during the COVID-19 pandemic. Several specific conclusions can be drawn from this study:
• The use of TRIZ and industry talks with digital education tools potentially motivated students to learn, thus improving their cognitive presence, social presence, and also achievement of POs.
• While correlations between industry talks and other variables were strong, the slightly low correlation between industry talks and social presence was perhaps due to the limited scope of topics covered by the talks and assignment.
• The inclusion of TRIZ and industry talks significantly affected learning motivation, cognitive presence, and social presence on the whole, with explained variations of about 70 to 80%.
The findings of this research should be generalisable across courses under the soft-applied disciplinary areas, such as health and social care, modern languages, business and education. Concerning the directions for future research, further studies that explain the moderating and mediating effects of TRIZ and industry talks on the other variable relationships could be explored to reinforce the findings of this research. Further investigations regarding control and experimental groups could be done to compare the performance between a class taught in a traditional setting and a class exposed with TRIZ knowledge and industry talks.
6Limitations
Although the sample size of this study was sufficient for the researchers to run the required analyses, it was comparatively lower than the sample size used for other similar studies that investigate factors affecting students’ learning motivation in blended learning settings. The researchers intend to extend their study to other blended learning courses in the future. This study emphasised on the participation of engineering students in an engineering law course, which is categorised under the soft-applied disciplinary area. Admittedly, the generalisability of this study’s results might be somewhat inexact in the context of other hard-pure, hard-applied, or soft-pure disciplines. Thus, future research should involve data collection from courses of varying disciplinary areas.
7Implications
The findings of this research provide important implications for online learning instructors and instructional designers. The inclusion of TRIZ and industry talks in teaching and learning could assist instructors in addressing the self-regulation challenges faced by students in an online learning environment [10]. In particular, the use of TRIZ tools and involvement of industry experts for talks might complement the use of prompts in supporting students’ self-regulation [84], which could increase online student engagement. For instance, the aspect of metacognitive prompts which requires reflection on online learning contents [85] can be systemised and given form using TRIZ tools such as function analysis, CEC analysis, technical contradictions, and inventive principles. Furthermore, in the midst of employment uncertainties due to the COVID-19 pandemic, linking students with industry experts through talks not only gives them a simulated experiential learning experience, but also confidence in the context of preparing them for employment as engineers [86].
Author contributions
CONCEPTION: Poh Kiat Ng and Voon Chet Koo
METHODOLOGY: Poh Kiat Ng, Voon Chet Koo and Yu Jin Ng
DATA COLLECTION: Poh Kiat Ng
INTERPRETATION OR ANALYSIS OF DATA: Poh Kiat Ng, Yu Jin Ng and Jian Ai Yeow
PREPARATION OF THE MANUSCRIPT: Poh Kiat Ng and Yu Jin Ng
REVISION FOR IMPORTANT INTELLECTUAL CONTENT: Poh Kiat Ng, Voon Chet Koo, Yu Jin Ng and Jian Ai Yeow
SUPERVISION: Poh Kiat Ng and Voon Chet Koo
Acknowledgments
The authors would like to thank all the students from the engineering law course (class of Trimester 1, 2020/2021) and also the Faculty of Engineering and Technology, Multimedia University (MMU) for their support in carrying out this study. The authors extend their appreciation to the Academic Development for Excellence in Programmes and Teaching (ADEPT) department and their talents from MMU for providing the authors with various training and workshops on BL, digital learning tools and the virtual classroom learning model. The authors also express their gratitude to Mr Choon Hong Lok for sharing his experience and insights on topics relevant to the study. Final special thanks go to Ms Chiew Fen Ng for her constructive criticism of the manuscript.
References
[1] | Raccanello D , Balbontín-Alvarado R , da Silva Bezerra D , Burro R , Cheraghi M , Dobrowolska B ,et al., Higher education students’ achievement emotions and their antecedents in E-learning amid COVID-19 pandemic: A multi-country survey. Learning and Instruction. 2022:101629. |
[2] | Bacher-Hicks A , Goodman J , Mulhern C , Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time, Journal of Public Economics (2021) ;193104345. |
[3] | Hussein E , Daoud S , Alrabaiah H , Badawi R , Exploring undergraduate students’ attitudes towards emergency online learning during COVID- A case from the UAE, Children and Youth Services Review (2020) ;119: :105699. |
[4] | Meulenbroeks R , Suddenly fully online: A case study of a blended university course moving online during the Covid-19 pandemic, Heliyon (2020) ;6: :e05728. |
[5] | Mishra L , Gupta T , Shree A , Online teaching-learning in higher education during lockdown period of COVID-19 pandemic, International Journal of Educational Research Open (2020) ;1: :100012. |
[6] | Rahiem MDH , Remaining motivated despite the limitations: University students’ learning propensity during the COVID-19 pandemic, Children and Youth Services Review (2021) ;120: :105802. |
[7] | Boelens R , Wever BD , Voet M , Four key challenges to the design of blended learning: A systematic literature review, Educational Research Review (2017) ;22: :1–18. |
[8] | Kaur M , Blended learning - its challenges and future, Procedia - Social and Behavioral Sciences (2013) ;93: :612–7. |
[9] | Ng ALY , Yang I , Academics and Learners’ Perceptions on Blended Learning as a Strategic Initiative to Improve Student Learning Experience, MATEC Web of Conferences (2017) ;87: (04005)1–5. |
[10] | Rasheed RA , Kamsin A , Abdullah NA , Challenges in the online component of blended learning: A systematic review, Computers and Education (2020) ;144: :103701. |
[11] | Law KMY , Geng S , Li T , Student enrollment, motivation and learningperformance in a blended learning environment: The mediating effectsof social, teaching, and cognitive presence, Computers andEducation (2019) ;136: :1–12. |
[12] | Lim J , Richardson JC Predictive effects of undergraduate students’ perceptions of social, cognitive, and teaching presence on affective learning outcomes according to disciplines. Computers and Education. 2021;161. doi: 10.1016/j.compedu.2020.104063. |
[13] | Rolim V , Ferreira R , Lins RD , Găsević D , A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry, Internet and Higher Education (2019) ;42: :53–65. |
[14] | Berdonosov V , Concept of the TRIZ Evolutionary Approach in Education, Procedia Engineering (2015) ;131: :721–30. |
[15] | Berdonosov VD , Redkolis EV TRIZ Evolutionary Approach: Didactics, Procedia Engineering (2015) ;131: :1105–12. |
[16] | Chang Y-S , Chien Y-H , Yu K-C , Chu Y-H , Chen MY-c , Effect of TRIZ on the creativity of engineering students, Thinking Skills and Creativity (2016) ;19: :112–22. |
[17] | Nakagawa T . Education and training of creative problem solving thinking with TRIZ/USIT, Procedia Engineering (2011) ;9: :582–95. |
[18] | Hitchings MG , Career Opportunities: Connecting Design Students With Industry, Procedia - Social and Behavioral Sciences (2016) ;228: :622–7. |
[19] | Therrien A , Lépy É , Boutet J-S , Bouchard K , Keeling A Place-based education and extractive industries: Lessons from post-graduate courses in Canada and Fennoscandia. The Extractive Industries and Society. 2021: 00989. |
[20] | Lin M-H , Chen H-C , Liu K-S , A Study of the Effects of Digital Learning on Learning Motivation and Learning Outcome, EURASIA Journal of Mathematics Science and Technology Education (2017) ;13: (7):1305–8215. |
[21] | Law KMY , Lee VCS , Yu YT , Learning motivation in e-learning facilitated computer programming courses, Computers and Education (2010) ;55: (1):218–28. |
[22] | Linnenbrink EA , Pintrich PR , Motivation as an enabler for academic success, School Psychology Review (2002) ;31: (3):313–27. |
[23] | Lynch DJ , Motivational factors, learning strategies and resources management as predictors of course grades, College Student Journal (2006) ;40: (2):423–8. |
[24] | Ng CF , Ng PK , A Review of Intrinsic and Extrinsic Motivations of ESL Learners, International Journal of Languages, Literature and Linguistics (2015) ;1: (2):98–105. |
[25] | Gero A , Improving Intrinsic Motivation among Sophomore Electrical Engineering Students by an Introductory Project, International Journal of Engineering Pedagogy (2012) ;2: (4):13–7. |
[26] | Park J , Seo M , Influencing Factors on Nursing Students’ Learning Flow during the COVID-19 Pandemic: A Mixed Method Research, Asian Nursing Research (2022) ;16: (1):35–44. |
[27] | Law K , Sandnes FE , Jian H-L , Huang Y-P , A Comparative Study of Learning Motivation among Engineering Students in South East Asia and Beyond, International Journal of Engineering Education (2009) ;25: (1):144–51. |
[28] | Yacob A , Saman MYM . Assessing Level of Motivation in Learning Programming among Engineering Students. International Conference on Informatics and Applications; 3-5 June 2012; Kuala Terengganu, Malaysia; 2012. |
[29] | Garrison DR E-Learning in the 21st century: A framework for research and practice. 2nd ed. New York, USA: Routedge; 2011. |
[30] | Kim J , Kwon Y , Cho D , Investigating factors that influence social presence and learning outcomes in distance higher education, Computers and Education (2011) ;57: (2):1512–20. |
[31] | Whiteside A , Dikkers AG , Lewis S The Power of Social Presence for Learning. EDUCAUSE Review 2014. Available from: https://er.educause.edu/articles/2014/5/thepower-of-social-presence-for-learning. [Accessed 20 May 2021]. |
[32] | Hostetter C , Community matters: Social presence and learning outcomes, Journal of the Scholarship of Teaching and Learning (2013) ;13: (1):77–86. |
[33] | Johnson RD , Gender Differences in E-Learning: Communication, Social Presence, and Learning Outcomes, Journal of Organizational and End User Computing (2011) ;23: (1):79–94. |
[34] | Richardson JC , Swan K , Examining Social Presence in Online Courses in Relation to Students’ Perceived Learning and Satisfaction, Journal of Asynchronous Learning Networks (2003) ;7: (1):68–88. |
[35] | So H-J , Brush TA , Student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: Relationships and critical factors, Computers and Education (2008) ;51: (1):318–36. |
[36] | Kehrwald B Understanding social presence in text-based online learning environments, Distance Education (2008) ;29: (1):89–106. |
[37] | Garrison DR , Online collaboration principles, Journal of Asynchronous Learning Networks (2006) ;10: (1):25–34. |
[38] | Garrison DR , Anderson T , Archer W , Critical inquiry in a text-based environment: computer conferencing in higher education, Internet and Higher Education (2000) ;2: (2-3):87–105. |
[39] | Garrison DR , Anderson T , Archer W , Critical thinking, cognitive presence, and computer conferencing in distance education, American Journal of Distance Education (2001) ;15: (1):7–23. |
[40] | Goh W , Dexter B , Self R , 4-Stage Online Presence Model: Model for Module Design and Delivery Using Web 2, 0 Technologies to Facilitate Critical Thinking Skills. International Journal on E-Learning (2014) ;13: (3):251–72. |
[41] | Gwee S , Damodaran S Use of Web 2.0 and Mobile Technologies for Developing Argumentative Skills. In: Brown T, Merwe Hvd, editors. The Mobile Learning Voyage From Small Ripples to Massive OpenWaters: International Conference on Mobile and Contextual Learning. Cham: Springer, Switzerland; (2015) . pp. 129–42. |
[42] | Joo YJ , Lim KY , Kim EK , Online university students’ satisfaction and persistence: Examining perceived level of presence, usefulness and ease of use as predictors in a structural model, Computers and Education (2011) ;57: (2):1654–64. |
[43] | Gaăsević D , Adesope O , Joksimović S , Kovanović V , Externally-facilitated regulation scaffolding and role assignment todevelop cognitive presence in asynchronous online discussions, Internet and Higher Education (2015) ;24: :53–65. |
[44] | Kovanović V , Gaăsević D , Joksimović S , Hatala M , Adesope O , Analytics of communities of inquiry: Effects of learningtechnology use on cognitive presence in asynchronous onlinediscussions, Internet and Higher Education (2015) ;27: :74–89. |
[45] | Kozan K , Richardson JC Interrelationships between and among social, teaching, and cognitive presence, Internet and Higher Education (2014) ;21: :68–73. |
[46] | Kozan K , The incremental predictive validity of teaching, cognitive and social presence on cognitive load, Internet and Higher Education (2016) ;31: :11–9. |
[47] | Barry K , Domb E , Slocum MS Slocum MS. What is TRIZ? The TRIZ Journal. 2010:603-32. |
[48] | Lim ISS , Teoh K-S , Eco-Efficient Product Design Using Theory of Inventive Problem Solving (TRIZ) Principles, American Journal of Applied Sciences (2010) ;7: (6):852–8. |
[49] | Hua Z , Yang J , Coulibaly S , Zhang B , Integration TRIZ with problem-solving tools: a literature review from to International Journal of Business Innovation and Research (2006) ;1: (1/2):111–28. |
[50] | Yeoh TS TRIZ: Systematic Innovation in Business and Management. 1st ed. Selangor, Malaysia: Firstfruits Publishing; (2014) . |
[51] | Yeoh TS , Yeoh TJ , Song CL TRIZ: Systematic Innovation in Business and Management. 1st ed. Selangor, Malaysia: Firstfruits Publishing; (2015) . |
[52] | Harlim J , Belski I , Long-term Innovative Problem Solving Skills: Redefining Problem Solving, International Journal of Engineering Education (2013) ;29: (2):280–90. |
[53] | Harlim J , Belski I , Learning TRIZ: Impact on confidence when facing problems, Procedia Engineering (2015) ;131: :95–103. |
[54] | Belski I , Baglin J , Harlim JM , Teaching TRIZ at University: A Longitudinal Study, International Journal of Engineering Education (2013) ;29: (2):346–54. |
[55] | Greenberg S , “Development of thinking skills” course: teaching TRIZ in academic setting, Procedia Engineering (2015) ;131: :219–23. |
[56] | Gonsalves D What’s the difference between an industry talk and a subject talk at an Open Day? 2019. Available from: https://bit.ly/37gYMt1. [Accessed 29 November 2020]. |
[57] | Fleischmann K , Developing on-campus work-integrated learning activities: The value of integrating community and industry partners into the creative arts curriculum, Asia-Pacific Journal of Cooperative Education (2015) ;16: (1):25–38. |
[58] | Engineering Accreditation Council. Engineering Programme Accreditation Standard. 1st ed. Selangor, Malaysia: Board of Engineers Malaysia; 2020. |
[59] | Markom M , Khalil MS , Misnon R , Othman NA , Abdullah SRS , Mohamad AB , Industrial Talk and Visit for Students, Procedia - Social and Behavioral Sciences (2011) ;18: :674–82. |
[60] | Nordin R , Bakar AAA , Zainal N , Husain H , Preliminary Study on the Impact of Industrial Talks and Visits towards the Outcome Based Education of Engineering Students, Procedia - Social and Behavioral Sciences (2012) ;60: :271–6. |
[61] | Chiang CL , Lee H , The Effect of Project-Based Learning on Learning Motivation and Problem-Solving Ability of Vocational High School Students, International Journal of Information and Education Technology (2016) ;6: (9):709–12. |
[62] | Jani HM , Teaching TRIZ Problem-Solving Methodology in Higher Education: A Review, International Journal of Science and Research (2013) ;2: (9):98–103. |
[63] | Pogrebnaya TV , Kozlov AV , Sidorkina OV , editors. Invention of knowledge in TRIZ-based education. 2013 International Conference on Interactive Collaborative Learning (ICL); 2013 25-27 Sept. 2013. |
[64] | Wits WW , Vaneker THJ , Souchkov V Full Immersion TRIZ in Education. In: Rizzi C, editor. Proceedings of the TRIZ Future Conference. Bergamo, Italy: Bergamo University Press; (2010) . pp. 269–76. |
[65] | Burns C , Chopra S , A Meta-analysis of the Effect of Industry Engagement on Student Learning in Undergraduate Programs, The Journal of Technology, Management, and Applied Engineering (2017) ;33: (1):1–20. |
[66] | Friesen M , Ibrahim N , McSorley G , Mattucci S , Engineers-in-Residence Programs as a Framework for Industry Engagement in Undergraduate Engineering Education: Challenges and Opportunities, Proceedings of the Canadian Engineering Education Association (CEEA-ACEG19) Conference (2019) ;9: :1–8. |
[67] | Male SA , King R , Enhancing learning outcomes from industry engagement in Australian engineering education, Journal of Teaching and Learning for Graduate Employability (2019) ;10: (1):101–17. |
[68] | Allen IE , Seaman CA . Likert Scales and Data Analyses. Quality Progress. 2007:64-65. |
[69] | Finstad K , Response Interpolation and Scale Sensitivity: Evidence against 5-Point Scales, Journal of Usability Studies (2010) ;5: (3):104–10. |
[70] | Matell MS , Jacoby J , Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and Validity, Educational and Psychological Measurement (1971) ;31: :657–74. |
[71] | Nunnally J , Bernstein L Psychometric theory. 1st ed. New York, USA: McGraw-Hill; 1994. |
[72] | Taber KS , The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education, Research in Science Education (2018) ;48: :1273–96. |
[73] | Awang Z , Afthanorhan A , Mohamad M , Asri MAM , An evaluation of measurement model for medical tourism research: the confirmatory factor analysis approach, International Journal of Tourism Policy (2015) ;6: (1):29–45. |
[74] | Sobri S Case Study: But I Have Known Him For Years! 2017. Available from: https://www.praktis.com.my/all-articles/case-study-but-i-have-known-him-for-years. [Accessed 20 May 2021]. |
[75] | Livotov P , Measuring motivation and innovation skills in advanced course in new product development and inventive problem solving with TRIZ for mechanical engineering students, Procedia Engineering (2015) ;131: :767–75. |
[76] | Sire P , Haeffelé G , Dubois S , TRIZ as a tool to develop a TRIZeducational method by learning it, Procedia Engineering (2015) ;131: :551–60. |
[77] | Castleberry A , Ward W , Stein S , Lifelong learning inspires the creative art of academic writing, Currents in Pharmacy Teaching and Learning (2019) ;11: (8):757–9. |
[78] | Day-Black C , Merrill EB , Konzelman L , Williams TT , Hart N . Gamification: An innovative teaching-learning strategy for the digital nursing students in a community health nursing course, The Association of Black Nursing Faculty Journal (2015) ;26: (4):90–4. |
[79] | Huang S-Y , Kuo Y-H , Chen H-C , Applying digital escape rooms infused with science teaching in elementary school: Learning performance, learning motivation, and problem-solving ability, Thinking Skills and Creativity (2020) ;37: :100681. |
[80] | Varthis S Students’ Perceptions of Blended Learning and its Effectiveness As a Part of Second Year Dental Curriculum [Dissertation]. New York, USA: Columbia University; 2016. |
[81] | Abdullah SRS , Takriff MS , Anuar N , Ismail M , Harun S , JKKP Experience in Conducting Integrated Project since Session /Procedia - Social and Behavioral Sciences (2012) ;60: :42–51. |
[82] | Bertoncelli T , Mayer O , Lynass M , Creativity, Learning Techniques and TRIZ, Procedia CIRP (2016) ;39: :191–6. |
[83] | Hillmayr D , Ziernwald L , Reinhold F , Hofer SI , Reiss KM , The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis, Computers and Education (2020) ;153: :103897. |
[84] | Schumacher C , Ifenthaler D , Investigating prompts for supporting students’ self-regulation –A remaining challenge for learning analytics approaches? Internet and Higher Education (2021) ;49: :100791. |
[85] | Ekmekci I , Nebati EE . TRIZ Methodology and Applications, Procedia Computer Science (2019) ;158: :303–15. |
[86] | Johan K , Perception of Students Towards Lecturers Teaching Engineering Courses With Industry Experience: A Case Study In Malaysia Technical University, Procedia - Social and Behavioral Sciences (2015) ;195: :925–31. |
Appendices
Appendix
Section 1 | ||
Age: | ||
Gender: Male, Female | ||
Engineering programme: | ||
Year in the engineering programme: | ||
Section 2 | ||
Indicators | Code | Items |
Learning | LM1 | I felt motivated when I completed the tasks distributed in the course successfully. |
Motivation | LM2 | I felt motivated to learn from the course. |
LM3 | I was interested in the course contents. | |
LM4 | The insightful knowledge from the course motivated me to study the course more. | |
LM5 | My instructor motivated me to learn from the course. | |
LM6 | My classmates motivated me to learn from the course. | |
LM7 | I felt more connected to this course as compared to other courses. | |
Social Presence | SP1 | The course provided chances for me to express my opinions (e.g. through Padlet, Google Docs, Google Meet, Google Classroom). |
SP2 | The course offered the opportunity for me to interact with fellow students (e.g. through online chats, video calls, online discussions). | |
SP3 | The course provided me with collaborative activities (e.g. through online group assignments, Padlet, Google Docs). | |
SP4 | I enjoyed participating in the course activities (e.g. through Padlet, Google Docs, Quizziz). | |
Cognitive Presence | CP1 | I was able to acquire knowledge from the course quickly. |
CP2 | I was able to identify the problems I encountered during the course. | |
CP3 | I was able to explore more information related to the course from other means of learning (e.g. through YouTube videos, online research, online discussions). | |
CP4 | I was able to link the information I learned from the course (e.g. assignments, Quizziz, Padlet, Google Docs). | |
CP5 | The course provided chances for me to reflect and ask questions about what I learned (e.g. through Google Meet, e-mail, online discussions). | |
CP6 | The course allowed me to explore and integrate ideas into solutions (e.g. through case studies, assignments, online discussions, Padlet). | |
TRIZ | TRIZ1 | Function analysis gave a clearer illustration of complex descriptions about the Malaysian legal system and case studies. |
TRIZ2 | Function analysis helped me systematically analyse the interactions among various components in the Malaysian legal system and case studies. | |
TRIZ3 | Cause-and-effect-chain (CEC) analysis assisted me in systematically breaking down potential causes to the main problem in case studies. | |
TRIZ4 | CEC analysis aided me in identifying the likely root cause of the main problem in case studies. | |
Industry Talks | IND1 | The industry talk helped improve my understanding of intellectual property (IP) law. |
IND2 | I was able to relate my lectures with real-world situations from the industry expert’s viewpoint. | |
IND3 | The industry talk exposed me to the practical aspects of intellectual property law. | |
IND4 | The industry talk developed my sense and awareness of intellectual property law in an industry context. |