Exam design: To assess the application of the course material in addition to knowledge and comprehension
​Abstract:
​Each student’s ability to apply the key course learnings, beyond just understanding the material, was assessed using a 30 question short answer, case-based exam design in a number of undergraduate and graduate courses. Such a design provides a foundation for not only assessing each student’s knowledge and comprehension, but also their higher-level thinking surrounding the major concepts being taught. Developing such higher-level thinking abilities (application, analysis, synthesis and evaluation), which requires critical and creative thinking, is a transferable skill which should be a central focus of the educational process. Hence, student assessments of these abilities are valuable to the student, professor and educational institution.

                       If a picture is worth a thousand words, is video worth 3,000 words/a million? 1,2

​I have recently embraced a wider application of video and audio recordings for my university lectures as well as student assignments.3 These include online lectures, video-based assignments and audio-based assignment feedback. The objectives were three-fold: to add a bit of fun and variation to the assignments, free up some valuable class time, and improve overall learning, retention and feedback. Judging by the student feedback to date, these innovations have been well received and have delivered these benefits.

Like many lecturers, I generally post my lecture slides and the rough audio recording of the lecture after class.4,5 However as a  result of two “snow days”, i.e. school closings due to weather, I took it upon myself to deliver these two lectures online by providing an audio overlay to the PowerPoint slides that I had prepared. There are a number of ways to accomplish this within PowerPoint or other software.6 It is best if you take time to fashion a script as the resulting recording tends to be shorter and tighter. Nevertheless, it can be done top-of-mind, mimicking a live lecture, which is quicker and more spontaneous.7 In these recordings, I still asked quick write questions; asking the students to pause and answer questions using online tools.8,9 Having prepared these recordings, in addition to previously producing a few formal topic-specific videos, it reminded me of the all the benefits of Massive Open Online Courses (MOOCs) and the flipped classroom concept.10,11 As a result, I plan to make more recordings. These recordings provide students with a well-focused, asynchronous method of learning and reviewing the core material, while at the same time freeing up class time for higher order discussion and debate.

Given the ease of preparing videos and slide-audio recordings, I have also introduced them on the student side. I have had my students video themselves (in a slides over the shoulder mode), usually by tripod-mounted smartphone, giving 6-10 minute presentations on course-relevant topics.12 The resulting MP4 recordings are then posted on a shared hard drive so that I and a number of their peers can provide feedback on their presentation style and the content of their presentation.13 Given that video recording is only a tool, the quality and production of the recording itself is not, and in my opinion should not, be assessed. The advantages of making and sharing these recordings include: providing a fun and alternate way to complete an assignment, saving valuable class time by replacing in-class presentations, allowing students to practice presenting as well as observing and critiquing their own style and that of others, and finally creating a permanent record that is richer and easier to review for exams than articles or slide decks alone.

In a recent class, I also asked my students to prepare and post a 10-20 second video of themselves stating their name as well as telling me why they were taking the course and what they hoped to learn. This allowed me to match names and faces, perfect name pronunciations, and learn a little about each person. This was invaluable and is a practice that I will continue in the future.

Finally, I have started posting short audio commentary to student submissions. I first came across this idea in a research paper written by a colleague.14 In that study, Dr. Rawle et al. found that 70% of students felt that audio feedback improved their learning. The audio feedback that I now provide compliments both the specific, line-by-line, in-text comments that I post as well as the four level scale of the specific rubrics (i.e. exemplary, accomplished, attained, unattained) that I complete and post for assignments. I plan to extend this use of audio commentary to live or recorded students submissions as a supplement to the graded rubric and general written comments. The advantage of such audio feedback is that broad and general comments can be provided, including the highlighting of major issues, themes, insights and even oversights. In addition, audio allows the proper tone of the comments to be conveyed.

I have found that the wide application of video and audio recordings for lectures as well as student assignments and grading can add significant value to learning outcomes as well as being well received by my students.

- - - - - - -

1From Shipper, F. (2013). If a Picture is Worth 1,000 Words, Is a Video Worth 3,000 Words? A Review of Video Resources Available for Use in Todays Management ClassroomIf a Picture is Worth 1,000 Words, Is a Video Worth 3,000 Words? A Review of Video Resources Available for Use in Todays Management Classroom. Academy of Management Learning & Education,12(4), 684-686. doi:10.5465/amle.2013.0304
2 From Yadav, A., Phillips, M. M., Lundeberg, M. A., Koehler, M. J., Hilden, K., & Dirkin, K. H. (2011). If a picture is worth a thousand words is video worth a million? Differences in affective and cognitive processing of video and text cases. Journal of Computing in Higher Education, 23(1), 15-37 doi:10.1007/s12528-011-9042-y
3 I currently lecture at the undergraduate (Institute for the Management of Innovation) and graduate level (Master of Biotechnology) at the University of Toronto Mississauga.
4 I usually supply my slides in PDF format to avoid compatibility issues. For general audio recording, I use the free program www.audacityteam.org and export it as an MP3 file however a smartphone will suffice. I generally do not videotape my lectures as I find the slide and audio combination superior to a “talking head with slides over the shoulder” movie. Although it takes one more step as discussed below, by dubbing the audio over the slides it is possible to create a useful MP4 recording.
5 Indeed, I often post a set of pre-lecture slides as well that are missing some of the key items to which I plan to direct questions, generally in a quick write format.
6 The quickest, editable, slide by slide method involves simply inserting the audio on each slide (PP: Insert/Media/Audio/Record Audio/Insert or Keynote: Insert/Record Audio/Inset). An audio icon then appears on each slide, that can be played by clicking or animated to start automatically. One advantage of this method for the students is that it can be viewed and selected slide by slide. The disadvantage is the correct version of the specific software is required.
For a cruder one-take recording, PP: Slide Show/Record Slideshow and the microphone is automatically on as you toggle slides. Keynote: Play/Record Slideshow and then toggle the microphone on and off as you advance the slides.
You can also import your slides into more sophisticated video tools like Apple’s iMovie and then dub audio on top to make a full MP4 video capable of running on any computer. I tend to do use this method when producing more formal videos so that they will run in a standalone manner. I have also used this method to dub a live audio recording from a presentation over the corresponding slide set (see my quick write presentation at https://youtu.be/Sk6eem-WfZU as an example).
7 A combination of first giving a spontaneous talk, recording it, transposing it by hand or through voice recognition software, and then using this transcript to produce a polished script is the ultimate but somewhat more timely way to go.
8 For more on quick writes, watch my video https://youtu.be/Sk6eem-WfZU.
9 Here I tend to use www.GoSoapBox.com but this could also be done with other online tools like Blackboard or Canvas, or even through email directly.
10 These are online courses supplied at little or no cost from companies like www.coursera.org, www.edx.org and www.udacity.com covering a multitude of topics including academic courses from major universities. I highly recommend exploring their courses, having taken a number myself.
11 The flipped classrooms concept revolves around having the primary learning being done outside class through readings and especially videos. This frees up the in-class time for more tutorial style discussion, clarification and hands-on problem solving. For more information see https://en.wikipedia.org/wiki/Flipped_classroom.
12 The one difficulty with these recording is their size. For a smartphone, a 10 minute video filmed at 720p requires 600 MB, whereas at 4K it is almost 4 GB. As a result, sharing files often requires the use of cloud-based storage like www.dropbox.com or https://drive.google.com.
13 To allocate peers and collect the feedback, I use the online tool https://app.peerscholar.com. Generally I have each student complete 5 peer reviews and then a self-evaluation.
14   Rawle, F., Thuna, M., Zhao, T., & Kaler, M. (2018). Audio Feedback: Student and Teaching Assistant Perspectives on an Alternative Mode of Feedback for Written Assignments. Canadian Journal for the Scholarship of Teaching and Learning, 9(2), 1-20. doi:10.5206/cjsotl-rcacea.2018.2.2 Retrieved from https://ir.lib.uwo.ca/cjsotl_rcacea/vol9/iss2/2.

Quick write - an easy to implement educational tool to improve student engagement and learning outcomes.

A live 25' recording and slides from my 9/Mar/2019 presentation at the UofT Faculty Assoc. (UTFA) Challenges and Strengths II conference.
​
​ https://youtu.be/Sk6eem-WfZU

Quick write summary:
- A short written response to a question or prompt. 
- Designed to spur creative or critical thinking though open-ended questioning.
- Inherently inclusive (everyone participates) and best used in a formative, reflective and interactive manner.
- Allows all individuals time to formulate their response. 
- Can be used with planned and spontaneous questions.
- Generally, 2 or 3 questions are asked per hour of lecture and 2 minutes or so are provided to complete the paper-based/hand-written response before discussing the responses.
- Many variations possible including exit/entry tickets, group work, drawings and figures, calculations etc.

I have been doing a fair amount of teaching over the last 19 months at the University of Toronto, primarily in the excellent Masters of Biotech programme (MBiotech). I taught 4 half courses, co-taught 3 others, ran two workshops and assisted in a lab course. As a result, I'm a little behind on my blogs including this one on my educational innovations, experiments and learnings. So here's update on what's new as well as what's ongoing with some tweaks.

1) Application of PeerScholar
In my undergraduate course, HMB301, the students are required to do a major 7-part modular report on various aspects of a company of their choosing. Each section is based on the previous week's lecture and covers the technology, market, management, competition, intellectual property and finances. This has been an extremely successful exercise. This year I added a peer review component, based on the cloud-based PeerScholar software. The students were required to post their assignment 2 days before it was ultimately due. Two peers then reviewed it for both grammar and content, posting their comments and suggestions within 24 hours. They were pass/fail graded for their effort/participation. 24 hours later, the author was required to submit their section for grading and formal feedback, hopefully incorporating the suggestions. Not only did the students find the peer feedback very valuable, the reviewers also had the chance to learn about another two companies as well as pick up some tips as to how their peers approached their analysis. This was verified by an informal follow-up survey.

2) Jeopardy
For a number of lectures, I have employed a PowerPoint-based version of the popular gameshow Jeopardy with some modifications. Briefly, an somewhat cryptic answer is given and the participants are asked to figure out the topic in the form of a question. My modification involves showing and discussing one or more standard lecture slides covering that particular topic after each answer/question. I can get through about 20 answers/questions (4 categories of 5 questions each) in about 45'. For longer classes we play a second round after a break and of course Final Jeopardy.

3) Expanded use of GoSoapBox
GoSoapBox.com is in effect a cloud-based clicker system. I use this in the class for polls and interspersed multiple choice questions to make the class more interactive. I have started using the quiz option to post 5 simple questions about the one (or two) pre-read(s) that I assign before every class (See Partially Flipped Classroom). Again this is graded as a simple pass/fail for the effort. I have found the students come to class better prepared, having read the pre-read which allows the lecture to cover the material in more depth and foster broader discussions. 

4) Twitter
I continue to use Twitter as a means to post current events relevant to the particular course I am teaching (See #BTC1820 for posts on agricultural biotech). My recent tweak (not tweet) was to post these twitter links in the Blackboard discussion forum and have the students, on a first-come-first-serve basis, write a brief summary of one tweet each. Again this was evaluated on a simple pass/fail basis. By providing a summary, the students did not need to read each tweet in its entirety (or not), but instead could read a short summary. The material covered by the tweets is not directly tested on the final exam, however many of the tweets are illustrative of important course concepts.

5) Final Exam
I do not like multiple choice exams. They are difficult to write, despite being easy to mark. In addition, in my opinion more effort is spent trying to trick students as oppose to extracting a representation of their knowledge and understanding. For that reason, I prefer short answer exams. The format that I have moved towards involves three sections in which the students are required to answer 10 of 12 questions, for a total of 30 questions over a 2-hour sitting. Giving some choice is much appreciated and reduces the "penalty" of not understanding one particular concept or from a complex or even poorly worded question. I have found that this is a fair amount of writing but not too much The marking on the other hand is a nightmare but that's the price. I can mark about 30 questions every 10 minutes so that's almost 7 hours for a class of 40! I do it page by page or the same 4 questions for all students before moving to the next page. This makes it quicker as you don't need to refresh your memory as to the question/answer and results in more consistent/fairer marking.

6) Interesting/intriguing questions
My newest and so far successful innovation (based on initial student feedback) involves assigning the class a fairly open, 2-3 page report expanding on something they came across or found novel, interesting, intriguing, or unexpected that arises from the pre-reads, lectures or even tweets. I supplied an ongoing list of items that I found and students could chose from those if they were stuck.  I expected a detailed analysis/ explanation. Topics include items such as: What are the mechanistic differences between C3, C4 and CAM plants? What supports the claim that we need a 70% increase in food calories by 2050? How do you go about culturing a as yet "unculturable" microbe?

So there's the update. I still have the students present to their peers in every class. I use blackboard extensively to manage all the outgoing information and incoming assignments. I post slide sets just  before class to aid note taking. I do encourage class participation and am still working on how to improve this. (One idea that I have yet to implement is my "Socratic hat" in which everyone's name is placed and over the course of a lecture or two - each student's name is drawn to answer a question or discuss an issue. If they opt not to, their name is simply returned to the hat for a future question.)  And, I keep thinking and reading about additional innovations I can bring to improve my teaching as well as the students' experience and most importantly - learning. 


    

I just finished teaching four 3-hour sessions on Project Management to graduate students at the UofT. The objective of the sessions was to provide the participants with a practical and functional understanding of the core principles and tools of project management that can be directly apply to their day-to-day work. Outlined below is the agenda:

I wanted the participants to learn through their own experiences, so I divided the session into  three main sections:
1) A set of exercises to build a simple (spaghetti and marshmallow) structures that involved sub-groups making designs and plans and then passing off for others to interpret and build. This was effective in illustrating the importance of good communication as well as the constraints of scope, quality, time and cost (there was a material budget).
2) An exercise involving a PERT diagram and a Gantt chart to "crash" a 12-week plan back to 10-weeks. This provided some experience with these two main tools as well as work breakdown structures, dependencies, the critical path a the condo again the constraints of time and cost (adding additional labour incurred escalating costs).
3) A 20-slide lecture to reinforce the experiences, learnings and discussions that arose from the exercises as well as provide some of the key concepts and terminology used in project management.

The feedback from this experiential experiment was very positive, so I'm looking to explore similar approaches in the future.

1) Issuing my slides in advance with words missing
I have seen this done very effectively, however it requires one to be well-prepared and a little upfront effort. It involves providing the presentation slides to the students in advance but with certain words and concepts left blank. Many students prefer to have the slides in advance so that they can take notes directly on them. On the other hand, teachers generally want to encourage active class participation, often asking questions in advance of covering the topic and the related slide. By distributing incomplete slides, both the student and teacher are well served.

2) Adding video or audio material outside class
Video or audio material either as pre-read or supplementary material can free up more of the limited and valuable class time to in depth discussion, debate and questioning. This is related to the practices involved in the flipped classroom and massive open online courses (MOOC’s). Again this requires some upfront planning and effort, especially if you are recording your own material.

3) Testing individuals orally
A 15’ interview/ discussion with each student grounded in a defined subject area or individual project would give each student the opportunity to demonstrate their knowledge and critical thinking abilities in a forum that can not be replicated by paper-based testing. The difficulty is the time commitment for teachers: In a class of 30, this would require the better part of two full days.

4) Holding small group discussions
Related to item 3), is the concept of meeting with small groups of students to discuss course material and especially pre-assigned cases to assist them in developing their abilities to think creatively and critically, infer, detect trends and debate. This often occurs in tutorials, but poor attendance is the norm. 

5) Adding interim milestones to larger projects
To get the best results, it is advantageous to have some checks to be sure each student is focusing on the right things, not missing major components, or building on a fatal flaw. Interim milestones such as a brief proposal, list of the key topics/issues to be examined, proposed table of contents, initial list of references, project plan, and rough draft/ preliminary report can all provide the instructor the opportunity to review the progress to date and suggest areas for focus, inclusion, improvement and correction. 

6) Employing peer review as part of interim project milestones
Related to item 5), is the idea of having at least some of these interim milestones reviewed by your peers. This works especially well when the task/assignment is similar but not the identical for each student as there will be common themes but each case will be different. Examples include having each student review a different novel, play, political/ social issue, historical event, famous person, company/stock, technology or science project. 

7) Completing a large project based on exceeding a standard
Following on from items 5) and 6), the concept follows more of a pass/ fail criteria like a driver’s test (as well as a graduate thesis). The project is deemed complete and a pass, once it meets a particular acceptance criteria for quality, thoroughness and insight (perhaps as defined by a rubric). Submissions that do not meet the criteria receive comments and guidance in order that the student can improve the weak components and re-submit the project (a number of times if necessary). This ensures that all students eventually complete a project that meets or exceeds a desired standard and that they have acquire the requisite skills and knowledge.     

8) Providing self-assesments 
A number of textbooks provide short answer and even multiple choice questions at the end of each chapter. Math texts in particular have always provided answers to selected questions at the back of the book. These are valuable tools for self-assessment and could be expanded by having teachers developing similar quizzes and problem sets with an accompanying answer sheet to their lecture material and other supporting materials (readings, books, videos etc.). Such self-assessment tools puts the focus on learning and off grades.   

9) Giving permission to try and fail
Unfortunately, most schooling is overly grade-driven. Students are taught set material and then tested on it. All their grades are accumulated into a final grade, so doing well on each and every test is important. Although such testing was devised to encourage student learning and provide feedback, in many cases the student focus becomes solely on test taking based on cramming and short-term memorization. Stress, competition (to stay ahead of the bell curve) and "getting it done" are the key outcomes not the hoped for enjoyment, interest, creative expression and actual learning. 

Experiential learning, learning from experience or learning by reflecting on doing, is seen as a valuable addition to this standard “chalk and talk” teaching/ testing paradigm. Examples include field trips, performances, lab work and even demonstrations. While these activities can indeed be more interesting and enjoyable, as well as promote learning, they tend to remain well structured to permit grading while discouraging creativity and risk-taking. 

A specific form of experiential learning that I have termed Experimental learning or learning by undertaking experiments, incorporates creativity but with that comes the risk of failures. Science fair type projects, art work, creative writing and even debating can fit this experimental category. Learning from all failures or missteps is extremely powerful and should not be discouraged. However, grading the outcomes of these experiments and creative activities can stifle them. If grading must be undertaken on these projects, it should focus on the process as opposed to the results.


I plan to try out items 1), 2), 4), 5), 8) and 9) in the course I am currently co-teaching. Items 3), 6), and 7) will have to wait as they are not as feasible in a large class and in some cases may not conform to the academic policies of a fairly conservative institution.

Follow-up note: I have recently and successfully implemented examples of: 1) in an IP lecture, 2) in a video on financial bonds, 4) in a GamBit course, 5) for a company analysis project and 9) in an exercise involving contacting potential employers. 

If you are applying for graduate school, a professional school (medicine, dentistry etc.) or even a job in your field of study upon graduation, you will need letters of recommendation/reference. Especially in an academic setting, these letters are a critical factor in the decision process. Strong letters of recommendation from a couple of your professors can weigh heavily in your favour. 

So how do you go about getting them? Get to know a couple of profs well. To achieve this, you need to start early and put in some time and effort.

Asking a professor that taught you, and 69 of your peers, two years ago is certainly not the way. Unless you really stood out in class or in tutorials, the chance of them remembering you, even if you were an “A” student, is slim. I can attest to this first hand, as I recently had five former students contact me under exactly these conditions.

The problem is that although the majority of professors, myself included, want to help their students, they can’t effectively help under these circumstances. The purpose of a letter of recommendation is to provide some specific information and insight about a candidate, their achievements and their accomplishments that can’t be found elsewhere. This requires specific knowledge and experience working with and observing the candidate. Without such knowledge, what results is a general and uninspiring recommendation indicating that you were studious and received good marks - period. In most cases, professors will attempt to politely decline providing a reference given that they know it will only be general.

So how can you get to know some of your profs and get them to know you? You need to put yourself in a position where you are interacting with them.

The most common and obvious approach is to do a research project or thesis with a professor as one of your later year course requirements. This I highly recommend and it will certainly result in a strong and specific recommendation (assuming you put in the effort).

So what else can you do, especially with respect to a second, non-project associated professor teaching a large class? Do something special.

Being an active participant in class and especially in prof-led tutorials (which many are not), both answering posed questions and asking insightful ones is an approach that shows your involved, thinking and doing the homework. This could be enough, but not usually. You need to step up your interaction outside of class. The means are consistent with the principles underlying networking and mentoring which include providing value and assistance (solicited and especially unsolicited) as well as accepting advice, while keeping in regular communication (in person as well as by phone or e-mail even tweets - whatever works for the two of you). See if there is an opportunity to volunteer in the professor’s lab, participate in a study, serve as a teaching assistant, or do a summer internship. Could the professor use some assistance doing some background research for a paper/presentation/lecture, conducting a survey, following the trends, news and recent reports in a related field, or reviewing the market/competitive landscape? When you come across an interesting article or book, tried some new software or app, or learned something in another class, even a teaching method/tool that could be of value to pass along?     

Increasing your interaction with one or two professors, outside a research project/thesis, will provide them the insight to write a strong and specific recommendation. However, this requires significant time and effort, so develop a plan and get started early. 

Some good articles on recommendation letters are:
http://undergrad.psy.ohio-state.edu/html/grad_letter.php
http://www.education.com/reference/article/college-officers-letters-recommendation/
http://mybiasedcoin.blogspot.ca/2007/10/importance-of-recommendation-letters.html
http://work.chron.com/importance-reference-letter-employment-6025.html
and an article on networking: 
http://www.businessinsider.com/the-give-and-take-of-networking-2011-6 

Double Play: In baseball, a play in which two players on the offense are retired (or put out) within one continuous play by the fielding of the defense. This usually occurs when a ground ball is hit to one of the infielders, who then turns a force out at second base out and a tag out at first base.

 

A friend of mine, Adrienne Alison, is one of two finalists in a competition to design and create a War of 1812 monument for Parliament Hill in Ottawa. Her sculpture involves seven figures representing the British Army and Navy, the Provincial Marine, the English-speaking and French-speaking militias, First Nations, and Métis. This undertaking involves both a major art project, developing a concept and sculpting seven life-size individuals as well as a significant history project, understanding the period, the inter-relations between these groups and incorporating proper facial features and uniforms. Indeed, Adrienne has expended considerable effort on both the artistic and historic aspects – a double play.

This got me thinking about homework assignments and major projects in high school. (It could be applied in college, university or even lower school grades.) Why couldn’t projects be done as double or even triple plays? What I mean is having two or more teachers of different subjects assign a common project. Examples include:
-       An art project about a historical figure or event (like the War of 1812).
-       A written research report/ presentation on an aspect of art, history or science.
-       A model of a scientific concept (like the structure of an atom.)
-       A debate on an aspect of science (like should genes be patentable?)
-       An analysis of the costs and process (business) of creating a sculpture.

Each teacher would then mark and comment on their respective aspect of the project, based on separate rubrics and perhaps a common one as well. For example, the English teacher would mark the project for creativity, grammar, paragraph structure; the Science or History teacher would mark the student’s understanding of the topic, and together they would mark the clarity and overall organization.

First, this would no doubt improve the efficiency of the student efforts; focusing on one well-researched and presented project would be required as opposed to two projects emphasizing different aspects of the curriculum. Second, due to the inherent integration it would improve the project quality and learning on all fronts. Third, it would be more representative of the workplace where individuals, like Adrienne, are frequently called on to integrate different fields. Most people whether artists, historians, journalists, business people, or scientists need to be able to write well, present, debate, research new topics, critically review, analyze, and develop creative solutions to problems within their domain of expertise. The ability to build models, draw diagrams or sketches, create videos as well as time and project management, budgeting, and statistics to name a few are all of great value and can be incorporated into major homework assignments and projects.  

Setting up the Double Play could be a real education game winner.

© Duncan Jones 2013 All rights reserved

I just finished teaching HMB301F, a third-year half course entitled “Biotechnology” at the University of Toronto, as a sessional lecturer. The course explored the interaction between science and business and their roles in the promise and reality of the Biotechnology industry. The course drew on the book Science Business by Gary Piscano (2006). From all accounts, the course was a great success - the majority of students enjoyed it and found it valuable, and I enjoyed teaching it. Outlined below are eight innovations that I introduced. 

1) TWITTER: In order to provide current content to the course, I set up a course specific twitter account (www.twitter.com) and retweeted about 80 relevant news stories and e-zine articles that I came across throughout the term (many from FierceBiotech).  I plan to maintain this account for those who are interested. 

2) TEAM-BASED LEARNING: One-third of the course involved team-based learning in which students were required to complete a reading assignment as homework each week and then answer two multiple choice quizzes in class. The first quiz was completed individually and the second as part of a randomly generated team of 5 students in which they discussed the reading and the questions being posed.  This proved to be a engaging and well-received process. For more information see www.teambasedlearning.org 

3) SCRATCH CARDS: For the team-based learning sessions as well as for class participation, IF-AT scratch cards were used (Immediate Feedback Assessment Technique by Epstein Educational www.epsteineducation.com).  Multiple choice questions were presented to the students, who then scratched their chosen answer. If correct, a star was revealed. If incorrect they scratched a second choice and so on until they uncovered the correct answer - hence immediate feedback. The students liked the feedback, the assignment of part marks depending on the number of answers scratched, and the fact that taking up the questions was no longer necessary.  

 4) SURVEYS: I surveyed the class using SurveyMonkey at the beginning of the course,  halfway through and near the end. The first survey explored the students’ interests and goals. As 84% were interested in learning more about careers, a lecture was dedicated to this. The second survey explored the course content, delivery and testing with minor adjustments being made based on the feedback. The third survey focused on the Guest speaker who was well received.

5) INQUIRY: As part of the research project on a biotechnology company of the students' choice, they were required to contact their company (e-mail/ phone) and ask about career opportunities. The response rate was 25% and significant information about the skills and expertise in highest demand (Scientists in their product area), as well as the preferred education (PhD, MBA) was collected and shared with the class.

6) TESTING:  Both the midterm and final exam included 25 short answer questions. The types of questions were equally divided between the six types outlined in Bloom’s Taxonomy (See www.nwlink.com/~donclark/hrd/bloom.html  and
 http://edorigami.wikispaces.com/Bloom%27s+Digital+Taxonomy). This involved the standard remembering (list, name, describe) and understanding (explain, interpret) types as well as higher-order questions requiring applying, analyzing, evaluating and creating (interpret, compare, hypothesize, critique).

7) BLACKBOARD: UofT provides Blackboard Learn as their Learning Management System (www.blackboard.com/Platforms/Learn/Overview.aspx)  (Learning Portal as they call it). I used this website/ portal to its full potential, posting each lecture before and a revised version after (as 1, 2 & 3 slides per page), as well as supporting documents and links, homework assignments and general announcements, and all grades. The students appreciated this rich, one-stop shop.

8) RECORDINGS: Both I and one of my students recorded each lecture independently. These were then made available to the class for review via Dropbox given their size (> 20 Mb/ 50’ lecture). In future, I would like to merge this audio with the PowerPoint slides and perhaps add some video as well. In doing so, I could further explore the flipped classroom model. One company that facilitates this is Echo360.