Is it time for more widespread reform of college teaching?

This series explores the state of the college lecture, and how technologies point to new models of undergraduate education.

Last month, we began inviting students across the countries to fire up their Web cameras or camera-phones to send us video commentaries about whether lectures work for them.

Chronicle.com/LectureFail displays a number of student comments, including a compilation, along with several faculty responses.  One caveat before a few thoughts… As I read/watch comments, I did interpret the question of #lecturefail to focus on learning experiences/courses which rely heavily or exclusively on lecture.  It does not seem that students are commenting on nor faculty are defending a moderate use of lecture as a learning tool.  The comments from both groups do not suggest a class in which a 30 minute lecture is followed by 30 minutes of discussion or project work, or a course experience that includes a lecture or two to support a 3 hour field experience; all comments seem to suggest a course which is entirely lecture based.

As a faculty member, as I watched several of the videos, I found my beliefs and attitudes to be more in line with the students than my faculty colleagues.  Personally, lectures are boring… for me… as a faculty member. I don’t like them, and pedagogically and historically, I find them to be an outmoded approach to teaching and learning. Why?First, historically.  At the very least, lectures serve a once-relevant but now obsolete purpose: to disseminate information and knowledge.  Dating back at least to medieval universities, the purpose of the lecture was to convey information from an original source to learners; there was ONE paper source of a particular text, so the professor read from the text and learners took notes – transcribing their own copy.  That purpose and function of the lecture was, at least, diminished with the advent of the printing press, and with the development of the internet and the ubiquitous availability of and access to information, it is entirely  unnecessary to disseminate information to learners.  They already have access to more information than can be consumed in a lifetime.  The range of available technologies makes more than a few traditional classroom learning methods obsolete, and in some cases, inconsistent with established and accepted theories and practices of teaching and learning.

Second, pedagogically.  I have firm beliefs regarding what constitutes effective teaching and learning; it can be defined explicitly and succinctly. Those beliefs are based on a thorough knowledge of learning theory and practice gained through my own teaching and classroom experiences – as a professor and as a student.  I do not believe a lecture inherently precludes an active, collaborative and authentic learning experience, but I believe it may certainly inhibit it.

For a lecture to be a meaningful and effective learning experience, students must have the ability and habits of a scholar’s mind to engage the content; they must be able independently to engage the lecture by actively processing new information, by relating and applying concepts to situations relevant to their academic or future professional career, by choosing to engage classmates or others in the learning process.  If they do not possess those skills, a lecture ends up being a very passive, solitary and uniquely academic experience – everything that is NOT effective teaching and learning.  Is it a 18-21 year old undergraduate student’s fault if they do not yet possess the habits or skills of a scholar?  It could be; there are certainly students that just don’t care and don’t make the effort.  However, a lecture focused on content does nothing to model, instruct or otherwise facilitate learners engaging a scholarly process of interacting with and applying new knowledge.  If a learner does not possess the meta-cognitive awareness to manage and direct their own learning experience during and following a lecture, that is not their failure alone.

My own experience was that I did not fully develop my own meta-cognitive awareness until I was several years into my graduate career; looking back, I had started engaging lectures more independently during the last year or two of my undergraduate education, but I was not fully aware of what I was doing or the process in which I was engaging.  During those times though, I was working with and learning from faculty that were deliberately teaching scholarly, academic habits of mind; they weren’t just delivering content.  Given that experience, I believe fewer undergraduate learners than faculty think or believe have progressed far enough into their academic careers to be fully prepared to make a lecture the learning experience it could be.  Further complicating the issue, perhaps, is that faculty DO possess the skills for a lecture to be truly and deeply meaningful, so the lecture, as an instructional tool, seems much more effective than it actually may be.

Professionally and personally, I’d prefer to assume that most of my undergraduate students have not yet had an opportunity to fully develop the requisite skills to learn actively, collaboratively, and authentically from a lecture.  For me, methods other than lecture are much more likely to facilitate an active, collaborative and authentic learning experience.  I believe other methods are more likely to be effective even though a lecture is much less work, on my part, than designing, developing, implementing and effectively facilitating an active, collaborative, and authentic learning experience for my students.

In discussion with colleagues, faculty have differing opinions regarding the validity of local incentives that have an impact on the course grade.  The argument that completing an end of course survey is not a valid assessment is quite accurate; in that instance, a portion of the student’s grade is based on something other than their performance in the classroom.  I absolutely understand and respect the argument.

With that said, if I can go from a 16% response rate to a 90%+ response rate just by offering extra points on the final exam, is the feedback for me not worth giving students a bump on the final exam?  If the extra points amount to a 1 point bonus on the course average? a half point?  1.5 points? 2 points?  What’s too much?  We are talking about a significant source of “quality control” for the course?  The teaching and learning environment I provide and facilitate is the service I’m providing; the end of course evaluations are a source of quality control for that service.  How many extra points is it worth to get feedback in a single section from 24 students rather than 4?

Your thoughts?

Back up a small step . . .

When evaluating the effectiveness of instruction, there are direct and indirect methods; the Texas A&M Office of Institutional Assessment defines those two concepts fairly well, for reference as needed.  Direct methods are those which “require students to produce work so that reviewers can assess how well students meet expectations.”  As a direct measure, “course embedded assessments” are those that students complete during a course; that is in contrast to assessment methods like certification/licensure exams, other standardized exams, student portfolios, or other work that students may do – in addition to coursework – for the purpose of assessment and evaluation.  However, in conversations focused on institutional evaluation of general education outcomes, I have noticed two different levels of the extent to which assessments are embedded.

First, some institutions are leveraging only the artifacts students produce during a course; they sample a number of sections of a particular course and all student work from a particular assignment.  For example, to evaluate general education outcomes, a lab report assignment from 15 different sections of a microbiology course may be sampled.  Those assignments are collected and graded by a group of faculty (other than the faculty teaching the course) against a rubric measuring level of student attainment regarding critical thinking.

Second, other institutions are leveraging the artifacts students produce during a course AND the assessment of the work also occurred during the course.  For example, to evaluate general education outcomes, the same lab report assignment from 15 different sections of a microbiology course may be sampled.  Those assignments AND the assessment of the work by the faculty member – against rubric criteria or an entirely separate rubric measuring level of students attainment regarding critical thinking – are aggregated.  In short, the assignment is not regraded outside the scope of the course; the institutional assessment effort relies on the grading of work by faculty teaching the course.

If you work with evaluation of general education program/outcomes in your institution, what is your understanding of “course embedded assessment?”  Which interpretation represents your institution’s approach?  What are the pros and cons of implementing each approach?

Working with a broad range of faculty and instructional design types, I believe there’s some confusion within education regarding Bloom’s Taxonomy.  Specifically, it’s often perceived and applied as a hierarchy rather than a taxonomy.  Quite bluntly, that is incorrect and counterproductive to effective teaching and learning.

The image to the right presents the concepts/labels with which many educators are familiar: remember, understand, apply, analyze, evaluate and create.  Unfortunately, many educators consider the particular levels of complexity to be a linear process through which students progress.  Less mature or sophisticated learners are perceived to function only at the lower order levels of thinking : remember and understand.  I’ve heard comments that college students in introductory or survey courses must first master remembering and understanding before moving on to higher order thinking.  Certainly, a certain amount of understanding is requisite before analyzing content, applying concepts, evaluating ideas, or creating new perspectives.  However, the design of instruction should not focus entirely on the lower levels of thinking simply because it’s an introductory course.

It is quite possible for younger, more immature and less sophisticated learners to think critically at higher levels.  One example. Spending time at a lake with my then 3rd grade daughter, we explored the concept of concentric circles by skipping rocks.  We discussed how the rock lands at a point in a smooth lake and causes ripples that expand outward, effecting the entire lake.  A few months later, my wife and I were discussing interpersonal relationships and dynamics at the dinner table.  Specifically, the dynamics of one relationship impacts another; for someone to think that actions toward one individual has zero effect on their relationship with other mutual relationships is naive.  My 3rd grade daughter was clearly listening; she chimed in that what we were talking about was very much like a rock landing in a lake.  While the rock only landed in one spot, it effected the entire lake, including areas in which the rock did not land.  EXACTLY.  That clearly demonstrated her ability to apply what she had learned while analyzing the new, entirely unrelated conversation she was hearing at the time.  It is counterproductive to suggest, think, or design learning experiences in a way that would preclude or inhibit my daughter from engaging in a conversation about interpersonal relationships or further discussing the concepts of fluid dynamics presented by skipping rocks until she gained more knowledge by remembering and understanding more ideas or concepts.  That higher order thinking enhanced the lakeside learning experience she had several months prior, and she learned something new about personal relationships in the process.

Bloom did not present a hierarchy through which we should try to walk students until they eventually reach higher order thinking; Bloom described lower and higher orders of thinking as a TAXONOMY.  The goal for ALL learning should be to help learners iteratively engage the various levels of thinking when learning new content.  Learning experiences should be designed in a manner that encourages learners to apply etc, as quickly as possible, the new knowledge they’ve remembered or understood.  Depending upon the sophistication and maturity of the learner, learning may require repetition to help remember and understand, but no learning experience – elementary, secondary or higher education classrooms – should end with simply understanding content.  There should *always* be an effort to apply, analyze, evaluate and create. Higher order thinking enriches, improves and makes more efficient the learning experience.

1. Objectives & Outcomes are distinct concepts. Blurring of the two may create significant issues for faculty and institutions regarding curriculum, assessment, and accreditation.
2. Instructional and curriculum design strategies exist regarding the development of effective objectives and effective outcomes. Two caveats. First, when using Bloom’s Taxonomy as a strategy, remember that it’s a taxonomy *not* a hierarchy. Second, outcomes written with a focus on knowledge and comprehension are arguably inappropriate for *many* college courses.
3. Given effective objectives and outcomes, assessment becomes the focus. A framework exists to facilitate continuous improvement of assessment methods.

When you enter phrases into the Google Books Ngram Viewer, it displays a graph showing how those phrases have occurred in a corpus of books (e.g., “British English”, “English Fiction”, “French”) over the selected years.

In short? Put words or phrases into the search box, separated by commas, select a time range, and the NGram viewer displays the frequency at which each appears in the corpus of text contained in the Google Books database.  A few quick searches, with predictable results, that I did when I first experimented with the tool included: (a) groovy, (b) laptop, and (c) hillbilly.

My question, “Where does @GoogleBooks plan for this project to go in the future? And, do they realize this could be a killer app for qualitative research?”  Imagine two things.

First, rather than only laying the tool over the top of the Google Books database, enable it’s use with any database; in particular, I’m thinking in terms of a research database that contains the texts and transcripts of interviews or other qualitative research data.

Second, add algorithms that flip the search function around; rather than user input dictating the results, develop algorithms that identify the most used words *and/or* phrases.  If you’re familiar with Wordles, think something similar to what Wordle.net does but with the added feature of identifying phrases and not just individual words.

The results could be an invaluable starting point for qualitative researchers with large quantities of data.  NGram used in that manner would identify for researchers the most frequently used words or phrases which takes a significant step toward identifying recurring themes across artifacts whether it be interview transcripts, texts, or documents etc.

photo credit: -Marlith-

Technological progress makes many things obsolete: horse drawn carriages as a means of regular transportation, broadcast television or printed newspapers as a primary or sole source of news and information, tests as reliable and valid forms of assessment . . .

Wait?  What?

Yep. Tests are an anachronism of an assessment era that is or should be fading into the past.  They no longer effectively serve the purpose they were intended to serve. Why?

Testing, as we know it (and as I understand it), grew up out of the post World War II, Skinnerian era of learning and cognitive psychology.  Tests were designed to estimate indirectly what educators could not observe directly: learner performance of knowledge, skill or ability in an authentic environment.  When teaching learners to become effective engineers, mathematicians, business managers, accountants, sociologists, psychologists, or even teachers, educators of years past had limited means by which they could observe learners performing in those respective capacities.  So, tests were designed to “estimate” the knowledge, skills, or abilities (KSA’s) a learner had acquired through a designed learning activity or course.  And, because tests were in fact estimates, it became necessary to evaluate the reliability and validity of those tests.  If educators could only estimate how well someone might perform after they’ve learned, it became essential to know that the instruments used to estimate were in fact (a) measuring the KSA’s they were designed to measure (validity), and (b) measuring those KSA’s accurately in each instance and over time (reliability).

Fortunately, a wide range of technologies have made it possible, as educators, to observe directly what once could not be  observed directly: learners performing KSA’s in a more authentic environment.  We know longer have to estimate how well:

• a future nurse may diagnose/triage a new patient; advanced simulation technologies ranging from virtual environments to complex simulation dummies allow faculty nurses to observe exactly how a learner performs;
• a future historian will be able to engage others in a meaningful discourse regarding the causes of three different wars in history and how that might relate to current politics; online discussions, including “voice boards,” ably capture such an ongoing discussion of which a faculty historian can later assess learner proficiency;
• a future (and of course current) computer user will be able to incorporate general education KSA’s regarding topics like computer and technological literacy; learners can simply record their own exhibition of that knowledge as they engage a salesman at the local electronics store to make an informed purchasing decision regarding their next computer;
• a future citizen and parent will be able to write while engaging their local city council or representatives to voice their valued opinion; the online world allows learners to post their comments in online public forums and allows faculty to observe and critique that written work (and to disagree if she so chooses).
• a future forensic accountant will be able to identify “the money trail” of an alleged money launderer; computer based simulations present complex problems using advanced algorithms and large databases to generate dynamic case-based scenarios and simulations.
• ( insert your example in the comments…. and the list goes on! )

So, in a significant number of instances, it is no longer necessary to estimate how well a learner may perform or exhibit KSA’s of a given discipline; the behavior can be observed much more directly in contexts that possess a good measure of authenticity.

From an educational perspective, continuing to embrace tests/exams/quizzes as a significant means of assessment is an anachronism; the usefulness of those tools has been displaced by technology.  In fact, it is possible to argue from a psychometric perspective that continued reliance on test-like instruments to *estimate* a learners performance rather than observing the actual performance is inherently less valid and less reliable.

The differences lie in the level of specificity each provides and the relationship of each to assessment methods and instructional activities.  Failure to understand and accommodate the differences can restrict academic freedom of faculty and complicate institutional efforts to manage curriculum and assessment.

Using the course I teach – COSC 1401 Introduction to Computers – I want to briefly illustrate the difference and the relationship between a learning outcome and a learning objective. 

OUTCOME

9. Demonstrate proficiency in the use of word processing, spreadsheet, database, and presentation applications.

OBJECTIVE

Given plain text of an existing research paper with 3-5 references cited in the bibliography, students will be able to format with 80% accuracy a MS Word document according to MLA guidelines including all document specifications (margins, headers, footers, use of styles, tab stops etc.) and use of the reference manager to manage the formatting of the bibliography.

The learning outcome provides a high level view of what a student will be able to accomplish by the end of the course; it points all faculty in the same direction.  However, from the learning outcome, do you have an idea of specific activities to be engaged?  or specific assessment methods that will be used? No.  In my opinion, that vagueness of learning outcomes is what enables a coherent curriculum while permitting academic freedom.  Both of those are critical to the profession.

In contrast, the objective is much more specific and serves to clarify, in part, what the outcome means.  From the objective, I believe you likely have a very good idea of the specific activities students will be engaging, and I believe you can easily begin to envision the assessment process as well.  If interested, feel free to describe the specific activities and assessment process in the comments.

Certainly, a number of learning objectives defined within the course may support and facilitate one learning outcome; that’s clear from the fact that there’s other tasks that must be addressed to be considered proficient in “word processing,” and there’s also tasks that must be addressed regarding spreadsheets, presentation software and databases.

The difference between an outcome and an objective is critical. Why?

Learning outcomes should be specified, at the very least, by institutions; that creates a coherent curriculum.  However, the development of learning objectives belongs in the hands of faculty or, at most, groups of faculty within an institution.  If the two are not clearly delineated, the terms “outcomes” and “objectives” may be (and are being, in some instances) used interchangeably.   That misplaces the  locus of control for the development of outcomes and objectives.  Institutions or, worse yet, accrediting or regulatory agencies specifying outcomes that are more akin to objectives creates a much more prescriptive curriculum than what is intended or desired.  That begins to restrict academic freedom and erode the professional creativity and responsibility of faculty, making the curriculum development process much more difficult.

Second, there should be many more objectives (tasks and assessments) IN a course than learning outcomes OF a course.  Without clear delineation between an outcome and an objective, the mission of professionals at varying points in the curriculum development process becomes clouded.  Without clarity in the differences between an outcome and an objective, a group charged with writing high level outcomes may quickly digress into writing objectives – thinking in terms of what activities and assessments could or should be included in a course.  That potentially leaves the curriculum development process with too many course outcomes that are actually objectives, and those objectives may be poorly written in an effort to not imply specific activities or assessment methods.

For example, imagine the objective in my example above being listed as an outcome and worded as, “Format a word processing document according to MLA guidelines.”  It’s too specific to be an effective learning outcome, and it’s not specific enough to be an effective learning objective.  Combine that with the additional poorly worded objectives necessary to address additional word processing tasks plus all of the tasks necessary for the other applications.  The result is at least 8-9 poorly worded objectives being listed as outcomes in lieu of the single learning outcome noted in the example. Of course, 30-40+ “outcomes” is unwieldy, so some will be removed from the curriculum leaving a confusing mish-mash* of objectives and outcomes to define the scope of a course.  Basically, it becomes a mess. (*mish-mash is a technical term in this instance.)

Further yet, with all of that confusion, imagine the difficulty imposed on institutions regarding assessment and accountability responsibilities.  Faculty reporting samples of assessment results of all course outcomes over a given period of time (4-5 years) is a manageable, necessary (yet still time consuming) process.  How difficult or impossible does that responsibility become with 20+ mish-mashed, poorly worded objectives rather than 7-8 well written outcomes each supported by well written, faculty developed objectives?

AND, all of those issues precede the issue of developing effective learning objectives.  That’s another, related-yet-separate ball of wax.

Long version and a few issues are described below.  After I explain all of this, please comment and tell me what sort of things I don’t know about QM or licensing issues etc that preclude an institution from doing this ;-)

Several posts, tweets and conversations over the past week prompted what is probably not a new/unique idea

• Barry Dahl tweeted . . .
  So, the alternative title for this post is: “Why so many versions of the same online course?” Many institutions have as many versions of an online course as they have faculty teaching that online course.  Why?  Faculty typically develop their own course content, so 14 faculty teaching ENGL 1301 online for a community college means that community college has 14 uniquely developed online versions of the course.  I’ve encountered only a few instances in which institutions strategically develop online course materials to be distributed to all faculty teaching the course.
• I had a conversation with a friend that will be teaching adjunct for the first time at a community college. She commented how much work she has to do to figure out the course for the first time, and that’s even with her having extensive teaching experience.  It’s STILL a new course and a great deal of preparation.
• I noticed an institution that lists the courses and faculty developers that have received QM certification for their courses.  I wondered how much was spent on the QM review and know that only one course and group of students each semester will benefit from that quality review.

The Plan.  Given an course that has received QM Certification, offer the faculty developer of the course a nominal fee per semester for the rights to distribute that version of the course to every faculty member – adjunct and full time – that is teaching the course for the college – online or on campus. Faculty using the course have license to edit or modify the course as needed, and it may be copied from one semester to the next, but that copy must be reported and licensed.

The Benefits.  First, for less than the salary cost of one course section, every faculty member teaching the course has access to use a QM certified course.  That likely improves the online content available to students in a great number of courses; certainly, that won’t be true in every instance, many faculty develop high quality courses worthy of QM certification.  However, at the very least, providing the QM certified course will (a) allow experienced faculty an opportunity to add more high quality content to an existing, quality course, and (b) more importantly, provide a greatly needed resource available to adjunct faculty.  Providing adjuncts with online course materials for use in on campus or online courses would be a tremendous improvement over the predominant “Here’s-a-textbook-and-an-institutional-syllabi-for-the-course” approach to providing resources to adjuncts.  IF even 50% of the adjuncts teaching ENGL 1301 begins using the QM certified course, the quality of instruction for that course will improve as those faculty focus more on actually teaching rather than trying to scope, sequence, and develop course materials.

Second, the issue sidesteps the sticky IPR issue and finds a win-win-win for everyone involved.  This idea immediately raises the question of ownership, “Who owns the developed online course in the first place?” Faculty will immediately tell you it’s their course; they developed it.  However, institutions may answer that question differently.  Institutional intellectual property policies become critical, and if the faculty member leveraged any institutional equipment or resources while developing the course, the institution may have a legitimate “work for hire” claim to the course.  Rather than trying to work those out, the institution licenses the course at a very reasonable cost which increases the return on that investment; the faculty member receives additional compensation for licensing the course; and other faculty get a tremendous head start for making online content available to their students or a resource to add additional high quality content to their course.

Third, if the adoption rate of a licensed QM course becomes significant enough, the institution greatly reduces the task at hand in addressing course quality.  Propagating QM certified course materials via a $1500/semester cost is much more scalable than facilitating a QM review for every faculty member that teaches a course and infinitely more feasible than managing to get QM quality courses developed for or by every adjunct.

The Issues.  The primary stumbling block will be (a) any licensing issues with QM and (b) monitoring the potential for the continued use of licensed materials beyond the scope of the license.  I’m not entirely familiar with QM licensing; if anyone has insight relevant to this idea, I’d appreciate that being shared in the comments.   The institution will have to diligently monitor the continued use of the licensed material to ensure that the licensing of content from faculty developers is not abused.  What other issues might there be?

Your thoughts/comments are of interest.

The discussions focus on learners applying new knowledge regarding computer technology to make selection decisions.  For example, given new knowledge of storage and hard drives, students must choose which hard drive specifications for a new computer they are purchasing.  Students must explain their choice within the context of their personal computing needs and use habits; I refer to it as a “goldilocks” computer – it’s not too much or too little of a computer for them.  The goal is a perfectly efficient purchase that maximizes the use of every dollar spent.

The two criteria focus on students’ “knowledge & decision-making” and their level of “engagement & participation” in the online discussions.  The descriptors are in the attached PDF version of the discussion rubric.  Your feedback is of interest.

In this instance, a student procrastinated on a paper and turned in a draft that relied entirely on complex primary historical documents; the paper presented advanced analysis of those documents requiring, arguably, the proficiency of a graduate student or professional historian.  When confronted by the professor, who “voiced concerns about the integrity of the document,” the student was unable to adequately defend or discuss the contents of the paper.  A failing grade was assigned, and a seemingly thorough institutional appeals process was followed resulting in the student’s degree being delayed given the “blatant nature” of the plagiarism.

Ultimately, however,

College authorities were unable “to identify additional sources from which [the student]  material used in [his] paper, the preponderance of the evidence indicates that other parts of the paper are not [his] work,”

. . .

[The Student] was never “confronted with the source from which he was charged with plagiarizing.”

For me, that’s a significant concern.  As I commented on the TCCTA blog, if the integrity of a faculty member were challenged, a significant burden of proof would be required by an institutional review process before rendering a decision that would tarnish a professor’s academic reputation and jeopardize their future, and that burden of proof would almost absolutely require that a plagiarized document be identified for comparison.  However, as the court’s decision in this case noted,

the College’s Student Academic Honesty Code (hereinafter the Code) does not define plagiarism to require that the source of the plagiarism be specifically identified. While the faculty handbook suggests that any plagiarism charge be accompanied by “a comparison of the source document with the plagiarized document,” such a submission is not mandated by the College’s rules and regulations…

The court ruled that “compelling circumstantial evidence exists,” and as it is described in paragraph 6 of the decision, I would agree. However, I have grave concerns with having two standards for defining plagiarism in scholarly work; I believe the proficiency and academic experience of the scholar is immaterial.  I don’t want students to plagiarize, and if I believe they are, I certainly want to pursue the issue.  But, should we not value student scholarship enough that we require a significant burden of proof to support accusations of plagiarism – that we should at least require a plagiarized document be produced as we would if challenging any other scholar?  How firm is the ground on which we choose to stand if we judge a student to have plagiarized based on the judgement of faculty?

Certainly, as faculty, I value the professional judgement of faculty, but I personally would not challenge the integrity of a student unless I had a source document in hand.  I will not challenge a student’s integrity based on suspicions and circumstantial evidence – no matter how compelling; by doing so, I feel I am placing my own integrity and professionalism at risk.

First, I believe Bloom’s Taxonomy is often mis-interpreted and misapplied by educators.  Repeatedly, I have experienced educators that interpret the lower levels of thinking to be appropriate for introductory and survey level college courses and that the higher order thinking skills are appropriate for advanced, or junior, senior, and graduate level courses.  The impact of that is that early college learners in those courses are limited to only rote knowledge experiences.  Of course, that’s a problem with the implementation of Bloom’s theory and not the theory itself.  However, it’s still important given the impact that Bloom’s has on the learning experience, and it leads into a related argument…

Second, Bloom’s Taxonomy – at least in it’s popular repetitions – fails to acknowledge that learners may perform at varying levels of proficiency within each type of higher order thinking skill.  It’s not that an early college learner is incapable of application, analysis, synthesis and evaluation; they simply will not perform with an expert level of proficiency in those higher order thinking skills; they should be expected to apply, analyze, synthesize, evaluate, or create, but they will do that at a novice level.  For example, a student in a first year micro-biology class can and should be expected to apply knowledge of cell structures and epidemiology to identify a particular organism; however, the level of difficulty of the problem should be appropriate for the first year micro-biology student and not require advanced declarative or procedural knowledge which typically requires advanced study in micro-biology.

Third, technology makes possible many more avenues for students to perform and to be assessed; the range of simulations and interaction that can be created through technology enables more authentic problem solving opportunities.  Plus, the increasing demands in academia and the workplace for learners to be better prepared suggest learners need more authentic learning experiences.  The combination of those to facts indicates learners need to be performing to apply knowledge in as close to “real life” situations as possible.  This supports Dr. Sugrue’s argument that “all objectives are at the use level (that is, “performance” objectives) and that learners will practice or be assessed on the particular performance in representative task situations.” We should be observing students performing as they will need to in the future – and measure that performance, at whatever level of expertise is appropriately and reasonably expected of that learner given their prior learning experiences.  Simply “knowing” or “comprehending” something is not enough.

Perhaps classroom assessment would benefit from focusing on simply engaging learners with active, collaborative and authentic learning experiences and measuring their performance according to the level of expertise the learners should exhibit in that environment.

First things first : A learning outcome and a learning objective are two distinct concepts.  The delineation of the two concepts is critical to effective design of learning and assessment for a course, and subsequently, the management of assessment practices.

Assessment of learner outcomes and performance has become increasingly important in a political and accreditation environment focused on evidence, data and accountability.  While institutions work to establish procedures to ensure learner assessment results are documented, reported and used to improve instruction, the success of those improvements is dependent upon or assumes the identification of quality learning outcomes, learning objectives, and learning activities.  Effective learning experiences begin and end with assessment derived from defined outcomes and objectives.  The focus question then is, “How do we develop effective learning outcomes and objectives?”   

First, the terms “learning outcome” and “learning objective” often are used interchangeably – even by consultants and statewide educational leadership.  However, within the formal practice of assessment and instructional governance, a distinction between the two is critical.  The differences lie in the level of specificity each provides and the relationship of each to assessment methods and instructional activities.  Failure to understand and accommodate the differences can restrict academic freedom of faculty and complicate institutional efforts to manage curriculum and assessment.  Each of the differences and the implications of both are discussed in depth.

Second, learning outcomes for a given course or program/discipline often have long been defined by experts within the respective field, and in some cases, the learning outcomes are dictated by governing or accrediting agencies.  So, the conversation focuses on developing effective learning objectives.  Methods from the instructional design field aid in the development of effective learning objectives.  Those methods will be described and discussed followed by interactive discussion among faculty applying the method to re/develop objectives for one of their own courses.