Versions Compared

Version Old Version 11 New Version 12
Changes made by

Mark Gallimore

Mark Gallimore

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Here, COLI is assembling a list of resources concerning artificial intelligence, and it's possible implications for pedagogy and scholarship.

...

  • descriptions of a book whose text or detailed summaries of the same are not in the AI's training data.  The AI might develop a plausible but false interpretation or summary based on the book's title, or what information it may have on the book's subject.
  • scientific or engineering explanations of complex phenomena.
  • biographies of non-famous individuals. (Try asking for a short biography of you and your title, if it is already publicly available on the web.  You may receive a fantastic, if false biography.)


Pedagogy 

Sources

LLM AIs have learned primarily on open-sourced content.  This might be on the internet, or books that are out of copyright.  There may be exceptions in unpublished training aids.  But much of what we assign is copyrighted content, out of necessity, since that is where specialized disciplinary knowledge is found.  Writing assignments that ask students to focus on these specialized resources will not be accessible to generative AIs.

...

Beyond your current assignment prompts, ask the AIs to perform the types of analyses that are core skillsets for your discipline.  Can it accurately perform calculations of a sort?  Can it interpret types of evidence commonly used by professionals?  Can it identify important elements in a given text, according to certain scholarly or professional priorities?  Can it discuss relevant literature on a particular topic, or describe debates within a discipline?  And with all of the above, can it provide depth, detail, or precision that you expect students to exhibit when completing assignments?  

Ask the AI

As a starting point, it can be helpful to ask the AIs directly what they cannot do, but that are typical learning outcomes of a discipline.  They might answer with good insights.  For example, Chat GPT suggests several things that undergraduate students should learn to do in history college classes, but that LLM AIs will not be able to do effectively:

History

  1. Primary source analysis: Assign students to analyze and interpret primary sources, such as letters, diaries, or government documents. This task requires critical thinking skills and the ability to contextualize information, which LLM AI models cannot fully replicate.

  2. Historiographical essay: Have students write an essay that evaluates the historiography of a specific historical event or period. This assignment requires students to synthesize and critically analyze a variety of sources, which LLM AI models may not be able to fully replicate.

  3. Historical interpretation: Assign students to write a short essay that offers an original interpretation of a specific historical event or figure. This task requires creativity, critical thinking, and the ability to make connections between different sources and information, which LLM AI models may not be able to fully replicate.

  4. Group research project: Assign students to work in groups to research and present on a specific historical topic. This task requires collaboration, communication, and the ability to synthesize information from multiple sources, which LLM AI models may not be able to fully replicate.

  5. Oral presentation: Ask students to give an oral presentation on a historical topic or event, and to answer questions from the class. This task requires public speaking skills, the ability to think on one's feet, and the ability to explain complex ideas in a clear and concise manner, which LLM AI models may not be able to fully replicate.

However, be aware that an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.)  Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity.  

...

Biology

  1. Laboratory experiments: Assign students to conduct experiments in a laboratory setting, where they can practice hands-on skills such as pipetting, culturing microorganisms, or using microscopes. LLM AI models cannot fully replicate the experience of conducting experiments in a physical laboratory environment.

  2. Data analysis and interpretation: Have students analyze and interpret data from a scientific study or experiment. This assignment requires critical thinking skills and the ability to draw conclusions from complex data, which LLM AI models may not be able to fully replicate.

  3. Literature review: Assign students to write a literature review that synthesizes and evaluates research on a specific biological topic. This task requires the ability to critically analyze and synthesize information from multiple sources, which LLM AI models may not be able to fully replicate.

  4. Field study: Ask students to conduct a field study of a specific biological ecosystem, and to write a report that details their observations and conclusions. This task requires observation skills, the ability to identify different species, and the ability to make connections between different aspects of an ecosystem, which LLM AI models may not be able to fully replicate.

  5. Oral presentation: Have students give an oral presentation on a specific biological topic or research study, and to answer questions from the class. This task requires public speaking skills, the ability to explain complex ideas in a clear and concise manner, and the ability to think on one's feet, which LLM AI models may not be able to fully replicate.

Management

  1. Case study analysis: Assign students to analyze and solve a real-world business case study. This task requires critical thinking skills, the ability to identify and evaluate different solutions, and the ability to make decisions based on incomplete information, which LLM AI models may not be able to fully replicate.

  2. Group project: Have students work in groups to develop a business plan or strategy for a hypothetical or real-world business. This task requires collaboration, communication, and the ability to synthesize information from multiple sources, which LLM AI models may not be able to fully replicate.

  3. Leadership reflection: Ask students to write a reflection paper on their own leadership style and how it has evolved throughout the course. This task requires self-awareness, the ability to identify and evaluate one's own strengths and weaknesses, and the ability to reflect on personal experiences, which LLM AI models may not be able to fully replicate.

  4. Role-playing exercise: Assign students to role-play different scenarios related to management and leadership, such as handling a difficult employee or negotiating a contract. This task requires interpersonal skills, the ability to read and respond to social cues, and the ability to think on one's feet, which LLM AI models may not be able to fully replicate.

  5. Oral presentation: Have students give an oral presentation on a specific business topic or strategy, and to answer questions from the class. This task requires public speaking skills, the ability to explain complex ideas in a clear and concise manner, and the ability to think on one's feet, which LLM AI models may not be able to fully replicate.

To generate answers similar to these, here's the prompt: What are some assignments for a undergraduate university (discipline) course that have students practice or demonstrate things LLM AIs cannot do for them?

Like a lot of responses from LLM AIs, these suggestions are typically vague.  And an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.)  Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity.  But these can be a good starting line for your process of thinking about assignments that are "AI proof"

Innovative Pedagogy

If you need to "AI-proof" your course, you have the opportunity to do something more.  Can you make your assignments more effective as opportunities for students to practice or demonstrate the skills embodied in your learning objectives and goals?  

Academic and professional disciplines across campus usually advertise some or another form of critical thinking and analysis skills as part of their learning outcomes.  This is also present in our general education programs, the Core Curriculum and All-College Honors Programs.  These tend to correlate with higher levels of Bloom's Taxonomy.  They can also be especially challenging to assess on classroom exams, at least in something approaching a real-world scenario.

But perhaps we can develop authentic assessments that challenge students to complete tasks poorly done, or altogether inaccessible to LLM AIs.  Many of these assignments may have been especially valuable before AIs existed.