Here, COLI is assembling a list of resources concerning artificial intelligence, and it's possible implications for pedagogy and scholarship. This resource is tentative, certainly incomplete, might age rapidly, and probably wrong in places.
...
Google BardAs an AI, how are you classified? "I am classified as a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories." Can you describe your training corpus? "Sure. My training corpus is a massive dataset of text and code that I was trained on. It includes text from a variety of sources, such as books, articles, websites, and code. The corpus is designed to teach me about the structure of language and how to use language to communicate effectively. The corpus is also designed to teach me about the world. It includes text about a variety of topics, such as science, history, and current events. This helps me to understand the world around me and to generate text that is relevant and informative. The corpus is constantly being updated with new text. This helps me to stay up-to-date on the latest information and to generate text that is accurate and up-to-date." |
...
This tentative guide is concerned with Large Language Model AIs, which are (at the time this is written) the most powerful artificial text generation tools available. Other tools, be it a customer service chatbot, a grammar assistant, or even the non-playable antagonist and their henchmen in video games, can be considered artificial intelligence, although they generally follow much simpler routines than LLM AIs. However, in COLI we expect to see in the next several months and years LLM AI-powered tools and features appear across the internet, in software and on mobile devices. So this distinction may not be relevant over time.
Pedagogy
Each discipline will need to determine the extent to which LLM AIs compel adaptation or alteration of their curriculum. Each faculty member will also likely need to consider the relationship between their teaching style and methods, and LLM AIs. However, here is a list of things that might spur some inspiration.
...
Experiment With AIs, Generally
...
Become Familiar With AIs
Try having conversations with the LLM AIs on topics both within your professional discipline, or just about anything else. Get a sense of how they respond.
Build Examples
If you ask AIs to perform the kinds of analysis you do in your discipline, you'll likely discover limitations. Either they simply don't do it very well, or their penchant for falsehood will surface. Document these examples to share with students. You might (as we suggest below) ask the AIs to try your assignments, but even before that, simply ask them to summarize lengthy texts that are less likely to be the topic of considerable analysis (good, bad, and everything in between) on the internet.
Here's an example: Allan Millett, Peter Maslowski, and William Fies organized their book For the Common Defense around six themes:
- First, national military considerations alone have rarely shaped military policies and programs.
- Second, American defense policy has traditionally been built upon pluralistic military institutions, most noticeably a mixed force of professionals and citizen-soldiers.
- Third, the popular belief that the United States has generally been unprepared for war, policymakers have done remarkably well in preserving the nation’s security.
- Fourth, the nation’s firm commitment to civilian control of military policy requires careful attention to civil-military relations.
- Fifth, the armed forces have become progressively more nationalized and professionalized.
- Finally, beginning in the mid-nineteenth century, but especially during the twentieth century, industrialization has shaped the way the nation has fought.
(Millett, Allan R., Maslowski, Peter, and William Feis. For the Common Defense: A Military History of the United States from 1607 to 2012. 3rd ed. New York: Free Press, 2012 pp. 14-15. Free Press. Kindle Edition.)
When asked to list these themes, ChatGPT had this to say:
|
Some of these are more or less close to the truth. Most suggest that ChatGPT made a reasonable but erroneous guess based on the book title and perhaps some other publicly available descriptions. Since the book has wide circulation and the earliest edition dates from the 1980s, this suggests that considerable quantities of long-form text, and especially copyrighted text, have not been included in the AI's training materials.
This example could be supplied to students (perhaps assigned the Millet, Maslowski, and Feis text) as a warning that the AI production cannot currently, at least, be trusted at face value for veracity.
With Your Students
When we say that AIs are likely a part of the professional future for many of our students, and we need to prepare them to work with or around AIs, what we might mean is simply making students aware of AI's current likely behaviors, and developing in students a habit of remaining aware of trends in AI. For example, if you are teaching economics or anthropology, you might periodically prompt AIs to discuss the day's class subject, content or activity, and then discuss with students how the AI responds.
...
To generate answers similar to these, here's the prompt: What are some assignments for a undergraduate university (discipline) course that have students practice or demonstrate things LLM AIs cannot do for them?
Like a lot of responses from LLM AIs, these suggestions are typically vague. And an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.) Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity. But these can be a good starting line for your process of thinking about assignments that are "AI proof"
...