Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Here, COLI is assembling a list of resources concerning artificial intelligence, and it's possible implications for pedagogy and scholarship.  This resource is tentative, certainly incomplete, might age rapidly, and probably wrong in places.  

...

Google Bard

As an AI, how are you classified?

"I am classified as a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories."

Can you describe your training corpus?

"Sure. My training corpus is a massive dataset of text and code that I was trained on. It includes text from a variety of sources, such as books, articles, websites, and code. The corpus is designed to teach me about the structure of language and how to use language to communicate effectively.

The corpus is also designed to teach me about the world. It includes text about a variety of topics, such as science, history, and current events. This helps me to understand the world around me and to generate text that is relevant and informative.

The corpus is constantly being updated with new text. This helps me to stay up-to-date on the latest information and to generate text that is accurate and up-to-date."

...

This tentative guide is concerned with Large Language Model AIs, which are (at the time this is written) the most powerful artificial text generation tools available.  Other tools, be it a customer service chatbot, a grammar assistant, or even the non-playable antagonist and their henchmen in video games, can be considered artificial intelligence, although they generally follow much simpler routines than LLM AIs.  However, in COLI we expect to see in the next several months and years LLM AI-powered tools and features appear across the internet, in software and on mobile devices.  So this distinction may not be relevant over time.  

Pedagogy 

Each discipline will need to determine the extent to which LLM AIs compel adaptation or alteration of their curriculum.  Each faculty member will also likely need to consider the relationship between their teaching style and methods, and LLM AIs.  However, here is a list of things that might spur some inspiration.

...

Experiment With AIs, Generally

...

Become Familiar With AIs

Try having conversations with the LLM AIs on topics both within your professional discipline, or just about anything else.  Get a sense of how they respond. 

Build Examples

If you ask AIs to perform the kinds of analysis you do in your discipline, you'll likely discover limitations.  Either they simply don't do it very well, or their penchant for falsehood will surface.  Document these examples to share with students.  You might (as we suggest below) ask the AIs to try your assignments, but even before that, simply ask them to summarize lengthy texts that are less likely to be the topic of considerable analysis (good, bad, and everything in between) on the internet.  

Here's an example: Allan Millett, Peter Maslowski, and William Fies organized their book For the Common Defense around six themes:

  1. First, national military considerations alone have rarely shaped military policies and programs.
  2. Second, American defense policy has traditionally been built upon pluralistic military institutions, most noticeably a mixed force of professionals and citizen-soldiers.
  3. Third, the popular belief that the United States has generally been unprepared for war, policymakers have done remarkably well in preserving the nation’s security.
  4. Fourth, the nation’s firm commitment to civilian control of military policy requires careful attention to civil-military relations.
  5. Fifth, the armed forces have become progressively more nationalized and professionalized.
  6. Finally, beginning in the mid-nineteenth century, but especially during the twentieth century, industrialization has shaped the way the nation has fought.

(Millett, Allan R., Maslowski, Peter, and William Feis. For the Common Defense: A Military History of the United States from 1607 to 2012. 3rd ed. New York: Free Press, 2012 pp. 14-15. Free Press. Kindle Edition.)

When asked to list these themes, ChatGPT had this to say: 

  1. Civil-Military Relations: The book explores the relationship between the military and civilian authorities. It examines how this relationship has evolved over time, the challenges that have arisen, and how they have been resolved.

  2. Technology and Innovation: The authors highlight the importance of technology and innovation in shaping the military history of the United States. They trace the evolution of military technology, from muskets to drones, and how it has influenced military tactics and strategy.

  3. Strategy and Operations: The book covers a range of military campaigns, including the Revolutionary War, the Civil War, and the Gulf War. It analyzes the strategic and operational decisions made by military leaders and the outcomes of these decisions.

  4. Military Leadership: The authors discuss the role of military leadership in the success or failure of military campaigns. They examine the qualities that make an effective military leader and the challenges that leaders face in times of war.

  5. Military Culture: The book explores the culture of the military and how it has evolved over time. It examines the values, norms, and beliefs that shape military organizations and the impact of this culture on military operations.

  6. The Impact of War on Society: Finally, the book examines the impact of war on American society. It discusses the social, economic, and political consequences of war, including the effects on veterans and their families.

Some of these are more or less close to the truth.  Most suggest that ChatGPT made a reasonable but erroneous guess based on the book title and perhaps some other publicly available descriptions.  Since the book has wide circulation and the earliest edition dates from the 1980s, this suggests that considerable quantities of long-form text, and especially copyrighted text, have not been included in the AI's training materials. 

This example could be supplied to students (perhaps assigned the Millet, Maslowski, and Feis text) as a warning that the AI production cannot currently, at least, be trusted at face value for veracity.

With Your Students

When we say that AIs are likely a part of the professional future for many of our students, and we need to prepare them to work with or around AIs, what we might mean is simply making students aware of AI's current likely behaviors, and developing in students a habit of remaining aware of trends in AI.  For example, if you are teaching economics or anthropology, you might periodically prompt AIs to discuss the day's class subject, content or activity, and then discuss with students how the AI responds.  

...

To generate answers similar to these, here's the prompt: What are some assignments for a undergraduate university (discipline) course that have students practice or demonstrate things LLM AIs cannot do for them?

Like a lot of responses from LLM AIs, these suggestions are typically vague.  And an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.)  Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity.  But these can be a good starting line for your process of thinking about assignments that are "AI proof"

...