Here, COLI is assembling a list of resources concerning artificial intelligence, and it's possible implications for pedagogy and scholarship.  This resource is tentative, certainly incomplete, might age rapidly, and probably wrong in places.  

...

Large Language Model AIs, or more properly their chat versions, such as ChatGPT, Google Bard, or Bing Chat, are designed to simulate human typed-text conversation.  They are computer programs that, in addition to code written by software engineers and developers, have been trained on large quantities of (mostly) human-generated text.  Much of this is the open internet, but other sources occasionally have been added to the training corpus as well.  Versions have been in development for years, but LLM AIs emerged into broader public attention in late 2022, when the firm OpenAI made available, for free use by anyone on the internet, ChatGPT, which was powered by the GPT-3 LLM.  

What LLM AIs Do

These AIs can create things or do by generating text.  Just some examples are:

• recipes for food dishes,

• lesson plane for secondary school science classes,
• a cover letter accompanying a job application,
• a thank-you note,
• an essay on the development of the Code Napoleon,
• simulate a fifteen-year-old blogger reviewing a video game,
• code a module or particular task within a computer program.

Each of these examples, and any other successful production, requires a carefully written prompt from the user.  This must properly describe the user's intent, 

Importantly, AIs have limits.  If you ask them to describe those limits, they will usually enumerate.  For example, when asked why it occasionally gets things wrong, ChatGPT replies that its answers will reflect shortcomings in its training data: biases, incomplete or wrong information, or ambiguity.  Plus, it may struggle to interpret language within that training corpus.  

Equally important, although something that LLMs might not be able to articulate themselves, is that they present simulations of humans, instead of possessing human concepts of truth or correctness.  If an LLM AI is prompted to answer a question for which it does not have training data, it may decline to answer, or it may provide a plausible, but fictional answer.  These are what AI developers refer to as "hallucinations."  Some examples of these fictions could be:

• descriptions of a book whose text or detailed summaries of the same are not in the AI's training data.  The AI might develop a plausible but false interpretation or summary based on the book's title, or what information it may have on the book's subject.
• scientific or engineering explanations of complex phenomena.
• biographies of non-famous individuals. (Try asking for a short biography of you and your title, if it is already publicly available on the web.  You may receive a fantastic, if false biography.)

Pedagogy LLM AI Google Bard describes itself as follows: 

Any successful LLM AI production requires a carefully written prompt from the user.  This must properly describe the user's intent, and if the AI's response is not to be dependent on its training corpus, it the user's prompt must supply relevant data as well.  Already, education and technology experts speak of "prompt engineering" as a probably skill necessary for students in the future, but it remains to be seen whether that is true, as these tools (more or less) evolve.

Limits to LLM AIs

Importantly, the current LLM AIs have limits.  If you ask them to describe those limits, they will usually enumerate.  For example, when asked why it occasionally gets things wrong, ChatGPT replies that its answers will reflect shortcomings in its training data: biases, incomplete or wrong information, or ambiguity.  Plus, it may struggle to interpret language within that training corpus.  

Equally important, although something that LLMs might not be able to articulate themselves, is that they present simulations of humans, instead of possessing human concepts of truth or correctness.  If an LLM AI is prompted to answer a question for which it does not have training data, it may decline to answer, or it may provide a plausible, but fictional answer.  These are what AI developers refer to as "hallucinations."  Some examples of these fictions could be:

• descriptions of a book whose text or detailed summaries of the same are not in the AI's training data.  The AI might develop a plausible but false interpretation or summary based on the book's title, or what information it may have on the book's subject.
• scientific or engineering explanations of complex phenomena.
• biographies of non-famous individuals. (Try asking for a short biography of you and your title, if it is already publicly available on the web.  You may receive a fantastic, if false biography.)

How AIs do or do not develop to overcome these limits will be important for education.  At any point, in what ways can students and faculty responsibly use these AIs?  In what ways are these AIs not appropriate, given their limitations? 

What Is Or Is Not An AI?

This tentative guide is concerned with Large Language Model AIs, which are (at the time this is written) the most powerful artificial text generation tools available.  Other tools, be it a customer service chatbot, a grammar assistant, or even the non-playable antagonist and their henchmen in video games, can be considered artificial intelligence, although they generally follow much simpler routines than LLM AIs.  However, in COLI we expect to see in the next several months and years LLM AI-powered tools and features appear across the internet, in software and on mobile devices.  So this distinction may not be relevant over time.  

Pedagogy 

Each discipline will need to determine the extent to which LLM AIs compel adaptation or alteration of their curriculum.  Each faculty member will also likely need to consider the relationship between their teaching style and methods, and LLM AIs.  However, here is a list of things that might spur some inspiration.

Sources

LLM AIs have learned primarily on open-sourced content.  This might be on the internet, or books that are out of copyright.  There may be exceptions in unpublished training aids.  But much of what we assign is copyrighted content, out of necessity, since that is where specialized disciplinary knowledge is found.  Writing assignments that ask students to focus on these specialized resources will not be accessible to generative AIs.

...

To generate answers similar to these, here's the prompt: What are some assignments for a undergraduate university (discipline) course that have students practice or demonstrate things LLM AIs cannot do for them?

Like a lot of responses from LLM AIs, these suggestions are typically vague.  And an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.)  Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity.  But these can be a good starting line for your process of thinking about assignments that are "AI proof"

...