An Updated version of this guide is published at the Canisius University AI and Society Initiative Site. This version is not updated after June 5th, 2023 |
---|
...
Google Bard As an AI, how are you classified? "I am classified as a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories." Can you describe your training corpus? "Sure. My training corpus is a massive dataset of text and code that I was trained on. It includes text from a variety of sources, such as books, articles, websites, and code. The corpus is designed to teach me about the structure of language and how to use language to communicate effectively. The corpus is also designed to teach me about the world. It includes text about a variety of topics, such as science, history, and current events. This helps me to understand the world around me and to generate text that is relevant and informative. The corpus is constantly being updated with new text. This helps me to stay up-to-date on the latest information and to generate text that is accurate and up-to-date." |
...
- descriptions of a book whose text or detailed summaries of the same are not in the AI's training data. The AI might develop a plausible but false interpretation or summary based on the book's title, or what information it may have on the book's subject. In COLI, we find that copyrighted works seem particularly troublesome in this respect. It is likely that such works, and text descriptions of those works, have not made it into LLM AI training data.
- scientific or engineering explanations of complex phenomena. One example is rocketry.
- biographies of non-famous individuals. Try asking for a short biography of you and your title, if it is already publicly available on the web. For example, ChatGPT had this to say about COLI Director Dr. Mark Gallimore: "Gallimore holds a Bachelor of Arts degree in English from St. Bonaventure University and a Master of Science degree in Education from Canisius College. He began his career in education as a high school teacher, teaching English and social studies in Buffalo, New York." While he has the highest respect for the aforementioned institutions and high school teachers, this biography of Dr. Gallimore is in every respect false.
We may say that LLM AIs "making stuff up" or "get it wrong," but they are not malfunctioning. They simulate human composition. Ars Technica's Benj Edwards offers a good explanation of this. But the short version was offered by OpenAI CEO Sam Altman, soon after ChatGPT became available in late 2022: "ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It's a mistake to be relying on it for anything important right now. It’s a preview of progress; we have lots of work to do on robustness and truthfulness."
...
This tentative guide is concerned with Large Language Model AIs, which are (at the time this is written) the most powerful artificial text generation tools available. Other tools, be it a customer service chatbot, a grammar assistant, or even the non-playable antagonist and their henchmen in video games, can be considered artificial intelligence, although they generally follow much simpler routines than LLM AIs. However, in COLI we expect to see in the next several months and years LLM AI-powered tools and features appear across the internet, in software and on mobile devices. So this distinction may not be relevant over time.
Pedagogy
Each discipline will need to determine the extent to which LLM AIs compel adaptation or alteration of their curriculum. Each faculty member will also likely need to consider the relationship between their teaching style and methods, and LLM AIs. However, here is a list of things that might spur some inspiration.
...
To generate answers similar to these, here's the prompt: What are some assignments for a undergraduate university (discipline) course that have students practice or demonstrate things LLM AIs cannot do for them?
Like a lot of responses from LLM AIs, these suggestions are typically vague. And an LLM AI–ChatGPT, for example-- may attempt to simulate the things it tells it can not do very well, if prompted by you (or a student.) Therefore, the faculty member needs strong command of any disciplinary knowledge involved in the assignment, if they are to assess student work for accuracy or integrity. But these can be a good starting line for your process of thinking about assignments that are "AI proof"
...