大语言模型，如ChatGPT、Huggingface，有问必答，而且，如果要求大语言模型对同一问题重复给出答案，它们不厌其烦地会给出同一问题的不同答案，这些答案尽管大致一样，但是其表述方式、内容排列和答案长短都不一样，就像考生回答考卷的同一问题时一样，尽管答案大致一样，但是其表述方式和内容排列不相同。大语言模型的出现为AI时代的学生开卷考试如何命题提出了难题，特别是网络授课、网上考试的开卷考试如何命题提出了难题。我恰好遇到了这一难题，我们学校的国际学生今年还是网上授课、网上开卷考试。
我将2022年我讲授网课的开卷考试试题输入ChatGPT，模型很快就给出了答案。答题的速度比我阅读速度快。而且，可以要求模型重复给出答案，每次答案都不一样，但都大体正确。大语言模型解释名词的速度更快。谢天谢地，2022年我为我讲授的课程开卷考试命题时还没有大语言模型。但是，今年如何命题遇到了难题。尽管今年课程设计已经通过翻转课堂（学生演讲）和习题降低了期终考试分数在学生成绩中的比重。但是期终考试要是要考的。解铃还须系铃人。于是，我咨询了ChatGPT：
我问道：
In an open-book examination in which the students are allowed to access the internet, that is to say; they have access to large Language Models like ChatGPT. Is it still wise to ask them to answer questions in such a test?
ChatGPT答道：
If students are allowed access to large language models like ChatGPT during an open-book examination, asking them questions that such models can directly answer might not be wise. Since these models have access to vast amounts of information, they can quickly provide accurate and detailed responses to a wide range of questions.
However, it's important to consider the learning objectives of the examination. If the goal is to assess the student's ability to find, evaluate, and synthesize information from different sources, then asking questions that require critical thinking and analysis would still be valuable. For instance, instead of asking for straightforward facts, the questions could focus on problem-solving, application of knowledge, interpretation of data, or providing reasoned arguments.
Additionally, open-book examinations that allow access to the internet can encourage students to develop important skills such as research, information literacy, and the ability to apply knowledge effectively. It reflects real-world scenarios where individuals have access to vast resources and need to navigate and utilize them efficiently.
To ensure fairness and discourage plagiarism, the assessment methods could involve questions that require higher-order thinking skills, individual interpretation, or practical application of concepts rather than relying solely on factual recall that can be easily retrieved from a language model.
Ultimately, the examination design should align with the educational objectives and promote the development of skills relevant to the subject matter.
译文：
我：在开卷考试中，允许学生上网，也就是说他们可以访问像ChatGPT这样的大型语言模型。你认为在这样的考试中让他们回答问题仍然是明智的吗?
ChatGPT：如果允许学生在开卷考试中使用像ChatGPT这样的大型语言模型，那么要他们回答可以由这些模型直接回答的问题可能是不明智的。由于这些模型可以访问大量的信息，因此模型可以快速地对各种各样的问题提供准确而详细的回答。
然而，重要的是要考虑考试评定的目标。如果目标是评估学生从不同来源寻找、评估和综合信息的能力，那么提出需要批判性思维和分析的问题仍然是有价值的。例如，这些问题可以侧重于解决问题、知识应用、数据解释或提供合理的论点，而不是询问直接的事实。
此外，允许上网的开卷考试可以鼓励学生发展重要技能，如研究、获取信息和有效应用知识的能力。它反映了现实世界的场景：个人可以访问大量资源，需要有效地导航和利用它们。
为了确保公平和防止抄袭，评估方法可以包括要求高阶思维技能、个人理解或实际应用概念的问题，而不是仅仅依赖于可以从语言模型中轻松检索的信息。
最终，考试的设计应该与教育目标保持一致，并促进与课程相关的技能的发展。
受大语言模型的启迪，我大致有了今年开卷考试命题的思路。
转载本文请联系原作者获取授权，同时请注明本文来自蒋志刚科学网博客。
链接地址：https://wap.sciencenet.cn/blog-68091-1390929.html?mobile=1
收藏
