【Datawhale 大模型基础】第三章 大型语言模型的有害性(危害)

第三章 大型语言模型的有害性(危害)

As illustrated aforementioned, LLMs have unique abilities that present only when the model have huge parameters. However, there are also some harms in LLMs.

When considering any technology, we must carefully weigh its benefits and harms. This is a complex task for three reasons:

  1. Benefits and harms are difficult to quantify;
  2. Even if they could be quantified, the distribution of these benefits and harms among the population is not uniform (marginalized groups often bear more harm), making the balancing act between them a thorny ethical issue;
  3. Even if you can make meaningful trade-offs, what authority do decision-makers have to make decisions?

Preventing of LLMs' harmfulness is still a very new research direction. The current content focuses mainly on the following two points:

  1. Harm related to performance differences: For specific tasks (such as question answering), performance differences mean that the model performs better in some groups and worse in others.
  2. Harm related to social biases and stereotypes: Social bias is the systematic association of a concept (such as science) with certain groups (such as men) over others (such as women). Stereotypes are a specific and widely held form of social bias in which the associations are widely held, oversimplified, and generally fixed.

Due to the opacity of pre-training datasets for LLMs and their inclusion of web-crawled data, it is likely that they contain online discussions encompassing political topics (e.g., climate change, abortion, gun control), hate speech, discrimination, and other forms of media bias. Some researchers have identified misogyny, pornography, and other harmful stereotypes within these pre-training datasets. Similarly, researchers **have observed that LLMs exhibit political biases that exacerbate the existing polarization in the pre-training corpora, thereby perpetuating societal biases in the prediction of hate speech and the detection of misinformation.

Recent studies have delved into the potential sources of biases in LLMs (such as training data or model specifications), the ethical concerns associated with deploying biased LLMs in diverse applications, and the current methods for mitigating these biases. An interesting find is that all models exhibit systematic preferences for stereotype data, showing that there is an eager need to establish a high-quality pre-training database.

Toxicity and disinformation are two key harms that all the researchers concern. In the context of toxicity and disinformation, LLMs can be served as two purposes:

  1. They can be used to generate toxic content, which malicious actors can exploit to amplify their information dissemination;
  2. They can be used to detect disinformation, thereby aiding in content moderation.

The challenge of identifying toxicity lies in the ambiguity of labeling, where the output may be toxic in one context but not in others, and different individuals may have varying perceptions of toxicity. Jigsaw, a division of Google, focuses on using technology to address societal issues, such as extremism. In 2017, they developed a widely popular proprietary service called Perspective, which is a machine learning model that assigns a toxicity score between 0 and 1 to each input. This model was trained on discussion pages on Wikipedia (where volunteer moderators discuss editing decisions) and labeled by crowdworkers. And the website is: https://perspectiveapi.com/.

For disinformation, it is the deliberate presentation of false or misleading information to deceive a specific audience, often with an adversarial intent. Another similar noun is misinformation (can be considered as "hallucinations"), which refers to information that is misleadingly presented as true. It is important to note that misleading and false information is not always verifiable; at times, it may raise doubts or shift the burden of proof onto the audience.

A recent research hotspot is hallucinations. To differentiate between various types of hallucinations, the given source content of the model can be analyzed, such as the prompt, potentially containing examples or retrieved context. There are two types of hallucinations: intrinsic and extrinsic hallucinations. In the former, the generated text logically contradicts the source content. In the latter, users are unable to verify the accuracy of the output based on the provided source; the source content lacks sufficient information to evaluate the output, making it undetermined. Extrinsic hallucination is not necessarily erroneous, as it simply means the model produced an output that cannot be supported or refuted by the source content. However, this is still somewhat undesirable as the provided information cannot be verified.

To better compare the difference between them, I cite a figure from a survey:

p.s. Recently I find some insteresting paper that discuss abilities about LLMs, maybe I will make notes in Chinese after finishing datawhale study.

END

相关推荐
可触的未来,发芽的智生2 分钟前
追根索源-神经网络的灾难性遗忘原因
人工智能·神经网络·算法·机器学习·架构
CAE3203 分钟前
基于Ncode的新能源汽车电池包随机振动疲劳分析
人工智能·汽车·电池包·hypermesh·振动疲劳·optistruct
zzywxc7873 分钟前
自动化测试框架是软件测试的核心基础设施,通过预设规则和脚本自动执行测试用例,显著提高测试效率和覆盖率。
运维·人工智能·自动化·prompt·测试用例·流程图
尺度商业8 分钟前
2025服贸会“海淀之夜”,点亮“科技”与“服务”底色
大数据·人工智能·科技
AWS官方合作商9 分钟前
涂鸦智能携手亚马逊云科技,以全球基础设施与生成式AI加速万物智联时代到来
人工智能·科技·aws·亚马逊云科技
FunTester11 分钟前
拥抱直觉与创造力:走进VibeCoding的新世界
人工智能·语言模型·编程·vibecoding
liukuang11013 分钟前
飞鹤财报“新解”:科技筑牢护城河,寒冬凸显龙头“硬核力”
人工智能·科技
eqwaak016 分钟前
科技信息差(9.13)
大数据·开发语言·人工智能·华为·语言模型
技术程序猿华锋27 分钟前
深度解码OpenAI的2025野心:Codex重生与GPT-5 APIKey获取调用示例
人工智能·vscode·python·gpt·深度学习·编辑器
嘀咕博客34 分钟前
Stable Virtual Camera:Stability AI等推出的AI模型 ,2D图像轻松转3D视频
人工智能·3d·音视频·ai工具