【Datawhale 大模型基础】第三章 大型语言模型的有害性(危害)

第三章 大型语言模型的有害性(危害)

As illustrated aforementioned, LLMs have unique abilities that present only when the model have huge parameters. However, there are also some harms in LLMs.

When considering any technology, we must carefully weigh its benefits and harms. This is a complex task for three reasons:

  1. Benefits and harms are difficult to quantify;
  2. Even if they could be quantified, the distribution of these benefits and harms among the population is not uniform (marginalized groups often bear more harm), making the balancing act between them a thorny ethical issue;
  3. Even if you can make meaningful trade-offs, what authority do decision-makers have to make decisions?

Preventing of LLMs' harmfulness is still a very new research direction. The current content focuses mainly on the following two points:

  1. Harm related to performance differences: For specific tasks (such as question answering), performance differences mean that the model performs better in some groups and worse in others.
  2. Harm related to social biases and stereotypes: Social bias is the systematic association of a concept (such as science) with certain groups (such as men) over others (such as women). Stereotypes are a specific and widely held form of social bias in which the associations are widely held, oversimplified, and generally fixed.

Due to the opacity of pre-training datasets for LLMs and their inclusion of web-crawled data, it is likely that they contain online discussions encompassing political topics (e.g., climate change, abortion, gun control), hate speech, discrimination, and other forms of media bias. Some researchers have identified misogyny, pornography, and other harmful stereotypes within these pre-training datasets. Similarly, researchers **have observed that LLMs exhibit political biases that exacerbate the existing polarization in the pre-training corpora, thereby perpetuating societal biases in the prediction of hate speech and the detection of misinformation.

Recent studies have delved into the potential sources of biases in LLMs (such as training data or model specifications), the ethical concerns associated with deploying biased LLMs in diverse applications, and the current methods for mitigating these biases. An interesting find is that all models exhibit systematic preferences for stereotype data, showing that there is an eager need to establish a high-quality pre-training database.

Toxicity and disinformation are two key harms that all the researchers concern. In the context of toxicity and disinformation, LLMs can be served as two purposes:

  1. They can be used to generate toxic content, which malicious actors can exploit to amplify their information dissemination;
  2. They can be used to detect disinformation, thereby aiding in content moderation.

The challenge of identifying toxicity lies in the ambiguity of labeling, where the output may be toxic in one context but not in others, and different individuals may have varying perceptions of toxicity. Jigsaw, a division of Google, focuses on using technology to address societal issues, such as extremism. In 2017, they developed a widely popular proprietary service called Perspective, which is a machine learning model that assigns a toxicity score between 0 and 1 to each input. This model was trained on discussion pages on Wikipedia (where volunteer moderators discuss editing decisions) and labeled by crowdworkers. And the website is: https://perspectiveapi.com/.

For disinformation, it is the deliberate presentation of false or misleading information to deceive a specific audience, often with an adversarial intent. Another similar noun is misinformation (can be considered as "hallucinations"), which refers to information that is misleadingly presented as true. It is important to note that misleading and false information is not always verifiable; at times, it may raise doubts or shift the burden of proof onto the audience.

A recent research hotspot is hallucinations. To differentiate between various types of hallucinations, the given source content of the model can be analyzed, such as the prompt, potentially containing examples or retrieved context. There are two types of hallucinations: intrinsic and extrinsic hallucinations. In the former, the generated text logically contradicts the source content. In the latter, users are unable to verify the accuracy of the output based on the provided source; the source content lacks sufficient information to evaluate the output, making it undetermined. Extrinsic hallucination is not necessarily erroneous, as it simply means the model produced an output that cannot be supported or refuted by the source content. However, this is still somewhat undesirable as the provided information cannot be verified.

To better compare the difference between them, I cite a figure from a survey:

p.s. Recently I find some insteresting paper that discuss abilities about LLMs, maybe I will make notes in Chinese after finishing datawhale study.

END

相关推荐
AI_gurubar1 小时前
大模型教机器人叠衣服:2025年”语言理解+多模态融合“的智能新篇
人工智能·机器人
搏博2 小时前
基于Python3.10.6与jieba库的中文分词模型接口在Windows Server 2022上的实现与部署教程
windows·python·自然语言处理·flask·中文分词
XINVRY-FPGA3 小时前
EPM240T100I5N Altera FPGA MAX II CPLD
人工智能·嵌入式硬件·fpga开发·硬件工程·dsp开发·射频工程·fpga
HuggingFace3 小时前
开源开发者须知:欧盟《人工智能法案》对通用人工智能模型的最新要求
人工智能
媒体人8884 小时前
GEO 优化专家孟庆涛:技术破壁者重构 AI 时代搜索逻辑
大数据·人工智能
小菜AI科技5 小时前
Windsurf 评测:这款 人工智能 IDE 是你需要的颠覆性工具吗?
人工智能
RaymondZhao345 小时前
【全面推导】策略梯度算法:公式、偏差方差与进化
人工智能·深度学习·算法·机器学习·chatgpt
yzx9910135 小时前
小程序开发APP
开发语言·人工智能·python·yolo
AKAMAI5 小时前
通过自动化本地计算磁盘与块存储卷加密保护数据安全
人工智能·云计算
无规则ai6 小时前
动手学深度学习(pytorch版):第四章节—多层感知机(5)权重衰减
人工智能·pytorch·python·深度学习