编程语言演化逻辑与新旧语言更替约束分析

注:本文为 "编程语言演化" 相关合辑。

英文引文,机翻未校。

中文引文,略作重排。

如有内容异常,请看原文。


编程语言演化逻辑与新旧语言更替约束分析

1 问题推演与现实观测

阅读《Hackers & Painters》中关于 Lisp 长期发展潜力的论述,可整理一组客观推演逻辑与行业观测现象。

依据摩尔定律,同等算力 CPU 的采购成本每 18 个月缩减至原有一半,硬件算力持续贬值;人力开发成本数值区间保持稳定,人力投入在软件开发总投入中的占比持续上升。结合 Andy and Bill's Law,软件开发流程中针对底层代码性能调优的资源投入持续收缩。

沿该逻辑推导,编程语言设计会同步出现两类变化:

  1. 语法入门门槛持续降低,句式表达贴合人类自然语言习惯;
  2. 程序运行阶段算力开销同步增加。

行业观测现象与上述推导存在偏差:当前工业场景大范围使用的编程语言均诞生于数十年前,初始语法规范适配早年算力储备有限的硬件环境,近年全新设计的编程语言未完成对存量老牌语言的替换。

一批通用编程语言集中诞生于 1990 ∼ 1995 1990 \sim 1995 1990∼1995 区间,发布时间记录如下:

  • Ruby( 1995 1995 1995)
  • Python( 1991 1991 1991)
  • R( 1993 1993 1993)
  • PHP( 1995 1995 1995)
  • Java、JavaScript( 1995 1995 1995)

2 编程语言发展四阶段划分(按开发驱动条件)

语言发布年份仅记录语法规范首次定稿时间,当前工业场景使用的语言标准均经过多轮版本迭代调整。编程语言发展历程可按开发驱动条件划分为三个完整阶段, 2012 2012 2012 年后出现企业生态驱动的第四阶段雏形。

2.1 第一阶段: 1950 ∼ 1970 1950 \sim 1970 1950∼1970,数值计算与数学表达驱动

该阶段语言开发目标围绕科学公式自动化编写,语法与运行机制包含大量可跨时代复用的设计,长期具备工程使用价值。

  1. FORTRAN( 1957 1957 1957)
    全称 IBM Mathematical Formula Translating System,开发目标替代手写汇编完成数值计算,设计同时兼容硬件适配与跨设备运行能力。 1990 1990 1990 年之前,该语言长期垄断数值计算场景; 1990 1990 1990 年末,对应场景开发工作逐步迁移至 C、C++。
  2. Lisp( 1958 1958 1958)
    开发背景为人工智能逻辑建模,以列表作为基础数据结构,原生搭载自动内存回收、可作为参数传递的函数机制。初代版本数值计算执行速度相较 FORTRAN 存在 10 ∼ 100 10 \sim 100 10∼100 倍差值,但语法结构适配逻辑描述类开发流程。
  3. COBOL( 1959 1959 1959)
    开发面向商业业务流程,不针对科学计算场景,语法句式贴合英文书面表达,仅设置数字、字符串两类基础数据类型。 1990 1990 1990 年末,全球约 2000 2000 2000 亿行业务程序基于该语言搭建,覆盖 80 % 80\% 80% 商业业务系统;大型主机硬件使用规模收缩后,该语言应用范围收窄,存量业务系统仍持续运行。

本阶段语言形成的语法、运行机制,陆续被后续各代编程语言吸收集成;受学习门槛、运行速度两类条件约束,自身应用规模逐步收缩。

2.2 第二阶段: 1970 ∼ 1990 1970 \sim 1990 1970∼1990,底层操作系统开发驱动

代表语言:Pascal( 1970 1970 1970)、C( 1972 1972 1972)、C++( 1983 1983 1983)。

此类语言程序执行效率稳定,语法逻辑可直接映射硬件运算流程,不存在多层抽象结构带来的额外算力开销;多年内持续维持工业使用规模,C、C++ 长期占据 TIOBE 榜单前三位。

1990 1990 1990 年前后问世的互联网配套语言,开发目标弥补 C、C++ 存在的三类局限:学习门槛偏高、静态编译迭代周期长、缺少动态运行特性;且全部依托 C、C++ 搭建底层运行时程序,时间线存在明确承接关系。

2.3 第三阶段: 1990 ∼ 2012 1990 \sim 2012 1990∼2012,互联网网页服务开发驱动

1990 ∼ 1995 1990 \sim 1995 1990∼1995 集中推出的动态语言,应用规模扩张依托互联网产业规模增长。

  1. Python、Ruby、PHP 适配网页服务后端开发:后端业务耗时多集中在 IO 交互环节,算力消耗占比偏低,牺牲部分运行速度可缩短整体开发周期,匹配工程落地约束;
  2. JavaScript 适配浏览器页面交互场景,语言完整开发流程依托网页前端需求生成。

2.4 第四阶段雏形: 2012 2012 2012 年后,企业自有生态平台驱动

科技企业针对自有软硬件生态开发专属编程语言,均为全新语法规范设计,形成稳定使用群体。

  • Google:Go( 2009 2009 2009)
  • Microsoft:C#( 2000 2000 2000)、F#( 2005 2005 2005)、TypeScript( 2012 2012 2012)【原文未标注 C#、F#、TypeScript 发布年份,补充如下】
  • Apple:Swift( 2014 2014 2014),IBM 完成 Swift 全生态适配
    同期具备完整工程落地能力的新式编程语言:Rust( 2010 2010 2010)、Zig( 2016 2016 2016)、Vlang( 2019 2019 2019)、Dart( 2011 2011 2011)、Solidity( 2014 2014 2014,区块链合约专用)、ReScript、MoonBit。

3 老牌语言难以被新式语言替换的客观约束

3.1 计算机底层硬件架构无颠覆性调整

CPU 运算逻辑、存储分层结构数十年未出现本质改动,早年语言适配硬件的运行规则、性能特征可直接延续使用。多核并行、SIMD 向量指令等新增硬件能力,可通过老牌语言版本更新补充配套语法特性完成适配,无需从零搭建完整编程语言。

3.2 不存在向自然语言单向靠拢的固定演化路径

  1. 人类完成一门编程语言系统化学习仅需数月,熟练掌握一门自然语言需要数年;若遵循前文推导逻辑,诞生时间更早的人类自然语言会优先完成淘汰,推导逻辑无法自洽。
  2. 1990 1990 1990 年后,硬件算力储备不再构成编程语言设计约束,算力提升不会推动语法持续向自然语言简化。语言设计需要同步平衡三组指标:开发耗时、程序运行速度、多人协作维护难度,不存在单向简化语法的演化规则。

3.3 存量开发资源抬高新式语言普及门槛

  1. 复用代码资产约束

    长期使用的语言配套足量工具库、开发框架、底层协议实现代码(TCP/IP、加密模块、图形界面、数据库驱动等),工程开发无需从零搭建底层运行模块;全新语言短周期内无法积累同等规模配套资源。

  2. 迭代成本低于完整重构成本

    现有语言出现设计缺陷时,行业通用处理方案为更新语言标准、新增语法特性,而非从零开发新式语言。C++20 新增模块、概念、协程机制,C++26 完善全流程模块运行体系,版本更新带来的语法调整幅度等效全新语言,同时兼容历史存量代码。

  3. 基础运行库维护人力约束

    现代软件配套基础库文件体积可达百兆级别,中小开发团队不具备长期维护一套完整全新基础运行库的人力储备;无全新颠覆性业务场景作为支撑时,新式语言缺少落地使用条件。

3.4 垂直行业与语言形成路径锁定关系

语言长期匹配固定行业开发场景,形成切换约束:

  • Python 匹配深度学习开发场景
  • Go 匹配云原生容器、K8s 基础设施开发场景
  • TypeScript 匹配网页前端工程场景
  • Kotlin 匹配 Android 移动端开发场景

企业存量业务系统完整替换编程语言,需要投入大量人力完成代码改写、功能测试、运维流程调整,多数业务无法承担对应成本。

4 21 世纪新式通用语言普及规模偏小的成因观测

4.1 语言产出数量对比观测

1980 1980 1980 年代后期,平均每月有实验性质编程语言推出;进入 21 世纪后,每年仅有少量完整规范的新式语言发布,TIOBE 榜单排名长期由 C、C++、Java、Python、JavaScript、C# 等老牌语言占据。

客观事实补充:编程语言开发工作未中断,仅大范围商用、全行业普及的新式语言数量偏少。各类小众实验语言持续产出,覆盖函数式语法、二维编程、文学类编程语言、Esolang 趣味语言、底层编译原型等方向,海外程序语言专业社区持续输出创新设计方案。

4.2 主流语言完整演化脉络

不存在脱离前置技术积累凭空开发的编程语言,每一类主流语言均由三类条件叠加生成:前代语言功能短板、对应时代硬件算力、落地业务场景。

  1. FORTRAN( 1957 1957 1957):解决汇编编写数值计算流程繁琐问题,验证高级语言执行速度可贴近手写汇编;代码缺少分层约束,大量使用 GOTO 跳转语句,大规模程序可读性偏低。
  2. ALGOL( 1958 1958 1958):提出代码块分层、局部变量作用域规则,现代代码花括号、缩进分层、局部变量定义语法均源自该语言;未实现大范围商用,语法规则成为后续全部语言的设计参考基准。
  3. C( 1972 1972 1972):配套 UNIX 操作系统开发,语法可直接对接硬件操作,规则简化,不增加多层校验逻辑;内存管理交由开发者手动操作,缓冲区越界、野指针等程序异常长期存在。
  4. C++( 1983 1983 1983):整合 Simula 面向对象分层能力与 C 底层执行速度,随工程需求持续新增模板、RAII、内存移动规则等语法,语法体系规模持续扩张。
  5. Java( 1995 1995 1995):规避 C++ 手动内存管理、未定义行为等运行风险,引入自动内存回收、跨设备虚拟机运行机制;通过语法约束降低运行异常概率,代码书写行数增加,虚拟机运行产生算力开销。
  6. Python( 1991 1991 1991):资源分配向缩短开发者耗时倾斜,吸收 ABC 语言缩进分层语法,简化原型开发流程,适配非专业开发人员;单线程执行算力存在约束。
  7. JavaScript( 1995 1995 1995):短期完成完整开发,融合 Scheme 函数语法、Self 原型继承规则,表层语法参考 Java,长期作为浏览器唯一原生脚本语言。
  8. Go( 2009 2009 2009):针对 C++ 编译耗时久、语法体系复杂问题开发,语法规则简化,原生搭载并行运行机制,适配云原生单文件静态编译部署需求。
  9. Rust( 2010 2010 2010):不搭载自动内存回收机制,依靠编译期所有权规则规避内存异常,同步实现底层硬件适配速度与程序稳定;语法学习流程步骤偏多。

4.3 语言设计多指标互斥约束

编程语言设计属于多指标平衡工作,各组优化目标存在互斥关系,不存在适配全部场景的统一标准。

  1. 执行速度与程序稳定互斥
    C 优先保障执行速度,内存风险管控交由开发者;Java 依靠自动回收机制降低运行异常,同步产生算力开销;Rust 通过编译期校验同步平衡两组指标,增加语法学习流程长度。
  2. 语法灵活度与协作维护难度互斥
    Python、Ruby 支持运行时动态调整类结构,小规模项目开发速度快;数十人协同开发的大型项目,易出现隐性逻辑覆盖问题;TypeScript 通过补充静态类型标注,收缩语法灵活范围,降低长期维护难度。
  3. 语法精简程度与逻辑表达范围互斥
    Go 删减复杂语法规则,泛型机制延迟补充;C++ 支持编译期元编程,逻辑表达覆盖范围更广,多人协作阅读、调试耗时同步上升。

4.4 动态、静态类型语法持续融合,创新空间收窄

硬件存储、算力资源稀缺的早年,FORTRAN、C、Pascal 采用静态类型规则,缩短编译执行耗时;Lisp 落地动态类型设计路线,后续 Python、Ruby、JavaScript 沿用对应规则。

现代两类语法持续融合:

  1. 动态语言补充静态类型标注工具:Python Type Hints、TypeScript;
  2. 新式静态语言完善类型自动推导机制,减少冗余类型标注代码:Kotlin、Swift。

两类语法设计边界持续收窄,独立创新空间收缩。

4.5 2026 2026 2026 年新增约束:AI 代码工具形成数据壁垒

  1. 主流代码大模型训练素材取自 GitHub 存量开源代码,Python、Java、JavaScript 累计百亿行级别程序样本,AI 工具可完成代码补全、缺陷排查、测试用例生成工作。
  2. 全新编程语言无历史代码样本,AI 工具无法提供配套开发辅助;开发者缺少切换使用动力,形成闭环约束:使用群体规模小 → 配套代码样本不足 → AI 适配度低 → 新增使用者数量持续收缩。
  3. AI 工具弱化新式语言"语法精简"优势:老牌语言样板代码行数偏多,使用者通过自然语言描述开发需求,即可由 AI 批量生成完整代码,无需专门学习轻量化新式语言。

5 编程语言长期演化两条可观测路径

5.1 存量老牌语言持续迭代更新

主流语言通过版本标准更新调整底层编译、运行规则,调整幅度等效全新语言。

  • C++ 通过模块、协程、概念调整代码组织、编译流程;
  • Python 2 与 Python 3 存在完整语法断层, 3.9 ∼ 3.11 3.9 \sim 3.11 3.9∼3.11 持续更新运行机制;
  • Swift、TypeScript 维持高频版本更新节奏,持续调整语法规则。

版本更新兼容存量业务代码,企业无需完整迁移开发语言,为行业主流升级方式。

5.2 自然语言作为上层需求载体,底层分化两类形式化语言

当前 AI 辅助开发完整链路:

使用者自然语言描述需求 → AI 转换为成熟编程语言代码 → 编译器生成硬件可执行指令

编程语言从使用者直接交互载体,转换为 AI 与硬件之间的中间转换载体。

自然语言存在语义歧义、模糊描述缺陷,无法直接编译运行,不会完全替代形式化编程语言;长期演化后语言体系分为两类:

  1. 通用底层语言:沿用现有配套资源,承担编译、硬件调度、程序运行职能;
  2. AI 专用中间语言:语法规则严谨,适配机器解析流程,人类阅读难度偏高,仅由 AI 自动生成,使用者仅通过自然语言完成需求描述。

6 边界补充说明

  1. 实验类编程语言开发未中断:全球程序语言研究社区持续输出创新原型,覆盖函数式、形式化校验、合约开发、硬件描述细分场景,仅难以突破资源壁垒进入通用工业开发领域。
  2. 垂直细分场景持续产出专用语言:区块链 Solidity、硬件描述语言、数据查询 SQL、自然语言转 SQL 工具链,场景专属需求支撑小众专用语言稳定使用。
  3. 行业语言竞争规则调整:早年竞争集中在语法、执行速度优化;当前新式语言扩大使用规模,需要绑定全新产业落地场景、兼容现有开发资源、配套完整 AI 辅助工具链,仅语法层面优化无法支撑大范围普及。

附录:相关文献

一、编程语言演化与历史

  1. Sammet, J. E. (1969). Programming Languages: History and Fundamentals . Prentice-Hall.

    编程语言历史与基础理论的权威综述,涵盖 FORTRAN、COBOL、ALGOL、Lisp 等早期语言的完整发展脉络。

  2. Wexelblat, R. L. (Ed.). (1981). History of Programming Languages . Academic Press.

    第一届 HOPL(History of Programming Languages)会议论文集,收录 FORTRAN、Lisp、COBOL、ALGOL、BASIC 等语言的创始人亲述历史。

  3. Bergin, T. J., & Gibson, R. G. (Eds.). (1996). History of Programming Languages II . Addison-Wesley.

    第二届 HOPL 会议论文集,涵盖 C、C++、Smalltalk、Ada、Prolog、SQL 等语言的历史回顾。

  4. Desmazières, A., Di Cosmo, R., & Lorentz, V. (2025). "50 Years of Programming Language Evolution through the Lens of a Unified Source Code Archive." HAL Open Science .

    基于 Software Heritage 超过 200 亿源文件的数据集,对 1970--2021 年间编程语言趋势进行量化分析,验证 C、Java、Python 等语言的长期主导地位。

二、Lisp 与《Hackers & Painters》

  1. Graham, P. (2004). Hackers & Painters: Big Ideas from the Computer Age . O'Reilly Media.

    其中"The Hundred-Year Language"等章节论述了 Lisp 的长期发展潜力与编程语言演化逻辑。

  2. Graham, P. (1993). On Lisp: Advanced Techniques for Common Lisp . Prentice Hall.

    Lisp 宏系统与元编程的经典教材,深入阐述 Lisp 作为"可编程编程语言"的设计理念。

  3. McCarthy, J. (1960). "Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I." Communications of the ACM , 3(4), 184--195.

    Lisp 语言的奠基论文,首次提出 S-表达式、递归函数求值等概念。

三、语言设计权衡与多指标约束

  1. Wirth, N. (1976). Algorithms + Data Structures = Programs . Prentice-Hall.

    Pascal 语言设计者 Wirth 的经典著作,系统阐述算法、数据结构与程序设计的关系,体现早期语言设计的简洁哲学。

  2. Stroustrup, B. (1994). The Design and Evolution of C++ . Addison-Wesley.

    C++ 创始人亲述语言设计决策与演化历程,是理解 C++ 复杂语法体系形成原因的文献。

  3. Kernighan, B. W., & Ritchie, D. M. (1978). The C Programming Language . Prentice Hall.

    C 语言标准教材,阐述 C 语言"接近硬件、高效、可移植"的设计哲学。

  4. Pike, R. (2012). "Go at Google: Language Design in the Service of Software Engineering." Proceedings of the 2012 ACM Symposium on New Ideas in Programming and Reflections on Software .

    Go 语言设计者 Rob Pike 阐述 Go 的设计动机------简化 C++ 的复杂性,适配大规模软件工程需求。

四、语言采用障碍与生态系统锁定

  1. Zeng, C., et al. (2019). "Identifying Barriers to Adoption for Rust through Online Discourse." Proceedings of the 9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018) . arXiv:1901.01001.

    通过 Reddit 和 Hacker News 的在线讨论,识别 Rust 语言采用的三类主要障碍:工具链推广不足、复杂指针别名模式表达困难、与现有语言生态集成的成本高。

  2. Fulton, S. R., et al. (2021). "Benefits and Drawbacks of Adopting a Secure Programming Language: Rust as a Case Study." Proceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2021) .

    通过 16 次深度访谈和 178 份问卷,系统分析 Rust 在企业级采用中的收益与障碍,包括学习曲线陡峭、依赖膨胀、编译耗时、库支持有限等。

  3. Tomas Petricek. (2022). "Cultures of Programming." Draft for HOPL IV .

    从数学文化、黑客文化和管理文化三个维度分析编程语言的设计动机差异,以 ALGOL、Lisp、COBOL 为例说明不同文化背景如何塑造语言特征。

五、COBOL 与遗留系统

  1. Thibodeau, P. (2013). "COBOL: The Most Important Language You've Never Heard Of." Computerworld .

    系统分析 COBOL 在金融、政府等关键行业的持续使用情况,以及迁移成本与风险。

  2. Micro Focus. (2022). "COBOL Application Footprint Survey."

    行业调研报告显示全球 COBOL 代码量已达 775--8500 亿行, 92 % 92\% 92% 的组织将其 COBOL 应用视为战略资产。

六、AI 与编程语言未来

  1. Chen, M., et al. (2021). "Evaluating Large Language Models Trained on Code." arXiv:2107.03374 .

    OpenAI Codex 论文,分析代码大模型的训练数据来源(主要为 GitHub 存量代码),以及模型性能与训练数据分布的关系。

  2. Bommasani, R., et al. (2021). "On the Opportunities and Risks of Foundation Models." arXiv:2108.07258 .

    斯坦福大学基础模型研究中心报告,讨论大语言模型对软件开发范式的潜在影响,包括"自然语言作为编程接口"的可能性与局限。

七、软件生态系统与网络效应

  1. Jansen, S., Finkelstein, A., & Brinkkemper, S. (2009). "A Sense of Community: A Research Agenda for Software Ecosystems." ICSE Companion 2009 .

    软件生态系统研究的奠基性论文,提出生态系统参与者的"桥梁"与"障碍"分析框架。

  2. Manikas, K., & Hansen, K. M. (2013). "Software Ecosystems --- A Systematic Literature Review." Journal of Systems and Software , 86(5), 1294--1306.

    软件生态系统领域的系统性文献综述,涵盖生态系统定义、分类、健康度评估等议题。


Andy and Bill's Law(安迪与比尔定律)

What Andy giveth, Bill taketh away.

安迪给予的,比尔拿走了。

The law originates from a humorous one-liner told in the 1990s during computing conferences. Andy in this law is Andy Grove, CEO of Intel and later Chairman of the board between 1987 and 2004. In those days Dennard scaling and the rapid evolution of semiconductor technology made a rapid growth of processing power for PCs possible and the single chip designs from Intel were quickly catching up with the much more expensive often multi-chip designs for workstation and server processors. New architectural features were added that made them more and more like the processors that were used in much bigger computers and capable of running multiple bigger applications safely together on a single machine, even for multiple users simultaneously. Bill in this law is Bill Gates, founder of Microsoft and its CEO and Chairman between 1975 and 2000. Especially in the '90s, when GUIs became popular, PC software rapidly expanded and always managed to use all available processing power, sometimes not feeling any faster than the previous version of a package on a previous generation of hardware. With some packages it felt as if you didn't really get that much more work done quickly even though your hardware became faster and faster. However, beyond this law is also a frustration of Andy Grove who felt that Bill Gates wasn't always making proper use of new performance-enhancing features of the new processors and hence not using the new hardware to the full potential. Intel introduced the 80286 in 1982 and it was first used by IBM in the IBM AT in 1984, but it took until 1987 before an OS appeared that was aimed at regular PC users and fully exploited the 80286, with OS/2, which was in fact mostly built by IBM. Though Microsoft should not be entirely to blame for this as it was not possible to use the extended mode (actually called protected mode) while keeping compatibility with older DOS software also, except via an undocumented trick (which is what OS/2 used). The 80386 launched in 1985 and was the first processor of Intel that offered a 32-bit instruction set, and it was also designed to run old 16-bit DOS programs well together with 32-bit software. Though there were some Unix variants that supported the CPU in 32-bit mode and a 32-bit version of OS/2 in early 1992, it took Microsoft until Windows NT 3.1 in July 1993 to come with an OS for typical PC use that fully exploited the 32-bit features and process isolation features for safe and robust multitasking of the 80386 (by then succeeded by the 80486 and Pentium).

这一定律源于 1990 年代计算机会议上流传的一句幽默俏皮话。定律中的 Andy 是 Andy Grove,Intel 的 CEO,后于 1987 年至 2004 年间担任董事会主席。在那个年代,Dennard 缩放定律和半导体技术的快速演进使得 PC 处理能力得以迅速增长,Intel 的单芯片设计迅速追赶上了昂贵得多且通常为多芯片设计的工作站和服务器处理器。新的架构特性不断加入,使它们越来越像大型计算机中使用的处理器,能够在一台机器上安全地同时运行多个大型应用程序,甚至支持多用户同时使用。定律中的 Bill 是 Bill Gates,Microsoft 的创始人,于 1975 年至 2000 年间担任 CEO 和董事会主席。尤其在 90 年代,随着 GUI 的普及,PC 软件迅速膨胀,总能用尽所有可用的处理能力,有时新版本在上一代硬件上运行起来并不比旧版本快多少。对某些软件包而言,尽管硬件越来越快,实际完成工作的速度似乎并没有显著提升。然而,这一定律背后也反映了 Andy Grove 的沮丧------他认为 Bill Gates 并未充分利用新处理器的性能增强特性,因而未能发挥新硬件的全部潜力。Intel 于 1982 年推出 80286,IBM 于 1984 年在 IBM AT 中首次采用该处理器,但直到 1987 年才出现面向普通 PC 用户并充分利用 80286 的操作系统------OS/2,而 OS/2 实际上主要由 IBM 开发。不过,Microsoft 不应为此承担全部责任,因为在保持与旧版 DOS 软件兼容的前提下,无法使用扩展模式(实际称为保护模式),除非借助一个未公开的技巧(这正是 OS/2 所采用的方法)。Intel 于 1985 年推出 80386,这是 Intel 首款提供 32 位指令集的处理器,同时其设计也能良好地运行旧的 16 位 DOS 程序与 32 位软件。尽管有一些 Unix 变体支持该 CPU 的 32 位模式,且 OS/2 的 32 位版本于 1992 年初问世,但 Microsoft 直到 1993 年 7 月的 Windows NT 3.1 才推出一款面向典型 PC 用户、充分利用 32 位特性以及 80386 进程隔离特性以实现安全稳健多任务处理的操作系统(此时 80386 已被 80486 和 Pentium 取代)。

It used to be common practice in much of the scientific computing community to ridicule Microsoft and to claim that UNIX and UNIX-derived operating systems are superior. A large part of the scientific computing community isn't doing any better though and we can think of several laws equivalent to Andy and Bill's law that apply to scientific computing.

科学计算界过去常常嘲笑 Microsoft,并声称 UNIX 及其衍生操作系统更为优越。然而,科学计算界的很大一部分人也并未做得更好,我们可以想到几条与安迪与比尔定律等效、适用于科学计算的定律。

What Andy giveth, Cleve taketh away.

安迪给予的,克利夫拿走了。

where Cleve is Cleve Moler who really started the development of efficient linear algebra libraries such as LINPACK and EISPACK, predecessors to LAPACK, but then also developed MATLAB as a user-friendly environment to experiment with those libraries and in 1984 was one of the founders of MathWorks, the company that went on to develop Matlab into what it is today. Matlab evolved into an excellent system to prototype numerical algorithms. However, its language is not nearly as efficient as traditional programming languages when it comes to execution efficiency and hence is a good way to slow down modern hardware. The one thing that Matlab does rather well, but only when used by a good Matlab programmer who understands how Matlab works, is exploiting the vector compute facilities of modern processors thanks to its use of optimised linear algebra libraries.

其中的 Cleve 是 Cleve Moler,他真正开启了高效线性代数库(如 LINPACK 和 EISPACK,二者为 LAPACK 的前身)的开发,随后又开发了 MATLAB,作为实验这些库的友好环境,并于 1984 年成为 MathWorks 的创始人之一,该公司将 MATLAB 发展为今日之形态。MATLAB 已演变为一个用于快速原型化数值算法的优秀系统。然而,在执行效率方面,其语言远不及传统编程语言高效,因此成为拖慢现代硬件运行速度的一种方式。MATLAB 做得相当好的一件事------但仅限于理解 MATLAB 工作原理的优秀 MATLAB 程序员------是利用现代处理器的向量计算能力,这得益于其对优化线性代数库的使用。

What Andy giveth, James taketh away.

安迪给予的,詹姆斯拿走了。

where James is James Gosling, the main developer of the Java programming language. Java, and many other programming languages from that area, may have good ideas to improve programmer productivity or make it easier to run code on multiple machines, but this also came at a cost of performance. The first versions of the Java virtual machine were just slow, and even later versions based on a just-in-time compiler (JIT compiler) are not that spectacular. Designers of just-in-time compilers have long promised us better performance than regular compilers as they can use runtime information to further improve the generated code, but the reality is that gathering that information and using it properly to improve the generated code is too expensive and cumbersome. Abstracting away too much of the memory system is also not a good idea as making proper use of the memory hierarchy is essential for performance and one cannot expect compilers to be able to do the necessary code transformations on their own. Integrating with code written in other programming languages that make it easier to write high-performance library routines is also very cumbersome. And getting garbage collection to work well in a distributed memory context also requires build-in support in the virtual machines for this type of parallelism and cannot be done via a simple library add-on (there has been an effort to do MPI for Java but that didn't work well because of this). Granted not everything about Java is bad though. The language did support concurrency in the base language and was hence ready for shared memory execution. And in theory the just-in-time compiler concept also allows to quickly adapt to new processor architectures, if it were not that the language lacked the features to enable to compiler to easily vectorise code, one of the features that influences performance of modern processors most on scientific code.

其中的 James 是 James Gosling,Java 编程语言的主要开发者。Java 以及同时代的许多其他编程语言,在提高程序员生产力或使代码更易在多台机器上运行方面可能有不错的理念,但这也以牺牲性能为代价。Java 虚拟机的最初版本运行缓慢,即便后来基于即时编译器(JIT compiler)的版本也并未带来显著提升。JIT 编译器的设计者长期以来承诺其性能优于常规编译器,因为它们可以利用运行时信息进一步优化生成的代码,但现实是,收集这些信息并正确利用以改进生成代码的代价过高且过程繁琐。对内存系统过度抽象也非良策,因为充分利用内存层次结构对性能至关重要,不能指望编译器自动完成必要的代码转换。与使用其他编程语言编写的高性能库代码集成也极为繁琐。此外,在分布式内存环境中实现高效的垃圾回收,还需要虚拟机内置对此类并行性的支持,无法通过简单的库扩展实现(曾有人尝试为 Java 实现 MPI,但因此未能成功)。不过,Java 并非一无是处。该语言在基础语言层面支持并发,因此已具备共享内存执行的能力。理论上,JIT 编译器的概念也允许快速适配新处理器架构,但前提是语言需具备使编译器易于向量化的特性------而这是影响现代处理器在科学代码上性能的最关键因素之一,Java 恰恰缺乏这一特性。

At some point Java gained some popularity in scientific computing basically because it became the first programming language taught at many universities and hence the one that beginning researchers were most familiar with, but it is mostly given here as an example of languages that try to abstract away to much of the underlying system architecture and hence tend to run with less than optimal efficiency.

Java 曾在科学计算领域获得一定 popularity,主要是因为它成为许多大学首门教授的编程语言,因而成为初涉研究者最熟悉的语言。但此处提及 Java,主要是作为一类试图过度抽象底层系统架构、因而运行效率难以达到最优的编程语言的示例。

What Andy giveth, Guido taketh away.

安迪给予的,吉多拿走了。

where Guido is Guido van Rossum, the original developer of the Python scripting language (back in 1989 already). Python for a long time was a scripting language only known by system administrators and the like, but from 2005 on, with the advent of NumPy, became more and more popular in scientific computing. The Python ecosystem exploded with lots of small and often badly tested packages, and the language designers got in the habit to break code with every minor release every 18 months. Moreover, the language is usually interpreted and the interpreter is extremely inefficient on much code. In fact, the Python designers aren't really to blame for this as the language was developed with a completely different purpose in mind and did a good job at that for a very long time. There have been several efforts to develop just-in-time compilers (and an ahead-of-time compiler to C) for Python. But as of today there is still no just-in-time compiler that does well on most Python code, and several companies that invested in the development of one have given up, though all just-in-time compilers will probably come with examples where they offer a 100x or 1000x speed increase over naively written pure Python code for small specific fragments. Cython is an example of an ahead-of-time compiler that needs some help as regular Python code doesn't really offer enough type information to generate efficient code. Numba and PyPy are two examples of just-in-time compilers where Numba seems to do best with code that heavily uses NumPy data structures while PyPy works better on non-NumPy code. Python 3.13 now also comes with a JIT, but it is disabled by default and the performance improvements so far are little if there are any at all.

其中的 Guido 是 Guido van Rossum,Python 脚本语言的最初开发者(早在 1989 年)。Python 长期以来只是系统管理员等人群知晓的脚本语言,但从 2005 年起,随着 NumPy 的出现,它在科学计算中变得越来越广泛。Python 生态系统急剧膨胀,涌现出大量小型且测试不充分的软件包,语言设计者养成了每 18 个月的小版本发布就破坏代码兼容性的习惯。此外,该语言通常采用解释执行,解释器在大量代码上的运行效率极低。事实上,Python 设计者对此并不应负主要责任,因为该语言最初是为完全不同的目的而开发的,并且长期以来在这方面表现出色。人们曾多次尝试为 Python 开发 JIT 编译器(以及将代码预编译为 C 的 ahead-of-time 编译器)。但迄今为止,仍没有一款 JIT 编译器能在大多数 Python 代码上表现良好,多家曾投资开发此类编译器的公司也已放弃,尽管所有 JIT 编译器都可能附带一些示例,展示它们在特定小段代码上相比朴素编写的纯 Python 代码可获得 100 倍甚至 1000 倍的加速。Cython 是一款 ahead-of-time 编译器的例子,但它需要额外辅助,因为常规 Python 代码未能提供足够的类型信息以生成高效代码。Numba 和 PyPy 是两个 JIT 编译器的例子:Numba 似乎在大量使用 NumPy 数据结构的代码上表现最佳,而 PyPy 在非 NumPy 代码上效果更好。Python 3.13 现在也内置了 JIT,但默认禁用,且迄今为止即便有性能提升,也十分有限。

Never mind that we also tend install Python and its packages from repositories that often only contain generic binaries compiled to run on as large a range of hardware as possible rather than binaries that exploit specific features of each processor to optimise performance. And the way Python packages are distributed, is also not very friendly to large parallel filesystems as Python packages tend to come as many small files...

更不用说,我们通常从软件仓库安装 Python 及其软件包,而这些仓库往往只包含为尽可能广泛的硬件范围编译的通用二进制文件,而非利用各处理器特定特性以优化性能的二进制文件。此外,Python 软件包的发布方式对大型并行文件系统也不太友好,因为 Python 软件包往往由大量小文件组成......

But here's the problem...(但问题在于......)

Of course it is easy to write bad performing code in any programming language, but the point is that there are languages where it is near impossible or even just impossible to write truly efficient code. One used to get away with that in the days that the performance-for-money ratio improved a lot with every new generation of hardware. But as we have discussed, these days are over for now and it may be a long time before they return. For now, progress will have to come from better, more efficient software that better exploits the features of current hardware, and we need computer languages that support that. We need computer languages that give the programmer sufficient control over data storage and data flows and hence help to exploit the hierarchy in memory in the node, and languages where parallelism is not an add-on but intrinsic to the language to make parallel programming less difficult.

当然,在任何编程语言中都容易写出性能低下的代码,但关键在于,有些语言几乎不可能甚至完全不可能写出真正高效的代码。在过去,每一代新硬件都能显著提升性价比,人们尚可容忍这一点。但正如我们所讨论的,那样的日子目前已告一段落,且短期内难以重现。眼下,进步必须来自更好、更高效的软件,以更好地利用当前硬件的特性;我们需要能够支持这一目标的编程语言。我们需要赋予程序员对数据存储和数据流充分控制权的编程语言,从而帮助利用节点内的内存层次结构;还需要将并行性内建于语言本身而非作为附加组件的语言,以降低并行编程的难度。

Before the '90s, computers were so expensive and so limited in what they could do compared to even a basic PC today that researchers had to pay a lot of attention to both good algorithms and a good implementation of those algorithms. After that, the attention to algorithms hasn't been that strong in all science domains, and the attention to a proper implementation has been poor, especially in the newer science domains. In some fields of scientific computing performance improvements have come as much from better numerical methods as from faster computers.

在 90 年代之前,计算机极其昂贵,与当今一台基础 PC 相比功能也十分有限,因此研究者必须高度重视优良算法及其高效实现。此后,并非所有科学领域对算法的关注都那么强烈,而对正确实现的关注更是不足,尤其在新兴科学领域。在某些科学计算领域,性能提升同样得益于更优的数值方法,而不仅仅是更快的计算机。

Given the slow-down of performance growth of hardware at a constant budget, it is clear that performance growth will have to come from better algorithms and better software.

鉴于在固定预算下硬件性能增长的放缓,显然性能增长将不得不依赖于更优的算法和更优的软件。

And a non-solution...(以及一个非解决方案......)

There may be a lot of discussion today about how quantum computers or computers with optical components will solve all problems. This is just a discussion by the hopeful. The reality today is that the quantum computer is still looking for an application in which it will excel, and currently needs to be attached to a rather powerful traditional computer also simply to turn the information that it produces in something useful. The cooling needed for a quantum computer is also extremely expensive as it needs to be cooled to nearly the absolute zero. Every fraction of a Kelvin above the absolute zero makes the results worse because of noise (which manifests itself as errors in the computation). Don't be fooled by the pictures you see of quantum computers: Most of these pictures don't even show the quantum computer nor all the hardware that is needed to measure the quantum state. The most popular pictures tend to show the nice-looking multistage cooling system.

如今可能有大量讨论认为量子计算机或光学计算机将解决所有问题。这不过是乐观者的空谈。现实是,量子计算机仍在寻找其能够脱颖而出的应用场景,目前还需要连接一台相当强大的传统计算机,仅仅是为了将其产出的信息转化为有用的结果。量子计算机所需的冷却也极为昂贵,因为它需要被冷却至接近绝对零度。绝对零度以上每升高几分之一开尔文,噪声就会使结果变差(噪声表现为计算中的错误)。不要被量子计算机的照片所迷惑:大多数照片甚至并未展示量子计算机本身,也未展示测量量子态所需的全部硬件。最受欢迎的照片往往只是展示外观精美的多级冷却系统。

The same holds for optical computers. The reality there is that we are still very far from integrated optical circuits, let alone ones that are powerful enough to do computations quicker than our current computers. The amount of money needed to develop that technology into something useful may turn out to be prohibitive unless we really hit barriers very hard.

光学计算机也是如此。现实是,我们距离集成光路还非常遥远,更不用说强大到比现有计算机更快完成计算的光路。将该项技术发展为实用形态所需的资金可能高得令人望而却步,除非我们真的遭遇了难以逾越的障碍。

In fact, we have seen this happening before. As we have discussed, flash memory has serious problems. Longevity is a big problem. And furthermore it is still a block-oriented medium limiting the ways in which it can be used. There were two promising technologies to replace it over time: the memristor and phase change memory. A form of the latter was brought to market as 3D XPoint by Micron and Intel in 2017. However, it was expensive compared to flash memory, partly also because there was a huge development cost that needed to be paid back with the relatively low initial volumes, and it would still have required a lot of funding to become truly price-volume competitive with flash memory. The technology was abandoned in 2022 because of that.

事实上,我们此前已见证过类似情况。正如我们所讨论的,闪存存在严重问题。寿命是一个大问题。此外,它仍然是面向块的介质,限制了其使用方式。曾有两种有望逐步取代它的技术:忆阻器(memristor)和相变存储器(phase change memory)。后者的一种形式于 2017 年由 Micron 和 Intel 以 3D XPoint 之名推向市场。然而,与闪存相比它价格昂贵,部分原因是巨额开发成本需要通过相对较低的初期销量来回收,且要成为与闪存在价格和容量上真正具有竞争力的产品,仍需要大量资金投入。该技术于 2022 年因此被放弃。

Moreover, those new computer architectures (and certainly quantum computers) will also require to rethink algorithms and implementations. So why should we put our hope on those new architectures if today we already refuse to think about algorithms and implementations?

此外,这些新型计算机架构(当然包括量子计算机)也将要求重新思考算法和实现。那么,如果今天我们已拒绝思考算法和实现,为何还要将希望寄托于这些新架构之上?


Reference