利用DuckDB luajit插件和openblas库对表中数据做矩阵运算

  1. 安装openblas库

    C:\d>wsl
    root@DESKTOP-59T6U68:/mnt/c/d# docker start gcc
    gcc
    root@DESKTOP-59T6U68:/mnt/c/d# docker exec -it gcc bash
    root@6ae32a5ffcde:/# cd /par
    root@6ae32a5ffcde:/par# whereis libopenblas.so
    libopenblas.so:
    root@6ae32a5ffcde:/par# apt install libopenblas0
    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    The following additional packages will be installed:
    libgfortran5 libopenblas0-pthread
    The following NEW packages will be installed:
    libgfortran5 libopenblas0 libopenblas0-pthread
    0 upgraded, 3 newly installed, 0 to remove and 117 not upgraded.
    Need to get 7534 kB of archives.
    After this operation, 47.4 MB of additional disk space will be used.
    N: Ignoring file 'debian.sources_org' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
    Do you want to continue? [Y/n]
    Get:1 https://mirrors.tuna.tsinghua.edu.cn/debian bookworm/main amd64 libgfortran5 amd64 12.2.0-14+deb12u1 [793 kB]
    Get:2 https://mirrors.tuna.tsinghua.edu.cn/debian bookworm/main amd64 libopenblas0-pthread amd64 0.3.21+ds-4 [6709 kB]
    Get:3 https://mirrors.tuna.tsinghua.edu.cn/debian bookworm/main amd64 libopenblas0 amd64 0.3.21+ds-4 [32.6 kB]
    Fetched 7534 kB in 0s (15.4 MB/s)
    debconf: delaying package configuration, since apt-utils is not installed
    Selecting previously unselected package libgfortran5:amd64.
    (Reading database ... 65497 files and directories currently installed.)
    Preparing to unpack .../libgfortran5_12.2.0-14+deb12u1_amd64.deb ...
    Unpacking libgfortran5:amd64 (12.2.0-14+deb12u1) ...
    Selecting previously unselected package libopenblas0-pthread:amd64.
    Preparing to unpack .../libopenblas0-pthread_0.3.21+ds-4_amd64.deb ...
    Unpacking libopenblas0-pthread:amd64 (0.3.21+ds-4) ...
    Selecting previously unselected package libopenblas0:amd64.
    Preparing to unpack .../libopenblas0_0.3.21+ds-4_amd64.deb ...
    Unpacking libopenblas0:amd64 (0.3.21+ds-4) ...
    Setting up libgfortran5:amd64 (12.2.0-14+deb12u1) ...
    Setting up libopenblas0-pthread:amd64 (0.3.21+ds-4) ...
    update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in auto mode
    update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in auto mode
    update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so.0 to provide /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (libopenblas.so.0-x86_64-linux-gnu) in auto mode
    Setting up libopenblas0:amd64 (0.3.21+ds-4) ...
    Processing triggers for libc-bin (2.36-9+deb12u13) ...

2.安装duckdb luajit插件linalg库,如果网络不畅安装失败,可以再试

复制代码
root@6ae32a5ffcde:/par# ./duckdb155
DuckDB v1.5.5 (Variegata)
Enter ".help" for usage hints.
memory D load luajit;
memory D SELECT * FROM luajit_module(mode := 'install', sql_name := 'linalg');
┌─────────┬─────────┬─────────┬───────────────────────────────────────────────────────────────────┬─────────┬──────────┐
│   ok    │  mode   │  phase  │                              message                              │ detail  │ sql_name │
│ boolean │ varchar │ varchar │                              varchar                              │ varchar │ varchar  │
├─────────┼─────────┼─────────┼───────────────────────────────────────────────────────────────────┼─────────┼──────────┤
│ false   │ install │         │ install 'linalg' failed: IO Error: Timeout was reached error for  │         │ linalg   │
│         │         │         │ HTTP HEAD to 'https://raw.githubusercontent.com/alitrack/duckdb-l │         │          │
│         │         │         │ uajit-libs/main/INDEX'                                            │         │          │
└─────────┴─────────┴─────────┴───────────────────────────────────────────────────────────────────┴─────────┴──────────┘
memory D SELECT * FROM luajit_module(mode := 'install', sql_name := 'linalg');
┌─────────┬─────────┬─────────┬──────────────────────────────────────┬──────────────────────────────────────┬──────────┐
│   ok    │  mode   │  phase  │               message                │                detail                │ sql_name │
│ boolean │ varchar │ varchar │               varchar                │               varchar                │ varchar  │
├─────────┼─────────┼─────────┼──────────────────────────────────────┼──────────────────────────────────────┼──────────┤
│ true    │ install │ install │ installed 'linalg' (UDF) --- cached at │ luajit_table('linalg') or luajit_s(' │ linalg   │
│         │         │         │  /root/.duckdb/luajit-libs/linalg.lu │ linalg', ...)                        │          │
│         │         │         │ a                                    │                                      │          │
└─────────┴─────────┴─────────┴──────────────────────────────────────┴──────────────────────────────────────┴──────────┘

3.计算常量和表中数据的运算结果

复制代码
memory D SELECT luajit_s('linalg', {'op':'matmul', 'a':[1,2,3,4], 'm':2, 'n':2,'b':[5,6,7,8], 'mb':2, 'nb':2});
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ luajit_s('linalg', main.struct_pack(op := 'matmul', a := main.list_value(1, 2, 3, 4), m := 2, n := 2, b := main.list │
│                                        _value(5, 6, 7, 8), mb := 2, nb := 2))                                        │
│                                                       varchar                                                        │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ {"c":[19,22,43,50]}                                                                                                  │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
memory D create table d (a float[],b float[]);
memory D insert into d select range(1,101)::float[],range(2,202,2)::float[];
memory D SELECT luajit_s('linalg', {'op':'matmul', 'a':a, 'm':2, 'n':50,'b':b, 'mb':50, 'nb':2})from d where length(a)=100;
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ luajit_s('linalg', main.struct_pack(op := 'matmul', a := a, m := 2, n := 50, b := b, mb := 50, nb := 2)) │
│                                                 varchar                                                  │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ {"c":[169150,171700,419150,426700]}                                                                      │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────┘

如果把内存数据库改成文件数据库,下次打开数据库时,已安装的luajia库会自动加载。

相关推荐
Elastic 中国社区官方博客1 小时前
Elasticsearch:什么是向量数据库?
大数据·运维·数据库·人工智能·elasticsearch·搜索引擎·ai
cspttty1 小时前
财务人员学数据分析考什么证
数据库
隔窗听雨眠2 小时前
MySQL索引机制深度解析:从B+树原理到高并发查询优化的完整指南
数据库·b树·mysql
数智化管理手记2 小时前
数据质量无法保障?自动化数据质量规则校验如何搭建方案?
大数据·数据库·自动化
我的xiaodoujiao2 小时前
使用PyMySQL模块技术操作MySQL数据库
数据库·python·测试工具·mysql
zmzmzmalo5 小时前
Linux ELF文件加载与内存管理揭秘
linux·网络·数据库
Super 含13 小时前
Android 启动优化(五):线程、GC 与 IO 为什么会拖慢启动?
java·服务器·数据库
ltl13 小时前
向量检索引擎选型:决策树、RAG 回链与开放问题
数据库
坚持学习前端日记13 小时前
Python SQLAlchemy ORM 从0到1精通实战手册(基础到复杂高阶)
数据库·python·oracle