又用AI修了一个开源数据库插件BUG
参考文档点击文末阅读原文打开; 推荐《最好的PostgreSQL学习镜像》;
编译vops插件报错,轻松用AI修了
以前遇到开源插件bug,你会怎么办?发issue求助作者?问数据库内核专家?发stackoverflow求助?去各种数据库群求助专家们?都不太靠谱,漫长的等待,还需要把复现方法说清楚,一来一回太耽误时间了!现在有了AI,自己搞定👍
问题: PolarDB 11 编译vops插件报错, 但PolarDB 15 编译vops不会报错.
vops.c: In function ‘vops_window_accumulate’:
vops.c:4338:62: error: macro "ereport" passed 4 arguments, but takes just 2
4338 | errhint("Update vops to version 1.1"));
| ^
In file included from /home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/postgres.h:53,
from vops.c:1:
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/elog.h:122: note: macro "ereport" defined here
122 | #define ereport(elevel, rest) \
|
vops.c:4335:9: error: ‘ereport’ undeclared (first use in this function)
4335 | ereport(ERROR,
| ^~~~~~~
vops.c:4335:9: note: each undeclared identifier is reported only once for each function it appears in
make: *** [<builtin>: vops.o] Error 1
为什么要在PolarDB中使用vops呢? 因为VOPS可以帮助PolarDB加速OLAP场景的性能. 更多详情可参考:
《PostgreSQL 向量化执行插件(瓦片式实现-vops) 10x提速OLAP》 《PostgreSQL VOPS 向量计算 + DBLINK异步并行 - 单实例 10亿 聚合计算跑进2秒》
复现方法
1、搭建PolarDB开发环境, 在开发环境中通过源码编译安装PolarDB 11.
1.1、拉取一个你熟悉的操作系统的PolarDB开发环境Docker镜像, 例如ubuntu22.04:
docker pull registry.cn-hangzhou.aliyuncs.com/polardb_pg/polardb_pg_devel:ubuntu22.04
1.2、创建并运行容器
docker run -d -it -P --shm-size=1g --cap-add=SYS_PTRACE --cap-add SYS_ADMIN --privileged=true --name polardb_pg_devel registry.cn-hangzhou.aliyuncs.com/polardb_pg/polardb_pg_devel:ubuntu22.04 bash
1.3、进入容器
# 进入容器
docker exec -ti polardb_pg_devel bash
1.4、克隆PolarDB 11源码,编译部署 PolarDB-PG 实例。
# 例如这里拉取 POLARDB_11_STABLE 分支;
# PS: 截止2024.9.24 PolarDB开源的最新分支为: POLARDB_15_STABLE
cd /tmp
git clone -c core.symlinks=true --depth 1 -b POLARDB_11_STABLE https://github.com/ApsaraDB/PolarDB-for-PostgreSQL
# 编译PolarDB 11并初始化实例
cd /tmp/PolarDB-for-PostgreSQL
./polardb_build.sh --without-fbl --debug=off
# 验证PolarDB-PG
psql -c 'SELECT version();'
version
--------------------------------
PostgreSQL 11.9 (POLARDB 11.9)
(1 row)
# 在容器内关闭、启动PolarDB数据库方法如下:
pg_ctl stop -m fast -D ~/tmp_master_dir_polardb_pg_1100_bld
pg_ctl start -D ~/tmp_master_dir_polardb_pg_1100_bld
2、在容器中下载vops源码:
cd /tmp
git clone -c core.symlinks=true --depth 1 https://github.com/postgrespro/vops
3、编译vops
cd /tmp/vops
USE_PGXS=1 make install
报错如下:
vops.c: In function ‘vops_window_accumulate’:
vops.c:4338:62: error: macro "ereport" passed 4 arguments, but takes just 2
4338 | errhint("Update vops to version 1.1"));
| ^
In file included from /home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/postgres.h:53,
from vops.c:1:
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/elog.h:122: note: macro "ereport" defined here
122 | #define ereport(elevel, rest) \
|
vops.c:4335:9: error: ‘ereport’ undeclared (first use in this function)
4335 | ereport(ERROR,
| ^~~~~~~
vops.c:4335:9: note: each undeclared identifier is reported only once for each function it appears in
make: *** [<builtin>: vops.o] Error 1
解决办法
给AI看看,从报错中可以看到是vops_window_accumulate调用ereport时, 传入的参数个数和elog.h头文件中定义的ereport参数个数不匹配.
vops.c
PG_FUNCTION_INFO_V1(vops_window_accumulate);
Datum
vops_window_accumulate(PG_FUNCTION_ARGS)
{
// 这里使用了4个参数
ereport(ERROR,
errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("vops aggregates are not supported in current vops version"),
errhint("Update vops to version 1.1"));
PG_RETURN_NULL();
}
PolarDB 11和PostgreSQL 11版本兼容, 两者的elog.h头文件是一样的.
/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server/utils/elog.h src/include/utils/elog.h
PostgreSQL 11代码分支如下:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=shortlog;h=refs/heads/REL_11_STABLE
经查, 在elog.h中ereport期望传入2个参数, 并且给出了New-style error reporting API介绍如下:
74 /*----------
75 * New-style error reporting API: to be used in this way:
76 * ereport(ERROR,
77 * (errcode(ERRCODE_UNDEFINED_CURSOR),
78 * errmsg("portal \"%s\" not found", stmt->portalname),
79 * ... other errxxx() fields as needed ...));
elog.h中定义的ereport:
122 #define ereport(elevel, rest) \
123 ereport_domain(elevel, TEXTDOMAIN, rest)
根据提示的New-style error reporting API, 修改一下vops_window_accumulate:
vops.c
PG_FUNCTION_INFO_V1(vops_window_accumulate);
Datum
vops_window_accumulate(PG_FUNCTION_ARGS)
{
ereport(ERROR,
// 把第二个参数包起来
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("vops aggregates are not supported in current vops version"),
errhint("Update vops to version 1.1"))
);
PG_RETURN_NULL();
}
重新编译就正常了.
$ cd /tmp/vops
$ USE_PGXS=1 make install
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -pipe -Wall -grecord-gcc-switches -I/usr/include/et -O3 -fPIC -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -c -o vops.o vops.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -pipe -Wall -grecord-gcc-switches -I/usr/include/et -O3 -fPIC -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -c -o vops_fdw.o vops_fdw.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -pipe -Wall -grecord-gcc-switches -I/usr/include/et -O3 -fPIC -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -c -o deparse.o deparse.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -pipe -Wall -grecord-gcc-switches -I/usr/include/et -O3 -fPIC -shared -o vops.so vops.o vops_fdw.o deparse.o -L/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib -Wl,-rpath,'$ORIGIN/../lib' -L/usr/lib/llvm-15/lib -Wl,--as-needed -Wl,-rpath,'/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib',--enable-new-dtags -L/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib -lpq
/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -Xclang -no-opaque-pointers -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -Wno-deprecated-non-prototype -O2 -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -flto=thin -emit-llvm -c -o vops.bc vops.c
/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -Xclang -no-opaque-pointers -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -Wno-deprecated-non-prototype -O2 -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -flto=thin -emit-llvm -c -o vops_fdw.bc vops_fdw.c
/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -Xclang -no-opaque-pointers -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -Wno-deprecated-non-prototype -O2 -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include -I. -I./ -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/server -I/home/postgres/tmp_basedir_polardb_pg_1100_bld/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -flto=thin -emit-llvm -c -o deparse.bc deparse.c
/usr/bin/mkdir -p '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib'
/usr/bin/mkdir -p '/home/postgres/tmp_basedir_polardb_pg_1100_bld/share/extension'
/usr/bin/mkdir -p '/home/postgres/tmp_basedir_polardb_pg_1100_bld/share/extension'
/usr/bin/install -c -m 755 vops.so '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/vops.so'
/usr/bin/install -c -m 644 .//vops.control '/home/postgres/tmp_basedir_polardb_pg_1100_bld/share/extension/'
/usr/bin/install -c -m 644 .//vops--1.0--1.1.sql .//vops--1.1.sql '/home/postgres/tmp_basedir_polardb_pg_1100_bld/share/extension/'
/usr/bin/mkdir -p '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode/vops'
/usr/bin/mkdir -p '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode'/vops/
/usr/bin/install -c -m 644 vops.bc '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode'/vops/./
/usr/bin/install -c -m 644 vops_fdw.bc '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode'/vops/./
/usr/bin/install -c -m 644 deparse.bc '/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode'/vops/./
cd'/home/postgres/tmp_basedir_polardb_pg_1100_bld/lib/bitcode' && /usr/lib/llvm-15/bin/llvm-lto -thinlto -thinlto-action=thinlink -o vops.index.bc vops/vops.bc vops/vops_fdw.bc vops/deparse.bc
为什么PostgreSQL 15和PolarDB 15不会报错呢? 因为这个接口改成了如下, 参数个数变成了1+N, N是可变的, 按需提供即可.
93 /*----------
94 * New-style error reporting API: to be used in this way:
95 * ereport(ERROR,
96 * errcode(ERRCODE_UNDEFINED_CURSOR),
97 * errmsg("portal \"%s\" not found", stmt->portalname),
98 * ... other errxxx() fields as needed ...);
...
157 #define ereport(elevel, ...) \
158 ereport_domain(elevel, TEXTDOMAIN, __VA_ARGS__)
现在可以在PolarDB 11中使用vops插件了:
$ psql
psql (11.9)
Type "help"forhelp.
postgres=# select version();
version
--------------------------------
PostgreSQL 11.9 (POLARDB 11.9)
(1 row)
postgres=# create extension vops;
CREATE EXTENSION
文末彩蛋:国产数据库周边生态
当然一款数据库要流行起来, 除了自己要强大, 还离不开生态. 用好周边生态工具, 管理水平战胜90%老司机!!! 下面简单介绍一下国产数据库周边生态.
1、管控软件
鸣嵩(前阿里云数据库总经理 / 研究员)等大佬们创业创办的云猿生, 核心产品是KubeBlocks. 他们的理念是让管理数据库和搭积木一样简单, 如果你要管理很多套并且种类(OLTP\OLAP\NoSQL\KV\TS\MQ等)很多的数据库产品, 推荐首选.
https://github.com/apecloud/kubeblocks
PG中文社区核心委员唐成老师的公司乘数开源的Clup, 专用管理PostgreSQL和PolarDB的集群管理软件, 如果你要管理很多套数据库, 推荐选择. 并且Clup还提供了企业版、自研的连接池、分布式存储、一体机、备份平台等, 是企业用户推荐之选.
https://www.csudata.com/
若航老司机开源的pigsty, 集成了300多个PG插件的PG集群和PolarDB集群管理软件, 如果你要管理很多套PG或PolarDB数据库, 且对插件有特别多的需求, 推荐选择.
https://pigsty.cc/zh/
2、审计监控诊断优化
翟总(曾经是我背后的男人)到海信聚好看后研发的 DBdoctor, 采用ebpf技术, 在对数据库几乎没有影响的情况下实时监控数据库和服务器的各项指标, 发现和诊断问题根因非常方便.
https://www.dbdoctor.cn/
天舟老哥的核心产品Bytebase 是位于您和数据库之间的中间件。它是数据库 DevOps 的 GitLab/GitHub,专为开发人员、DBA 和平台工程师打造。
https://bytebase.cc/docs/introduction/what-is-bytebase/
PawSQL, SQL优化和诊断产品.
D-Smart, Oracle老前辈白老大出品, 专注企业级市场, 将业界顶级DBA经验的产品化作品, 产品功能包括数据库监控、诊断、优化等.
https://www.modb.pro/db/567140
3、国产数据库IDE
IDE是开发者的必备工具,例如社区有pgAdmin, 国产IDE则可以看看老程序猿达刚老师的DeskUI:
https://www.deskui.com
4、数据同步&迁移&备份恢复
NineData, 老领导出去创业做的产品, 产品涵盖了数据同步、迁移、备份、比对、devops、chatDBA等.
https://www.ninedata.cloud/home
DSG, 非常老牌的数据库同步迁移企业级产品, 支持各种数据库的异构和同构迁移, 用他们的话说, 没有dsg搞不定的迁移, 比goldengate还牛.
https://www.dsgdata.com/
公开课
如果你对PolarDB学习感兴趣可以阅读这个公开课系列:
除了PolarDB还非常值得关注的几款PG栈国产数据库:
HaloDB(基于PG兼容PostgreSQL、Oracle、MySQL. http://www.halodbtech.com/ )、 IvorySQL(基于开源PG兼容PG、Oracle. https://www.ivorysql.org/zh-cn/ )、 ProtonBase(云原生分布式数仓. https://protonbase.com/ )、 成都文武数据库(https://ww-it.cn)
参考文档点击阅读原文获得
感谢关注我的github (https://github.com/digoal/blog) 及视频号: