这么多人合作的PostgreSQL开源项目, 如何保持统一的代码风格?
digoal约 9 分钟参考文档点击文末阅读原文打开; 推荐《最好的PostgreSQL学习镜像》;
这么多人合作的PostgreSQL开源项目, 如何保持统一的代码风格?
PostgreSQL是全球合作的开源项目, 这么多贡献者, 怎么能保持统一的代码风格呢?
看PolarDB 开发镜像的build项目时, 在Dockerfile中发现了几条“奇怪”的命令, 感觉上对用户没有任何作用, 为什么要加这几条呢?
https://github.com/ApsaraDB/polardb-pg-docker-images/blob/main/Dockerfile-devel-ubuntu24.04
COPY misc/pg_bsd_indent_$TARGETARCH /usr/bin/pg_bsd_indent
# install perl module for TAP test
RUN perl -MCPAN -e "CPAN::Shell->notest('install', 'SHANCOCK/Perl-Tidy-20230309.tar.gz')"
咨询了研发人员原来是开发者使用的, 继承自PostgreSQL的传统, 用来保持统一的代码风格.
具体体现在2个环节中,pgindent调用了pg_bsd_indent和pgperltidy来检查代码风格, 同时依赖一个似乎是代码涉及到的所有符号表(typedefs.list)的文件.
以下截取自src/tools/pgindent/README
1 DOING THE INDENT RUN BEFORE A NORMAL COMMIT:
1、把pg_bsd_indent放到PATH路径中, 例如
export GITHUB_PROXY=https://ghproxy.com/
git clone --depth 1 -b REL_17_STABLE https://github.com/postgres/postgres
cd postgres
./configure --without-icu
cd src/tools/pg_bsd_indent/
make -j8
2、使用检查被修改过或新增的C文件
例如, 使用pgindent对所有PostgreSQL文件进行检查
export PATH=src/tools/pg_bsd_indent:$PATH
cd postgres
src/tools/pgindent/pgindent .
Usage:
pgindent [OPTION]... [FILE|DIR]...
Options:
--help show this message and quit
--commit=gitref use files modified by the named commit
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
--excludes=PATH file containing list of filename patterns to ignore
--indent=PATH path to pg_bsd_indent program
--diff show the changes that would be made
--check exit with status 2 if any changes would be made
The --excludes and --commit options can be given more than once.
3、使用git status检查是否有产生变化, 被检查修改过的文件的老文件会被保存(pgindent leaves *.BAK files behind if it has trouble, while
perltidy leaves *.LOG files behind. 后面这个是src/tools/pgindent/pgperltidy产生的.)
$ git status
On branch REL_17_STABLE
Your branch is up to date with 'origin/REL_17_STABLE'.
nothing to commit, working tree clean
4、If pgindent wants to change anything your commit wasn't touching, stop and figure out why. If it is making ugly whitespace changes around typedefs your commit adds, you need to add those typedefs tosrc/tools/pgindent/typedefs.list.
生成typedefs.list的具体方法参考:
http://adpgtech.blogspot.com/2015/05/running-pgindent-on-non-core-code-or.html
5、If you have the patience, it's worth eyeballing the "git diff" output for any egregiously ugly changes. See below for cleanup ideas.
6、Do a full test build:
make -s clean
make -s all # look for unexpected warnings, and errors of course
make check-world
Your configure switches should include at least--enable-tap-tests or else much of the Perl code won't get exercised. The ecpg regression tests may well fail due to pgindent's updates of header files that get copied into ecpg output; if so, adjust the expected-files to match.
2 AT LEAST ONCE PER RELEASE CYCLE:
Download the latest typedef file from the buildfarm:
wget -O src/tools/pgindent/typedefs.list https://buildfarm.postgresql.org/cgi-bin/typedefs.pl
This step resolves any differences between the incrementally updated version of the file and a clean, autogenerated one. (See https://buildfarm.postgresql.org/cgi-bin/typedefs.pl?show_list for a full list of typedef files, if you want to indent some back branch.)
Run pgindent as above.
Indent the Perl code using
perltidy:
src/tools/pgindent/pgperltidy .
If you want to use someperltidy version that's not in yourPATH, first set thePERLTIDY environment variable to point to it.
Reformat the bootstrap catalog data files:
./configure # "make" will not work in an unconfigured tree
cd src/include/catalog
make reformat-dat-files
cd ../../..
When you're done, "
git commit" everything including the typedefs.list file you used.Add the newly created commit(s) to the
.git-blame-ignore-revsfile so that "git blame" ignores the commits (for anybody that has opted-in to using the ignore file). Follow the instructions that appear at the top of the.git-blame-ignore-revsfile.
Another "git commit" will be required for your ignore file changes.
参考
https://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list
https://www.pgbuildfarm.org/cgi-bin/typedefs.pl?branch=REL_17_STABLE
src/tools/pgindent/README
src/tools/pg_bsd_indent/README
http://adpgtech.blogspot.com/2015/05/running-pgindent-on-non-core-code-or.html
Running pgindent on non-core code, or development code
Running pgindent is not nearly as hard as some people seem to think it is. The hardest part of getting a workable set of typedefs to use. That's why the buildfarm now constructs these lists automatically for each live branch.
But that doesn't help if you're working on non-core code. Here's what I did to get a working typedefs list for the Redis FDW code:
objdump -W redis_fdw.so |\
egrep -A3 DW_TAG_typedef |\
perl -e ' while (<>) { chomp; @flds = split;next unless (1 < @flds);\
next if $flds[0] ne "DW_AT_name" && $flds[1] ne "DW_AT_name";\
next if $flds[-1] =~ /^DW_FORM_str/;\
print $flds[-1],"\n"; }' |\
sort | uniq > redis_fdw.typedefs
This is a slight adaptation of what the buildfarm code does on Linux to get a typedefs list.
After that, indenting the code was a matter of just doing this:
pgindent --typedefs=redis_fdw.typedefs redis_fdw.c
What if you're developing a piece of core code and you'd like to run pgindent on it, but you've introduced some new typedefs, so pgindent mucks up the indentation by adding extraneous spaces. You have a couple of options. Let's assume that what you're working on is backend code. Then you could run the above extraction on the built backend - it doesn't have to be installed, just run it againstsrc/backend/postgres. Then use that to run pgindent against each of the files you're working on. You don't have to run it separately for each file - you can name as many files to indent as you like on the command line.
If you do that, look at the results carefully. It's possible that the absence of some platform-dependent typedef has mucked up your file. So a safer procedure is to grab the latest typedefs list from the buildfarm server and combine it with the typedefs list you just constructed, like this:
wget -q -O - "http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?branch=HEAD" |\
cat - mytypedefs | sort | uniq > mynewtypedefs
and then use that file to pgindent your code.
None of this is as easy as it might be. But none of it is very hard either.
Addendum
If you only have a handful of new typedefs, you can pass them on the command line to pgindent, like this:
pgindent --typedefs=mytypedefs --list-of-typedefs="typedef1 typedef2" myfile1.c myfile2.c
文末彩蛋:国产数据库周边生态
当然一款数据库要流行起来, 除了自己要强大, 还离不开生态. 用好周边生态工具, 管理水平战胜90%老司机!!! 下面简单介绍一下国产数据库周边生态.
1、管控软件
鸣嵩(前阿里云数据库总经理 / 研究员)等大佬们创业创办的云猿生, 核心产品是KubeBlocks. 他们的理念是让管理数据库和搭积木一样简单, 如果你要管理很多套并且种类(OLTP\OLAP\NoSQL\KV\TS\MQ等)很多的数据库产品, 推荐首选.
https://github.com/apecloud/kubeblocks
PG中文社区核心委员唐成老师的公司乘数开源的Clup, 专用管理PostgreSQL和PolarDB的集群管理软件, 如果你要管理很多套数据库, 推荐选择. 并且Clup还提供了企业版、自研的连接池、分布式存储、一体机、备份平台等, 是企业用户推荐之选.
https://www.csudata.com/
若航老司机开源的pigsty, 集成了300多个PG插件的PG集群和PolarDB集群管理软件, 如果你要管理很多套PG或PolarDB数据库, 且对插件有特别多的需求, 推荐选择.
https://pigsty.cc/zh/
2、审计监控诊断优化
翟总(曾经是我背后的男人)到海信聚好看后研发的 DBdoctor, 采用ebpf技术, 在对数据库几乎没有影响的情况下实时监控数据库和服务器的各项指标, 发现和诊断问题根因非常方便.
https://www.dbdoctor.cn/
天舟老哥的核心产品Bytebase 是位于您和数据库之间的中间件。它是数据库 DevOps 的 GitLab/GitHub，专为开发人员、DBA 和平台工程师打造。
https://bytebase.cc/docs/introduction/what-is-bytebase/
PawSQL, SQL优化和诊断产品.
D-Smart, Oracle老前辈白老大出品, 专注企业级市场, 将业界顶级DBA经验的产品化作品, 产品功能包括数据库监控、诊断、优化等.
https://www.modb.pro/db/567140
3、国产数据库IDE
IDE是开发者的必备工具，例如社区有pgAdmin, 国产IDE则可以看看老程序猿达刚老师的DeskUI:
https://www.deskui.com
4、数据同步&迁移&备份恢复
NineData, 老领导出去创业做的产品, 产品涵盖了数据同步、迁移、备份、比对、devops、chatDBA等.
https://www.ninedata.cloud/home
DSG, 非常老牌的数据库同步迁移企业级产品, 支持各种数据库的异构和同构迁移, 用他们的话说, 没有dsg搞不定的迁移, 比goldengate还牛.
https://www.dsgdata.com/
公开课
如果你对PolarDB学习感兴趣可以阅读这个公开课系列:
除了PolarDB还非常值得关注的几款PG栈国产数据库:
HaloDB(基于PG兼容PostgreSQL、Oracle、MySQL. http://www.halodbtech.com/ )、 IvorySQL(基于开源PG兼容PG、Oracle. https://www.ivorysql.org/zh-cn/ )、 ProtonBase(云原生分布式数仓. https://protonbase.com/ )、 成都文武数据库（https://ww-it.cn）
参考文档点击阅读原文获得
感谢关注我的github (https://github.com/digoal/blog) 及视频号: