ReaderMX.
PostgreSQL码农集散地

PostgreSQL 18 preview - 异步IO支持io_uring, IO性能大幅提升

Pdigoal约 6 分钟

PostgreSQL 18 preview - 异步IO支持io_uring, IO性能大幅提升

PostgreSQL 18又提交了一系列补丁, 展示了 PostgreSQL 在异步 I/O 方面的持续改进，特别是通过引入 Linux 特有的 io_uring 机制来提升性能，同时注重代码的安全性和一致性。

io_uring 性能优势：

  • 相比基于 worker 的 AIO (io_method=worker) 有显著性能提升
  • 特别适合大量小 I/O 操作场景：
    • 减少 worker 模式下的上下文切换开销
    • 避免 I/O worker 数量限制成为瓶颈

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=96da9050a57aece4a48ab34a84bc3b3412708a20

aio: Be more paranoid about interrupts  
author  Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:06:54 +0000 (16:06 -0400)  
committer   Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:06:54 +0000 (16:06 -0400)  
commit  96da9050a57aece4a48ab34a84bc3b3412708a20  
tree    db94f67398834e9f1d1dd2fb4038b31c8643d635    tree  
parent  47a1f076a7c9789134a29dc738db0152e5f71b4c    commit | diff  
aio: Be more paranoid about interrupts  

As reported by Noah, it's possible, although practically very unlikely, that  
interrupts could be processed in between pgaio_io_reopen() and  
pgaio_io_perform_synchronously(). Prevent that by explicitly holding  
interrupts.  

It also seems good to add an assertion to pgaio_io_before_prep() to ensure  
that interrupts are held, as otherwise FDs referenced by the IO could be  
closed during interrupt processing. All code in the aio series currently runs  
the code with interrupts held, but it seems better to be paranoid.  

Reviewed-by: Noah Misch <[email protected]>  
Reported-by: Noah Misch <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f321ec237a54912b28d81acc4bfc4e434d9339c7

aio: Pass result of local callbacks to ->report_return  
author  Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:06:54 +0000 (16:06 -0400)  
committer   Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:06:54 +0000 (16:06 -0400)  
commit  f321ec237a54912b28d81acc4bfc4e434d9339c7  
tree    1195dca27d47a700c158e85cddfb979b1a696d45    tree  
parent  96da9050a57aece4a48ab34a84bc3b3412708a20    commit | diff  
aio: Pass result of local callbacks to ->report_return  

Otherwise the results of e.g. temp table buffer verification errors will not  
reach bufmgr.c. Obviously that's not right. Found while expanding the tests  
for invalid buffer contents.  

Reviewed-by: Noah Misch <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9469d7fdd2bcf2327ad91a1022885953a461942a

aio: Rename pgaio_io_prep_* to pgaio_io_start_*  
author  Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:10:29 +0000 (16:10 -0400)  
committer   Andres Freund <[email protected]>    
Wed, 26 Mar 2025 20:10:29 +0000 (16:10 -0400)  
commit  9469d7fdd2bcf2327ad91a1022885953a461942a  
tree    6b4f45f8031a7ac98af79878244401a4e1238e1f    tree  
parent  f321ec237a54912b28d81acc4bfc4e434d9339c7    commit | diff  
aio: Rename pgaio_io_prep_* to pgaio_io_start_*  

The old naming pattern (mirroring liburing's naming) was inconsistent with  
the (not yet introduced) callers. It seems better to get rid of the  
inconsistency now than to grow more users of the odd naming.  

Reported-by: Noah Misch <[email protected]>  
Discussion: https://postgr.es/m/[email protected]  

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=8eadd5c73c44708ecd45b9fd3ac54a550511d16f

aio: Add liburing dependency  
author  Andres Freund <[email protected]>    
Tue, 18 Mar 2025 18:40:05 +0000 (14:40 -0400)  
committer   Andres Freund <[email protected]>    
Wed, 26 Mar 2025 23:45:32 +0000 (19:45 -0400)  
commit  8eadd5c73c44708ecd45b9fd3ac54a550511d16f  
tree    45074275a931018bf5854dcae95b64a22ee16da0    tree  
parent  f056f75dafd0025d26efaca026a87f14c079a130    commit | diff  
aio: Add liburing dependency  
Will be used in a subsequent commit, to implement io_method=io_uring. Kept  
separate for easier review.  
Reviewed-by: Noah Misch <[email protected]>  
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt  

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c325a7633fcb33dbd73f46ddbbe91e95ddf3b227

aio: Add io_method=io_uring  
author  Andres Freund <[email protected]>    
Tue, 18 Mar 2025 18:40:05 +0000 (14:40 -0400)  
committer   Andres Freund <[email protected]>    
Wed, 26 Mar 2025 23:49:13 +0000 (19:49 -0400)  
commit  c325a7633fcb33dbd73f46ddbbe91e95ddf3b227  
tree    0aa7ab5b32bd5dd26eef0aff6b4d4b200a771d75    tree  
parent  8eadd5c73c44708ecd45b9fd3ac54a550511d16f    commit | diff  
aio: Add io_method=io_uring  
Performing AIO using io_uring can be considerably faster than  
io_method=worker, particularly when lots of small IOs are issued, as  
a) the context-switch overhead for worker based AIO becomes more significant  
b) the number of IO workers can become limiting  
io_uring, however, is linux specific and requires an additional compile-time  
dependency (liburing).  
This implementation is fairly simple and there are substantial optimization  
opportunities.  
The description of the existing AIO_IO_COMPLETION wait event is updated to  
make the difference between it and the new AIO_IO_URING_EXECUTION clearer.  
Reviewed-by: Noah Misch <[email protected]>  
Reviewed-by: Jakub Wartak <[email protected]>  
Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt  
Discussion: https://postgr.es/m/[email protected]  
Discussion: https://postgr.es/m/stj36ea6yyhoxtqkhpieia2z4krnam7qyetc57rfezgk4zgapf@gcnactj4z56m  

AI 解读

PostgreSQL AIO 补丁系列解读

这一系列补丁涉及 PostgreSQL 的异步 I/O (AIO) 功能改进，主要围绕安全性、功能完善和新特性引入。以下是各补丁的详细解读：

1. 关于中断处理的更严格保护 (aio: Be more paranoid about interrupts)

问题背景：

  • 在 pgaio_io_reopen() 和 pgaio_io_perform_synchronously() 之间可能存在中断处理的风险
  • 虽然概率极低，但可能导致 FD 在中断处理期间被关闭

解决方案：

  • 显式地保持中断禁用状态以防止这种情况
  • 在 pgaio_io_before_prep() 中添加断言确保中断被保持

评审意见：

  • Noah Misch 报告并评审了此问题
  • 虽然当前代码已经保持中断，但添加额外保护是更安全的做法

2. 本地回调结果传递修复 (aio: Pass result of local callbacks to ->report_return)

问题发现：

  • 在扩展测试无效缓冲区内容时发现
  • 临时表缓冲区验证错误等本地回调结果未正确传递到 bufmgr.c

修复内容：

  • 确保本地回调结果能正确传递到报告系统

3. 函数命名规范化 (aio: Rename pgaio_io_prep_* to pgaio_io_start_*)

命名问题：

  • 原有命名模仿 liburing 的命名方式
  • 与即将引入的调用方不一致

变更内容：

  • 将 pgaio_io_prep_* 系列函数重命名为 pgaio_io_start_*
  • 在更多用户使用前解决命名不一致问题

4. 引入 liburing 依赖 (aio: Add liburing dependency)

目的：

  • 为后续实现 io_method=io_uring 做准备
  • 单独提交以便于评审

技术细节：

  • liburing 是 Linux 特有的高性能异步 I/O 库
  • 此补丁仅添加依赖，不包含功能实现

5. 实现 io_uring 支持 (aio: Add io_method=io_uring)

性能优势：

  • 相比基于 worker 的 AIO (io_method=worker) 有显著性能提升
  • 特别适合大量小 I/O 操作场景：
    • 减少 worker 模式下的上下文切换开销
    • 避免 I/O worker 数量限制成为瓶颈

实现特点：

  • Linux 特有功能，依赖 liburing
  • 当前实现较为基础，有大量优化空间
  • 更新了 AIO_IO_COMPLETION 等待事件描述以区别于新的 AIO_IO_URING_EXECUTION

讨论要点：

  • 性能对比和优化机会是主要讨论话题
  • 多个开发者参与了评审和讨论

这一系列补丁展示了 PostgreSQL 在异步 I/O 方面的持续改进，特别是通过引入 Linux 特有的 io_uring 机制来提升性能，同时注重代码的安全性和一致性。

KEEP THE THREAD GOING

更多来源文章