LoongArch单机Ceph Bcache加速4K随机写性能测试

news2025/7/15 7:05:56

LoongArch单机Ceph Bcache加速4K随机写性能测试

两块HDD做OSD

[root@ceph01 ~]# fio -direct=1 -iodepth=128 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=600 -group_reporting -name=mytest -filename=/dev/rbd0
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.22
Starting 1 thread
Jobs: 1 (f=1): [w(1)][100.0%][w=1605KiB/s][w=401 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=1): err= 0: pid=83763: Mon Oct 16 03:44:45 2023
  write: IOPS=404, BW=1620KiB/s (1659kB/s)(950MiB/600262msec); 0 zone resets
    slat (usec): min=3, max=116, avg= 5.76, stdev= 4.41
    clat (msec): min=36, max=947, avg=316.05, stdev=71.26
     lat (msec): min=36, max=947, avg=316.06, stdev=71.26
    clat percentiles (msec):
     |  1.00th=[  180],  5.00th=[  215], 10.00th=[  239], 20.00th=[  264],
     | 30.00th=[  279], 40.00th=[  296], 50.00th=[  309], 60.00th=[  326],
     | 70.00th=[  342], 80.00th=[  363], 90.00th=[  397], 95.00th=[  435],
     | 99.00th=[  542], 99.50th=[  609], 99.90th=[  793], 99.95th=[  810],
     | 99.99th=[  944]
   bw (  KiB/s): min=  232, max= 3072, per=100.00%, avg=1622.36, stdev=394.76, samples=1198
   iops        : min=   58, max=  768, avg=405.58, stdev=98.69, samples=1198
  lat (msec)   : 50=0.02%, 100=0.01%, 250=14.65%, 500=83.73%, 750=1.44%
  lat (msec)   : 1000=0.15%
  cpu          : usr=0.11%, sys=0.30%, ctx=16672, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,243095,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=1620KiB/s (1659kB/s), 1620KiB/s-1620KiB/s (1659kB/s-1659kB/s), io=950MiB (996MB), run=600262-600262msec

Disk stats (read/write):
  rbd0: ios=0/242996, merge=0/0, ticks=0/76720597, in_queue=76980152, util=100.00%

【Bcache】一块SSD加速两块HDD(OSD)

Ceph中使用bcache_4K随机写测试_1块SDD加速2块HDD

image-20231013003009484

[root@ceph01 ceph]# fio -direct=1 -iodepth=128 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=600 -group_reporting -name=mytest -filename=/dev/rbd0
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.22
Starting 1 thread
Jobs: 1 (f=1): [w(1)][10.7%][w=12.1MiB/s][w=3097 IOPS][eta 08m:56s]
mytest: (groupid=0, jobs=1): err= 0: pid=37245: Thu Oct 12 22:08:43 2023
  write: IOPS=4065, BW=15.9MiB/s (16.7MB/s)(1024MiB/64475msec); 0 zone resets
    slat (usec): min=3, max=173, avg= 5.80, stdev= 3.99
    clat (msec): min=9, max=336, avg=31.47, stdev=21.69
     lat (msec): min=9, max=336, avg=31.48, stdev=21.69
    clat percentiles (msec):
     |  1.00th=[   17],  5.00th=[   19], 10.00th=[   21], 20.00th=[   24],
     | 30.00th=[   26], 40.00th=[   27], 50.00th=[   28], 60.00th=[   29],
     | 70.00th=[   31], 80.00th=[   33], 90.00th=[   37], 95.00th=[   48],
     | 99.00th=[  146], 99.50th=[  180], 99.90th=[  268], 99.95th=[  284],
     | 99.99th=[  334]
   bw (  KiB/s): min= 4216, max=20288, per=100.00%, avg=16284.49, stdev=3300.36, samples=128
   iops        : min= 1054, max= 5072, avg=4071.00, stdev=825.06, samples=128
  lat (msec)   : 10=0.01%, 20=8.54%, 50=87.00%, 100=2.20%, 250=2.13%
  lat (msec)   : 500=0.14%
  cpu          : usr=0.95%, sys=3.21%, ctx=28915, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=15.9MiB/s (16.7MB/s), 15.9MiB/s-15.9MiB/s (16.7MB/s-16.7MB/s), io=1024MiB (1074MB), run=64475-64475msec

Disk stats (read/write):
  rbd0: ios=0/261860, merge=0/0, ticks=0/8185334, in_queue=8198080, util=100.00%

【Bcache】两块SSD加速两块HDD(OSD)

image-20231014231026800

[root@ceph01 ceph]# fio -direct=1 -iodepth=128 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=600 -group_reporting -name=mytest -filename=/dev/rbd0
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.22
Starting 1 thread
Jobs: 1 (f=1): [w(1)][10.5%][w=15.8MiB/s][w=4047 IOPS][eta 08m:57s]
mytest: (groupid=0, jobs=1): err= 0: pid=363441: Sat Oct 14 11:19:49 2023
  write: IOPS=4158, BW=16.2MiB/s (17.0MB/s)(1024MiB/63037msec); 0 zone resets
    slat (usec): min=3, max=624, avg= 5.76, stdev= 4.10
    clat (msec): min=10, max=359, avg=30.77, stdev=18.74
     lat (msec): min=10, max=359, avg=30.78, stdev=18.74
    clat percentiles (msec):
     |  1.00th=[   17],  5.00th=[   19], 10.00th=[   21], 20.00th=[   23],
     | 30.00th=[   25], 40.00th=[   27], 50.00th=[   28], 60.00th=[   29],
     | 70.00th=[   31], 80.00th=[   33], 90.00th=[   38], 95.00th=[   51],
     | 99.00th=[  111], 99.50th=[  153], 99.90th=[  262], 99.95th=[  266],
     | 99.99th=[  359]
   bw (  KiB/s): min= 3768, max=21016, per=100.00%, avg=16674.80, stdev=3007.06, samples=125
   iops        : min=  942, max= 5254, avg=4168.57, stdev=751.74, samples=125
  lat (msec)   : 20=9.54%, 50=85.45%, 100=3.49%, 250=1.38%, 500=0.13%
  cpu          : usr=0.99%, sys=3.24%, ctx=28037, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=16.2MiB/s (17.0MB/s), 16.2MiB/s-16.2MiB/s (17.0MB/s-17.0MB/s), io=1024MiB (1074MB), run=63037-63037msec

Disk stats (read/write):
  rbd0: ios=0/261835, merge=0/0, ticks=0/7989253, in_queue=7996052, util=100.00%

【Bcache】两块SSD加速两块HDD(OSD)+两块SSD加速block.db和block.wal

[root@ceph01 ~]# fio -direct=1 -iodepth=128 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=600 -group_reporting -name=mytest -filename=/dev/rbd0
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.22
Starting 1 thread
Jobs: 1 (f=1): [w(1)][10.5%][w=16.3MiB/s][w=4162 IOPS][eta 08m:57s]
mytest: (groupid=0, jobs=1): err= 0: pid=73003: Mon Oct 16 02:34:47 2023
  write: IOPS=4124, BW=16.1MiB/s (16.9MB/s)(1024MiB/63559msec); 0 zone resets
    slat (usec): min=3, max=109, avg= 5.69, stdev= 3.75
    clat (msec): min=10, max=294, avg=31.03, stdev=17.27
     lat (msec): min=10, max=294, avg=31.03, stdev=17.27
    clat percentiles (msec):
     |  1.00th=[   17],  5.00th=[   19], 10.00th=[   21], 20.00th=[   23],
     | 30.00th=[   25], 40.00th=[   27], 50.00th=[   28], 60.00th=[   30],
     | 70.00th=[   32], 80.00th=[   34], 90.00th=[   40], 95.00th=[   52],
     | 99.00th=[  110], 99.50th=[  136], 99.90th=[  226], 99.95th=[  249],
     | 99.99th=[  284]
   bw (  KiB/s): min= 6200, max=20376, per=100.00%, avg=16508.00, stdev=2659.19, samples=126
   iops        : min= 1550, max= 5094, avg=4126.88, stdev=664.77, samples=126
  lat (msec)   : 20=9.65%, 50=85.11%, 100=3.87%, 250=1.33%, 500=0.04%
  cpu          : usr=1.00%, sys=3.13%, ctx=25141, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=16.1MiB/s (16.9MB/s), 16.1MiB/s-16.1MiB/s (16.9MB/s-16.9MB/s), io=1024MiB (1074MB), run=63559-63559msec

Disk stats (read/write):
  rbd0: ios=0/261407, merge=0/0, ticks=0/8062837, in_queue=8075472, util=100.00%

两块SSD做OSD

image-20231013225413334

[root@ceph01 ~]# fio -direct=1 -iodepth=128 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=600 -group_reporting -name=mytest -filename=/dev/rbd0
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
fio-3.22
Starting 1 thread
Jobs: 1 (f=1): [w(1)][4.3%][w=34.7MiB/s][w=8883 IOPS][eta 09m:35s] 
mytest: (groupid=0, jobs=1): err= 0: pid=125901: Fri Oct 13 11:06:55 2023
  write: IOPS=10.2k, BW=39.8MiB/s (41.7MB/s)(1024MiB/25751msec); 0 zone resets
    slat (nsec): min=3310, max=78980, avg=5364.31, stdev=3425.80
    clat (usec): min=2965, max=33393, avg=12565.10, stdev=3428.92
     lat (usec): min=2970, max=33400, avg=12570.90, stdev=3428.60
    clat percentiles (usec):
     |  1.00th=[ 6652],  5.00th=[ 7963], 10.00th=[ 8717], 20.00th=[ 9765],
     | 30.00th=[10552], 40.00th=[11207], 50.00th=[11994], 60.00th=[12780],
     | 70.00th=[13829], 80.00th=[15139], 90.00th=[17171], 95.00th=[19006],
     | 99.00th=[22676], 99.50th=[24511], 99.90th=[27657], 99.95th=[28705],
     | 99.99th=[31589]
   bw (  KiB/s): min=33628, max=44247, per=99.99%, avg=40717.55, stdev=2014.23, samples=51
   iops        : min= 8407, max=11061, avg=10179.18, stdev=503.57, samples=51
  lat (msec)   : 4=0.01%, 10=23.01%, 20=73.64%, 50=3.35%
  cpu          : usr=2.23%, sys=7.41%, ctx=20485, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
     issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=128

Run status group 0 (all jobs):
  WRITE: bw=39.8MiB/s (41.7MB/s), 39.8MiB/s-39.8MiB/s (41.7MB/s-41.7MB/s), io=1024MiB (1074MB), run=25751-25751msec

Disk stats (read/write):
  rbd0: ios=0/260926, merge=0/0, ticks=0/3220267, in_queue=3224032, util=99.89%

结论

测试环境为单机双副本!!

测试IOPSBWlat
两块HDD做OSD4041659kB/s316.06ms
【Bcache】一块SSD加速两块HDD(OSD)406516.7MB/s31.48ms
【Bcache】两块SSD加速两块HDD(OSD)415817.0MB/s30.78ms
【Bcache】两块SSD加速两块HDD(OSD)+两块SSD加速block.db和block.wal412416.9MB/s31.03ms
两块SSD做OSD10.2k41.7MB/s12.57ms
对比数据1640567MB/s7.80ms

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/1102951.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

System.exit()方法参数

说明文档:System (Java Platform SE 8 ) 终止当前正在运行的Java虚拟机。该参数用作状态代码;按照惯例,非零状态码表示异常终止。 此方法调用类Runtime中的exit方法。此方法从不正常返回。 调用System.exit(n)实际上等…

这应该是关于回归模型最全的总结了(附原理+代码)

本文将继续修炼回归模型算法,并总结了一些常用的除线性回归模型之外的模型,其中包括一些单模型及集成学习器。 保序回归、多项式回归、多输出回归、多输出K近邻回归、决策树回归、多输出决策树回归、AdaBoost回归、梯度提升决策树回归、人工神经网络、随…

个人博客系统的总结

个人博客系统 1、项目背景: 个人博客系统的兴起和发展是与信息技术和互联网的迅猛发展密切相关的。随着互联网的普及和数字化时代的到来,越来越多的人开始使用互联网平台来表达自己的观点、分享知识和展示个人创作。个人博客系统作为一种在线的个人信息…

C# CodeFormer Inpainting 人脸填充

效果 项目 代码 using Microsoft.ML.OnnxRuntime; using Microsoft.ML.OnnxRuntime.Tensors; using OpenCvSharp; using System; using System.Collections.Generic; using System.Drawing; using System.Drawing.Imaging; using System.Windows.Forms;namespace CodeFormer_D…

Qt系列-QSplitter使用笔记

Qt拆分窗口可以使用QSplitter,也可以使用比较容易使用布局控件来完成,如果有需要对界面进行切割,可以使用QSplitter进行简易的控件拼接。 注意:QSplitter 继承自QFrame,而QFrame又继承自QWidget,这点很重要…

数据结构----算法--五大基本算法

数据结构----算法–五大基本算法 一.贪心算法 1.什么是贪心算法 在有多个选择的时候不考虑长远的情况,只考虑眼前的这一步,在眼前这一步选择当前的最好的方案 二.分治法 1.分治的概念 分治法:分而治之 将一个问题拆解成若干个解决方式…

代码随想录第44天 | ● 1143.最长公共子序列 ● 1035.不相交的线 ● 53. 最大子序和 动态规划

1143.最长公共子序列 //*** param {string} text1* param {string} text2* return {number}*/ var longestCommonSubsequence function(text1,text2) {let dp new Array(text2.length1)dp.fill(0)for(let i 1; i < text1.length; i) {// 这里pre相当于 dp[i - 1][j - 1]…

这些 channel 用法你都用起来了吗?

channel 是什么&#xff1f; channel 是GO语言中一种特殊的类型&#xff0c;是连接并发goroutine的管道 channel 通道是可以让一个 goroutine 协程发送特定值到另一个 goroutine 协程的通信机制。 关于 channel 的原理&#xff0c;channel通道需要注意的地方&#xff0c;之前…

Kotlin中布尔类型、字符类型、字符串类型和数组类型

在Kotlin中&#xff0c;布尔类型、字符类型、字符串类型和数组类型是常用的数据类型之一。下面我将对它们进行详细描述并提供示例代码。 布尔类型&#xff08;Boolean&#xff09;&#xff1a; 布尔类型表示逻辑值&#xff0c;只有两个可能的取值&#xff1a;true和false。在K…

我的创作纪念日———C/C++之动态内存管理

个人主页&#xff1a;点我进入主页 专栏分类&#xff1a;C语言初阶 C语言程序设计————KTV C语言小游戏 C语言进阶 C语言刷题 欢迎大家点赞&#xff0c;评论&#xff0c;收藏。 一起努力&#xff0c;一起奔赴大厂。 目录 1.前言 2.为什么要有动态内存分配…

浅谈工业企业能源管理软件的应用

安科瑞 崔丽洁 摘要&#xff1a;在工业企业中&#xff0c;能源是企业正常工作的根本保障&#xff0c;同时也是工业企业成本的主要构成部分。工业企业在相应的工作和使用的能源包括水、电、气等。在过去&#xff0c;企业对能源消耗的关注度比较低&#xff0c;更多的强调安全与保…

二分查找算法(Python)

目录 1、概念 2、思路 3、实现算法 1、概念 二分查找又称折半查找&#xff0c;它是一种效率较高的查找方法 原理&#xff1a;首先&#xff0c;假设表中元素是按升序排列&#xff0c;将表中间位置记录的关键字与查找关键字比较&#xff0c;如果两者相等&#xff0c;则查找成…

从入门到进阶 之 ElasticSearch 文档、分词器 进阶篇

&#x1f339; 以上分享 ElasticSearch 文档、分词器 进阶篇&#xff0c;如有问题请指教写。&#x1f339;&#x1f339; 如你对技术也感兴趣&#xff0c;欢迎交流。&#x1f339;&#x1f339;&#x1f339; 如有需要&#xff0c;请&#x1f44d;点赞&#x1f496;收藏&#…

通达OA 2016网络智能办公系统 handle.php SQL注入漏洞

一、漏洞描述 北京通达信科科技有限公司通达OA2016网络智能办公系统 handle.php 存在sql注入漏洞&#xff0c;攻击者可利用此漏洞获取数据库管理员权限&#xff0c;查询数据、获取系统信息&#xff0c;威胁企业单位数据安全。 二、网络空间搜索引擎查询 fofa查询 app"T…

攻防演练蓝队|Windows应急响应入侵排查

文章目录 日志分析web日志windows系统日志 文件排查进程排查新增、隐藏账号排查启动项/服务/计划任务排查工具 日志分析 web日志 dirpro扫描目录&#xff0c;sqlmap扫描dvwa Python dirpro -u http://192.168.52.129 -b sqlmap -u "http://192.168.52.129/dvwa/vulnera…

牛客:FZ12 牛牛的顺时针遍历

FZ12 牛牛的顺时针遍历 文章目录 FZ12 牛牛的顺时针遍历题目描述题解思路题解代码 题目描述 题解思路 通过一个变量来记录当前方向&#xff0c;遍历矩阵&#xff0c;每次遍历一条边&#xff0c;将该边的信息加入到结果中 题解代码 func spiralOrder(matrix [][]int) []int {…

小程序-uni-app:将页面(html+css)生成图片/海报/名片,进行下载 保存到手机

一、需要描述 本文实现&#xff0c;uniapp微信小程序&#xff0c;把页面内容保存为图片&#xff0c;并且下载到手机上。 说实话网上找了很多资料&#xff0c;但是效果不理想&#xff0c;直到看了一个开源项目&#xff0c;我知道可以实现了。 本文以开源项目uniapp-wxml-to-can…

Mysql——查询sql语句练习

一、单表查询 素材&#xff1a; 表名&#xff1a;worker-- 表中字段均为中文&#xff0c;比如 部门号 工资 职工号 参加工作 等 1、显示所有职工的基本信息。 select * from worker; 2、查询所有职工所属部门的部门号&#xff0c;不显示重复的部门号。 select distinct 部门号…

开发万岳互联网医院APP:技术要点和关键挑战

随着移动技术和互联网的飞速发展&#xff0c;互联网医院APP成为医疗领域的一项重要创新。这些应用程序为患者和医生提供了更多便利和互动性&#xff0c;但开发互联网医院APP也伴随着一系列的技术要点和关键挑战。本文将探讨互联网医院APP的技术要点以及在开发过程中需要面对的挑…

AFL模糊测试+GCOV覆盖率分析

安全之安全(security)博客目录导读 覆盖率分析汇总 目录 一、代码示例 二、afl-cov工具下载 三、编译带覆盖率的版本并启动afl-cov 四、AFL编译插桩并运行afl-fuzz 五、结果查看 AFL相关详见AFL安全漏洞挖掘 GCOV相关详见GCOV覆盖率分析 现将两者结合&#xff0c;即进…