Starcoin 并行执行压测报告

Owned by Bob
Last updated: 9月 06, 2023
4 min read

概述和目标

本测试的主要目的是为了测试Starcoin TurboSTM相关性能，以达到提高TPS的效果。并且对比Aptos的相关性得知提升的具体数值，以及可以针对TurboSTM的产出结果来进行一些建议。

计划

  1. 先参考Aptos给出的论文结果来选择使用哪些基准参数，并且在实际的本地开发环境中进行测试。

  2. 在Starcoin中进行benchmark

  3. 对比两者差异

  4. 压测指标参考 Aptos BlockSTM 论文中4.1节相关讨论。

测试执行

Aptos 压测

论文的结果

所有的压测数据均摘自论文中4.1节相关讨论，对比了BSTM、LiTM、BOHM这几类的性能，其代码在

  1. 环境：Amazon Web Services c5a.16xlarge instance (AMD EPYC CPU and 128GB memory)

  2. 系统：Ubuntu18.04

图3. BSTM、LiTM、BOHM和顺序执行在不同的区块大小的性能

 

图4. BSTM和顺序执行，在不同的区块大小的性能

 

通过上述的指标可知，性能压测基准指标为：线程数、区块大小。

 

实际工程产生的结果

基于上述文档中提供的代码工程 https://github.com/danielxiangzl/Block-STM ，block_stm分支中执行以下命令，并产生以下结果：

执行环境：

系统环境： macOS X 14.0

CPU：Apple M1 Max 10 Core  

内存：32 GB LPDDR5

 

 

CPUS = 10 PARAMS: num_account = 2, block_size = 1000 TPS: [3937, 4149, 4166, 4201, 4201, 4201, 4219, 4366, 4444, 4504] AVG TPS = 4238 PARAMS: num_account = 10, block_size = 1000 TPS: [7407, 7462, 7518, 7751, 7751, 7812, 7874, 7936, 8000, 8064] AVG TPS = 7757 PARAMS: num_account = 100, block_size = 1000 TPS: [18181, 20833, 20833, 21739, 22222, 22222, 22222, 23809, 24390, 25000] AVG TPS = 22145 PARAMS: num_account = 1000, block_size = 1000 TPS: [23255, 24390, 25000, 25641, 26315, 26315, 26315, 27027, 27027, 27777] AVG TPS = 25906 PARAMS: num_account = 10000, block_size = 1000 TPS: [17241, 17857, 18867, 21276, 21276, 23809, 23809, 25000, 25641, 27027] AVG TPS = 22180 PARAMS: num_account = 2, block_size = 10000 TPS: [3846, 4332, 4347, 4384, 4389, 4478, 4488, 4547, 4562, 4574] AVG TPS = 4394 PARAMS: num_account = 10, block_size = 10000 TPS: [8169, 8257, 8650, 8650, 8695, 8787, 8810, 9099, 9124, 9363] AVG TPS = 8760 PARAMS: num_account = 100, block_size = 10000 TPS: [31948, 32258, 32786, 34013, 34364, 34722, 35842, 35842, 36363, 37037] AVG TPS = 34517 PARAMS: num_account = 1000, block_size = 10000 TPS: [39370, 39525, 39682, 40816, 40983, 41666, 43290, 43859, 43859, 44052] AVG TPS = 41710 PARAMS: num_account = 10000, block_size = 10000 TPS: [27100, 30030, 32051, 37313, 39215, 41493, 43478, 45248, 46296, 46728] AVG TPS = 38895

Starcoin 压测

PullRequest:

现有的工具

目前Starcoin压测工具包存在两类压测工具，一类是基于criterion 库的 benchmark压测，另外是自己写的压测程序

 

执行PR

并行执行数据

CPUS = 10 PARAMS: num_account = 2, block_size = 1000 TPS: [12500, 13513, 14705, 16129, 19230, 19230, 20833, 20833, 21276, 21276] AVG TPS = 17952 PARAMS: num_account = 10, block_size = 1000 TPS: [8928, 12820, 15384, 16666, 16949, 16949, 17241, 17857, 18518, 18867] AVG TPS = 16017 PARAMS: num_account = 100, block_size = 1000 TPS: [14285, 15151, 15625, 16393, 16949, 17857, 17857, 17857, 20000, 22222] AVG TPS = 17419 PARAMS: num_account = 1000, block_size = 1000 TPS: [8333, 15625, 15873, 16129, 17241, 18518, 19607, 20000, 21739, 21739] AVG TPS = 17480 PARAMS: num_account = 10000, block_size = 1000 TPS: [16393, 16949, 17543, 17543, 18181, 18518, 20000, 20000, 20833, 22727] AVG TPS = 18868 PARAMS: num_account = 2, block_size = 10000 TPS: [52356, 65789, 69930, 72992, 72992, 74074, 74074, 76923, 76923, 78125] AVG TPS = 71417 PARAMS: num_account = 10, block_size = 10000 TPS: [65359, 70422, 71428, 72992, 72992, 72992, 73529, 75187, 75757, 75757] AVG TPS = 72641 PARAMS: num_account = 100, block_size = 10000 TPS: [62893, 66225, 72463, 73529, 75757, 75757, 75757, 76335, 78125, 78740] AVG TPS = 73558 PARAMS: num_account = 1000, block_size = 10000 TPS: [71942, 73529, 73529, 74074, 74626, 74626, 75757, 75757, 76335, 78740] AVG TPS = 74891 PARAMS: num_account = 10000, block_size = 10000 TPS: [64935, 69444, 70422, 72992, 73529, 75187, 75757, 76923, 77519, 77519] AVG TPS = 73422

 

串行执行

CPUS = 10 PARAMS: num_account = 2, block_size = 1000 TPS: [12358, 12358, 12358, 12358, 12358, 12512, 12670, 12670, 12670, 12670] AVG TPS = 12498 PARAMS: num_account = 10, block_size = 1000 TPS: [12060, 12060, 12207, 12207, 12207, 12207, 12358, 12358, 12670, 12670] AVG TPS = 12300 PARAMS: num_account = 100, block_size = 1000 TPS: [12358, 12358, 12512, 12670, 12670, 12670, 12670, 12670, 12670, 12670] AVG TPS = 12591 PARAMS: num_account = 1000, block_size = 1000 TPS: [12358, 12358, 12512, 12512, 12670, 12670, 12670, 12670, 12670, 12833] AVG TPS = 12592 PARAMS: num_account = 10000, block_size = 1000 TPS: [12060, 12060, 12358, 12358, 12512, 12512, 12670, 12670, 12670, 12670] AVG TPS = 12454 PARAMS: num_account = 2, block_size = 10000 TPS: [15315, 15675, 15700, 15724, 15749, 15849, 15849, 15874, 16001, 16027] AVG TPS = 15776 PARAMS: num_account = 10, block_size = 10000 TPS: [15433, 15457, 15651, 15799, 15799, 15950, 16001, 16001, 16052, 16052] AVG TPS = 15819 PARAMS: num_account = 100, block_size = 10000 TPS: [15505, 15626, 15700, 15774, 15849, 15849, 15950, 15950, 16027, 16078] AVG TPS = 15830 PARAMS: num_account = 1000, block_size = 10000 TPS: [15553, 15651, 15675, 15675, 15700, 15724, 15774, 15799, 15824, 15976] AVG TPS = 15735 PARAMS: num_account = 10000, block_size = 10000 TPS: [15386, 15433, 15433, 15529, 15553, 15553, 15577, 15749, 15749, 15799] AVG TPS = 15576

总结和建议

TODO