开源地址:GitHub - topling/cspp-memtable
编译
CSPP MemTable 是 ToplingDB 的组件,跟 ToplingDB 是松耦合的,所以未使用 submodule,而是在编译 ToplingDB 的时候,会自动 clone 并获取最新版。
clone 并编译 ToplingDB(with CSPP MemTable)
sudo yum -y install git libaio-devel gcc-c++ gflags-devel zlib-devel bzip2-devel
git clone https://github.com/topling/toplingdb
cdtoplingdb
make -j`nproc`db_benchDEBUG_LEVEL=0
cp sideplugin/rockside/src/topling/web/{style.css,index.html}${/path/to/dbdir}
cp sideplugin/rockside/sample-conf/lcompact_community.yaml .
exportLD_LIBRARY_PATH=`find sideplugin -name lib_shared`
性能测试
memtablerep_bench 性能测试
./memtablerep_bench -benchmarks=fillrandom,readrandom,readwrite\
-memtablerep='cspp:{"mem_cap":"16G","use_hugepage":false}'\
-write_buffer_size=536870912\
-item_size=0-num_operations=10000000
memtablerep_bench 测试的是 MemTable 本身的性能,虽然相比直接测试 CSPP 算法损失了不少性能,但相比 SkipList 仍体现出巨大的优势。
db_bench 性能指标
# change ./lcompact_community.yaml
# 1. path items (search /dev/shm), if you have no fast disk(such as on a cloud server), use /dev/shm
# 2. change max_background_compactions to your cpu core num
# command option -json can accept json and yaml files, here use yaml file for more human readable
./db_bench -json lcompact_community.yaml -num 10000000 -disable_wal=true \
-value_size10 -batch_size=1000 -benchmarks=fillrandom,readrandom
# you can access http://127.0.0.1:8081 to see webview
# you can see this db_bench is much faster than RocksDB
db_bench 包含了完整的读写流程,CSPP MemTable 的性能优势会被拉平很多,通过把 batch_size 改大,value_size 改小,CSPP MemTable 的优势会体现得更突出:
cspp fillrandom : 13.751 micros/op 4654214 ops/sec; 106.5 MB/s
skiplist fillrandom : 39.414 micros/op 1623731 ops/sec; 37.2 MB/s