RocksDB安装部署
RocksDB是使用C++开发的开源嵌入式数据库，长安链采用golang开发，为了集成RocksDB，目前长安链使用了gorocksdb第三方开源的golang库，gorocksdb需要依赖RocksDB的库文件，因此如果选用RocksDB作为存储数据库，需要在本地安装RocksDB的环境，并使用条件编译来集成RocksDB。过程如下：
1.RocksDB安装：https://github.com/facebook/rocksdb/blob/master/INSTALL.md
2.gorocksdb安装：https://github.com/tecbot/gorocksdb
3.使用rocksdb需要通过-tag方式启动，build方式：
go build -tags=rocksdb
Linux下Rocksdb环境安装
1 安装依赖
安装gcc、zlib、snappy、lz4等依赖工具
yum -y install lrzsz git gcc gcc-c++ lz4-devel
yum -y install snappy snappy-devel zlib zlib-devel bzip2 bzip2-devel lz4 lz4-devel zstd
2 安装cmake
gflags-2.2.2对cmake版本有要求，所以需要指定版本的cmake
curl -O https://cmake.org/files/v3.6/cmake-3.6.0-Linux-x86_64.tar.gz
mv cmake-3.6.0-Linux-x86_64.tar.gz /opt/
cd /opt/
tar -xvzf cmake-3.6.0-Linux-x86_64.tar.gz
yum remove cmake
cat >>/etc/profile <<EOF
export PATH=\$PATH:/opt/cmake-3.6.0-Linux-x86_64/bin
EOF
source /etc/profile
3 安装gflags
wget -O gflags-2.2.2.tar.gz https://github.com/gflags/gflags/archive/v2.2.2.tar.gz
tar -xvzf gflags-2.2.2.tar.gz
cd gflags-2.2.2/
mkdir build
cd build/
cmake -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=ON -DINSTALL_HEADERS=ON -DINSTALL_SHARED_LIBS=ON -DINSTALL_STATIC_LIBS=ON ..
make
make install
cat >>/etc/profile <<EOF
export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/usr/local/lib
EOF
source /etc/profile
4 下载并安装rocksdb
wget -O rocksdb-5.18.3.tar.gz https://github.com/facebook/rocksdb/archive/v5.18.3.tar.gz
tar -xzvf rocksdb-5.18.3.tar.gz
cd rocksdb-5.18.3
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local/rocksdb ..
make
make install
cat >>/etc/profile <<EOF
export CPLUS_INCLUDE_PATH=\$CPLUS_INCLUDE_PATH:/usr/local/rocksdb/include/
export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/usr/local/rocksdb/lib64/
export LIBRARY_PATH=\$LIBRARY_PATH:/usr/local/rocksdb/lib64/
EOF
source /etc/profile
5 使用测试
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build]# cd tools/
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ll
total 2608
drwxr-xr-x 12 root root 4096 Dec 25 10:38 CMakeFiles
-rw-r--r-- 1 root root 269 Dec 25 10:38 CTestTestfile.cmake
-rw-r--r-- 1 root root 18973 Dec 25 10:38 Makefile
-rw-r--r-- 1 root root 988 Dec 25 10:38 cmake_install.cmake
-rwxr-xr-x 1 root root 232649 Dec 25 10:55 db_repl_stress
-rwxr-xr-x 1 root root 233443 Dec 25 10:55 db_sanity_test
-rwxr-xr-x 1 root root 1338230 Dec 25 10:55 db_stress
-rwxr-xr-x 1 root root 48060 Dec 25 10:55 ldb
-rwxr-xr-x 1 root root 207347 Dec 25 10:55 rocksdb_dump
-rwxr-xr-x 1 root root 207358 Dec 25 10:55 rocksdb_undump
-rwxr-xr-x 1 root root 8571 Dec 25 10:55 sst_dump
-rwxr-xr-x 1 root root 350147 Dec 25 10:55 write_stress
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]#
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]#
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]#
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ./ldb -help
ldb - RocksDB Tool
commands MUST specify --db=<full_path_to_db_directory> when necessary
The following optional parameters control if keys/values are input/output as hex or as plain strings:
--key_hex : Keys are input/output as hex
--value_hex : Values are input/output as hex
--hex : Both keys and values are input/output as hex
The following optional parameters control the database internals:
--column_family=<string> : name of the column family to operate on. default: default column family
--ttl with 'put','get','scan','dump','query','batchput' : DB supports ttl and value is internally timestamp-suffixed
--try_load_options : Try to load option file from DB.
--ignore_unknown_options : Ignore unknown options when loading option file.
--bloom_bits=<int,e.g.:14>
--fix_prefix_len=<int,e.g.:14>
--compression_type=<no|snappy|zlib|bzip2|lz4|lz4hc|xpress|zstd>
--compression_max_dict_bytes=<int,e.g.:16384>
--block_size=<block_size_in_bytes>
--auto_compaction=<true|false>
--db_write_buffer_size=<int,e.g.:16777216>
--write_buffer_size=<int,e.g.:4194304>
--file_size=<int,e.g.:2097152>
Data Access Commands:
put <key> <value> [--ttl]
get <key> [--ttl]
batchput <key> <value> [<key> <value>] [..] [--ttl]
scan [--from] [--to] [--ttl] [--timestamp] [--max_keys=<N>q] [--start_time=<N>:- is inclusive] [--end_time=<N>:- is exclusive] [--no_value]
delete <key>
deleterange <begin key> <end key>
query [--ttl]
Starts a REPL shell. Type help for list of available commands.
approxsize [--from] [--to]
checkconsistency
Admin Commands:
dump_wal --walfile=<write_ahead_log_file_path> [--header] [--print_value] [--write_committed=true|false]
compact [--from] [--to]
reduce_levels --new_levels=<New number of levels> [--print_old_levels]
change_compaction_style --old_compaction_style=<Old compaction style: 0 for level compaction, 1 for universal compaction> --new_compaction_style=<New compaction style: 0 for level compaction, 1 for universal compaction>
dump [--from] [--to] [--ttl] [--max_keys=<N>] [--timestamp] [--count_only] [--count_delim=<char>] [--stats] [--bucket=<N>] [--start_time=<N>:- is inclusive] [--end_time=<N>:- is exclusive] [--path=<path_to_a_file>]
load [--create_if_missing] [--disable_wal] [--bulk_load] [--compact]
manifest_dump [--verbose] [--json] [--path=<path_to_manifest_file>]
list_column_families full_path_to_db_directory
dump_live_files
idump [--from] [--to] [--input_key_hex] [--max_keys=<N>] [--count_only] [--count_delim=<char>] [--stats]
repair
backup [--backup_env_uri] [--backup_dir] [--num_threads] [--stderr_log_level=<int (InfoLogLevel)>]
restore [--backup_env_uri] [--backup_dir] [--num_threads] [--stderr_log_level=<int (InfoLogLevel)>]
checkpoint [--checkpoint_dir]
write_extern_sst <output_sst_path>
ingest_extern_sst <input_sst_path> [--move_files] [--snapshot_consistency] [--allow_global_seqno] [--allow_blocking_flush] [--ingest_behind] [--write_global_seqno]
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]#
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# pwd
/opt/rocksdb-5.18.3/build/tools
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ll /tmp/
total 32
srwxrwxrwx 1 root root 0 Dec 18 23:04 agent_cmd.sock
drwxr-xr-x 2 root root 4096 Dec 25 10:14 commandnotfound
-rw-r--r-- 1 root root 53 Dec 17 10:44 cpuidle_support.log
-rw-r--r-- 1 root root 2513 Dec 17 10:43 cvm_init.log
-rw-r--r-- 1 root root 297 Dec 17 10:44 net_affinity.log
-rw-r--r-- 1 root root 26 Dec 17 10:44 nv_gpu_conf.log
-rw-r--r-- 1 root root 155 Dec 17 10:44 setRps.log
drwx------ 3 root root 4096 Dec 17 10:44 systemd-private-e9868203df724169bf07a791d55819cd-ntpd.service-mA2nuw
-rw-r--r-- 1 root root 2017 Dec 17 10:44 virtio_blk_affinity.log
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ./ldb --db=/tmp/test_db --create_if_missing put a1 b1
OK
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ./ldb --db=/tmp/test_db scan
a1 : b1
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ./ldb --db=/tmp/test_db get a1
b1
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# ./ldb --db=/tmp/test_db get a2
Failed: NotFound:
[root@VM-219-157-centos /opt/rocksdb-5.18.3/build/tools]# cd /tmp/test_db/
[root@VM-219-157-centos /tmp/test_db]# ll
total 920
-rw-r--r-- 1 root root 26 Dec 25 11:13 000003.log
-rw-r--r-- 1 root root 16 Dec 25 11:13 CURRENT
-rw-r--r-- 1 root root 37 Dec 25 11:13 IDENTITY
-rw-r--r-- 1 root root 0 Dec 25 11:13 LOCK
-rw-r--r-- 1 root root 16246 Dec 25 11:14 LOG
-rw-r--r-- 1 root root 19272 Dec 25 11:13 LOG.old.1608866049557780
-rw-r--r-- 1 root root 15784 Dec 25 11:14 LOG.old.1608866049562808
-rw-r--r-- 1 root root 16246 Dec 25 11:14 LOG.old.1608866058035881
-rw-r--r-- 1 root root 15788 Dec 25 11:14 LOG.old.1608866058041253
-rw-r--r-- 1 root root 16250 Dec 25 11:14 LOG.old.1608866062086538
-rw-r--r-- 1 root root 15784 Dec 25 11:14 LOG.old.1608866062091559
-rw-r--r-- 1 root root 13 Dec 25 11:13 MANIFEST-000001
-rw-r--r-- 1 root root 4744 Dec 25 11:13 OPTIONS-000005
6 安装zstd
zstd是facebook为适配rocksdb开发的zstandard数据压缩工具，如果不安装该软件，会导致gorocksdb安装失败。
安装步骤如下：
cd /usr/local
git clone https://github.com/facebook/zstd.git
cd zstd
make
make install
7 安装gorocksdb
通过1-6安装步骤后，rocksdb会被安装在 usr/local/rocksdb 这个目录下，我们使用go版本的rocksdb需要依赖于该路径。
目前使用的gorocksdb为：github.com/tecbot/gorocksdb
安装上述的安装路径，使用下面的命令即可：
CGO_CFLAGS="-I/usr/local/rocksdb/include" \
CGO_LDFLAGS="-L/usr/local/rocksdb -lrocksdb -lstdc++ -lm -lz -lbz2 -lsnappy -llz4 -lzstd" \
go get github.com/tecbot/gorocksdb
如果安装目录有变化，则修改对应的/usr/local/rocksdb对应的路径