修改dfuse.yaml,添加eosq-api-endpoint-url

start:
  args:
  - abicodec
  - apiproxy
  - blockmeta
  - booter
  - dashboard
  - dgraphql
  - eosq
  - eosws
  - merger
  - mindreader
  - relayer
  - search-archive
  - search-forkresolver
  - search-indexer
  - search-live
  - search-router
  - statedb
  - tokenmeta
  - trxdb-loader
  flags: 
    apiproxy-nodeos-http-addr: :9888
    blockmeta-eos-api-upstream-addr: :9888
    eosws-nodeos-rpc-addr: :9888
    eosq-api-endpoint-url: http://39.106.103.152:8080
    common-chain-core-symbol: "8,RES"
    common-chain-id: cf057bbfb72640471fd910bcb67639c22df9f92470936cddc1ade0e2f2e7dc4f

参考

https://github.com/dfuse-io/dfuse-eosio/issues/204

查看本地端口开启情况

sudo ufw status
状态： 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere
8001                       ALLOW       Anywhere
80 (v6)                    ALLOW       Anywhere (v6)
8001 (v6)                  ALLOW       Anywhere (v6)

关闭防火墙

sudo ufw disable
防火墙在系统启动时自动禁用
sudo ufw status
状态：不活动

开启防火墙，允许访问特定端口

~ sudo ufw enable 
在系统启动时启用和激活防火墙
sudo ufw allow 81
规则已添加
规则已添加 (v6)

sudo ufw status
状态： 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere
8001                       ALLOW       Anywhere
81                         ALLOW       Anywhere
80 (v6)                    ALLOW       Anywhere (v6)
8001 (v6)                  ALLOW       Anywhere (v6)
81 (v6)                    ALLOW       Anywhere (v6)

不允许访问特定端口

sudo ufw deny 81
规则已更新
规则已更新 (v6)

sudo ufw status
状态： 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere                  
8001                       ALLOW       Anywhere                  
81                         DENY        Anywhere                  
80 (v6)                    ALLOW       Anywhere (v6)             
8001 (v6)                  ALLOW       Anywhere (v6)             
81 (v6)                    DENY        Anywhere (v6)

部署条件

部署系统： ubuntu 18.04

矿池交互图

编译源代码

git clone https://github.com/btccom/btcpool.git

安装依赖

sudo apt-get update
sudo apt-get install -y git build-essential autotools-dev libtool autoconf automake pkg-config cmake \
                   openssl libssl-dev libcurl4-openssl-dev libconfig++-dev \
                   libboost-all-dev libgmp-dev libmysqlclient-dev libzookeeper-mt-dev \
                   libzmq3-dev libgoogle-glog-dev libhiredis-dev zlib1g zlib1g-dev \
                   libsodium-dev libprotobuf-dev protobuf-compiler

注意：不再建议libevent-dev从软件源安装。 libevent的发行版和稳定版本将在sserver中导致死锁错误（问题＃75）。建议您使用下面的命令从它的master分支手动构建libevent。

有时一个或两个软件包由于依赖关系问题而失败，您可以尝试aptitude

sudo apt-get update
sudo apt-get install -y aptitude
sudo aptitude install git build-essential autotools-dev libtool autoconf automake pkg-config cmake \
                   openssl libssl-dev libcurl4-openssl-dev libconfig++-dev \
                   libboost-all-dev libgmp-dev libmysqlclient-dev libzookeeper-mt-dev \
                   libzmq3-dev libgoogle-glog-dev libhiredis-dev zlib1g zlib1g-dev \
                   libsodium-dev libprotobuf-dev protobuf-compiler
# Input `n` if the solution is `NOT INSTALL` some package.
# Eventually aptitude will give a solution that downgrade some packages to allow all packages to be installed.

安装ZooKeeper

安装依赖

sudo apt-get install -y zookeeper zookeeper-bin zookeeperd

创建所需目录

mkdir -p /data/work/zookeeper
mkdir /data/work/zookeeper/version-2
touch /data/work/zookeeper/myid
sudo chown -R zookeeper:zookeeper /data/work/zookeeper

设置机器id

sudo echo 1 > /data/work/zookeeper/myid

修改配置

sudo vim /etc/zookeeper/conf/zoo.cfg

initLimit=5
syncLimit=2
clientPort=2181
clientPortAddress=127.0.0.1
dataDir=/data/work/zookeeper
#伪分布式
server.1=127.0.0.1:2888:3888

启动服务

#start/stop service
sudo service zookeeper restart
#service zookeeper start/stop/restart/status

安装Kafka

安装依赖

sudo apt-get install -y default-jre

安装java 1.8

sudo apt-get update
sudo apt-get install openjdk-8-jdk
update-alternatives --display java
sudo update-alternatives --config java
java -version

安装kafka

mkdir -p /data/work/kafka
cd /data/work/kafka
wget https://archive.apache.org/dist/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz
tar -zxf ./kafka_2.11-0.11.0.2.tgz --strip 1

编辑配置

vim /data/work/kafka/config/server.properties

broker.id=1
offsets.topic.replication.factor=1
message.max.bytes=20000000
replica.fetch.max.bytes=30000000
log.dirs=/data/work/kafka-logs
listeners=PLAINTEXT://127.0.0.1:9092
#伪分布式
zookeeper.connect=127.0.0.1:2181

#start server
cd /data/work/kafka
nohup /data/work/kafka/bin/kafka-server-start.sh /data/work/kafka/config/server.properties > /dev/null 2>&1 &

遇到问题

yw_manager@web204_234:/data/work/kafka$ /data/work/kafka/bin/kafka-server-start.sh /data/work/kafka/config/server.properties
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/data/work/kafka/bin/../logs/kafkaServer-gc.log instead.
Unrecognized VM option 'PrintGCDateStamps'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

解决方案

降级java 1.8

从其master分支构建libevent

注意：版本2.1.9-beta之前的libevent将在sserver中导致死锁错误（问题＃75）。请使用release-2.1.9-beta和更高版本。

wget https://github.com/libevent/libevent/releases/download/release-2.1.10-stable/libevent-2.1.10-stable.tar.gz
tar zxf libevent-2.1.10-stable.tar.gz
cd libevent-2.1.10-stable
./autogen.sh
./configure --disable-shared
make -j$(nproc) && sudo make install

编译librdkafka-v0.9.1

wget https://github.com/edenhill/librdkafka/archive/0.9.1.tar.gz
tar zxvf 0.9.1.tar.gz
cd librdkafka-0.9.1
./configure && make -j$(nproc) && sudo make install

# if you want to keep static libraries only
sudo rm -v /usr/local/lib/librdkafka*.so /usr/local/lib/librdkafka*.so.*

下载Bitcoin源码，并建立链接到BTCPool

支持ETH的矿池用BTC选项

mkdir work
sudo mv work/ /data/
cd /data/work

wget -O bitcoin-0.16.0.tar.gz https://github.com/bitcoin/bitcoin/archive/v0.16.0.tar.gz
tar zxf bitcoin-0.16.0.tar.gz

git clone https://github.com/btccom/btcpool.git
cd btcpool
mkdir build
cd build

# Release build:
cmake -DJOBS=4 -DCHAIN_TYPE=BTC -DCHAIN_SRC_ROOT=/data/work/bitcoin-0.16.0 ..
make -j$(nproc)

# Debug build:
cmake -DCMAKE_BUILD_TYPE=Debug -DCHAIN_TYPE=BTC -DCHAIN_SRC_ROOT=/data/work/bitcoin-0.16.0 ..
make -j$(nproc)

常见问题

使用当前最新版本bitcoin v0.21.0提示如下错误，暂时未跟进该问题，先用v0.16.0版本测试

CMake Error at CMakeLists.txt:267 (message):
  /data/work/bitcoin-0.21.0/src/crypto/libbitcoin_crypto.a not exists!

估计是本地已安装boost版本问题，本项目依赖boost v1.65，删除本地已安装boost，重新运行第一步，安装依赖

/usr/local/include/boost/thread/pthread/condition_variable.hpp:131: undefined reference to `boost::this_thread::interruption_point()'
collect2: error: ld returned 1 exit status
CMakeFiles/jobmaker.dir/build.make:114: recipe for target 'jobmaker' failed
make[2]: *** [jobmaker] Error 1
CMakeFiles/Makefile2:125: recipe for target 'CMakeFiles/jobmaker.dir/all' failed
make[1]: *** [CMakeFiles/jobmaker.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

运行测试

./unittest

I0125 16:19:24.673416  7530 TestMain.cc:73] BTC unittest version 2020.06.09-11-fix-jobmaker-kafka-17-g0127a750
[==========] Running 91 tests from 24 test cases.
[----------] Global test environment set-up.
[----------] 1 test from BitcoinUtils
[ RUN      ] BitcoinUtils.GetBlockRewardBitcoin
[       OK ] BitcoinUtils.GetBlockRewardBitcoin (1 ms)
[----------] 1 test from BitcoinUtils (1 ms total)
...
[----------] Global test environment tear-down
[==========] 91 tests from 24 test cases ran. (4711 ms total)
[  PASSED  ] 91 tests.

初始化btcpool

初始化目录

cd /data/work/btcpool/build
bash ../install/init_folders.sh

设置全节点

docker for Parity

https://github.com/btccom/btcpool/tree/master/docker/eth-parity/latest

docker for Geth

https://github.com/btccom/btcpool/tree/master/docker/eth-geth/v1.8.23-btcpool-patched

安装mysql

sudo apt-get update
sudo apt-get install mysql-server
systemctl status mysql.service

初始化mysql数据库以及表

cd /data/work/btcpool/install
sudo mysql -uroot -p

CREATE DATABASE bpool_local_db;
USE bpool_local_db;

SOURCE bpool_local_db_ETH.sql;

CREATE DATABASE bpool_local_stats_db;
USE bpool_local_stats_db;
SOURCE bpool_local_stats_db.sql;

参考文档

https://github.com/btccom/btcpool-ABANDONED/blob/master/docs/INSTALL-ZooKeeper.md
https://github.com/btccom/btcpool-ABANDONED/blob/master/docs/INSTALL-Kafka.md
https://github.com/btccom/btcpool/blob/master/docs/INSTALL-BTCPool.md
https://www.pianshen.com/article/5122807736/
https://en.bitcoin.it/wiki/Merged_mining_specification
https://github.com/btccom/btcpool-go-modules/tree/master/mergedMiningProxy
https://blog.csdn.net/a1291985595/article/details/108799549

问题

由于同步节点由dfuseeos本身管理和运行，因此从测试的角度来看，dfuseeos的稳定性会对同步节点产生影响。如何避免这种关联导致的异常退出？

./dfuseeos start
Starting dfuse for EOSIO with config file './dfuse.yaml' 
Launching applications: abicodec,apiproxy,blockmeta,booter,dashboard,dgraphql,eosq,eosws,merger,mindreader,relayer,search-archive,search-forkresolver,search-indexer,search-live,search-router,statedb,tokenmeta,trxdb-loader 
Your instance should be ready in a few seconds, here some relevant links:

  Dashboard:        http://localhost:8081

  Explorer & APIs:  http://localhost:8080
  GraphiQL:         http://localhost:8080/graphiql

instance stopped, attempting restore from source (operator/operator.go:154) {"source": "snapshot", "command": "nodeos --config-dir=./mindreader --data-dir=/home/surou/Documents/Test_Dfuse/eosio/eos/programs/dfuseeos/dfuse-data/mindreader/data --pause-on-startup"}
<4>warn  2021-01-21T02:43:51.432 nodeos    chain_plugin.cpp:1199         plugin_initialize    ] 13 St13runtime_error: "state" database dirty flag set (log_plugin/to_zap_log_plugin.go:107) 
command terminated with non-zero status (superviser/superviser.go:179) {"status": {"Cmd":"nodeos","PID":4049750,"Exit":2,"Error":{"Stderr":null},"StartTs":1611197031417829539,"StopTs":1611197031434658318,"Runtime":0.016828781,"Stdout":null,"Stderr":null}}
<3>error 2021-01-21T02:43:51.433 nodeos    main.cpp:153                  main                 ] database dirty flag set (likely due to unclean shutdown): replay required (log_plugin/to_zap_log_plugin.go:107) 
cannot find latest snapshot, will replay from blocks.log (superviser/snapshot.go:153) 
restarting node from snapshot, the restart will perform the actual snapshot restoration (operator/operator.go:393) 
Received termination signal, quitting 
Waiting for all apps termination... 
app trxdb-loader triggered clean shutdown 

解决方案

第一条建议是mindreader独立于其余堆栈运行。这将大大减少dfuse-eosio的异常退出（由于其他部分）而影响mindreader操作的可能性，这对于node-manager管理nodeos进程的应用程序也是如此。

下一步是通过拍摄快照和自动恢复来定义良好的恢复策略。即使没有为EOSIO设置dfuse，nodeos也存在不干净关机的风险，例如由于内存不足错误，服务器意外重启以及其他原因。

如果您还没有自动快照获取机制，则本部分中的建议是node-manager在侧面独立运行应用程序。它将包含链的数据和状态的另一个同步副本，也可以用于服务Nodeos RPC API。这个程序负责定期拍摄自动快照。

# Storage bucket with path prefix where state snapshots should be done. Ex: gs://example/snapshots
node-manager-snapshot-store-url: <storage location, local path or supported cloud provider bucket>
# Enables restore from the latest snapshot when `nodeos` is unable to start.
node-manager-auto-restore-source: snaphost
#  If non-zero, a snapshot will be taken every {auto-snapshot-modulo} block.
node-manager-auto-snapshot-modulo: 100000 # Decrease for network with heavier traffic to take snapshot more often and shrink time to catch up from latest snapshot to HEAD
# If non-zero, after a successful snapshot, older snapshots will be deleted to only keep that number of recent snapshots
node-manager-number-of-snapshots-to-keep: 5 # Uses 0 to keep them all, useful for eventually regenerating dfuse merged blocks in parallel (not very likely but possible) 

当这些快照存在时，您现在可以将mindreader应用程序配置为使用它们，以在无法启动该nodeos过程（也几乎可以通过快照还原解决）时自动使用它们进行还原，mindreader会在过去启动并赶上来。所需的添加设置为：

# Storage bucket where `node-manager` wrote its snapshot, must be shared with `mindreader` app.
mindreader-snapshot-store-url: <storage location, local path or supported cloud provider bucket>
# Enables restore from the latest snapshot when `nodeos` is unable to start.
mindreader-auto-restore-source: snaphost

一切都可以在同一台计算机上运行，​​并可以启动不同的进程。例如，它也可以被容器化以在Kubernetes中运行。

另一个选择是使用该mindreader-stdin应用程序。此应用程序与mindreader应用程序类似，但不管理nodeos流程。相反，它nodeos通过stdin管道消耗深层数据，调用看起来像nodes -c | dfuseeos start mindreader-stdin <flag or -c config.yaml file>（可能不是确切的调用，如果需要，可以将您链接到文档）。

转载自：https://github.com/dfuse-io/dfuse-eosio/issues/202

编译源代码

下载代码

git clone https://github.com/dfuse-io/dfuse-eosio

安装go

wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin

检查go版本

go version

安装yarn

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt update && sudo apt install yarn

GOPATH

由于最后编译安装用到了go install，所以需要确认下GOPATH已设置

go env

最终dfuseeos会生成在$GOPATH/bin,如果没设置，临时设置下

export GOPATH=/home/当前用户/go/bin
export PATH=$PATH:$GOPATH

安装Go-bindata

go get -u github.com/jteeuwen/go-bindata/...

开始编译

./scripts/build.sh -f -y

安装dfuse定制版本的nodeos

wget https://github.com/dfuse-io/eos/releases/download/v2.0.8-dm-12.0/eosio_2.0.8-dm.12.0-1-ubuntu-18.04_amd64.deb
sudo apt install ./eosio_2.0.8-dm.12.0-1-ubuntu-18.04_amd64.deb

Depends: libicu60 but it is not installable

如果安装时提示此错误，先安装下依赖

echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic main restricted" | sudo tee /etc/apt/sources.list
sudo apt update && sudo apt install libicu60

测试dfuseeos

初始化相关配置

./dfuseeos init

启动测试

./dfuseeos start

could not locate box "dashboard-build"

如果出现此错误，