您正在查看: surou 发布的文章

Ubuntu 开启防火墙/开启端口

查看本地端口开启情况

sudo ufw status
状态: 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere
8001                       ALLOW       Anywhere
80 (v6)                    ALLOW       Anywhere (v6)
8001 (v6)                  ALLOW       Anywhere (v6)

关闭防火墙

sudo ufw disable
防火墙在系统启动时自动禁用
sudo ufw status
状态:不活动

开启防火墙,允许访问特定端口

~ sudo ufw enable 
在系统启动时启用和激活防火墙
sudo ufw allow 81
规则已添加
规则已添加 (v6)

sudo ufw status
状态: 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere
8001                       ALLOW       Anywhere
81                         ALLOW       Anywhere
80 (v6)                    ALLOW       Anywhere (v6)
8001 (v6)                  ALLOW       Anywhere (v6)
81 (v6)                    ALLOW       Anywhere (v6)

不允许访问特定端口

sudo ufw deny 81
规则已更新
规则已更新 (v6)

sudo ufw status
状态: 激活

至                          动作          来自
-                          --          --
80                         ALLOW       Anywhere                  
8001                       ALLOW       Anywhere                  
81                         DENY        Anywhere                  
80 (v6)                    ALLOW       Anywhere (v6)             
8001 (v6)                  ALLOW       Anywhere (v6)             
81 (v6)                    DENY        Anywhere (v6)

btcpool 编译安装,以及ETH私池搭建

部署条件

部署系统: ubuntu 18.04

矿池交互图

编译源代码

git clone https://github.com/btccom/btcpool.git

安装依赖

sudo apt-get update
sudo apt-get install -y build-essential autotools-dev libtool autoconf automake pkg-config cmake \
                   openssl libssl-dev libcurl4-openssl-dev libconfig++-dev \
                   libboost-all-dev libgmp-dev libmysqlclient-dev libzookeeper-mt-dev \
                   libzmq3-dev libgoogle-glog-dev libhiredis-dev zlib1g zlib1g-dev \
                   libsodium-dev libprotobuf-dev protobuf-compiler
从其master分支构建libevent

注意:版本2.1.9-beta之前的libevent将在sserver中导致死锁错误(问题#75)。请使用release-2.1.9-beta和更高版本。

wget https://github.com/libevent/libevent/releases/download/release-2.1.10-stable/libevent-2.1.10-stable.tar.gz
tar zxf libevent-2.1.10-stable.tar.gz
cd libevent-2.1.10-stable
./autogen.sh
./configure --disable-shared
make -j$(nproc) && sudo make install
编译librdkafka-v0.9.1
wget https://github.com/edenhill/librdkafka/archive/0.9.1.tar.gz
tar zxvf 0.9.1.tar.gz
cd librdkafka-0.9.1
./configure && make -j$(nproc) && sudo make install

# if you want to keep static libraries only
rm -v /usr/local/lib/librdkafka*.so /usr/local/lib/librdkafka*.so.*

下载Bitcoin源码,并建立链接到BTCPool

支持ETH的矿池用BTC选项

mkdir /data/work
cd /data/work

wget -O bitcoin-0.16.0.tar.gz https://github.com/bitcoin/bitcoin/archive/v0.16.0.tar.gz
tar zxf bitcoin-0.16.0.tar.gz

git clone https://github.com/btccom/btcpool.git
cd btcpool
mkdir build
cd build

# Release build:
cmake -DJOBS=4 -DCHAIN_TYPE=BTC -DCHAIN_SRC_ROOT=/data/work/bitcoin-0.16.0 ..
make -j$(nproc)

# Debug build:
cmake -DCMAKE_BUILD_TYPE=Debug -DCHAIN_TYPE=BTC -DCHAIN_SRC_ROOT=/data/work/bitcoin-0.16.0 ..
make -j$(nproc)
常见问题

使用当前最新版本bitcoin v0.21.0提示如下错误,暂时未跟进该问题,先用v0.16.0版本测试

CMake Error at CMakeLists.txt:267 (message):
  /data/work/bitcoin-0.21.0/src/crypto/libbitcoin_crypto.a not exists!

估计是本地已安装boost版本问题,本项目依赖boost v1.65,删除本地已安装boost,重新运行第一步,安装依赖

/usr/local/include/boost/thread/pthread/condition_variable.hpp:131: undefined reference to `boost::this_thread::interruption_point()'
collect2: error: ld returned 1 exit status
CMakeFiles/jobmaker.dir/build.make:114: recipe for target 'jobmaker' failed
make[2]: *** [jobmaker] Error 1
CMakeFiles/Makefile2:125: recipe for target 'CMakeFiles/jobmaker.dir/all' failed
make[1]: *** [CMakeFiles/jobmaker.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

运行测试

./unittest
I0125 16:19:24.673416  7530 TestMain.cc:73] BTC unittest version 2020.06.09-11-fix-jobmaker-kafka-17-g0127a750
[==========] Running 91 tests from 24 test cases.
[----------] Global test environment set-up.
[----------] 1 test from BitcoinUtils
[ RUN      ] BitcoinUtils.GetBlockRewardBitcoin
[       OK ] BitcoinUtils.GetBlockRewardBitcoin (1 ms)
[----------] 1 test from BitcoinUtils (1 ms total)
...
[----------] Global test environment tear-down
[==========] 91 tests from 24 test cases ran. (4711 ms total)
[  PASSED  ] 91 tests.

初始化btcpool

初始化目录

cd /data/work/btcpool/build
bash ../install/init_folders.sh

设置全节点

docker for Parity

https://github.com/btccom/btcpool/tree/master/docker/eth-parity/latest

docker for Geth

https://github.com/btccom/btcpool/tree/master/docker/eth-geth/v1.8.23-btcpool-patched

初始化mysql数据库以及表

cd /data/work/btcpool/install
mysql -h xxx -u xxx -p

CREATE DATABASE bpool_local_db;
USE bpool_local_db;

SOURCE bpool_local_db_ETH.sql;

CREATE DATABASE bpool_local_stats_db;
USE bpool_local_stats_db;
SOURCE bpool_local_stats_db.sql;

参考文档

https://github.com/btccom/btcpool/blob/master/docs/INSTALL-BTCPool.md
https://en.bitcoin.it/wiki/Merged_mining_specification
https://github.com/btccom/btcpool-go-modules/tree/master/mergedMiningProxy
https://blog.csdn.net/a1291985595/article/details/108799549

dfuse - database dirty flag set (likely due to unclean shutdown)

问题

由于同步节点由dfuseeos本身管理和运行,因此从测试的角度来看,dfuseeos的稳定性会对同步节点产生影响。如何避免这种关联导致的异常退出?

./dfuseeos start
Starting dfuse for EOSIO with config file './dfuse.yaml' 
Launching applications: abicodec,apiproxy,blockmeta,booter,dashboard,dgraphql,eosq,eosws,merger,mindreader,relayer,search-archive,search-forkresolver,search-indexer,search-live,search-router,statedb,tokenmeta,trxdb-loader 
Your instance should be ready in a few seconds, here some relevant links:

  Dashboard:        http://localhost:8081

  Explorer & APIs:  http://localhost:8080
  GraphiQL:         http://localhost:8080/graphiql

instance stopped, attempting restore from source (operator/operator.go:154) {"source": "snapshot", "command": "nodeos --config-dir=./mindreader --data-dir=/home/surou/Documents/Test_Dfuse/eosio/eos/programs/dfuseeos/dfuse-data/mindreader/data --pause-on-startup"}
<4>warn  2021-01-21T02:43:51.432 nodeos    chain_plugin.cpp:1199         plugin_initialize    ] 13 St13runtime_error: "state" database dirty flag set (log_plugin/to_zap_log_plugin.go:107) 
command terminated with non-zero status (superviser/superviser.go:179) {"status": {"Cmd":"nodeos","PID":4049750,"Exit":2,"Error":{"Stderr":null},"StartTs":1611197031417829539,"StopTs":1611197031434658318,"Runtime":0.016828781,"Stdout":null,"Stderr":null}}
<3>error 2021-01-21T02:43:51.433 nodeos    main.cpp:153                  main                 ] database dirty flag set (likely due to unclean shutdown): replay required (log_plugin/to_zap_log_plugin.go:107) 
cannot find latest snapshot, will replay from blocks.log (superviser/snapshot.go:153) 
restarting node from snapshot, the restart will perform the actual snapshot restoration (operator/operator.go:393) 
Received termination signal, quitting 
Waiting for all apps termination... 
app trxdb-loader triggered clean shutdown 

解决方案

第一条建议是mindreader独立于其余堆栈运行。这将大大减少dfuse-eosio的异常退出(由于其他部分)而影响mindreader操作的可能性,这对于node-manager管理nodeos进程的应用程序也是如此。

下一步是通过拍摄快照和自动恢复来定义良好的恢复策略。即使没有为EOSIO设置dfuse,nodeos也存在不干净关机的风险,例如由于内存不足错误,服务器意外重启以及其他原因。

如果您还没有自动快照获取机制,则本部分中的建议是node-manager在侧面独立运行应用程序。它将包含链的数据和状态的另一个同步副本,也可以用于服务Nodeos RPC API。这个程序负责定期拍摄自动快照。

# Storage bucket with path prefix where state snapshots should be done. Ex: gs://example/snapshots
node-manager-snapshot-store-url: <storage location, local path or supported cloud provider bucket>
# Enables restore from the latest snapshot when `nodeos` is unable to start.
node-manager-auto-restore-source: snaphost
#  If non-zero, a snapshot will be taken every {auto-snapshot-modulo} block.
node-manager-auto-snapshot-modulo: 100000 # Decrease for network with heavier traffic to take snapshot more often and shrink time to catch up from latest snapshot to HEAD
# If non-zero, after a successful snapshot, older snapshots will be deleted to only keep that number of recent snapshots
node-manager-number-of-snapshots-to-keep: 5 # Uses 0 to keep them all, useful for eventually regenerating dfuse merged blocks in parallel (not very likely but possible) 

当这些快照存在时,您现在可以将mindreader应用程序配置为使用它们,以在无法启动该nodeos过程(也几乎可以通过快照还原解决)时自动使用它们进行还原,mindreader会在过去启动并赶上来。所需的添加设置为:

# Storage bucket where `node-manager` wrote its snapshot, must be shared with `mindreader` app.
mindreader-snapshot-store-url: <storage location, local path or supported cloud provider bucket>
# Enables restore from the latest snapshot when `nodeos` is unable to start.
mindreader-auto-restore-source: snaphost

一切都可以在同一台计算机上运行,​​并可以启动不同的进程。例如,它也可以被容器化以在Kubernetes中运行。

另一个选择是使用该mindreader-stdin应用程序。此应用程序与mindreader应用程序类似,但不管理nodeos流程。相反,它nodeos通过stdin管道消耗深层数据,调用看起来像nodes -c | dfuseeos start mindreader-stdin <flag or -c config.yaml file>(可能不是确切的调用,如果需要,可以将您链接到文档)。

转载自:https://github.com/dfuse-io/dfuse-eosio/issues/202

dfuse-eosio安装及使用

编译源代码

下载代码

git clone https://github.com/dfuse-io/dfuse-eosio

安装go

wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin

检查go版本

go version

安装yarn

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt update && sudo apt install yarn

GOPATH

由于最后编译安装用到了go install,所以需要确认下GOPATH已设置

go env

最终dfuseeos会生成在$GOPATH/bin,如果没设置,临时设置下

export GOPATH=/home/当前用户/go/bin
export PATH=$PATH:$GOPATH

安装Go-bindata

go get -u github.com/jteeuwen/go-bindata/...

开始编译

./scripts/build.sh -f -y

安装dfuse定制版本的nodeos

wget https://github.com/dfuse-io/eos/releases/download/v2.0.8-dm-12.0/eosio_2.0.8-dm.12.0-1-ubuntu-18.04_amd64.deb
sudo apt install ./eosio_2.0.8-dm.12.0-1-ubuntu-18.04_amd64.deb

Depends: libicu60 but it is not installable

如果安装时提示此错误,先安装下依赖

echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic main restricted" | sudo tee /etc/apt/sources.list
sudo apt update && sudo apt install libicu60

测试dfuseeos

初始化相关配置

./dfuseeos init

启动测试

./dfuseeos start

could not locate box "dashboard-build"

如果出现此错误,

gobfuscate的混淆原理

近期,在看360net lab发布的一篇文章Blackrota, a heavily obfuscated backdoor written in Go时,看到了go语言的混淆手段,在文章中,攻击者使用开源的混淆工具gobfuscate对语言的源代码进行混淆,在对混淆之后的代码进行编译,以提高分析的难度。

根据gobfuscate的描述,gobufuscate会在源码级别混淆如下的数据:

  1. 包名
  2. 全局变量名
  3. 函数名
  4. 类型名
  5. 方法名

我们使用如下的代码来测试以下gobfuscate的混淆:

package main

import(
    "fmt"
    "io/ioutil"
    "net/http"
)
var url = "https://www.baidu.com/";
func httpget() string {
    client := &http.Client{}
    req, _ := http.NewRequest("GET",url,nil)
    req.Header.Set("Connection","Keep-Live")
    res,err := client.Do(req)
    if err != nil{
        fmt.Println("do error\n")
        return "NULL"
    }
    defer res.Body.Close()
    body,err := ioutil.ReadAll(res.Body)
    return string(body)
}
func main(){

    var body = httpget()
    fmt.Println(string(body))
}

首先gobfuscate生成的二进制文件符号是被抹除掉的:

使用go_parser对生成的二进制文件的符号进行解析,可以发现其源代码路径也被混淆了,但是源代码的文件名并没有被混淆。


函数名使用随机字符串混淆了。

对于每一个字符串,都会产生一个对用的解密函数,使用xor会字符串进行混淆。

转载自:https://mp.weixin.qq.com/s/X0iLtov4bH-HY4nhwo-zQQ