您正在查看: 2019年12月

mongo 同步节点出错,导致数据脏后得处理

讨论前提

方案讨论的前提是在,本地有一份低于mongo数据库块高度得链数据备份
如果没有得话,此文章无效,只能清数据库,整体重放了。所以体现了,定期备份得重要性!

方案实施

查询mongo 数据,确认最后写入得块高度
blocks表,查看最后插入记录得 block_num 块号。
然后查询下其余各表

  • block_states
  • action_traces
  • transaction_traces

将该块号下得所有记录按照交易id,块号都删掉。
因为不能保证最后一个块得数据是写入完整得。
其他的表,插入时有更新判断,不会造成重复,所以无需删除,如果有自行新加得表,自行处理。

启动同步节点

使用之前得低于mongo块高度得链数据备份,启动同步节点
添加--mongodb-block-start参数,指定从哪个块开始,才解析块数据写入mongo.
假设此时mongo删去最后一块后,块高度为1000,那执行开始位置为 1000 + 1

nodeos --mongodb-block-start 1001

开始启动后,节点开始同步,等到1001块时,开始mongo插件开始解析块数据,并写入mongo

附加

由于mongo插件缺少判断,导致提前插入数据,修改如下
https://github.com/EOSIO/eos/blob/3c553db73864ea19458512b4669cf9942dc59f57/plugins/mongo_db_plugin/mongo_db_plugin.cpp#L705

void mongo_db_plugin_impl::process_applied_transaction( const chain::transaction_trace_ptr& t ) {
   try {
      // always call since we need to capture setabi on accounts even if not storing transaction traces
      if( start_block_reached ) { // 新加判断
           _process_applied_transaction( t );
       }
   } catch (fc::exception& e) {
      elog("FC Exception while processing applied transaction trace: ${e}", ("e", e.to_detail_string()));
   } catch (std::exception& e) {
      elog("STD Exception while processing applied transaction trace: ${e}", ("e", e.what()));
   } catch (...) {
      elog("Unknown exception while processing applied transaction trace");
   }
}

获取更全的nodeos日志信息

新建logging.json日志配置文件

{
    "includes": [],
    "appenders": [{
        "name": "stderr",
        "type": "console",
        "args": {
            "stream": "std_error",
            "level_colors": [{
                "level": "debug",
                "color": "green"
            }, {
                "level": "warn",
                "color": "brown"
            }, {
                "level": "error",
                "color": "red"
            }]
        },
        "enabled": true
    }, {
        "name": "stdout",
        "type": "console",
        "args": {
            "stream": "std_out",
            "level_colors": [{
                "level": "debug",
                "color": "green"
            }, {
                "level": "warn",
                "color": "brown"
            }, {
                "level": "error",
                "color": "red"
            }]
        },
        "enabled": true
    }],
    "loggers": [{
        "name": "default",
        "level": "info",
        "enabled": true,
        "additivity": false,
        "appenders": ["stderr"]
    }, {
        "name": "net_plugin_impl",
        "level": "debug",
        "enabled": true,
        "additivity": false,
        "appenders": ["stderr"]
    }]
}

并将其放在config所在目录,重新运行nodeos,即可在终端看到更详细的日志。

通过RPC推送交易后的返回数据,只为参考方便提取相关字段

const result = await api.transact({
        actions: [{
            account: "eosio.token",
            name: 'transfer',
            authorization: [
            {
                actor: "bcskillsell",
                permission: 'active',
            }],
            data: {
                from:"bcskillsell",
                to:"bcskillbuy",
                quantity: '12.00000000 EOS',
                memo: 'bcskill.com'
            },
        }]
      }, {
        blocksBehind: 3,
        expireSeconds: 30,
      });
console.log('\n\nTransaction pushed!\n\n' + JSON.stringify(result, null, 2));
{
  "transaction_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
  "processed": {
    "id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
    "block_num": 32815165,
    "block_time": "2019-12-03T08:28:53.500",
    "producer_block_id": null,
    "receipt": {
      "status": "executed",
      "cpu_usage_us": 427,
      "net_usage_words": 18
    },
    "elapsed": 427,
    "net_usage": 144,
    "scheduled": false,
    "action_traces": [
      {
        "receipt": {
          "receiver": "eosio.token",
          "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
          "global_sequence": 36674138,
          "recv_sequence": 279646,
          "auth_sequence": [
            [
              "bcskillsell",
              205
            ]
          ],
          "code_sequence": 1,
          "abi_sequence": 1
        },
        "act": {
          "account": "eosio.token",
          "name": "transfer",
          "authorization": [
            {
              "actor": "bcskillsell",
              "permission": "active"
            }
          ],
          "data": {
            "from": "bcskillsell",
            "to": "bcskillbuy",
            "quantity": "12.00000000 EOS",
            "memo": "bcskill.com"
          },
          "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
        },
        "context_free": false,
        "elapsed": 166,
        "console": "",
        "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
        "block_num": 32815165,
        "block_time": "2019-12-03T08:28:53.500",
        "producer_block_id": null,
        "account_ram_deltas": [],
        "except": null,
        "inline_traces": [
          {
            "receipt": {
              "receiver": "bcskillsell",
              "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
              "global_sequence": 36674139,
              "recv_sequence": 122,
              "auth_sequence": [
                [
                  "bcskillsell",
                  206
                ]
              ],
              "code_sequence": 1,
              "abi_sequence": 1
            },
            "act": {
              "account": "eosio.token",
              "name": "transfer",
              "authorization": [
                {
                  "actor": "bcskillsell",
                  "permission": "active"
                }
              ],
              "data": {
                "from": "bcskillsell",
                "to": "bcskillbuy",
                "quantity": "12.00000000 EOS",
                "memo": "bcskill.com"
              },
              "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
            },
            "context_free": false,
            "elapsed": 5,
            "console": "",
            "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
            "block_num": 32815165,
            "block_time": "2019-12-03T08:28:53.500",
            "producer_block_id": null,
            "account_ram_deltas": [],
            "except": null,
            "inline_traces": []
          },
          {
            "receipt": {
              "receiver": "bcskillbuy",
              "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
              "global_sequence": 36674140,
              "recv_sequence": 906,
              "auth_sequence": [
                [
                  "bcskillsell",
                  207
                ]
              ],
              "code_sequence": 1,
              "abi_sequence": 1
            },
            "act": {
              "account": "eosio.token",
              "name": "transfer",
              "authorization": [
                {
                  "actor": "bcskillsell",
                  "permission": "active"
                }
              ],
              "data": {
                "from": "bcskillsell",
                "to": "bcskillbuy",
                "quantity": "12.00000000 EOS",
                "memo": "bcskill.com"
              },
              "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
            },
            "context_free": false,
            "elapsed": 6,
            "console": "",
            "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
            "block_num": 32815165,
            "block_time": "2019-12-03T08:28:53.500",
            "producer_block_id": null,
            "account_ram_deltas": [],
            "except": null,
            "inline_traces": []
          }
        ]
      }
    ],
    "except": null
  }
}

EOS 整个链停止,重新强制引导启动

BP节点全部停止,各节点重新启动后,无法正常出块

最近因为非技术原因,导致测试链所有BP节点都停止了。
由于producer_plugin中的安全检查,导致BP节点启动后,一直处于start_block_result::waiting
导致整个链网络瘫痪。

强制引导启动

先准备一台非目前BP节点所在的机器做强制引导节点。然后把目前出块账户的 2/3 +1 个BP账号都配置到此节点。
并把目前其他的BP节点都关闭pkill nodeos

修改强制引导节点的链程序,绕过安全检查

https://github.com/EOSIO/eos/blob/3c553db73864ea19458512b4669cf9942dc59f57/plugins/producer_plugin/producer_plugin.cpp#L1350
将下面代码注释掉,并重新编译链程序

if (_pending_block_mode == pending_block_mode::speculating) {
      auto head_block_age = now - chain.head_block_time();
      if (head_block_age > fc::seconds(5))
         return start_block_result::waiting;
   }

修改BP节点的配置为同步节点的配置

先将其余BP节点的config,改为同步节点的配置,也就是把强制引导节点的p2p-peer-address加到各BP节点的config,并把各BP节点的producer-namesignature-provider都注释掉。

强制启动引导节点

添加-e,强制启动出块

nodeos -e

此时强制出块节点应该已出块。

启动各BP节点

将其余BP节点分别启动,此时开始同步区块,等到追上强制出块节点高度。

迁移BP账号

关闭强制出块节点,把BP账号从config里面每注释掉一部分,就把对应的BP节点停止,并修改config,把对应的BP账号放开。此时重新启动强制出块节点和新迁移BP的节点,现在强制出块节点除去注释掉的部分出块账号。其余BP账号出块正常,且能从新迁移BP节点同步对应的出块BP账号的出块。新迁移的BP节点,出块也正常。
依次类推,迁移其余BP节点账号,等其余BP节点都出块正常后,及此次强制引导启动链网络结束。

备注

能出现全部节点停止的情况,基本都是特殊公链-“私链”,所以具有大部分BP的掌控权,我们先只针对此情景提出修复方案。

EOS 同步Mongo,各表数据解释

使用场景

由于需要对链上数据进行大量自定义得查询,一般来说还是通过部署同步节点,然后使用mongo插件将链上数据同步到Mongo进行使用更直接和方便。(虽然有其他得替代方案比如(Hyperion-History-API)(history-tools),测试下来,mongo更直接些,如果以后有其他方案更新,我们在做文章跟进,此处我们只讨论同步mongo得方案)

mongo 插件配置

必须开启以下配置,才能获取以下讨论中谈及得数据

  • mongodb-store-block-states = true
  • mongodb-store-blocks = true
  • mongodb-store-transactions = true
  • mongodb-store-transaction-traces = true
  • mongodb-store-action-traces = true

Mongo 数据表介绍

使用官方得mongo插件同步后,库数据表如下

  • account_controls
  • accounts
  • action_traces
  • block_states
  • blocks
  • pub_keys
  • transaction_traces
  • transactions

// 稍后补充

附加

action_traces表默认没有"trx_id"得索引,如果想根据"trx_id"查询得话,可以自行添加索引,不然会很慢。

db.action_traces.getIndexes()
db.action_traces.createIndex({"trx_id":1, "background":true})