Starcoin v1.12.3 Released with Improved Integration Testing for Complex Move

Starcoin has released version v1.12.3. Version 1.12 focuses on improving the integration-test capability of the (mpm) Move Package Manager and optimizing the command-line development experience for developers.

As the DApp's Move contract becomes more and more complex, we face some complex test scenarios:

scene one

Project A depends on Starcoin Move Framework, and Project B depends on Project A. At this time, if the Project B needs to test locally, it needs the integration test tool to provide the following capabilities:

Project B needs to be able to test the environment locally and deploy the Move contract of project A to the account address of project A. or
Project B forks a branch from the test network or the main network as a test environment, so that project B can directly test based on the contracts and states that have been deployed on the network.

scene two

The contract of project A relies on the historical state on the chain for some verification, such as obtaining the user's STC balance at a certain historical height. At this time, integration tests need to provide the following capabilities:

Call chain related APIs.
The post-test depends on the output of the pre-test.

The version released this time mainly focuses on these two complex scenarios to improve integration testing and command-line tools, and mainly provide or enhance the following features:

Command output and template variables, support for both integration testing and command line tools.
Use arbitrary addresses in integration tests for testing.
Enhanced integration testing capabilities of fork mode, developers can test locally based on a certain high state of the mainnet or testnet.
Added package and deploy integration testing support to facilitate integration testing through two-phase upgrades and DAO upgrades.

Detailed description and usage scenarios

`mpm integration-test`

1. Command output and template variables

For instructions with output results (such as faucet, call, call-api, etc.), the output results will be saved in the running environment, and subsequent instructions can obtain these results, and use template variables to use some fields in the results as follow-up the input parameters of the instruction.

This feature can meet the requirement that some tests need to rely on the output result of the pre-order command as the input parameter.

Example of use:

//# init -n dev --debug

//# faucet --addr creator --amount 12345000000

//# call 0x1::Account::balance --type-args 0x1::STC::STC --args {{$.faucet[0].txn.raw_txn.decoded_payload.ScriptFunction.args[0]}}

//# call-api state.get_resource ["{{$.faucet[0].txn.raw_txn.decoded_payload.ScriptFunction.args[0]}}","0x00000000000000000000000000000001::Account::Balance<0x00000000000000000000000000000001::STC::STC>",{"decode":true}]

//# run --signers creator --args {{$.call[0]}}u128 --args {{$.call-api[0].json.token.value}}u128 --args {{$.faucet[0].txn.raw_txn.decoded_payload.ScriptFunction.args[0]}}
script{
     fun main(_sender: signer, balance_from_call: u128, balance_from_api:u128, to: address){
          assert!(@creator==to, 101);
          assert!(balance_from_api == balance_from_call, 102);
          assert!(balance_from_call == 12345000000, 103);
     }
}

The --debug option can display the command output on the command line for easy debugging.
$.faucet[0], $.call[0], $.call-api[0] The array index in the command is the call number of the corresponding command in the entire test file, starting from 0.

Output format of common commands:

faucet

Example output:

{
    "output":{
        "events":[
            ...
        ],
        "gas_used":"367377",
        "status":"Executed",
        "write_set":[
            ...
        ]
    },
    "txn":{
        "authenticator":{
            "Ed25519":{
                "public_key":"0x4cb5abf6ad79fbf5abbccafcc269d85cd2651ed4b885b5869f241aedf0a5ba29",
                "signature":"0xd3d8df4f5f4a2faf1bcfacfb40e981d6307543c0ab0a4482b1e3fc27fcdb896fc74dd983620110c14afd558c5d14e7553096d7be4ec3d162628addaf10dc0201"
            }
        },
        "raw_txn":{
            "chain_id":254,
            "decoded_payload":{
                "ScriptFunction":{
                    "args":[
                        "0x662ba5a1a1da0f1c70a9762c7eeb7aaf",
                        12345000000
                    ],
                    "function":"peer_to_peer_v2",
                    "module":"0x00000000000000000000000000000001::TransferScripts",
                    "ty_args":[
                        "0x00000000000000000000000000000001::STC::STC"
                    ]
                }
            },
            "expiration_timestamp_secs":"3610",
            "gas_token_code":"0x1::STC::STC",
            "gas_unit_price":"1",
            "max_gas_amount":"40000000",
            "payload":"0x02000000000000000000000000000000010f5472616e73666572536372697074730f706565725f746f5f706565725f76320107000000000000000000000000000000010353544303535443000210662ba5a1a1da0f1c70a9762c7eeb7aaf1040c0d1df020000000000000000000000",
            "sender":"0x0000000000000000000000000a550c18",
            "sequence_number":"0"
        },
        "transaction_hash":"0xcccd3fcae2b32f699713d63216651c945127b88e99b5befe705b64b48f1a741e"
    }
}

scenes to be used:

Get the address of the faucet target account: {{$.faucet[0].txn.raw_txn.decoded_payload.ScriptFunction.args[0]}}
Get the amount of faucet: {{$.faucet[0].txn.raw_txn.decoded_payload.ScriptFunction.args[1]}}

call

The output of the call command depends on the output of the called contract function.

scenes to be used:

Get the result of the call command

call-api

The output result of the call-api command depends on the result returned by the called API interface.

scenes to be used:

Get the result of the call-api command

block

Example output:

{
    "author":"0x907ce56940b2c38ac54200e1400976a9",
    "author_auth_key":null,
    "chain_id":{
        "id":251
    },
    "number":6487001,
    "parent_gas_used":0,
    "parent_hash":"0x58d3b6aa35ba1f52c809382b876950b6038c4ce9fa078358c0fcf0b072e5ae3d",
    "timestamp":1658479121636,
    "uncles":0
}

scenes to be used:

Get block height: {{$.block[0].number}}
get block parent_hash: {{$.block[0].parent_hash}}

package

Example output:

{
    "file":"/tmp/.tmpnok0M6/0x3c6b00fadc6b4f37fa6e2c6c0949e97c89b00c07c0d1f1761671e6fe18792676.blob",
    "hex":"0x662ba5a1a1da0f1c70a9762c7eeb7aaf0146a11ceb0b040000000601000203020505070107080b0813100c2307000000010000000004746573740568656c6c6f662ba5a1a1da0f1c70a9762c7eeb7aaf000100000001020000",
    "package_hash":"0x3c6b00fadc6b4f37fa6e2c6c0949e97c89b00c07c0d1f1761671e6fe18792676"
}

scenes to be used:

Get the package file path: {{$.package[0].file}}
Get the package hash: {{$.package[0].package_hash}}

Troubleshooting：

call-api parameter exception:

Error: Invalid command. Got error error: Invalid value "xxxx" for '<params>': expected `,` or `]` at line 1 column 3

Cause: The input parameter did not match the expected parameter.

Solution:

Check whether the parameters conform to the interface definition;
The call-api data parameter is a json list, and there can be no spaces between list elements

2. Test with arbitrary address

The init initialization command option adds --addresses named=address to specify any address for testing, no longer need to specify public-keys.

//# init -n dev --addresses alice=0x662ba5a1a1da0f1c70a9762c7eeb7aaf

3. Support call-api calls in fork mode from remote nodes

Version 1.11 can fork from a certain height of a remote node, keep the state before the fork, and update the state based on the state of the fork. However, after updating the state, calling the call-api command will read the state from the remote node, and cannot obtain the locally updated state after fork.

1.12 supports call-api calls in fork mode.

Example of use

//# init --rpc <http://barnard.seed.starcoin.org>  --block-number 6487000

//# faucet --addr creator --amount 100000000000

//# block

//# call-api chain.get_block_by_number [6487001]

//# run --signers creator --args {{$.call-api[0].header.number}}u64  --args {{$.call-api[0].header.block_hash}}
script{
    use StarcoinFramework::Vector;
    use StarcoinFramework::Debug;
    fun main(_sender: signer, block_number: u64, block_hash: vector<u8>){
        
        Debug::print(&block_hash);
        assert!(block_number == 6487001, 100);
        assert!(Vector::length(&block_hash) == 32, 101);
        // 0x2e12...fc8e is the block hash of number 6487001 on barnard network.
        assert!(x"2e121adfe4bacb96de73e8b1afa5a67b8276bc4319fc3b4c0f346d12751ffc8e" != block_hash, 1002);
    }
}

During initialization, the state tree will be updated, and dozens of state tree nodes need to be read. These nodes must be obtained remotely through rpc. If the network is slow, the initialization process will be longer, up to 2-3 minutes.

4. Added package and deploy commands

In version 1.11, the publish command can be used to publish the module, which automatically compiles the Move module and builds the package transaction.

But sometimes, we need to know the Package hash in an integration test, such as a two-phase upgrade. The new package and deploy commands meet this need.

Example of use:

//# init -n dev

//# faucet --addr creator --amount 100000000000

//# package
module creator::test {
    public fun hello(){}   
}

//# deploy {{$.package[0].file}}

//# run --signers creator
script{
    use creator::test;
    fun main(_sender: signer){
        test::hello();
    }
}

CLI

1. Command output and template variables

The output results of the command will be saved in the running environment, and subsequent commands can obtain these results, and use template variables to use some fields in the results as input parameters of subsequent commands.

This feature can meet the requirement that some tests need to rely on the output result of the pre-order command as the input parameter.

Example of use:

starcoin% chain list-block
starcoin% chain get-block {{$.chain[0].ok[0].block_hash}}

2. dry-run mode

It supports dry-run mode, which can obtain the status from remote nodes, and then execute dry-run locally, which is convenient for debugging.

dry-run mode reads the latest state from the remote node, and then executes transactions based on the latest state of the node, but does not change at the same time

Example of use:

Connect to the remote node through the command starcoin --connect ws://barnard.seed.starcoin.org:9870 --local-account-dir ~/.starcoin/barnard/account_vaults consoleand enter the command line mode, then in the command line Just add the --dry-run option when executing a transaction.

account transfer --receiver 0x662ba5a1a1da0f1c70a9762c7eeb7aaf -v 2998645035 --dry-run

Transaction result:

{
  "ok": {
    "dry_run_output": {
      "events": [
        ...
      ],
      "explained_status": "Executed",
      "gas_used": "284531",
      "status": "Executed",
      "write_set": [
        ...
      ]
    },
    "execute_output": null,
    "raw_txn": {
      "chain_id": 251,
      "decoded_payload": {
        "ScriptFunction": {
          "args": [
            "0x662ba5a1a1da0f1c70a9762c7eeb7aaf",
            2998645035
          ],
          "function": "peer_to_peer_v2",
          "module": "0x00000000000000000000000000000001::TransferScripts",
          "ty_args": [
            "0x00000000000000000000000000000001::STC::STC"
          ]
        }
      },
      "expiration_timestamp_secs": "1661695240",
      "gas_token_code": "0x1::STC::STC",
      "gas_unit_price": "1",
      "max_gas_amount": "10000000",
      "payload": "0x...",
      "sender": "0x13ef2286ebcb79d268415660221de46a",
      "sequence_number": "3"
    },
    "raw_txn_hex": "0x..."
  }
}

The execution and result of the transaction can be checked in this way, but the modified state is not updated to the blockchain.

3. Resource paging query and filtering

The state list resource can be paged and queried, and can be filtered by resource_type and event.

When there are too many resources under the account, using paging query can improve the query speed, reduce the content displayed on the terminal, and make it more convenient to display.

Using resource_type, you can quickly query the specified resource and reduce unnecessary information display.

Command line arguments:

starcoin% state list resource -h
state-list-resource 

USAGE:
    state list resource [OPTIONS] <ADDRESS>

ARGS:
    <ADDRESS>    account address

OPTIONS:
    -h, --help                             Print help information
    -i, --start-index <START_INDEX>        
    -n, --block-number <BLOCK_NUMBER>      Get state at a special block height
    -s, --max-size <MAX_SIZE>              
    -t, --resource-type <RESOURCE_TYPE>

Paging query:
state list resource <address> -i <start index> -s <number of resources>
Filter based on resource_type:
state list resource <address> -t <resouce type>

Example of use:

Query the Token balance, the resource type is 0x1::Account::Balance

starcoin% state list resource 0xa2e5e803294a89e8a627f85729444c7f -t 0x1::Account::Balance
{
  "ok": {
    "resources": {
      "0x00000000000000000000000000000001::Account::Balance<0x00000000000000000000000000000001::STC::STC>": {
        "json": {
          "token": {
            "value": 12000108858
          }
        },
        "raw": "0x3a2143cb020000000000000000000000"
      }
    }
  }
}

Major Changes Pull Request

[mpm]

pre-version compatibility check by @yuliyu123 in #3522
[mpm] Support integration test template variable by @jolestar in #3533
mpm integration-test support fork from remote and call-api by @0xpause in #3600
[mpm] Integration test support package and deploy command by @jolestar in #3631
[mpm] add some integration test by @jolestar in #3635

[rpc]

add pagination for state.list_resource by @jiangying000 in #3526
[rpc][state.list_resource] add resource type filter by @jiangying000 in #3586
[rpc] improve state.list_resource filters by @YusongWang in #3615
[type] provide type_tag_match and contract event filter support struct tag match by @jolestar in #3638

[cli]

Change the default devnet data dir by @YusongWang in #3570
cli: dev get-coin don't check account by @YusongWang in #3589
[cli] dry run at cli side and print warn log when transaction execute failed by @jolestar in #3597
[scmd]Support jpst template expression in command line and starcoin testsuit by @jolestar in #3583
provide install script for the latest starcoin and mpm by @yuliyu123 in #3610
[cli]list resource support filter struct_type and start_idx max_size by @YusongWang in #3639

[StarcoinFramework]

StarcoinFramework latest， include DAOSpacke，deployed to Halley test network

Update StarcoinFramework for DAO Space by @jolestar in #3564
Update Framework to halley by @WGB5445 in #3604

Full Changes：Release v1.12.3 · starcoinorg/starcoin (github.com)

Contributors：

v1.12 English

Starcoin v1.12.3 Released with Improved Integration Testing for Complex Move

scene one

scene two

Detailed description and usage scenarios

mpm integration-test

1. Command output and template variables

2. Test with arbitrary address

3. Support call-api calls in fork mode from remote nodes

4. Added package and deploy commands

CLI

1. Command output and template variables

2. dry-run mode

3. Resource paging query and filtering

Major Changes Pull Request

`mpm integration-test`