OpenZeppelin is a well-known and trusted company in the blockchain world. It provides secure open-source smart contracts and libraries, which accelerate the blockchain development worldwide. For more than two years, developers from all over the world have been using OpenZeppelin’s v4 contracts to build applications. The v4 contracts were maintained, well-documented, and audited over a long period. However, On the 5th of October, a new journey began with the release the v5 of OpenZeppelin contracts. In this article, we will explain - with an example - why it is not safe to upgrade existing deployments from OpenZeppelin Contract v4 to v5.

TLDR

The v5 of the contracts includes significant changes and improvements. Some contracts and libraries have been restructured, while others have been removed. However, the most crucial reason for the incompatibility is that the v5 contracts of OpenZeppelin integrate a significant change to the storage layer. The contracts in v5 adopt the namespaced storage layout, which is defined in ERC-7201. This change implies that the new logic of v5 contracts will query existing data and write new data in incorrect storage slots, making an upgrade from v4 to v5 not safe. Developers have several options to update an existing contract, such as reinitializing the contract, deploying it as a new proxy, or choosing to continue with updated v4 contracts. Each option has its own unique challenges and requirements.

Prerequisites

We consider the knowledge of upgradeable smart contracts and rudimentary knowledge of the Ethereum Virtual Machine (EVM) storage as a requirement for understanding this article. The following tutorials give a good overview about upgradability of smart contracts:

• https://www.youtube.com/watch?v=JgSj7IiE4jA
• https://www.youtube.com/watch?v=kWUDTZhxKZI
• https://www.youtube.com/watch?v=YJZV9uiDbJI

For the EVM storage layer we recommend the following articles:

• https://steveng.medium.com/ethereum-virtual-machine-storage-layout-beb9a72a07e9
• https://www.adrianhetman.com/unboxing-evm-storage/
• https://medium.com/@Knownsec_Blockchain_Lab/knowsec-blockchain-lab-depth-understanding-of-evm-storage-mechanism-and-security-issues-50509acea373

Comparing how state variables are stored in OpenZeppelin upgradeable contracts in v4 versus v5

The Ethereum Virtual Machine (EVM) stores state variables in a key-value store with 256-bit keys and 256-bit values. These state variables are stored in a compact, continuous manner. Therefore, the order of the state variables in a smart contract determines their position in the storage.

OpenZeppelin v4 upgradeable contracts primarily use the default approach for storage management. The storage slots for state variables are determined by their sequential order in the contract. Therefore, base contracts often include a ‘storage buffer’ — defined as uint256[49] __gap; — to reserve storage slots. This will allow future versions of that contract to use up those slots without affecting the storage layout of child contracts. This approach of storage handling complicates contract management and upgradeability by increasing the risk of mistakes and storage conflicts, where new upgrades may unintentionally overwrite existing storage slots.

To simplify upgradability and minimize storage conflicts, OpenZeppelin v5 adopted the namespace storage layout, which has been proposed officially as ERC-7201 on June 20, 2023. The ERC-7201 recommends using pseudorandom locations derived from a namespace as the basis for new storage trees in base contracts. The root slot for storage is calculated using a specific formula. This approach does not require changes on the EVM level and can be implemented by grouping state variables within a struct and using assembly language to access specific storage slots directly, as shown in the example:

In the example smart contract x and y are state variables. However, they are not stored on the first and second slots in the storage as default. They will be stored in the storage tree under the given root MAIN_STORAGE_LOCATION = 0x183a…b500, which is calculated using the suggested formula in ERC-7201.

When a v4 upgradeable contract is deployed on the network, the state variables of the contract are initialized, stored, and managed using the default storage pattern based on their order in the contracts. However, if the implementation contract is upgraded using v5 contracts, a critical issue occurs. The new v5 contracts are designed to interact with storage in a different way, using the namespaced storage layout introduced in ERC-7201. As a result, these updated contracts will not recognize or access the original storage slots used by the old contract. This mismatch in storage access will lead to dangerous and incorrect states where variables may not reflect the intended values, and functions may behave unpredictably, leading to loss of funds or compromised security.

Unsafe Upgradability Example

The following ExampleV4 contract uses OpenZeppelin v4 and imports the following contracts Initializable, OwnableUpgradeable, and UUPSUpgradeable. It sets the value of exampleStateVariable, the address of the owner, the address of implementation contract, and the status of the contract to Initialized during the initialization process when it is deployed as proxy.

To check the storage keys of the state variables, we can deploy it as proxy using Remix IDE and then debug it using the Debugger.

As shown in the picture, slot 0x000…000 (slot 0) has the value 0x01 marking the contract as initialized. At Slot 0x000….033 (Slot 51) the, address of the owner of the contract is stored. At 0x000…0c9 (Slot 201) the value 7 of the exampleStateVariable is stored (Note: The variables are not sorted at slots 0, 1, 2 due to the buffer gaps between the contracts). At Slot 0x36089…82bbc the address of the implementation contract stored (Note: The slot of the implementation contract address was already hardcoded in v4 in a similar way to the namespaced storage layout to prevent conflicts).

For demonstration purposes, we will update the implementation contract to ExampleV5 using OpenZeppelin v5 contracts, without introducing new logic. To execute the upgrade, we must import the new v5 contracts, make minor modifications to the original contract to align with the new structure, and then deploy it. Then, we invoke the upgradeTo function on the proxy contract, providing the address of the new implementation contract as an argument.

After upgrading the contract we can call the two state variables in the proxy contract to check their values.

The value of exampleStateVariable is now 1 and not 7 because the upgraded contract reads the value from the first storage slot, which was the same slot of the initialized status. Furthermore, the owner is the zero address 0x000…000 because the new implemented contract following v5 of OpenZeppelin looks for the address of the owner at another storage slot. Inspecting the v5 contract, we will find that the owner value is read from slot 0x9016…9300, which is empty.

The same issue applies to the Initialization status of the contract. It is read from slot 0xf0c5…6a00 and not from 0x000…000 as it was in v4 contracts

This means our very simple updated contract has incorrect values, is not initialized, and has the zero address as Owner.

At this point, anyone would be able to initialize it again giving the address of owner to control the contract. Even if the original owner still managed to initialize the contract before anyone else the contract will stay in an unsafe state. To explain this we will re-initialize and debug the storage of the upgraded contract using Remix.

As displayed on the above storage snapshot, the original (v4) storage slots at 0x000…0c9, 0x000…033 are not empty. Therefore, new state variables added to the contract could unintentionally use these slots, creating invalid values.

Going Forward with Existing Contracts

For developers, finding the best way to update existing contracts is crucial. There are several options for updating deployed v4 upgradable contracts, some of these options are:

• Reinitializing the Contract: This process involves clearing the storage and migrating the data within the contract. While this approach might be feasible for simple contracts, for complex contracts with dynamic data types such as arrays, strings, and mappings, the storage becomes overly complicated, making migration within the contract impractical, if not impossible.
• Deploying as a Proxy: Similar to the Social Migration approach for non-upgradable contracts, this method involves deploying the new contract as a new proxy with a new address, followed by migrating the data and users to this new contract. Depending on the contract type and existing data, this process can become complicated. Additionally, it requires users’ involvement.
• Updating using v4 Contracts: Likely the most advisable approach is to continue using v4 contracts. This method eliminates the need for data migration or storage restructuring. Developers need to maintain their existing OpenZeppelin v4 contracts, extending or updating only the necessary components for their specific needs when required.

Summary

In Ethereum, upgradeable contracts are complicated due to potential storage conflicts. The introduction of a new storage layout aims to make the upgrade process easier and cleaner. However, old upgradable v4 contracts will face issues with the new layout, as their state variables could be dislocated, leading to an invalid contract state. Therefore, a native upgrade from v4 contracts to v5 is considered as not safe and is not recommended, requiring careful selection of strategies for updating v4 Upgradeable Contracts.

Why Upgrading OpenZeppelin Smart Contracts from Version 4 to Version 5 is Unsafe was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Due to its immutability and censorship resistance, the Blockchain seems to be a suitable place to manage identities and credentials. However, querying the data on the blockchain in a trusted and efficient way is one of the most challenging issues every Web3 developer has to face while developing a DApp. Therefore, this article describes a simple approach to revoke verifiable credentials and a decentralized and efficient way to index and query those revoked credentials using the Graph protocol.

We consider the knowledge of Self-Sovereign Identity (SSI) and rudimentary knowledge of the Ethr DID method as a requirement for understanding this article.

Ethr DID and Credentials

The Ethr DID method uses Ethereum addresses as identifiers. By default, the DID of the issuer/subject does not require an on-chain registration and it is constructed from the Ethereum address and controlled by its private key. This feature minimizes the effort of integrating Identities in many on-chain use cases.

When an off-chain verifiable credential is created, the issuer entity must set the “issuer” property to its Ethr DID and the “credentialSubject.id” property to the Ethr DID of the subject. Example:

"verifiableCredential": {
    "@context": [..],
    "issuer": {
      "id": "did:ethr:0xf1232f840f3ad7d23fcdaa84d6c66dac24efb198"
    },
    "issuanceDate": "2022-07-2T12:00:00.000Z",
    "credentialSubject": {
      "id": "did:ethr:0x435df3eda57154cf8cf7926079881f2912f54db4",
      "degree": "Bachelor Of Arts"
    },
    "proof": { ... }
  }

After that as usual the issued credential will be used by the subject to generate a verifiable presentation. The verifier will verify the presentation and accept or reject it depending on its business logic.

Credential Revocation

Verifiable Credentials are similar to certificates, some of them have an expiration date and some don’t. However, in both cases, the issuer and the subject must be able to revoke them. To enable this critical requirement, we need a tamper-proof revocation data registry that is always available and cannot be censored by any of the mentioned parties. This can be achieved using the Blockchain. Such a registry can be implemented as a simple smart contract that stores a reference of the credential and a reference of the entity who revoked it.

The revocation process:

• Get the hash value of the issued verifiable credential using the keccak256 hash function after converting the credentials to bytes. This value will be used as the on-chain reference to the credential
• Send a transaction to the revocation smart contract to call the revoke method revoke(bytes32 digest)giving the extracted hash value as input
• The entity that revoked the credential is the address of the invoker and it can be matched with the issuer or subject of the credential based on their DIDs

The additional important key that will help use indexing those revocations is that when a credential is revoked, the revocation smart contract triggers an event that includes the keccak256 hash of that credential and the address of the entity that revoked it.

In the Github Repository, we prepared a simple project that:

• Deploys a revocation smart contract that stores references to the credential and the entity that revoked it.
• Uses the did-jwt-vc and ethr-did libraries to issue 3 off-chain verifiable credentials by 3 different issuers and revoke them on-chain

Please check the ./ssi-contracts folder and follow the instructions in the README file. After following the steps you should have the address of the revocation registry smart contract deployed on the Goerli network. The 3 revoked transactions can be found on Etherscan using the address of the deployed contract.

Why do we need to index the revoked Credentials?

In a verification scenario, the verifier gets a verifiable presentation that includes 1 to n verifiable credentials from the subject. In addition to the normal validation process of the credentials, the verifier must also check the on-chain status of the credential at any time easily and efficiently. To do that the verifier could request the status of each credential by calling the methodrevoked(address issuer, bytes32 digest) in the revocation registry contract. However, this approach of getting the information is limited, costly, and not efficient for many reasons, some of which are:

• It requires knowledge of the Blockchain and Web3 technology
• It is slow and limited because of the latency, number of requests, node capability
• It requires running a blockchain node or relaying on a node provider
• Duplicate requests for the same credentials are sent over and over again
• It requires the verifier to be online all the time

And this is when indexing comes into play to provide an easy, fast and simple way to iterate and query data. Indexing is especially needed in the Blockchain area where its underlying design makes it difficult to search for relevant data.

But How can we index the Blockchain?

One approach is iterating all transactions and checking if the transaction interacts with our revocation contract. This method is very slow and resource-intensive. The other approach which is used by most developers is to subscribe to events that are triggered by the target contract. This method is much easier, supported by tools and libraries like web3js, and is more resource-saving. Now we come to the second important question of who should do the indexing? There are some possible answers:

1. The verifier himself. But it is not a good solution because each verifier needs to build up and maintain its own logic and infrastructure. Furthermore, the server of the verifier that is responsible for the indexing is a Single-point-of-failure

2. External ingestion service that indexes the data on behalf of the verifier. Such a service could be very useful. However, it creates one of the most critical issues which is the lack of TRUST. Verifiers need to trust the ingestion service and relying on it eliminates the advantage of using the Blockchain

3. Decentralized indexing protocol like The Graph protocol, which enables decentralized applications to query complex data from the blockchain without having to develop and operate proprietary indexing servers or use a centralized service provider. This is the preferred solution in most use cases because:

• It is distributed and decentralized. Therefore it removes the Single-point-of-failure and the missing trust issue
• It does not require advanced Web3 knowledge. Developers use only GraphQL requests to query the data from already implemented and indexed Subgraphs
• The technology behind it has been designed specifically for the Blockchain. Therefore, it is quick and easy to query and filter
• Cost-efficient because indexers are competing with each other to provide the best service for the lowest price
• Can be used by many verifiers around the world with minimal latency because the nodes are distributed and public

The creators of the Graph protocol recognized the indexing and querying problem at the end of 2017 and started working to solve it by defining and implementing a decentralized indexing protocol. The introduction page of the Graph protocol offers detailed information about the protocol and you can also join the free and very useful courses provided by The Graph Academy.

Develop and Deploy a Revocation Subgraph

A Subgraph defines which data should be indexed from a blockchain, and how it should be stored. In our simple example, the core data the verifier requires is the hash of the credential and who revoked it. To make things easier for verifiers we could also add the timestamp and block number.

To keep the article simple we will deploy the subgraph on a hosted service. Hosted service can be seen today as a good and easy place to start with the development of Subgraphs because it is free to use and reduces the deployment and testing complexity of Subgraphs. Developers can at any time migrate their Subgraphs to the decentralized network.

The following steps to develop and deploy the Subgraph are described in detail with all the related files in this repository :

• First, initialize a new sample project using the graph cli
• Then modify some of the configurations like adding a startBlock and checking the address of the contract
• After that comes the main part of the development which is to define the entities and implement the handle method that will be executed when an event is emitted by a smart contract
• In the end, build the project and deploy it on the hosted service using the graph cli and an access token

When the Subgraph is deployed and synchronized we can query the indexed data using simple GraphQL requests.

Example: open https://api.thegraph.com/subgraphs/name/<github-name>/credential-revocation-graph in browser and run the following query to get the hashes of all credentials that are revoked by Issuer-A from ./ssi-contracts

The results should contain only a single revoked credential unless you modified the scripts inside ./ssi-contract or used an already existing contract for this query. You can also compare the results with the logged data of the executed script in./ssi-contract

Summary

In this article, we described one of many approaches on how to revoke issued credentials. This outlined approach benefits from the Ethr DID characteristic and it depends on the Blockchain as an immutable registry accessible for all parties. We also talked about the likely best way to index and query revoked credentials without relying on centralized services. And at the end, we showed how to develop a simple Subgraph that could run on the Graph protocol decentralized indexing network and index the required data.

51nodes GmbH is a provider of crypto-economy solutions based in Stuttgart, Germany.

51nodes supports companies and other organizations in realizing their Blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (DApps), integration of blockchain with industry applications, and tokenization of assets.

Indexing and Querying Revoked Verifiable Credentials was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Today, one doesn’t even have to know anything about Solidity programming or blockchains at all to create a product or service that is built on top of a blockchain. Nowadays there are services like Tatum and Moralis, providing almost everything you need to instantly start developing your product or service instead of managing all the “behind the scenes” things on your own.

This post explains how to get started with blockchain development using the platforms of Tatum and Moralis. I discuss pros and cons and what you can build with them today.

Introduction

The two main features of Moralis and Tatum are a Web API and an SDK for application development based on a public blockchain. With the Web API we can retrieve all the necessary data from a blockchain and with the SDK we can easily use code to interact with the blockchain to make a transaction or call a function of a smart contract. The blockchain data retrieved by the Web API can be accessed with the SDK as well. Without using such a service a developer must run his own full node to know the blockchains state in order to make a correct transaction based on the current state.

Moralis and Tatum work on multiple public blockchains like Ethereum, Binance Smart Chain, and Polygon. Thus, one can retrieve data from all these blockchains using a single Web API. While Moralis “only” covers 6 blockchains, the Tatum API covers around 40. Both APIs can be used to access the main network and a test network of a blockchain like for example Rinkeby or Ropsten for Ethereum.

Behind the scenes, services like Tatum and Moralis run a full node of every blockchain they support. Having access to all blockchains’ data, they are able to aggregate it so that one can access all users’ balances, token balances or NFTs in one place. The API can then be used within the respective SDK to help developers creating their product or service. The following picture shows the general architecture of Moralis.

Web API comparison of Moralis and Tatum

Using a Web API to get all necessary blockchain data saves developers a lot of time and money. Retrieving blockchain data via Tatum and Moralis is free of charge. Both APIs are able to:

• Get all Transactions an address has made.
• Get the current block of any chain.
• Get the native balance or balance of any ERC-20 token.
• Get all NFT information of a user or a contract.
• Get and write data from and to IPFS.

Both, Moralis and Tatum cover all “basic” API requests. Differences are listed in the following.

Requests specific to Moralis’ API

• Get all Events emitted at a specific smart contract.
• Get liquidity pair reserves for a Uniswap based exchange (DeFi).
• Resolve an unstoppable domain and return the address.

Good thing to know: The Moralis Web API differentiate between all Ethereum Virtual Machine (EVM) blockchains like Ethereum, Binance Smart Chain, Avalanche and Polygon and the new created Solana Web API for Solana. For Solana, one can “only” retrieve users’ current balance, transactions, NFTs and tokens at the moment.

Requests specific to Tatum’s API

• Generate a wallet and get the private key, the mnemonics and the address.
• Deploy a standard ERC-20, ERC-721 and ERC-1155 contract.
• Interact with virtual accounts and the off chain ledger — I will cover this later in more detail.

How to use the APIs

After you registered for Moralis you can see on the left side of the screen the Web3 API Button. Once clicked you can use the whole Web API within the browser (see figure below). The x-api-key and the whole curl request are shown when you execute a request. Using the API from outside the browser like for example with Postman or curl is also possible.

Using Tatum’s API works a bit different. After the registration you will be redirected to your dashboard. Here you can find your API keys — one for all main networks and one for all test networks.

To see the whole API you have to go back to the homepage to find the documentation of the API. Sadly, it is not possible to use the API within the browser. However, Tatum provides request code examples in 13 programming languages (see below).

Pricing

Both APIs have a very good free version that can be used immediately after registration. The following figures show the number of requests possible with the APIs, depending on the plan you may choose.

Web API conclusion

Most importantly first: both API’s from Moralis and Tatum are working great and as expected and I can recommend both to you. Moralis has a little bit less supported blockchains and features than Tatum but looks more modern on the other hand. With both API’s you have access for all “basic” requests and if you really need something advanced you should look at both API’s and compare if one of these API’s is already dealing with your problem. If so just choose it and try it out.

How to use Moralis’ blockchain SDK

Moralis describes itself as “the ultimate Web3 Development Platform” and is often said to be “the firebase of crypto”. Regarding help in blockchain software development, a JavaScript SDK is available. You can either use it as a plain npm module named “moralis” or you can use the “react-moralis” module to use the SDK within your React project. All the following paragraphs will focus on EVM blockchains but like already mentioned Moralis also support Solana.

Moralis server

Starting a new project with Moralis requires you to create a Moralis Server within your Dashboard.

Creating a Moralis server requires only to select a name, your region, main network or test network and which blockchains should be included and listened to. In the free version of Moralis, you can run 3 of these servers at the same time but you must revoke them manually after 7 days.

After you created your Moralis server, you can open either its details or its dashboard. Opening the details will show you your Server URL and your Application ID. These two keys have to be imported in your SDK to use it correctly. In the dashboard you can see all your additional databases Moralis is creating for you.

In the following paragraphs I will show use cases of these additional databases and cover some of the more advanced features you can do with your Moralis server that is helpful for Dapp development.

Authenticating users

The Moralis SDK provides a simple way to track all users of a Dapp. Either users can register themselves with a username and password or do it the web3 way by using Metamask. The following code example shows how to connect your Dapp with your Moralis server and how to authenticate users the web3 way.

const Moralis = require('moralis');

const serverUrl = "YOUR_SERVER_URL";
const appId = "YOUR_APP_ID";

Moralis.start({ serverUrl, appId });

Moralis.authenticate()

The authenticate function will open the user’s Metamask window and request a signature from the user by clicking the “Sign” button. Note that there is no gas fee involved in signing because it is not a transaction.

Now, here’s the cool part. Once a new user logged in, you can observe a new row added to your “User” table in the Moralis dashboard with the address of the user. Furthermore, the Moralis server creates additional tables like for example “EthTransactions”, where you can see all transactions made of all users on the blockchain networks you selected when creating the server.

Because your tables can get fast quite big, your whole database is capable of managing queries and filters in order to only get and show the desired data.

Listening to events

One feature only Moralis provides, is listening to events of a specific contract. To this respect, open your server details and go to the “sync” tab. Now either choose to “follow” a specific address or a specific event. If you choose to follow an event, you only have to paste its name with parameter, its ABI as JSON and the contract address. After creation the data will synchronize automatically and the result can be found within a newly generated table in your server. A more detailed tutorial can be found here.

Cloud-based queries

Complex database queries can be run on the Moralis server in the cloud. For example, if you have a table with movie ratings with 1–5 stars and you would like to know the average film rating, you normally would first get all ratings and then calculate the average. To directly get the answer, just deploy and call the query as a JavaScript function within your Moralis server. For more information about the correct syntax check out Moralis’ documentation here.

Getting data with Moralis’ SDK

Let’s take a closer look at the actual JavaScript SDK and how to retrieve blockchain data. Moralis differentiates between their “normal” API for all EVM blockchains and their Solana API. The following paragraphs refer to the EVM blockchain API.

To get some data, there are 4 entry points:

• With Moralis.Web3API.native.functionName(options) we can access the Web API data within our SDK. Click here for more information.
• With the useMoralisQuery(‘tablename’) function we can get access to all the data within a specific table of our Moralis server. With the React SDK it should also be possible to enable live updates whenever your table you reference to updates. However, this didn’t work for me while testing.
• With the Moralis.executeFunction(‘options’) function we can interact with a smart contract directly in order to get the state of a public variable or the result of a read only function. Click here for more information.
• With the Moralis.Cloud.run(‘function_name’) function we can execute our cloud functions and get the result back.

Using these methods, you can retrieve all blockchain data required for building your Dapp. Of course, all these possibilities can be combined within your Dapp. For example, many requests like getting the specific ERC-20 balance of an address can be resolved using any of these four possibilities.

Creating transactions with Moralis’ SDK

Knowing how to get blockchain data, let’s look at how to make a transaction with the Moralis SDK. The general approach of doing a transaction with the Moralis SDK is by conducting the actual transaction via Metamask. This way, the user and the developer don’t have to deal with any private keys, because Metamask handles them. Metamask provides the SDK with the information, which network the user is currently connected to, so we don’t have to deal with any blockchain network code. The SDK creates a correct transaction with the currently connected network on its own. Thus, the user only has to click on a button within the Dapp and Metamask to accept the transaction.

Making a transaction with the Moralis SDK is a one liner of code. Only the parameters of a transaction, e.g., the transaction amount, the receiver and the contract address have to be passed as JSON parameter to the Moralis.transfer(options) function. Note the different parameter settings in the code examples below to create a transaction for respectively sending a native coin, an ERC-20 token or an NFT. For more information click here.

1) Transfer native coin options
const options = 
{type: "native", 
 amount: Moralis.Units.ETH("0.5"), 
 receiver: "0x.."
}

2) Transfer ERC20 Token options
const options = 
 {type: "erc20",
 amount: Moralis.Units.Token("0.5", "18"),
 receiver: "0x..",
 contractAddress: "0x.."
}

3) Transfer ERC721 Token options
const options = 
{type: "erc721",
 receiver: "0x..",
 contractAddress: "0x..",
 tokenId: 1
}

// Call the Moralis.transfer function with options as parameter
const transaction = await Moralis.transfer(options);

// You can wait until its confirmation and do sth. with the result
const result = await transaction.wait();

Interacting with smart contracts

For interacting with a function of a smart contract, you have to create an options variable as JSON string that includes the contract address, the function name, the ABI of the function, and the parameters. Now call the Moralis.execute(options) function with the options string as parameter. That’s everything it takes to create a transaction with the Moralis SDK as shown in the following code example.

let options = {
 contractAddress: '0x...',
 functionName: 'FUNCTION_NAME',
 abi: [ FUNCTION_ABI_AS_JSON ],
 params: {
     parameter1: value1,
    }
};

const tx = await Moralis.executeFunction(options);
const result = await tx.wait();

Moralis conclusion

Moralis appears modern and helps building a Dapp. The Web API works great and using the SDK, it becomes a one liner to handle user authentication, getting data from any blockchain or creating valid transactions. Also, the documentation is very well written and detailed. They also have a very active Youtube Channel.

A limitation of Moralis is that it only deals with reading from and writing to a blockchain. Creating your own wallet or deploying a new smart contract are not possible. Therefore, scripting can get complicated because to send a transaction you have to deal with Metamask in the first place. But if you are only looking for a SDK that helps you build the JavaScript/backend part of your Dapp then choosing Moralis makes a good choice.

How to use Tatum’s blockchain SDK

Tatum describes itself as “The ultimate blockchain development platform” that offers a unified framework for 40+ blockchain protocols to reduce the complexity of blockchain development. While Moralis specialized in building Dapps, Tatum is a more general tool with a lot of methods needed to deal with a blockchain. Methods for creating and managing your own wallets and private keys, deploying smart contracts and interacting with them are included in the Tatum JavaScript SDK and their API.

Maybe because of the broad scope of Tatum, it feels a little bit unfinished. They feature a lot of services and endpoints instead of specializing on one specific. Thus, their Web API documentation is so big that it takes more than 2–3 seconds to load and render the whole page. Also, a lot of things are missing completely, and some other things are not properly explained yet in the documentation. However, Tatum told me that they are working on it already. So as well as in the Moralis section of this article, I will focus in the following paragraphs on EVM blockchains only, since the Tatum API is most developed for these kind of blockchains.

In the following paragraphs, I will show you the most important things you can put to use already today with Tatum.

Initializing Tatum’s SDK

Tatum’s SDK is as well written for JavaScript and can be downloaded as npm module with the name ‘@tatumio/tatum’. Once installed you can import it within your project and use one of the hundreds of functions Tatum provides. Calling any of these functions is a one liner. But compared to Moralis you don’t pass all the options as a single JSON string, but rather each parameter is passed individually.

Generating a wallet

By creating a wallet from scratch, we can script any transaction we want because we know the private key and can sign transactions on our own. To create a wallet with the Tatum SDK you need to import the generateWallet function and the Currency Enum. Once you call the function with the desired cryptocurrency, like for example Bitcoin or Ethereum, you will get your 24-mnemonic words and your xpub for your newly generated HD-wallet.

With these two variables, Tatum can create new addresses with the corresponding private keys for your new wallet using the generateAddressFromXPub and generatePrivateKeyFromMnemonic functions, which you have to import as well. The following code demonstrates the whole process.

1) const {generateWallet, generateAddressFromXPub, generatePrivateKeyFromMnemonic, Currency} = require("@tatumio/tatum");

2) const btcWallet = await generateWallet(Currency.BTC, false);

3) const btcAddress = generateAddressFromXPub(Currency.BTC, false,    "YOUR_XPUB", "HD_WALLET_INDEX");

4) const btcPrivateKey = await   generatePrivateKeyFromMnemonic(Currency.BTC, false, "YOUR_MNEMONICS", "HD_WALLET_INDEX")

In the first line of code, we import the SDK and all required functions. In the second line, we generate a Bitcoin wallet and get the mnemonic and the xpub. In the third line, we create a new address from the xpub and in the fourth line, we create the private key for it. Because we are dealing with a HD-Wallet you must also pass the index of it as a parameter.

Tatum’s Key Management System (KMS)

To create a transaction, we need our newly created private key to be passed as a parameter to each function. Because it is never a good idea to send your private key across the internet, Tatum provides two approaches: (1) You can sign your transaction locally — this way you must have your private key stored locally on your computer, or (2) you can use the Tatum Key Management System (KMS). KMS is a cloud solution by Tatum, allowing to import your private keys encrypted with a password of your choice to let Tatum sign transactions for you within their service. This way, the private key doesn’t have to be sent across the internet because Tatum signs your transaction and only sends the signature of the transaction. For more information, click here.

Accessing blockchain data

Currently, the Tatum SDK does not support yet to get all the blockchain data being available via the API. So to access blockchain data via Tatum, one should consult the Web API docs and make a HTTP request. This method can be used to build your own Dapp. I recommend to create a separate function for each request, in which you pass all necessary parameters and do the actual HTTP request within this function. This way, you create your own little framework for one-liner requests to get any necessary data from the blockchain.

Interacting with a smart contract

Interacting with a smart contract doesn’t work yet with the SDK at the time of writing. Instead, you must use the API as described here.

Transfering coins and tokens

At least the transfer functions for native coins, ERC-20 tokens, NFTs and ERC-1155 tokens exist already in Tatum’s SDK and are documented. The following code example shows how to create an ERC-20 token transaction. For more information, check out the documentation.

import {sendEthOrErc20Transaction, ethGetTransaction, Currency} from '@tatumio/tatum';

const transaction = await sendEthOrErc20Transaction({
chain: Currency.'CHAIN_TO_USE', 
fromPrivateKey: 'YOUR_PRIVATE_KEY',
contractAddress: 'CONTRACT_ADDRESS',
digits: DIGITS_OF_TOKEN,
amount: "AMOUNT_TO_SEND--0.1 for example",
to: "TO_ADDRESS"
});

const transaction = await ethGetTransaction(transaction.txId);

Deploying a standard smart contract

Tatum’s SDK allows deploying standard smart contracts for EVM-compatible blockchains. Thus, one can deploy a smart contract with custom parameters for token standards listed below. Detailed information on parameters to use regarding using one of the token standards are linked to the list:

• ERC-20
• ERC-721 (NFT)
• ERC-1155

Virtual accounts

Virtual accounts are more or less the equivalent of a Moralis server. It’s a separate private ledger maintained by Tatum to enable, e.g., setting up webhook notifications when at a specific address a new transaction is observed. While for Moralis you must paste your address to synchronize and the webhook URL endpoint in your server’s dashboard, in Tatum you must set it up using the SDK or API. Moralis’ dashboard has great usability but Moralis provides no API for specifying webhooks. Tatum on the other hand has no dashboard but provides an API and SDK for the setup.

Unfortunately, Tatum’s SDK did not work properly during my experiments and I had to use the API again to test the webhooks. The steps for setting up a virtual account in Tatum and to subscribe to blockchain events are:

• Create a new wallet and save your ‘xpub’
• Create a new virtual account with the ‘xpub’ as one of the parameters. Only a private key corresponding to this ‘xpub’ is allowed to do a transaction within this virtual account.
• Connect the address to sync with your virtual account
• Initialize a subscription that listens to incoming events and send the data to the endpoint of your choice.

Tatum conclusion

Tatum’s SDK is a not a complete product at the moment, given its claims. Tatum aims at offering much more features than Moralis for interacting with a blockchain, especially when starting from scratch. Tatum is also more flexible by supporting more blockchains and already provides more usable features that can be helpful for scripting: Generating a wallet, managing keys, or deploying a new smart contract works great. The documentation is quite okay, but I would like to see Tatum to improve on that in the future. For functions not present in the SDK yet, you can use Tatum’s API instead, which works fine. To conclude, Tatum already provides a decent service and given the assumption that they will improve their documentation and SDK, I can recommend using Tatum’s SDK and API for developing a Dapp.

Other blockchain SDKs and APIs worth considering

Besides Moralis and Tatum, several other services exist that provide a Web API and a SDK for Dapp development. But none of them are as advanced as Moralis and Tatum and can be used only for some parts of your Dapp probably.

• Idexo: It is another service, offering an SDK for blockchain development that feels like the very small brother of Tatum. Idexo provides only an SDK and is very limited regarding the number of blockchains and features. When trying to evaluate it, I had problems with registration and login. What Idexo is most known for is their Telegram bot that allow users to mint an NFT within Telegram.
• The Graph: This is a Web API, focussing on complex and automated queries on Ethereum and IPFS.
• NOWNodes: They offer a Web API that is comparable to Tatum. The service supports also 40+ blockchains and a lot of endpoints are supported for each blockchain.
• Infura: Infura provides some tools and endpoints to interact with the Ethereum blockchain and IPFS.
• Cosmos: If you even want to build a custom blockchain, then the Cosmos SDK might be worthwhile. Creating an own blockchain gets almost as simple as deploying a new token with it.

Conclusion

It’s astonishing to experience how easy blockchain development has become due to tools and services like Moralis and Tatum. I recommend to use Moralis for developing a Dapp and to use Tatum for close interaction with different blockchains. I expect more to come as these services are evolving quickly. On top of that many other services exist to keep an eye on.

51nodes GmbH based in Stuttgart is a provider of crypto economy solutions. 51nodes supports companies and other organizations in realizing their Blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (Dapps), integration of blockchain with industry applications, and tokenization of assets.

Thanks to Achim Klein and Jan-Paul Buchwald for their help in the course of developing this article.

Getting Started with Blockchain Development: Pros and Cons of Tatum vs. Moralis was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Vor diesem Hintergrund streben Selbstsouveräne Identitäten (SSI) die Verknüpfung der Nutzerfreundlichkeit einer Google-ID mit einem hohen Grad an Nutzerkontrolle und Privatsphäre an.

Was genau ist SSI?

SSI bilden die neueste Entwicklungsstufe digitaler Identitäten. Eine Identität meint alle Attribute einer Person, Organisation, oder Gegenstands, die dieses Subjekt definieren. Attribute beschreiben die Charakteristika eines Subjekts. Eine digitale Identität meint hier die digitale Repräsentation dieser Attribute. Je nach Verwendungskontext können verschiedene Teilidentitäten benötigt werden, die nur einen Teil der Attribute umfassen.

Ein Credential ist ein fälschungssicherer und überprüfbarer Nachweis von Attributen eines Subjekts. Credentials werden in einem persönlichen digitalen Wallet gehalten. Das Subjekt ist der Holder der Nachweise, nachdem es diese von einem Issuer beschafft hat. Ein Issuer stellt Credentials nach einem Schema aus, das eine übergreifende Verwendung und allgemeingültige Interpretation erlaubt. Ein Verifier fordert, z.B. für den Zugriff auf einen Dienst, einen Nachweis einer Berechtigung (ein Credential) von dem zugreifenden Subjekt. Über ein Verifiable Data Registry werden dazu Identifikatoren ausgetauscht, die auf Subjekte, deren Credentials und deren Issuer verweisen können. Das Verifiable Data Registry enthält keine subjektbezogenen Daten.

Welchen Mehrwert bringt SSI?

SSI überwindet das inkonsistente und — für Nutzer als auch Diensteanbieter — aufwändige Identitätsmanagement in isolierten Datensilos und gleichzeitig die Abhängigkeit von bisherigen Single-Sign-On-Identitätsmanagern wie Google und Twitter, die die Anmeldevorgänge der Nutzer kontrollieren, auswerten und verwerten. Mit SSI haben Subjekte wie Personen oder Maschinen eine einzige Identitätsverwaltung (zumindest für einen gewissen Namensraum). Die einer Identität zugeordneten Berechtigungsnachweise müssen nur einmal eingeholt werden und können dann nach Bedarf passgenau für beliebig viele Anmelde- und Nachweisprozesse eingesetzt werden. Analog zur Plastikkärtchen-Identität können die Nachweise geprüft werden, ohne den Herausgeber zu involvieren.

Für Diensteanbieter vereinfachen sich Onboardingprozesse, das sogenannte Know-your-Customer und die Compliance zu den damit verbundenen Regularien. Zudem sind die bereitgestellten Identitätsinformationen aktuell. Durch die Vereinfachung der Onboardingprozesse werden diese auch mit einer höheren Wahrscheinlichkeit als bislang von den Nutzern abgeschlossen.

Nutzer von Diensten (beliebige Subjekte, auch Maschinen) können ihre Identität in Plug-and-Play-Manier einsetzen und ihre Identität einfach zwischen verschiedenen Diensten portieren. Das Verwalten und Aktuellhalten von dutzenden Nutzerkonten und das Merken der zugehörigen Passwörter entfällt, da nur noch ein Zugang zum persönlichen Wallet erforderlich ist. Konsequenterweise reduzieren sich auch IT-Sicherheitsprobleme, die durch die Verwendung des gleichen Passworts für verschiedene Nutzerkonten entstehen.

Welche Herausforderungen gibt es?

Einschränkend kann vermerkt werden, dass die Vorzüge und Nutzbarkeit einer selbstsouveränen Identität sich zunächst auf diejenigen Dienstanbieter beziehen, die auf einer Verifiable Data Registry präsent sind — ähnlich den mobilen Anwendungen auf den App Stores. In Analogie dazu ist ein SSI-Netzwerk umso wertvoller für seine Nutzer, je mehr Dienste darüber nutzbar sind. Schließlich ist hier ein Wachstums- und Konsolidierungsprozess zu erwarten, der über die Durchsetzung von bestimmten Angeboten entscheidet.

Ein kritischer Gedanke ist, dass mit dem Verifiable Data Registry eine ähnliche Abhängigkeit der Identitäts-Subjekte wie bislang zu den Single-Sign-On-Anbietern aus den Sozialen Medien entstünde. Dieser Gedanke führt jedoch in die Irre, denn das Verifiable Data Registry kann mit Hilfe von Distributed Ledger-Technologie in einem Blockchain-basierten Netzwerk betrieben werden. Mehrere dezentrale Rechenknoten, die von unterschiedlichen Akteuren betrieben werden können, würden das Netzwerk bilden. Oftmals sind hier auch Non-Profit-Organisationen wie Sovrin oder die Energy Web Foundation aktiv. Ein deutlicher Kontrast zu den bisherigen Single-Sign-On-Anbietern.

Ein ausschlaggebender Faktor für den Erfolg von SSI wird sein, dass die Issuer von Credentials das erforderliche Vertrauen herstellen können. Weiterhin wird, wie so oft, die Nutzererfahrung und die daraus resultierende Akzeptanz entscheidend sein. Lösungsanbieter wie Spherity nutzen eine Cloud, um ein nutzerfreundliches Identitätswallet bieten zu können, das auch Private Keys verwahrt und vor dem ungewollten Verlust der Identität schützt.

Wer treibt SSI voran?

Die Bundesregierung und die Europäische Union sind unter anderem Treiber von SSI. Sie fördern das Entwickeln der Konzepte, Technologie, Standards und Regulierungen bis hin zu Pilotierungen in verschiedensten Branchen und Bereichen.

Das Bundesministerium für Wirtschaft und Energie fördert z.B. das „Schaufenster sichere digitale Identitäten“. Eines der Schaufensterprojekte ist IDunion. IDunion arbeitet an der Schaffung eines offenen Ökosystems für dezentrale, selbstsouveräne Identitäten. Dazu wird mit Blockchain-Technologie ein verteiltes Identitätsnetzwerk aufgebaut. Das Projekt wird von 47 namhaften Forschungs- und Industriepartnern vorangetrieben. 51nodes ist Partner von IDunion.

Die Europäische Union finanziert z.B. das „European Self-Sovereign Identity Framework Lab“. 20 Teilprojekte kümmern sich um die Schaffung und Erweiterung eines Open Source-Softwareframeworks für SSI. Weitere 42 Teilprojekte entwickeln kommerzielle Komponenten and Dienste unter Nutzung des SSI-Frameworks.

Anwendungsbeispiel: Wie kann SSI der Energiewirtschaft helfen?

Vor dem Hintergrund der Energiewende stehen die Betreiber von Übertragungs- und Verteilnetzen für elektrischen Strom vor großen Herausforderungen und neuen gesetzlichen Pflichten. Über eine Million (privater) dezentraler Stromerzeuger müssen effizient und effektiv in bestehende energiewirtschaftliche Prozesse (z.B. der Regelleistung und des Engpassmanagements im Redispatch 2.0) integriert werden.

Aktuell sind energiewirtschaftliche Prozesse gekennzeichnet durch produktspezifische und teilweise aufwendige Registrierungsprozesse an verschiedenen Stellen. Beispielsweise kann man hier das Marktstammdatenregister der Bundesnetzagentur, die Regelleistungsplattform der Übertragungsnetzbetreiber und die Plattform connect+ zur Kooperation und zum Datenaustausch der Netzbetreiber nennen. Hinzu kommt eine geringe Motivation der Betreiber von kleinen Energieerzeugungsanlagen und der Sicherung der Aktualität der Daten an den unterschiedlichen Registrierungsorten.

Die Frage von erforderlichen Nachweisen betrifft z.B. die Art der Stromerzeugung (z.B. EEG-Anlage) bzw. für welches energiewirtschaftliche Produkt die Anlage eingesetzt werden darf und kann. Die Identifikatoren (Energy Identification Code) für Anlagen werden zentral durch den BDEW vergeben. Weiterhin gibt es Vorgaben zur Marktkommunikation durch den Regulator, die mit Hilfe einer Zertifikatinfrastruktur für Signierung und Verschlüsselung erfolgt. D.h. die Zertifikate müssen bei allen genannten Stellen eingepflegt und aktuell gehalten werden. In einem “Internet of Energy”, wenn Millionen von Endkunden mit ihren dezentralen Flexibilitätsanlagen (z.B. Elektromobilität, Heimspeicher, Wärmepumpen, etc.) auf den Energiemarkt drängen, kommen diese Systeme zur Abwicklung der Massenprozesse für Registrierungs- und Veränderungsanforderungen in der notwendigen Geschwindigkeit an ihre Grenzen.

Der Ansatz von SSI verspricht demgegenüber die Vision der Integration von Millionen von dezentralen Anlagen in die Prozesse und IT-Systeme der Netzwirtschaft in unaufwändiger Plug-and-Play-Manier, ohne dass Mehrfachanmeldungen und parallele Konsistenzhaltung in redundanten isolierten Identitätsmanagementsystemen weiter erforderlich wären. Damit sollten Hürden für Anlagenbetreiber sinken und die Prozesse der Netzwirtschaft können schlanker und reibungsärmer aufgestellt werden. Schließlich würde ein wichtiger Beitrag für das Gelingen der Energiewende geleistet.

Aktuell arbeitet 51nodes gemeinsam mit dem Übertragungsnetzbetreiber TransnetBW an der Erörterung von SSI im Kontext der Energiewirtschaft. Bislang konnten schon mehr als zehn potenzielle Anwendungsfälle für SSI identifiziert werden.

Interesse an SSI?

51nodes ist ein Anbieter von Lösungen für die Crypto Economy. 51nodes unterstützt Unternehmen und andere Organisationen bei der Umsetzung ihrer SSI- und Blockchain-Projekte. Sie haben Fragestellungen zu digitalen Identitäten und Anknüpfungspunkte zur Nutzung von SSI? Kommen Sie gerne auf uns zu!

Weiterführende Informationen zu SSI

(1) Self-Sovereign Identity: The Ultimate Beginners Guide (Webseite)
(2) Self-Sovereign Identity: Grundlagen, Anwendungen und Potenziale portabler digitaler Identitäten (Whitepaper des Fraunhofer FIT)
(3) Chancen der Self-Sovereign Identities aus Sicht von Unternehmen für das Identity & Access Management (Webseite)
(4) Ein Überblick über das SSI-Ökosystem (wissenschaftlicher Artikel)
(5) Eine technische Umsetzungsplattform für SSI in der Energiewirtschaft: Energy Web Decentralized Operating System (Technische Konzeptdokumentation)

Das Potenzial von Selbstsouveränen Identitäten am Beispiel der Energiewirtschaft was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Smart contracts are codified contracts using rules and algorithms that can automatically trigger and incur electronic transactions of cryptocurrencies like Ether. The expected economic impact of smart contracts is very large as they allow for improving the efficiency of many existing business models and they also allow for completely new business models. For instance payouts of insurance contracts can be automated using smart contracts or autonomous cars could also pay tolls and parking fees automatically.

While smart contracts are already well known amendments from blockchain approaches like Ethereum, these approaches suffer from shortcomings like low throughput, and high transaction costs. For these reasons, the IOTA foundation had set out to first propose a new and more scalable transaction system based on the so called “Tangle”. Secondly, IOTA version 2.0 is currently under development, and it includes an approach to smart contracts. Smart contracts under IOTA 2.0 are promised to be built on an infrastructure that both scales well, and incurs low transaction costs.

Before the background of a promising new approach for smart contracts, I started to explore how to set up a private IOTA network for development and how to develop and deploy a smart contract on it. As the IOTA developer documentation is lagging it was quite a challenge to get a working solution. Thus, the main contributions of this article are, (1) a proper description how to set up the network and how to enable smart contracts, and (2) a fully working demonstration smart contract, which implements a simple prediction market in which multiple network participants can predict and bet on a certain outcome of an event.

The outline of this article is, I give (1) an overview of IOTA and the network’s smart contract integration, (2) a description of how to set up an environment for developing and testing IOTA smart contracts, (3) an implementation of a simple prediction market as a smart contract, (4) some notes and insights on IOTA and smart contracts, and (5) a conclusion.

The IOTA Network and Smart Contracts

IOTA is about a new kind of public and permissionless distributed ledger for exchanging value and data. The IOTA network has been designed to overcome the main bottlenecks of Blockchain-based distributed ledgers. Due to the organization of transactions (of value) in a chain, there is just one end to append new ones, which makes it slow. Thus, in contrast to the Blockchain-based approaches the distributed ledger of IOTA is organized in a different way. The Blockchain, which is central to Bitcoin or Ethereum for instance, is replaced by the Tangle. The Tangle connects transactions via edges in a directed (and acyclical) graph (see next figure). In contrast to the Blockchain, there are multiple nodes (representing transactions) on which new edges to new nodes, i.e., transactions, can be appended.

Beside the data structure, there are further key differences to classic Blockchain-based approaches. The consent mechanism requires no miners because all users help in validating transactions. Therefore, transactions can be essentially conducted with zero fees. The current IOTA network still requires a central coordinator, defining trusted transaction milestones, which other transactions need to reference to be also trusted. To achieve true decentralization, the planned update to IOTA 2.0 should overcome this limitation.

The key properties and key promises of IOTA are of being

• highly scalable by a new data structure allowing for parallel transactions
• requiring few resources and being suitable for devices and sensors
• having zero-fee transactions
• running fast transactions
• finally approving messages within seconds
• providing a distributed network, which is robust against attacks

These properties would enable a layer of trust and a very scalable and efficient messaging system for the “machine economy” with a very large amount of devices connected to the Internet. The current version of IOTA on the Mainnet is 1.5, which went live in April 2021. Version 1.5 is seen as an intermediate step in maturing the IOTA technology and proving its usability and practical value. This would form the basis for smart contracts that use the messaging capabilities of the IOTA network. Smart contracts are supposed to be introduced in version 2.0 the latest and might be even backported to 1.5. At the time of writing, smart contracts are subject to development.

The design for IOTA 2.0 foresees different layers (see figure above) that separate the general underlying messaging, transacting, and validating infrastructure from the application layer, which comprises also smart contracts, which would then use layer 1 capabilities to realize their features. A main reason for separating smart contracts into another layer is to not compromise the messaging functionality. Smart contracts run on Wasp nodes on layer 2 in connection with Goshimmer on layer 1, responsible for conducting transactions and messaging (see next figure).

Due to the separation of Wasp and Goshimmer, they need a technical capability to connect and interact with each other. Therefore, Goshimmer nodes contain a “txstream” plugin, which needs to be activated for connecting Wasp nodes. Smart contracts are developed in the Rust programming language. Rust is known to generate very performant and memory efficient executables while preventing the developer from many classes of errors by a helpful compiler and a well-designed language. To test smart contracts and run unit tests against smart contract functionality, the “solo” environment written in Go can be used.

Setup

The described setup defines a private IOTA 2.0 network you can run on your laptop. The network hosts a layer 1 Goshimmer node for messaging and a layer 2 Wasp node for running smart contracts. On this basis, the next chapter shows how to develop and deploy a smart contract to this network.

Environment

I used Ubuntu 20.04 LTS with latest updates and upgrades and

• installed git
• installed rust 1.53.0 for developing native IOTA smart contracts by
  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
• installed wasm-pack 0.10.0 for compiling smart contracts into “WebAssembly” binaries by
  curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

Installed dependencies for rocksdb required by Goshimmer as its underlying database engine:

• sudo apt-get install libgflags-dev
• sudo apt-get install libsnappy-dev
• sudo apt-get install zlib1g-dev
• sudo apt-get install libbz2-dev
• sudo apt-get install liblz4-dev
• sudo apt-get install libzstd-dev

Cloned and built Goshimmer 0.7.5 from the develop branch (in the version of 2021–08–16) according the following description:

• git clone -b develop https://github.com/iotaledger/goshimmer.git
• git checkout tags/v0.7.5
• cd goshimmer
• go build -tags rocksdb

Prepared Goshimmer for transaction handling
Save the config.json file to your Goshimmer directory to

• enable the txstream plugin, which allows Goshimmer to communicate with Wasp nodes
• disable the portcheck plugin

To check whether Goshimmer synchronizes its time and whether messaging works, open up the dashboard on your local machine by http://127.0.0.1:8081/dashboard. It needs to display

• TangleTime Synced: Yes
• Message: DBVBaNbSEsq8D1SuNd7ULLeSPTXLwQBwfW1agWKnFX23 (as an example) — that is, the message must not read 1111111111111111111111111 .. (containing only ones)

Before running Goshimmer, delete the message database of previous (erroneous) attempts ( if any) by rm -rf mainnetdb in the mainnetdb subdirectory of Goshimmer. The database will be automatically generated again with a fresh start of Goshimmer.

Also in case tangle time does not synchronize (see the dashboard on your local machine by http://127.0.0.1:8081/dashboard or the Goshimmer log: “can’t issue payload: tangle not synced”), shutdown Goshimmer, delete the message database, and restart Goshimmer.

Run Goshimmer in its directory as follows — after having created the ./assets/snapshotTest.bin file as described subsequently:

./goshimmer --autopeering.seed=base58:8q491c3YWjbPwLmF2WD95YmCgh61j2kenCKHfGfByoWi --node.enablePlugins=bootstrap,prometheus,spammer,"webapi tools endpoint",activity,snapshot,txstream --messageLayer.startSynced=true --autopeering.entryNodes= --node.disablePlugins=clock --messageLayer.snapshot.file=./assets/snapshotTest.bin --messageLayer.snapshot.genesisNode= --metrics.manaResearch=false --mana.enableResearchVectors=false --mana.snapshotResetTime=true --statement.writeStatement=true --statement.writeManaThreshold=1.0 --config=./config.json

Create a cli wallet

The cli wallet can send IOTA funds to a Wasp wallet, where it is used to deploy smart contracts. We generate an initial transaction with funds for our cli wallet.

Install the cli-wallet in a new directory

• wget https://github.com/iotaledger/goshimmer/releases/tag/v0.7.5 download cli-wallet-0.7.5_Linux_x86_64.tar.gz
• tar -xf cli-wallet-0.7.5_Linux_x86_64.tar.gz

Set reuse_addresses=true in the config.json of cli-wallet:

To create a new wallet run ./cli-wallet init , returning

IOTA 2.0 DevNet CLI-Wallet 0.2 GENERATING NEW WALLET … [DONE] ================================================================ !!! PLEASE CREATE A BACKUP OF YOUR SEED !!! !!! !!! !!! E7owJWtDBGSUAZUWQkn1kHG5zUy2PLQf6eEr3RoMCJs7 !!! !!! !!! !!! PLEASE CREATE A BACKUP OF YOUR SEED !!! ================================================================ CREATING WALLET STATE FILE (wallet.dat) … [DONE]

Note your SEED for allocating funds to this wallet.

We generate a custom genesis snapshot, with the transaction that allocates the funds.

Go to the Goshimmer installation directory and then to the following subdirectory ./tools/genesis-snapshot

Paste the seed of the previously generated cli wallet to the following command

go run main.go --token-amount 3500000 --seed E7owJWtDBGSUAZUWQkn1kHG5zUy2PLQf6eEr3RoMCJs7 --snapshot-file snapshotTest.bin

Now,

• go to your Goshimmer directory and inside of it run
• mkdir assets
• cp ./tools/genesis-snapshot/snapshotTest.bin ./assets/snapshotTest.bin to provide the generated snapshotTest.bin file to Goshimmer.

Setting up a Wasp node for smart contracts

I installed Wasp from the master branch in the state of 2021–08–03.

• git clone https://github.com/iotaledger/wasp.git
• check out a workable state of the code from the repository. I used this commit.
• go build -tags rocksdb
• go build -tags rocksdb ./tools/wasp-cli

We need to transfer funds to the Wasp wallet by creating the wallet in the first place by ./wasp-cli init

We need to get the address of the wallet by ./wasp-cli balance , returning something like

Address index 0 Address: 1Ah4cqMPdrDGx6Htapk7NZUxxcYHsP1C3oAugEYHVmACj

To send funds to this wallet, paste your address into this command and run it in the cli-wallet’s directory:

./cli-wallet send-funds -amount 40000 -dest-addr 1Ah4cqMPdrDGx6Htapk7NZUxxcYHsP1C3oAugEYHVmACj

Now, ./wasp-cli balance returns a balance of 40,000 IOTA.

Finally, configure wasp-cli to be able to connect to the local Goshimmer node and to form a committee of one local Wasp node by saving the wasp-cli.json file to the directory of your wasp-cli.

Deploying a chain

Smart contracts are deployed on a chain, which needs to be deployed first:

./wasp-cli chain deploy --committee=0 --quorum=1 --chain=predmarketchain --description="Prediction Market"

where

• committee=0 specifies to use one Wasp node only, which handles smart contracts.
• quorum=1 says one Wasp node is enough here — for development and testing

Now we have to provide funds to the chain by

./wasp-cli chain deposit IOTA:1000 --chain=predmarketchain ,

reducing the wasp wallet’s balance by 1,000 IOTA.

A Prediction Market Smart Contract

The private IOTA network is now used to develop and deploy a smart contract. I report the design and implementation of a simple prediction market in Rust, how to build and deploy it, and finally how to use it.

Design

A classic example of realizing a smart contract is given by a prediction market. A prediction market is a virtual electronic market allowing to predict outcomes of future events by placing a monetary bet on a certain outcome. Such events could be sports events, political events, future prices of stocks, or other events with uncertain future outcomes. For instance, the outcome of a political election could be subject to predictions on a prediction market. A simple binary question to be answered by prediction market participants could be “Will candidate/team A win?” — with possible outcomes being “yes” or “no”.

Our design of a prediction market for demonstration purposes is simple. We omit a book maker and a pricing mechanism. Formally, we do not pose a question with predefined possible outcomes. Instead, market participants can bet basically on any outcome of an event with an arbitrary amount of tokens until the time for predictions is over. Afterwards, the winning outcome is determined and winning bets placed on the correct outcome receive back their share on the overall amount of tokens placed in bets. Assume, in total 700 tokens were bet on “no” and 300 tokens in total were bet on “yes”, and “yes” is the actual outcome. A single bet on “yes” with 100 tokens receives (100/300)*(700+300) = 333 tokens, making a win of 233 tokens.

Realizing this design as a smart contract in the IOTA network allows to deploy one contract per question to be answered. The account deploying the contract is in control and has to specify the time until when bets can be placed on outcomes. The actual question to be answered has to be communicated in third party channels. Any network participant can then look up the contract and call a function to place a bet on an outcome by sending some IOTA from their wallet. Finally, after the time for predictions and bets has passed, the deploying account has to call a function to close the prediction market and to provide the actual outcome of the event and correct answer to the question. This triggers the evaluation of all bets with regard to the correct answer. Accounts with the correct answer receive the winning amount of IOTAs in a transaction.

Note that in a real world and more production-like scenario, one might consider using an oracle to provide the outcome of an event. Oracles can stream off-chain data (about events) into the Tangle, so smart contracts can use this data in their evaluations.

Implementation

Smart contracts for the IOTA network can be implemented in Rust and then compiled to a WebAssembly file.

Our demonstration smart contract implemented in Rust can be viewed and cloned from this repository. The smart contract first exposes three functions for (1) initializing a prediction market, (2) placing a bet on an outcome, and (3) closing the prediction market for determining winners. When the contract is loaded, the mentioned functions’ implementations are made publicly available under the first string’s name, e.g., “initmarket”.

The first function should be called by the account deploying the contract for initialization. Optionally, the function can set an end time for betting using the parameter BETENDUTC, which is a date-and-time string in ISO format, assuming time in UTC. In case the parameter is omitted, bets can be placed at any time until the closemarket function is called (see below).

The second function allows to place a bet on a certain outcome value of an event, provided as parameter BETVALUE, e.g. “yes”. The amount to bet is the amount of IOTA sent with the function call. Bets must be placed in time before the betenddatetime has passed, which was set on initialization of the market. To save incoming bets for future evaluation to determine winners, I use two structs, defined in the beginning. That is, I define a hash map, mapping a betting account’s id to a Bet struct, which defines the amount of tokens and the outcome value of the bet. These custom structs are used instead of the built-in map offered by the context object of the function because only a proper hash map allows iterating over all keys and elements stored. As custom objects are not accommodated by the state stored in the context of the function, we need to jsonify it to produce a string, which can then be stored in the state.

The third function closes the prediction market and is to be called by the contract owner. Calls by other accounts will fail. The function can only be run after the specified end time has passed for predictions, and the function to close the market can be called successfully only once. The function requires a BETVALUE parameter, specifying the winning outcome, e.g., “yes”. The function runs through the stored bets, determines winning bets and the amount of IOTA coins they receive, and sends the IOTA to the wallets of the winners.

Build

To build the smart contract, one can pull the accompanying repository on github with the following structure:

* Cargo.toml 
* src/lib.rs 
* pkg/ 
* target/

The full Rust code of the smart contract is contained in a file called lib.rs. The structure and the naming of files follows standard conventions for Rust. To define dependencies of the smart contract code, the Cargo.toml file reads as follows:

The most important dependency is wasmlib, which is the IOTA’s smart contract library, allowing to produce smart contracts as compiled WebAssembly files. Furthermore, serde is used for serializing custom state objects to json strings for storage in the smart contract’s state. chrono brings some date and time related functionality required to control the end of a prediction market.

To build the smart contract, run wasm-pack build in the directory where Cargo.toml resides. The other directories like pkg and target are created automatically in the build process. The compiled WebAssembly file is located in the pkg directory and named predictionmarket_bg.wasm.

Preparation

We proceed by deploying our simple smart contract compiled as a WebAssembly wasm file.

Note: please adapt the path to the wasm file if required.

./wasp-cli chain deploy-contract wasmtime predictionmarket "Prediction Market SC" ./prediction-market-smart-contract/pkg/predictionmarket_bg.wasm --chain=predmarketchain --upload-quorum=1 -d --address-index=0

outputing:

uploaded blob to chain -- hash: 6wVabTkRUUGrQzEj8s4yuPC8dfaHGLsoHLXsqvveSw4hPosted on-ledger transaction BChQcWEmMptqRM1z4C9ZffTQXPebg1MMiEYENuAnV7KV containing 1 request:
- Request 4RkBSF6BAHgfamyJnXFy4r1YoVU9wvZ9b6uvXUjt2VWAfF5
Waiting for tx requests to be processed...
Posted off-ledger request 2JdtBzxjP4Bc6Tj2SnZPSctoYDHxuCAvPWJwn6ufMShJ6Lw

Now, functions can be called on the contract. First, the same wasp-cli that deployed the contract needs to call the initmarket function. There are two possibilities:

a) do not specify a specific an end date for the prediction market to simplify testing and development by this call ./wasp-cli chain post-request predictionmarket initmarket --chain=predmarketchain

b) specify a specific end date and time for the prediction market. The iso format is used and UTC is assumed. In this way, all bets must be placed before this time and the market can be only closed after this time. Run ./wasp-cli chain post-request predictionmarket initmarket string BETENDUTC string "2021-09-08 23:00" --chain=predmarketchain

Before placing a bet, we check the wallet’s balance with ./wasp-cli balance

returning

Address index 0
Address: 1BAgmaSN1RYk5rbbxMK21CZo8t2zQ3EFeMwDPnMhQdQbs
Balance:
IOTA: 28653
------
Total: 28653

For the deployed prediction market, we assume two possible outcomes “yes” and “no” on which bets can be submitted. To place a bet with 10 IOTA on “no”, we run

./wasp-cli chain post-request predictionmarket bet string BETVALUE string no --chain=predmarketchain -t IOTA:10

and afterwards, by ./wasp-cli balance we see the wallet's balance reduced by 10 IOTA.

Address index 0
Address: 1BAgmaSN1RYk5rbbxMK21CZo8t2zQ3EFeMwDPnMhQdQbs
Balance:
IOTA: 28643
------
Total: 28643

We now introduce and prepare the setup for four more participants to the prediction market. Each participant requires another wasp wallet. Thus, we create four new subdirectories waspwallet2, waspwallet3, waspwallet4, waspwallet5 and copy wasp-cli and wasp-cli.json to those directories by repeating with regard to these directories

The last command initializes a new wallet.

In case the contract’s code is changed, it needs to be re-deployed using the first and main wasp-cli. You also need to re-deploy the chain on which the contract is deployed first. Due to the redeployment of the contract, the copied versions of wasp-cli.json (in waspwallet2 to waspwallet5 directories) miss the new address of the deployed predmarketchain chain. In this case, we need to get it back in there to be able to run bets on the same prediction market from another account. Either you noted the address of the chain when it was created, such as

activating chain nZBwoJi5q7KGk8D2cgm16PWrdM6aL2qTdCY27HHjZgrK.. OK.

or you perform a cat wasp-cli.json in the directory of your first wasp wallet, giving you some information like this (among other information):

{
  "chains": {
    "predmarketchain": "nZBwoJi5q7KGk8D2cgm16PWrdM6aL2qTdCY27HHjZgrK",
  },

Now you need to provide this address of the predmarketchain to the new wasp wallets’ configuration files. To do this, you can run the following command for the new wallets in the respective directories (e.g. in waspwallet2 through waspwallet5 in our example).

./wasp-cli set chains.predmarketchain nZBwoJi5q7KGk8D2cgm16PWrdM6aL2qTdCY27HHjZgrK

Please adapt the chain’s actual address nZBwoJi5q7KGk8D2cgm16PWrdM6aL2qTdCY27HHjZgrK to yours.

Finally, all four new wasp wallets need to be funded. First, find out their address by running (for each new waspwallet subdirectory)

waspwallet2/wasp-cli address

Then, provide the funds using cli-wallet (in the directory where it resides on your computer) by running

./cli-wallet send-funds -amount 40000 -dest-addr 1Ah4cqMPdrDGx6Htapk7NZUxxcYHsP1C3oAugEYHVmACj

and replacing the address 1Ah4cqMPdrDGx6Htapk7NZUxxcYHsP1C3oAugEYHVmACj with the actual address found by running ./wasp-cli balance before in the respective subdirectories (of waspwallet2 to waspwallet5).

Simulation of a Prediction Market

Now we are ready to place bets in the deployed contract’s prediction market on behalf of these four new participants. So, in the respective subdirectories we run say

• cd waspwallet2
• ./wasp-cli chain post-request predictionmarket bet string BETVALUE string yes --chain=predmarketchain -t IOTA:100
• cd ../waspwallet3
• ./wasp-cli chain post-request predictionmarket bet string BETVALUE string no --chain=predmarketchain -t IOTA:50
• cd ../waspwallet4
• ./wasp-cli chain post-request predictionmarket bet string BETVALUE string yes --chain=predmarketchain -t IOTA:200
• cd ../waspwallet5
• ./wasp-cli chain post-request predictionmarket bet string BETVALUE string yes --chain=predmarketchain -t IOTA:500

Finally the contract owner (with the first wallet) can close the prediction market by running in the directory of the first wallet

./wasp-cli chain post-request predictionmarket closemarket string BETVALUE string no --chain=predmarketchain

In this example, the actual outcome is specified to be “no”. When running this command from a different wasp wallet’s directory, we obtain a log output on the Wasp node

You are not authorised to close the prediction market - only contract creator is allowed to close the market.

When successfully closing the prediction market, the Wasp node’s log outputs

CLOSEMARKET is executed:
the winning value is: "no"
total amount of bets placed on "no" is 60 IOTA
total amount of bets placed on "yes" is 800 IOTA
total amount of bets over all values: 860 IOTA
1FbCCHv9if3xbnRg3wJ7SY1kBFSdhNFR6Ax6haw6PhYDL placed a bet on "no", which is a WIN
bet amount: 10 IOTA; won amount: 143 IOTA; of total amount placed a bet on 860; where total amount per winning value: 60
transferring won amount of IOTA to: 1FbCCHv9if3xbnRg3wJ7SY1kBFSdhNFR6Ax6haw6PhYDL
1FZtVTCi2GDuQ1oMGZqpT38akLpcMiMv6a8MVKNJYYdsr placed a bet on "yes", which is not a win
1F81pGLKLhPb5ANFSGWQ7UPMSnPdahNZaZkgrcyaFXvpu placed a bet on "no", which is a WIN
bet amount: 50 IOTA; won amount: 716 IOTA; of total amount placed a bet on 860; where total amount per winning value: 60
transferring won amount of IOTA to: 1F81pGLKLhPb5ANFSGWQ7UPMSnPdahNZaZkgrcyaFXvpu
17jdFbAhWwF79fBEia6A8AYMTmMYncipaFhDTjsqUbMfp placed a bet on "yes", which is not a win
1BAgmaSN1RYk5rbbxMK21CZo8t2zQ3EFeMwDPnMhQdQbs placed a bet on "yes", which is not a win
consensus/action.go:338 postTransaction: POSTED TRANSACTION: 4Aw6PzQGkk6MFzPVAXZkz8RGiPocYxuxhA9o7qgeDN7h, number of inputs: 2, outputs: 3
EVENT: state was synced to block index #11, approving output: [0]4Aw6PzQGkk6MFzPVAXZkz8RGiPocYxuxhA9o7qgeDN7h
STATE TRANSITION TO #11. requests: 1, chain output: [0]4Aw6PzQGkk6MFzPVAXZkz8RGiPocYxuxhA9o7qgeDN7h

Revisiting the bets, we had placed the following ones:

Bets number 1 and 3 were on “no” and the bet was on the actual outcome “no”. So the total amount of IOTA was 860, the bets on “no” were only 60 IOTA in total. The share of bet number 1 is 10/60 and share of bet number 2 is 50/60. So, wasp wallet 1 receives 1/6 of 860 IOTA, i.e, 143 IOTA. And wasp wallet 3 receives 5/6 of 860 IOTA, i.e, 716 IOTA. Note that when transferring funds, a minimum transaction of fee of 1 IOTA is deducted from the amount to be transferred.

Note that running the closemarket function of the smart contract a second time leads to an error message in the Wasp node’s log:

the prediction market was already closed

Limitations

There are some limitations of the presented prediction market

• Only one contract per chain can be deployed because the bets are not stored per contract identification in the chain’s state
• All bets are stored on-chain, so they are public
• Each account (given by a wasp wallet) can place only one bet per deployed prediction market contract
• The actual question asked by the prediction market and the possible outcomes have to be conveyed informally
• Bets are against other market participants — there is no market maker

Insights

Along the way of producing this article and experimenting with IOTA smart contracts, many small insights were gained. With the hope that these insights will be helpful, I share them in the following.

Transaction Time

Running transactions in the described setup on a laptop can take several seconds when waiting for the wasp-cli commands to finish. However, they can be run asynchronously when using the wasp-cli, i.e., commands can return immediately after issuing a command. Of course, transactions can run in parallel. When running scripted wasp-cli requests, one has to take care not to create too much load on the Wasp node because otherwise requests will fail with “time out”-errors. To evaluate transaction times in a production setup on the Devnet or Mainnet, further investigations are required.

Transaction Fees

In principle fees for transactions and deployments are configurable. However, there are some minimum fees that apply such as for deploying a chain (100 IOTA), deploying a smart contract, and posting state changing requests to a smart contract (1 IOTA). With default minimum fees in mind, transaction costs in the IOTA network should be very low compared to transaction fees in the Ethereum network. Note that chain owners could also increase transaction fees.

Production Readiness

On a first shot, nothing really worked when trying to set up a private IOTA network based on public documentation. That is, one has to find workable versions of the software, find a proper configuration and parameters, and find working ways for funding wallets and deploying contracts. The IOTA 2.0 software is under development and special notice has to be taken on which versions to run and which version of the different pieces (Wasp, GoShimmer, cli) to combine. The documentation in IOTA’s repositories is lagging behind the development process. However, the IOTA community is very helpful and can be reached on Discord. So, getting the network up and running depended digging into material on the github repositories, third party information (e.g., on youtube), own experiments, and asking questions on IOTA’s discord server. Once the IOTA private network and the smart contract are up and running, you are ready for testing and experiments. Further steps taken by IOTA to bring the smart contract implementation and IOTA 2.0 to production readiness can be looked up in their roadmap.

Developing Smart Contracts

Using Rust as a language for developing smart contracts might help producing error-free and stable smart contracts because Rust is well-known for its strict and sophisticated compiler which together with the strong typing and the language design eliminates whole classes of errors common in other languages. Also, the compiler provides helpful error messages. Therefore, using Rust can provide some advantages over the Solidity language, which is usually used for developing Ethereum smart contracts. However, Rust might be unfamiliar for smart contract developers being used to Solidity.

Conclusion

This article discussed smart contracts as a new and upcoming feature in IOTA 2.0. Because IOTA has a different approach to conduct and store transactions in a more parallel way with fewer resources than Ethereum, a higher scalability and lower fees are promises that come with this approach. Our experimental setup of a private IOTA network and simple prediction market IOTA smart contract shows that it is already possible to use IOTA smart contracts for development and testing. As the process of setting up the IOTA network and running smart contracts was a bit challenging, I believe this article and example code can provide useful input and support for other developers and the community interested in IOTA smart contracts.

51nodes GmbH based in Stuttgart is a provider of crypto economy solutions. 51nodes supports companies and other organizations in realizing their Blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (DApps), integration of blockchain with industry applications, and tokenization of assets.

Thanks to Majd Turfa and Jan-Paul Buchwald for their help in the course of developing this article, experiments, and the setup.

Exploring IOTA 2.0 Smart Contracts in a Private Network: Developing a Prediction Market was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Note: This article provides a general and slightly technical introduction to Parachains and indicates how to get started building a Substrate based blockchain which can be launched as Parachain in the Kusama & Polkadot ecosystem. It will explain some parts of Polkadot but it won’t get too much into specific details like staking, slashing or other important mechanics in the ecosystem. Check out the official wiki to learn more about Polkadot.

Status quo

First of all it is important to mention that the Parachain functionality is not live yet. This functionality is the last missing piece of the Polkadot whitepaper and there is a public rollout plan available that provides an overview about the progress. However, since August 2020 the Parachain functionality is being tested in the official Parachain Testnet Rococo which in V0 only ran with prototype code. In December 2020 Rococo V1 was launched and recently Plasm Network announced to be the first project that joined the Rococo V1 Testnet which surely is a big milestone as the codebase of Rococo V1 will also be used in Kusama and Polkadot. Meanwhile other possible future Parachains like KILT followed and announced their own Roadmap.

Kusama vs. Polkadot

Although Kusama can be seen as the pre-production environment for Polkadot you should be aware that Kusama might and probably will evolve independently from Polkadot. If your project is in an early stage or you want to experiment with Parachains in a real world environment before moving it into production you will probably prefer Kusama over Polkadot first. You might also be interested in testing specific features in Kusama which aren’t yet available on Polkadot.

Relay Chain & Parachains

The Relay Chain is the central chain of Polkadot. All validators of Polkadot are staked on the Relay Chain in DOT and validate for the Relay Chain. It has deliberately minimal functionality and its main responsibility is to coordinate the system as a whole, including Parachains. Other specific work is delegated to the Parachains that serve different purposes. All Parachains that are connected to the Relay Chain share the same security. Polkadot also has a shared state between the Relay Chain and all connected Parachains to ensure the validity of the entire system. Smart Contracts are not supported on the Relay Chain but can be supported by Parachains.

It’s expected that each Parachain will be as light and application specific as possible and serve a specific use case.

In The Path of a Parachain Block it is described in detail how a new Parachain Block is produced.

You may notice the term Parathread in the picture above. The current (optimistic) assumption is that the Relay Chain supports up to 100 Parachains and some of these Parachain slots will be permanently reserved to serve the network on system level (e.g. Governance). Thus there probably won’t be enough slots available for all projects which is why they need to be obtained in a so-called auction (more on that below). Whereas Parachains get a guaranteed high throughput by bonding DOT tokens, Parathreads follow a pay as you go model where a fee in DOT is being paid to validators for creating blocks. In some cases this even makes sense if there aren’t many or high frequent state updates expected (e.g. DNS, Oracles).

Parachains can become Parathreads and vice versa as they share the same technical base.

Parachain Slot Auction & Parachain Crowdloans

In order to become a Parachain in Kusama or Polkadot a slot must be obtained in a Parachain Slot Auction. Each slot duration is capped to 2 years and divided into 6-month lease periods. Parachains may lease more than one slot over time in order to extend their lease to Polkadot past the 2 year slot duration. Depending on the auction it might be possible that a slot is owned by 4 different Parachains during the 2 year slot duration.

Parachains don’t need to always inhabit the same slot. As long as a Parachain inhabits any slot it can continue as Parachain.

Parachain candidates can place bids in unpermissioned candle auctions that rely on verifiable random functions (VRFs) where the original mechanism has been slightly modified to be secure on a blockchain. The auction can be divided into two phases:

• Opening Phase
• Closing Phase

No matter in what phase the auction is, the bids are public. From the first block until the last block during the Closing Phase a winner will be determined randomly and nobody knows which block determines the winner until the Closing Phase is finished.

Projects will also be able to launch a Crowdloan Campaign with specific parameters which allows them to loan DOTs from the community to obtain a Parachain slot without having to bond all the required DOTs on their own.

In the video above core developer Shawn Tabrizi explains both variants very well.

Substrate

After giving an overview about the Polkadot ecosystem it’s time to introduce Substrate. It is a modular framework that enables you to create purpose-built blockchains by composing custom and/or pre-built components.

Substrate is the foundation of all blockchains in the Polkadot ecosystem and requires knowledge in Rust.

In the Substrate Developer Hub (link below) you can learn everything that is required to get started developing your own Substrate-based chain.

Official Substrate Documentation for Blockchain Developers · Substrate Developer Hub

A few important things to know about Substrate:

• The runtime of Substrate is referred to as the “state transition function” which contains the business logic that defines the behavior of the blockchain. You as a developer can define storage items that represent the state of the blockchain and functions that allow users to make changes to the state.
• Substrate ships with FRAME (Framework for Runtime Aggregation of Modularized Entities) which is a set of modules (called Pallets) and support libraries that simplify runtime development. Pallets are individual modules within FRAME that host domain-specific logic.
• To enable forkless runtime upgrade capabilities, Substrate uses runtimes that are built as WebAssembly (WASM) bytecode.

To learn more about how to perform a forkless runtime upgrade you can take a look at the official tutorial.

Pallets vs. Smart Contracts

For people heading over from other chains and ecosystems like Ethereum it might be obvious that application specific logic is always written in Smart Contracts which in turn are deployed on the blockchain. But as you already learned the idea of the Polkadot ecosystem is to have different Parachains that are as light and as application specific as possible.

There are basically 3 possibilities to add custom logic to a Substrated-based blockchain:

1. Write custom Pallets and include it in the runtime
   - https://substrate.dev/docs/en/tutorials/create-a-pallet/
   - https://substrate.dev/recipes/pallets-intro.html
2. Include the Contracts Pallet in the runtime and write contracts in ink! (Rust based eDSL)
   - https://paritytech.github.io/ink-docs
   - https://substrate.dev/substrate-contracts-workshop
3. Include Frontier (Pallets that serve as Ethereum compatibility layer) to the runtime and write contracts in Solidity
   - https://github.com/paritytech/frontier
   - https://substrate.dev/frontier-workshop

Custom Pallets are the best choice if you are building a greenfield project that serves a specific purpose and needs to run its own network.

Substrate already provides a lot of Pallets which you can combine together as needed, including your custom Pallet(s) in order to build a lightweight and unique runtime for your application specific blockchain:

• Default Pallets of Substrate (Note: The link points to the v3.0.0 version of Substrate which was released a few days ago. Most tutorials are still based on v2.0.0)
• NFT Pallet (There is also a WIP reference implementation that showcases CryptoKitties on Substrate)

I also discovered a Substrate Marketplace where you are able to find different Pallets (at least the default ones) which you can include into your runtime.

If you aim to write contracts in ink! or port a Solidity based application from Ethereum over to a Substrate-based Parachain you don’t necessarily need to launch your own Parachain as there already exist projects that plan to launch a Parachain and include the required Pallets in their runtime:

• Edgeware (ink! based contracts)
• Moonbeam (Solidity contracts)

Note: To check out how Moonbeam works you can clone the following repository: https://github.com/51nodes/moonbeam-playground

Polkadot compatibility with Cumulus

When you have written (and hopefully tested ;-)) your custom Pallets you need to make sure that you build a Polkadot compatible runtime that exposes an interface for validating its state transition and provides interfaces to send and receive messages of other Parachains.

Cumulus is an extension to Substrate that makes it easy to transform any Substrate-built runtime into a Polkadot-compatible Parachain. It is still in development, but the idea is that it should be simple to take a Substrate chain and add the Parachain code by importing the crates and adding a single line of code. You can also dive deeper into that topic and learn more about the mechanics of Polkadot by reading the Parachain Implementers Guide.

If you have built your own Parachain and want to join the official Rococo Testnet you have to follow this guide. The auction mechanism mentioned above is not active yet and Rococo is currently controlled by the development leads at Parity Technologies. For joining the network you have to run at least 1 Collator for your Parachain and 1 Validator node for Rococo.

Note: Learn how to setup a local Polkadot network based on Rococo by following the Readme of our repository: https://github.com/51nodes/polkadot-local-rococo

Final Thoughts

Substrate has a clean design and makes it very easy to build an application specific blockchain for everyone familiar with Rust. If you decide to run your own Parachain (or Parathread) you should also think about how to incentivise other players to run a Collator node in your network — for example by introducing your own network token.

Although Parachains are not live on Kusama and Polkadot yet we can already see many projects that aim to become a Parachain as soon as possible. It will be interesting to watch the battle of those projects to secure their exclusive Parachain slot on Kusama and Polkadot. Personally I am very interested to see how the auction mechanism works and which projects will secure their Parachain slot via a Crowdloan Campaign.

I am also looking forward to play around with the interoperability features that Polkadot provides when the implementation is mature enough.

51nodes GmbH based in Stuttgart is a provider of Crypto Economy solutions.

51nodes is member of the Substrate Delivery Partners program and supports companies and other organizations in realizing their blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (dApps), integration of blockchain with industry applications, and tokenization of assets.

Kusama & Polkadot: Build an application specific Blockchain and launch a Parachain! was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

By implementing SSI solutions we will be able to:

• dramatically reduce (governmental) bureaucracy by providing efficient and frictionless processes
• exchange and verify (personal) data in a safe way and thus avoid the growing amount of data breaches
• eliminate fraud
• get rid of centralized logins using different authentication providers like Google, Facebook, Apple, Twitter and many more (why is that important?)

In the future, any type of certificate, attestation or authorization can be issued and verified digitally in a simple yet secure manner.

Imagine that every card we are holding in our personal wallet right now can be issued digitally and instead of having it physically in our pocket we will have it stored on our smartphone. If a service or a business needs to verify our age or any other personal or financial data we will be able to prove this within seconds by just pushing a button on our smartphone. The beauty here is that we will be able to do this (almost) completely independent from specific vendors and technologies. This is all possible thanks to Verifiable Credentials (VCs).

Sounds too good to be true? We encourage you to make yourself familiar with this topic and get in touch with us if you are interested! Here some good reading material to start with:

• Where to begin? (Decentralized Identity Foundation Blog)
• SSI101: An Introductory Course (Spherity Blog)
• Ultimate Beginners Guide (Tykn.tech)

Characteristics

For an identity to be self-sovereign, at least following statements must be fulfilled:

• Each individual owns their own data and does NOT rely on a central party to prove claims about themself.
• Each individual has full control on what personal information they are sharing and with whom.
• The SSI must be interoperable. Each individual can use their identity across multiple platforms and locations.

A Self-Sovereign Identity is portable, private and secure.

Trust over IP and the role of blockchain

On 5 May 2020, the Linux Foundation announced a new addition to its roster of global open source ecosystem projects: the Trust over IP Foundation. The mission of this new Foundation is to simplify and standardize how trust is established online so that everyone can feel safe, secure, and private in all of our digital interactions — whether between individuals, businesses, governments, or any “thing” on the Internet of Things.

Layer 1 can be implemented on any distributed system and doesn’t necessarily have to be a blockchain or a distributed ledger. But of course the high availability and cryptographic verifiability of blockchains enable strong roots of trust and won’t serve as single point of failure.

The most interesting work is currently happening on Layer 2 and Layer 3. If you want to dive deeper into that topic you should get in touch with the different working groups of the Digital Identity Foundation and take a closer look at Hyperledger Aries. It is for example possible to prove facts that are written in a VC without having to reveal all the information the VC contains. This is called selective disclosure and can be achieved by making use of Zero-Knowledge Proofs (ZKP).

Solution approaches and initiatives

In Germany alone we have many players working on exploring and implementing SSI solutions. The German ministry for economic affairs and energy (BMWi) is currently running an innovation competition called Secure Digital Identities where many different smaller and bigger companies collaborate in consortiums to transform the way we will manage our digital identity in the future.

For the project Mobility Blockchain Platform (MBP) of the Daimler Mobility AG Blockchain Factory, which we have accompanied since the beginning, SSI plays also a very important role. We worked together with companies like evan.network, Blockchain Helix, Spherity and Ontology to provide users access to services of the MBP with an SSI based solution.

Status quo & outlook

From the perspective of a company that builds and makes use of SSI based solutions we can certainly say that SSI is here to stay and that we can already build upon a set of technologies and frameworks to make our vision of a Crypto Economy reality.

Of course there are some technical and practical challenges like communication, interoperability and key management (including key recovery) which is being heavily worked on. But we are confident that it is just a matter of time until we can hide this complexity and make it as easy as possible for businesses and individuals to make use of SSI solutions.

The biggest challenge right now is to explain the value that SSI represents to businesses and people.

In the book “Self-Sovereign Identity Decentralized Digital Identity and Verifiable Credentials”, which isn’t finished yet and will approximately be published in summer 2021, the authors have written a whole chapter to guide people how to explain SSI to businesses!

51nodes GmbH based in Stuttgart is a provider of Crypto Economy solutions.

51nodes supports companies and other organizations in realizing their blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (dApps), integration of blockchain with industry applications, and tokenization of assets.

Self-Sovereign Identity (SSI) as building block of the Crypto Economy was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Given the growing number and adoption of distributed ledger solutions and the assumption that the future digital infrastructure will be based on a variety of networks rather than one single blockchain, the importance of interoperability between different distributed ledger networks becomes evident. However, inter-ledger interoperability is still in its infancy and focuses on the transfer of tokens or more generalized assets. It quickly hits limits when it gets to the exchange of identities and data related to individual business processes or logic. For example, using the account and identity of a consumer or a business verified in blockchain network A to interact and sign business transactions in blockchain network B, is not possible using today’s inter-ledger approaches.

In the following, a pragmatic approach how to handle interoperability of identities and claims based on W3C Decentralized Identifiers (DIDs) and the related Verifiable Credential concept is introduced. By adding a thin abstraction layer on the application level, business applications using distributed ledger networks that support DIDs can easily be enhanced for supporting multiple DID methods related to different networks.

Solution Description

Note: the following description requires basic knowledge of the concepts of Decentralized Identifiers (DID) and Verifiable Credentials (VC) / Verifiable Presentations (VP). For a brief introduction see the W3C Primer. In the context of the suggested approach, we only consider public DIDs, contrary to private (“pairwise”) DIDs which are only used and shared as part of direct communication between two parties.

DID Resolution

Public DIDs can be resolved to a DID document that gives information about the validity, public keys or other means of authentication, and additional attributes of the respective identity. Each DID method must specify how DIDs can be resolved in a deterministic, immutable manner. Usually, providing a reference implementation of a DID resolver and adding it to the universal resolver initiative for universal resolving is part of the DID method specification. The DID resolution step gives a standardized way to verify and authenticate an identity, and systems or applications that identify users based on DIDs can support any DID method by adding a respective resolver of this DID method into their code or dispatch to the universal resolver.

Verifiable Credentials and Presentations

For the exchange of more specific claims, proofs and signatures related to a DID, Verifiable Credentials are issued and signed by a trusted party (also identified by the issuer DID) to a particular subject DID. The subject DID can then present one or more VCs to a verifier using a Verifiable Presentation, which basically adds the subject’s signature to the presented claims as a means of authentication.

Although VCs and VPs are supposed to be a standard, there are different implementation options and technical representations, which causes some challenges to interoperability in real-world applications. The most commonly used representations are the JSON(-LD) format described and used in examples in the W3C documents, but in practice the JWT (JSON Web Token) mapping is also widely used. By supporting those two alternatives, most current DID and related VC/VP implementations can be handled.

Multi DID Method Handler Architecture

Based on the previously described elements, we can define an architecture pattern adding a thin layer for the orchestration of DID, VC and VP related operations supporting different DID methods and related configuration properties.

The following diagram shows the suggested architecture in more detail.

The business logic of an application interacts with the DID orchestration layer through an interface that is purely based on general DID and VC/VP elements. The interface is implemented by a single orchestrator that handles the orchestration and dispatching of operations based on the DID method of the involved DIDs. The operations related to a particular DID method are dispatched to a specific handler that is registered for this DID method. Handlers all implement the same operations and can be realized or looked up using a plugin mechanism to allow extensibility for new DID methods. Only the handler for a particular DID method connects to the underlying blockchain or DLT network of this DID method if necessary. The configuration of the component can contain the information about supported DID methods or handlers, the own DID and account information for write operations, as well as node endpoints or other blockchain related properties.

Operations provided by the interface could include:

• Resolve DID document for DID
• Validate signature for DID and message
• Validate VC issued by DID
• Validate VP signed by DID
• Update DID document for own DID
• Sign message with own DID
• Issue new VC with own DID as issuer
• Issue new VP for list of VCs with own DID as signer

The interface could be distinguished in a read-only part allowing resolution and validation only, and a write part requiring to have direct or indirect (via custodian mechanisms) access to an account private key in order to generate signatures or authenticate. Applications that only perform validation could be restricted to the read-only part with a simplified configuration.

The interface must be defined in a way to be able to treat VCs and VPs both as JSON or strings (JWT), which may cause some challenges for the implementation, but should be possible in most object oriented and type-safe programming languages.

Example

To make it more comprehensible how an implementation of this approach would look like in practice, let’s have a look at the typical VC and VP example scenario of a university degree credential issued by the university to a student and verified by an employer e.g. as part of a recruitment process.

It is assumed that the student uses application 1 (which could be a general self-sovereign identity app) and has registered DID a with DID method A, which is related to blockchain network A and uses JWT as the format for VCs and VPs. The university uses application 2 that implements the described DID, VC & VP abstraction component. The university has registered DID b with DID method B related to DLT network B using JSON as the format for VCs and VPs, which is configured as the own account. The employer uses application 3 also implementing the DID, VC & VP abstraction, however only in a read-only manner without the need for an own DID.

The flow could then look like the following:

1. The student (application 1) requests a Verifiable Credential from the university (application 2) by providing a proof of the physical identity and a signature by DID a.
   Note: for the sake of comprehensibility, in this example the university (application 2) checks the physical identity of the student itself. In more realistic scenarios, the student could have already run through an identification by a third party identity provider that mapped the student’s DID to relevant properties of the physical identity, and only present a reference to the id provider or respective Verifiable Credentials issued by the id provider.
2. Using the abstraction component, the business logic in application 2 can validate DID a and its signature, dispatching the validation to the handler for DID method A.
3. The business logic of application 2 checks the physical identity of the student, see note in step 1.
4. After successful verification, application 2 issues a Verifiable Credential using its own DID b as the issuer and DID a as the subject via the handler for DID method B.
5. Application 2 returns the VC issued in step 4 to the student (application 1).
6. The student (application 1) prepares a Verifiable Presentation containing the VC from the previous step, signed by DID a and using the JWT format according to DID method A. It sends the VP to the employer (application 3).
7. Using the abstraction component, the business logic in application 3 can validate both DIDs and signatures of DID a (7a) and DID b (7b) with the respective handlers. If all is valid in terms of that the DIDs exist, the signatures are correct and there is no expiration or revocation of the credentials, the abstraction component returns the validated VC information.
8. The business logic of application 3 can now further validate the VC content from a functional perspective. The validation should include a check whether the issuer DID of the VC is trusted.

Evaluation

With the suggested approach, an application can easily be extended to support multiple DID methods based on a common, reusable pattern. As an additional benefit, business logic is kept independent of particular DID methods or DID, VC and VP processing.

Assuming that the DID, VC and VP concept establishes as the standard for decentralized, self-sovereign identity, it is likely that DIDs from different systems or networks will interact. Since the main usage scenarios for DIDs and VCs are user authentication (“single sign-on”) and verification (KYC — Know Your Customer and digital signatures), providers implementing those kinds of authentication and verification methods will want to support consumers from as many ecosystems and related DID methods as possible. The suggested approach is a suitable, rather light-weight solution for achieving such an interoperability on a DID level which should be flexible enough to be integrated into any existing application or environment. This differs from other solution approaches to DID interoperability like DIDComm or Hyperledger Aries, which require to adjust the software architecture and workflows towards an agent-based model and may not be suited for every case or come with high costs.

A drawback of the approach is a certain initial effort for implementing handlers to support new DID methods. This could be eased by moving the abstraction to a reusable component like a library or SDK, or even a separate micro-service, that can be shared by many applications. The general complexity due to dependencies to multiple networks and related configuration effort cannot be avoided, but should be mitigated in the implementation by sound error and exception handling. Also, the approach suffers from the current gaps and room for interpretations in the DID, VC and VP specification. In practice, this leads to DID methods that do not fully comply to the specification or implement different, incompatible extensions or variants. It is expected, though, that with increasing adoption the specification will become more robust and implementations will align in the future.

Summary and Outlook

This article described an architectural approach to realize interoperability between DIDs of different networks that can be well integrated into most existing application architectures.

A similar approach has been successfully prototyped and shown as a demo in a mobility use case allowing interoperability between the Evan network (Ethereum based business blockchain network) and the Ontology blockchain as part of the 8th Startup Autobahn program. You can watch the recording of the breakout session during the virtual expo to get more details. Gratitude and acknowledgements go to the project partners that gave valuable input and made the solution possible, in particular Dr. Harry Behrens of the Daimler Mobility Blockchain Factory and the team of Ontology.

Future work on this topic may include the adaptation in real business applications to understand the needs and adjust the approach based on learnings from more usage scenarios, providing open source reference implementations or libraries in different programming languages, as well as the implementation and test of handlers for the most widely used DID methods.

51nodes GmbH based in Stuttgart is a provider of crypto economy solutions.

51nodes supports companies and other organizations in realizing their Blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (DApps), integration of blockchain with industry applications, and tokenization of assets.

Decentralized Identifiers based Interoperability Architecture was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

Authors: Julian Voelkel, Majd Turfa, Jan-Paul Buchwald

TL;DR

In this article, we introduce a new DID method that is used to identify and resolve data schemas. By supporting different underlying decentralized storage mechanisms (currently public IPFS and Evan IPFS) and an optional declaration and validation of the schema type, it provides flexibility and allows future extensions. We believe the schema DID method can build the foundation for a decentralized schema registry with globally unique and immutable schema identifiers to be used across centralized and decentralized systems.

Introduction and motivation

Whenever pieces of data with a certain semantic meaning are exchanged between different parties, it is essential to agree on a format and structure of this data. This is usually achieved via schemas or schema definitions, and within the history of software development, many representations of such schemas evolved. Most common schemas in use are related to XML (XSD — XML Schema Definition), JSON (JSON Schema), DDL (SQL Databases), GraphQL (GraphQL Schema), but also to embedded protocols (Protobuf) or particular products and frameworks (Thrift, Avro).

While a schema perfectly describes the structure of a piece of data and can be used to validate data, a further question is how to share those schemas between different parties or components using them. The obvious solution is a Schema Registry, meaning a repository to store and identify schemas. Today schema registries such as Confluent Schema Registry or Amazon EventBridge Schema Registry are used to store and address schemas in a particular namespace. This kind of schema registry is based on internal identification and central storage, which makes it dependent on a single party and narrows the usage scope immensely. So, we asked ourselves: how to share and reuse schema definitions across multiple systems or even globally?

The goal was to design a Decentralized Schema Registry based on a globally unique identifier allowing the use of immutable schemas which are accessible to the public or a set of relevant parties (depending on the chosen network).

Solution

Our solution for the previously stated challenge is a new DID method for identifying schemas, combined with a (preferably decentralized and immutable) storage mechanism. A schema DID should allow a globally unique resolution of the schema by providing a service endpoint to “get” or download the actual schema content. The DID format should be open to support multiple storage mechanisms or networks and provide a way to also expose the schema type.

The following gives some more explanation on the DID method and related software components we defined and implemented.

Schema Registry DID Method

The DID is comprised of multiple elements separated by colons (:):

• The general DID and method prefix did:schema
• The method-specific id consisting of:
  1. A mandatory identifier for the storage network, chosen from a list of supported storage mechanisms and networks.
  2. An optional identifier for the type or format of the schema, chosen from a list of supported types.
  3. The mandatory schema-hash identifying the schema file in the given storage network, represented as a string.

The full DID method specification can be found on GitHub. The schema DID method was also added to the official registry of DID methods in the W3C method registry.

Reference Library

To make the DID method available for other developers, we implemented a Typescript based library that supports the registration and extraction of schemas on different distributed networks by the two functions registerSchema and getSchema.

• The registerSchema method is used to store the schema on one of the supported networks and it returns the DID of the schema. The schema type is a mandatory parameter, and the given schema content is validated to comply to the given type.
• The getSchema method gets a DID as Input and returns the schema that is stored behind the given DID. If the DID contains a schema type, the library validates the content and treats it as “not found” if it does not comply to the type.

For the first version of the library, the supported schema types are JSON-Schema and XSD, the supported storage networks are the public IPFS and Evan IPFS, which is a permissioned IPFS network provided by the evan.network. The library also provides a function to initialize the library by setting the configuration needed to connect to the required networks and other utility functions to handle the DID.

Data stored on IPFS gets usually removed after some time, and only pinned files are stored permanently on the node. Since each node in IPFS networks handles this individually and pinning may incur costs, the pin functionality can be enabled or disabled in the library configuration.

The library is published on npm as @51nodes/decentralized-schema-registry, and the source code is available on GitHub. Detailed instructions on how to use the library are described in the README file. The project also contains a simple JavaScript example.

DID Resolver

Using the provided library, we also implemented a DID resolver as a NestJS application. It provides a GET endpoint to get the DID document of a schema DID. An example of a returned DID document is

The DID document includes a service endpoint that points to a gateway that provides the underlying schema content. For the public IPFS this is currently an infura public gateway, and for the evan.network IPFS there is a central gateway from evan.network, both loading a single resource identified by hash from the respective IPFS network.

The NestJS Application is dockerized and the Docker image is published under 51nodes/schema-registry-did-resolver on Docker Hub. To build and run the driver locally, read the instructions with the provided source code which are published on GitHub.

The Docker image of the resolver was also added as a driver to the universal DID resolver, which aims to provide a central resolution service for all kinds of DID methods (currently over 30).

Independent from the universal resolver the driver provides a GET request to download the saved schema directly under the endpoint http://localhost:8080/1.0/identifiers/{did}/get

Benefits and Usage Scenarios

The main benefit of the schema DID is to allow a stable identifier and public registry for schema files or objects which could replace existing proprietary (siloed) schema registries or other schema storage solutions. Applications can use DIDs for referencing schemas stored in different locations and only need a standardized resolver (or use the universal resolver) to load the schema contents, allowing not only interoperability, but also much better reuse across systems and applications.

In particular, schema DIDs could be useful in the following scenarios:

Schema references and validation for distributed data storage

Schema DIDs can reference schemas for payload data in Blockchain or smart contract transactions. Further this approach enables the use of structured data in distributed file or data storages themselves, or could be used for schemas of Verifiable Credentials.

Cross-organization processes and communication

Cross-organizational interactions become even more important in times of decentralized systems. Collaboration is the key for enhancing processes. Schemas stored in a distributed manner would enable cross-organizational standardized data interfaces accessible to everyone. This could elevate the collaboration to another level.

Outlook and next steps

Firstly, we are happy to get feedback on the solution and would love to see first adoption in real-world applications. Of course, we plan to use the DID based decentralized schema registry in our own future projects as well. We also want to have a closer look on the relations and possible usage in linked data such as JSON-LD or IPLD and in composite or nested schemas.

For the schema DID framework itself, we are looking for other networks that could be used as alternative decentralized storage networks. Some of the interesting projects in this area that we will keep an eye on are FileCoin, BigChainDB, IPDB, Sia and arweave. We are looking as well to support other schema types like GraphQL-Schema for GraphQL which is getting more attention lately. Furthermore, implementing the library with other programming languages like Java and Rust would help to make the schema DID method available to a wider number of developers. For all those topics, we welcome interested open source developers to participate and help to grow the solution.

51nodes GmbH based in Stuttgart is a provider of crypto economy solutions.

51nodes supports companies and other organizations in realizing their Blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (DApps), integration of Blockchain with industry applications, and tokenization of assets.

Decentralized Schema Registry was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.

But what type of business processes are we talking about? When diving deeper into that topic one will quickly realize that almost all business processes that are considered to be automated when talking about hyperautomation are internal processes.

Why is that? Isn’t there a lot more potential to automate processes across multiple independent companies?

We call these intercompany processes* and we think that by utilizing blockchain in combination with other techniques it will be possible to fully automate them in the future.

But what exactly is the role of blockchain in the context of hyperautomation?

*By our definition intercompany processes are processes where multiple independent companies are involved

Existing problems and the need for an intercompany transaction network (ICTN)

Business processes across companies are characterized by the fact that numerous process steps require the integration of different IT systems and the involvement of different roles. Today, these processes run via several communication media such as telephone, e-mail, Internet and paper-based correspondence. Due to a lack of system integration they are by definition costly, error-prone and usually lack consistent monitoring. Media and system discontinuities also mean that information from different systems has to be consolidated again and again at great expense.

Typical characteristics of intercompany processes:

• error-prone
• inconsistent data
• difficult to monitor
• high consolidation effort
• expensive

Today there doesn’t exist a solution to that problem which allows various companies to integrate their systems in a convenient and efficient way to get away from the unintended characteristics mentioned above.

If we think about automating intercompany processes, the following topics come to our minds and need to be tackled:

Missing standards and/or high integration costs

• Integration needs to be done for each business partner (!)
• Integration often not worth doing for small and medium-sized enterprises (SMEs) due to economical reasons

Lack of trust

• Business partners are at the same time often competitors and companies usually don’t want to share sensitive data with them

Missing data consistency

• There is no instrument to ensure data consistency in a shared business process where multiple companies are involved

Confidential and secure message system

• Communication and contract data should only be shared with involved parties

A solution that allows every business partner involved in a shared business process, e.g. in a supply chain, to get a realtime view for the current state of the process is desired. And wouldn’t it be great if it were possible to automate accounting related tasks which would allow to perform automated audits?

The solution to these questions which combines multiple techniques including blockchain is what we call an intercompany transaction network (ICTN).

We think that by leveraging blockchain technology in a common framework it will be possible to fully automate intercompany business processes without creating new monopolies and data silos.

The components required to enable an ICTN will be discussed later. First we want to recap past approaches that aimed to solve this problem by making use of blockchain technology.

Past approaches & consortium blockchain solutions

If we take a closer look at past pilot projects where a blockchain was involved many of these tried to replace and extend existing business process implementations by coding a similar logic in smart contracts.

The promise was that business processes could now take place across companies in realtime without media disruption and thus much more efficiently while ensuring data consistency. Since sensitive data is involved in the processes, these should not be publicly visible and for this reason often solutions like Quorum (modified Ethereum) or Hyperledger Fabric were chosen as the solution as these provide some important privacy features.

However, although most of these promises have been verified in pilot projects to some extent there are several reasons why it is difficult and risky to replace existing solutions with such a blockchain solution.

A blockchain network based on Quorum, Hyperleger Fabric or any similar solution always requires a consortium setup which is hard to establish and to maintain as the initial consortium and joining members all need to agree upon the governance model. This is typically a lengthy process that requires at least following questions to be answered:

• Who controls the network?
• What are the rules for new participants to join?
• How are critical errors in smart contracts dealt with?
• How will protocol updates be governed?

Besides the difficulty to establish one or more consortium chains with different partners there are other aspects to be considered:

• Is there enough incentive for other businesses to join the network? (we need network effects!)
• Is the technology mature enough to replace existing systems?
• Isn’t this just another data silo that is being built?

I guess none of the companies CIOs would be willing to take these risks and try to replace existing systems with such a solution as long as it isn’t battle tested. And here we haven’t even talked about performance, IT security policies and general compliance topics which must not be forgotten.

We need a solution that can tackle the existing problems without having to reinvent the wheel and this is where ICTN comes into place.

Idea & key components of ICTNs

The idea behind an ICTN is to combine multiple different techniques — including blockchain — in a common framework. An ICTN will allow companies to establish fully automated, secure and private intercompany business processes without media disruption while ensuring data consistency across all involved parties.

It is very important to mention that an ICTN can be integrated in any system landscape, no matter how heterogenous the ERP systems of the involved companies are. While the sensitive business data stays in the existing IT systems we make use of cryptographic methods to prove to all involved parties that certain steps in an intercompany business process have been completed correctly.

Whenever in the intercompany process an agreement between one or multiple parties takes place, zero-knowledge proofs are used to ensure that previously defined business rules are met. Thus every party involved in the overall process can be sure that the state of the process is consistent and up to date.

This is the role of the blockchain in the context of hyperautomation as we see it today.

In an ICTN the blockchain is the immutable state machine where zero-knowledge proofs are stored irreversible and can always be verified by all parties involved in an intercompany business process.

Another major advantage is that auditors could also be integrated in an ICTN which means that it would be possible to fully automate audits.

We identified following key components which are necessary to make an ICTN possible:

Secure and Private Communication

To enable an intercompany transaction network the companies involved in the processes need a communication protocol that allows them to send and receive messages in a secure and private manner. Although the protocol is used by all participants, messages required for certain steps in the process should stay confidential between involved parties and not be visible to other players.

Zero-knowledge

The zero-knowledge component is required to create and verify cryptographic proofs which contain transaction data for a certain step in a business process. The cryptographic proofs ensure the previously defined business rules are met and they are stored on the immutable state machine (blockchain). The proofs can be verified by all parties involved in the business process and this allows them to get a consistent view of the overall process while ensuring that confidential data between certain business partners stays confidential.

Zero-knowledge cryptography can also enable confidential value transfer between multiple parties. This way payments or transfers of tokenized business assets could be performed confidential on a public blockchain.

Immutable state machine / Blockchain

The role of the blockchain in an intercompany transaction network is to provide an always accessible and immutable state of records of business relevant proofs.

Company identity registry

The company identity registry is another important piece in an ICTN because there needs to be a way to verify that a business partner really is who it claims to be. This could for example be a global registry which stores DIDs of verified companies. If you aren’t familiar with DIDs we encourage you to read about the decentralized identity standard.

Another optional component we identified is a so called “Gas Station Network”. Since a public blockchain is considered to be used the companies would need to pay for the transaction fees to store the proofs on the blockchain. We expect that many companies don’t want to hold a cryptocurrency like ETH to cover the fees. This is why relayers might be necessary which take the transactions of the companies and publish them on behalf of the companies.

Does this idea sound familiar to you? Then you probably heard or read about the Baseline Protocol which has been hyped and advertised a lot in the recent weeks and months.

It was announced back in March 2020 by EY and a first Proof of Concept was developed in cooperation with Microsoft and ConsenSys.

“Blockchains will do for networks of enterprises and business ecosystems what ERP did for the single company.” (Paul Brody, EY)

If this statement by EY is correct, we cannot even begin to imagine how big this market will be.

Although there a still some open questions and challenges that need to be tackled we think that Baseline Protocol deserves to be hyped. We think so because we identified similar key components for our definition of an ICTN and it is an approach that can be integrated in the existing system landscape and hopefully will be seen very soon in a production environment.

The role of 51nodes

We are always trying to be at the forefront of the development that takes place in the crypto space and we think that the Baseline Protocol is a promising approach that can bring our vision of an intercompany transaction network into reality. Of course we will keep our eyes always open and look for other solutions.

If you are interested in learning more about this idea or you want to integrate and verify a solution like Baseline Protocol we would be pleased to assist you in your project!

51nodes GmbH based in Stuttgart is a provider of Crypto Economy solutions.

51nodes supports companies and other organizations in realizing their blockchain projects. 51nodes offers technical consulting and implementation with a focus on smart contracts, decentralized apps (dApps), integration of blockchain with industry applications, and tokenization of assets.

The role of blockchain in the context of hyperautomation was originally published in 51nodes on Medium, where people are continuing the conversation by highlighting and responding to this story.