diff --git a/foundations/whitepapers/ton.mdx b/foundations/whitepapers/ton.mdx index 734030eb7..1bec19210 100644 --- a/foundations/whitepapers/ton.mdx +++ b/foundations/whitepapers/ton.mdx @@ -26,25 +26,25 @@ This text is not intended to be the ultimate reference with respect to all imple The *Telegram Open Network (TON)* is a combination of the following components: -- A flexible multi-blockchain platform (*TON Blockchain*; cf. [Chapter 2](#2-ton-blockchain)), capable of processing millions of transactions per second, with Turing-complete smart contracts, upgradable formal blockchain specifications, multi-cryptocurrency value transfer, support for micropayment channels and off-chain payment networks. *TON Blockchain* presents some new and unique features, such as the "self-healing" vertical blockchain mechanism (cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)) and Instant Hypercube Routing (cf. [2.4.20](#2-4-20-instant-hypercube-routing%3A-“fast-path”-for-messages)), which enable it to be fast, reliable, scalable and self-consistent at the same time. +- A flexible [multi-blockchain platform](#2-ton-blockchain) (*TON Blockchain*), capable of processing millions of transactions per second, with Turing-complete smart contracts, upgradable formal blockchain specifications, multi-cryptocurrency value transfer, support for micropayment channels and off-chain payment networks. *TON Blockchain* presents some new and unique features, such as the "self-healing" [vertical blockchain](#2-1-17-correcting-invalid-shardchain-blocks) mechanism and [Instant Hypercube Routing](#2-4-20-instant-hypercube-routing:-“fast-path”-for-messages), which enable it to be fast, reliable, scalable and self-consistent at the same time. -- A peer-to-peer network (*TON P2P Network*, or just *TON Network*; cf. [Chapter 3](#3-ton-networking)), used for accessing the TON Blockchain, sending transaction candidates, and receiving updates about only those parts of the blockchain a client is interested in (e.g., those related to the client's accounts and smart contracts), but also able to support arbitrary distributed services, blockchain-related or not. +- A [peer-to-peer network](#3-ton-networking) (*TON P2P Network*, or just *TON Network*), used for accessing the TON Blockchain, sending transaction candidates, and receiving updates about only those parts of the blockchain a client is interested in (e.g., those related to the client's accounts and smart contracts), but also able to support arbitrary distributed services, blockchain-related or not. -- A distributed file storage technology (*TON Storage*; cf. [4.1.8](#4-1-8-example%3A-keeping-files-off-chain%3B-ton-storage)), accessible through *TON Network*, used by the TON Blockchain to store archive copies of blocks and status data (snapshots), but also available for storing arbitrary files for users or other services running on the platform, with torrent-like access technology. +- A [distributed file storage](#4-1-8-example%3A-keeping-files-off-chain%3B-ton-storage) technology (*TON Storage*), accessible through *TON Network*, used by the TON Blockchain to store archive copies of blocks and status data (snapshots), but also available for storing arbitrary files for users or other services running on the platform, with torrent-like access technology. -- A network proxy/anonymizer layer (*TON Proxy*; cf. [4.1.11](#4-1-11-example%3A-ton-proxy-is-a-fog-service) and [3.1.6](#3-1-6-channel-as-a-tunnel-identifier)), similar to the $I^2P$ (Invisible Internet Project), used to hide the identity and IP addresses of *TON Network* nodes if necessary (e.g., nodes committing transactions from accounts with large amounts of cryptocurrency, or high-stake blockchain validator nodes who wish to hide their exact IP address and geographical location as a measure against DDoS attacks). +- A network proxy/anonymizer layer (*TON Proxy*; [4.1.11](#4-1-11-example%3A-ton-proxy-is-a-fog-service) and [3.1.6](#3-1-6-channel-as-a-tunnel-identifier)), similar to the $I^2P$ (Invisible Internet Project), used to hide the identity and IP addresses of *TON Network* nodes if necessary (e.g., nodes committing transactions from accounts with large amounts of cryptocurrency, or high-stake blockchain validator nodes who wish to hide their exact IP address and geographical location as a measure against DDoS attacks). -- A Kademlia-like distributed hash table (*TON DHT*; cf. [3.2](#3-2-ton-dht%3A-kademlia-like-distributed-hash-table)), used as a "torrent tracker" for *TON Storage* (cf. [3.2.10](#3-2-10-distributed-“torrent-trackers”-and-“network-interest-groups”-in-ton-dht)), as an "input tunnel locator" for *TON Proxy* (cf. [3.2.14](#3-2-14-locating-abstract-addresses)), and as a service locator for *TON Services* (cf. [3.2.12](#3-2-12-locating-services)). +- A [Kademlia-like distributed hash table](#3-2-ton-dht%3A-kademlia-like-distributed-hash-table)(*TON DHT*), used as a [torrent tracker](#3-2-10-distributed-“torrent-trackers”-and-“network-interest-groups”-in-ton-dht) for *TON Storage* as an [input tunnel locator](#3-2-14-locating-abstract-addresses) for *TON Proxy*, and as a [service locator](#3-2-12-locating-services) for *TON Services*. -- A platform for arbitrary services (*TON Services*; cf. [Chapter 4](#4-ton-services-and-applications)), residing in and available through *TON Network* and *TON Proxy*, with formalized interfaces (cf. [4.3.14](#4-3-14-public-interfaces-of-a-smart-contract)) enabling browser-like or smartphone application interaction. These formal interfaces and persistent service entry points can be published in the TON Blockchain (cf. [4.3.17](#4-3-17-locating-user-interfaces-via-ton-dns)); actual nodes providing service at any given moment can be looked up through the *TON DHT* starting from information published in the TON Blockchain (cf. [3.2.12](#3-2-12-locating-services)). Services may create smart contracts in the TON Blockchain to offer some guarantees to their clients (cf. [4.1.7](#4-1-7-mixed-services%3A-partly-off-chain%2C-partly-on-chain)). +- A platform for [arbitrary services](#4-ton-services-and-applications) (*TON Services*), residing in and available through *TON Network* and *TON Proxy*, with [formalized interfaces](#4-3-14-public-interfaces-of-a-smart-contract) enabling browser-like or smartphone application interaction. These formal interfaces and persistent service [entry points](#4-3-17-locating-user-interfaces-via-ton-dns) can be published in the TON Blockchain; actual nodes providing service at any given moment can be [looked up](#3-2-12-locating-services) through the *TON DHT* starting from information published in the TON Blockchain. Services may create smart contracts in the TON Blockchain to offer some [guarantees](#4-1-7-mixed-services%3A-partly-off-chain%2C-partly-on-chain) to their clients. -- *TON DNS* (cf. [4.3.1](#4-3-1-ton-dns%3A-a-mostly-on-chain-hierarchical-domain-name-service)), a service for assigning human-readable names to accounts, smart contracts, services and network nodes. +- [TON DNS](#4-3-1-ton-dns%3A-a-mostly-on-chain-hierarchical-domain-name-service), a service for assigning human-readable names to accounts, smart contracts, services and network nodes. -- *TON Payments* (cf. [Chapter 5](#5-ton-payments)), a platform for micropayments, micropayment channels and a micropayment channel network. It can be used for fast off-chain value transfers, and for paying for services powered by *TON Services*. +- [TON Payments](#5-ton-payments), a platform for micropayments, micropayment channels and a micropayment channel network. It can be used for fast off-chain value transfers, and for paying for services powered by *TON Services*. -- TON will allow easy integration with third-party messaging and social networking applications, thus making blockchain technologies and distributed services finally available and accessible to ordinary users (cf. [4.3.24](#4-3-24-ton-www)), rather than just to a handful of early cryptocurrency adopters. We will provide an example of such an integration in another of our projects, the Telegram Messenger (cf. [4.3.19](#4-3-19-a-light-wallet-and-ton-entity-explorer-can-be-built-into-telegram-messenger-clients)). +- TON will allow easy integration with third-party messaging and social networking applications, thus making blockchain technologies and distributed services finally available and [accessible](#4-3-24-ton-www) to ordinary users, rather than just to a handful of early cryptocurrency adopters. We will provide an example of such an integration in another of our projects, the [Telegram Messenger](#4-3-19-a-light-wallet-and-ton-entity-explorer-can-be-built-into-telegram-messenger-clients). -While the TON Blockchain is the core of the TON project, and the other components might be considered as playing a supportive role for the blockchain, they turn out to have useful and interesting functionality by themselves. Combined, they allow the platform to host more versatile applications than it would be possible by just using the TON Blockchain (cf. [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain%3F) and [4.1](#4-1-ton-service-implementation-strategies)). +While the TON Blockchain is the core of the TON project, and the other components might be considered as playing a supportive role for the blockchain, they turn out to have useful and interesting functionality by themselves. Combined, they allow the platform to host more versatile applications than it would be possible by just using the TON Blockchain (cf. [2.9.13](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”) and [4.1](#4-1-ton-service-implementation-strategies)). --- @@ -56,7 +56,7 @@ For simplicity, we speak here about *the* TON Blockchain, even though in princip ### 2.1 TON Blockchain as a Collection of 2-Blockchains -The TON Blockchain is actually a *collection* of blockchains (even a collection of *blockchains of blockchains*, or *2-blockchains*—this point will be clarified later in [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)), because no single blockchain project is capable of achieving our goal of processing millions of transactions per second, as opposed to the now-standard dozens of transactions per second. +The TON Blockchain is actually a *collection* of blockchains (even a collection of *blockchains of blockchains*, or *2-blockchains*—this point will be clarified [later](#2-1-17-correcting-invalid-shardchain-blocks)), because no single blockchain project is capable of achieving our goal of processing millions of transactions per second, as opposed to the now-standard dozens of transactions per second. #### 2.1.1. List of blockchain types @@ -64,11 +64,11 @@ The blockchains in this collection are: - The unique *master blockchain*, or *masterchain* for short, containing general information about the protocol and the current values of its parameters, the set of validators and their stakes, the set of currently active workchains and their "shards", and, most importantly, the set of hashes of the most recent blocks of all workchains and shardchains. -- Several (up to $2^{32}$) *working blockchains*, or *workchains* for short, which are actually the "workhorses", containing the value-transfer and smart-contract transactions. Different workchains may have different "rules", meaning different formats of account addresses, different formats of transactions, different virtual machines (VMs) for smart contracts, different basic cryptocurrencies and so on. However, they all must satisfy certain basic interoperability criteria to make interaction between different workchains possible and relatively simple. In this respect, the TON Blockchain is *heterogeneous* (cf. [2.8.8](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems)), similarly to the EOS (cf. [2.9.7](#2-9-7-eos)) and PolkaDot (cf. [2.9.8](#2-9-8-polkadot)) projects. +- Several (up to $2^{32}$) *working blockchains*, or *workchains* for short, which are actually the "workhorses", containing the value-transfer and smart-contract transactions. Different workchains may have different "rules", meaning different formats of account addresses, different formats of transactions, different virtual machines (VMs) for smart contracts, different basic cryptocurrencies and so on. However, they all must satisfy certain basic interoperability criteria to make interaction between different workchains possible and relatively simple. In this respect, the TON Blockchain is [heterogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems), similarly to the [EOS](#2-9-7-eos-%5B5%5D%3B-https%3A%2F%2Feos-io) and [PolkaDot](#2-9-8-polkadot-%5B17%5D%3B-https%3A%2F%2Fpolkadot-io%2F) projects. -- Each workchain is in turn subdivided into up to $2^{60}$ *shard blockchains*, or *shardchains* for short, having the same rules and block format as the workchain itself, but responsible only for a subset of accounts, depending on several first (most significant) bits of the account address. In other words, a form of sharding is built into the system (cf. [2.8.12](#2-8-12-sharding-support)). Because all these shardchains share a common block format and rules, the TON Blockchain is *homogeneous* in this respect (cf. [2.8.8](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems)), similarly to what has been discussed in one of Ethereum scaling proposals.[1](#fn1) +- Each workchain is in turn subdivided into up to $2^{60}$ *shard blockchains*, or *shardchains* for short, having the same rules and block format as the workchain itself, but responsible only for a subset of accounts, depending on several first (most significant) bits of the account address. In other words, a form of [sharding](#2-8-12-sharding-support) is built into the system. Because all these shardchains share a common block format and rules, the TON Blockchain is [homogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems) in this respect, similarly to what has been discussed in one of Ethereum scaling proposals.[1](#fn1) -- Each block in a shardchain (and in the masterchain) is actually not just a block, but a small blockchain. Normally, this "block blockchain" or "vertical blockchain" consists of exactly one block, and then we might think this is just the corresponding block of the shardchain (also called "horizontal blockchain" in this situation). However, if it becomes necessary to fix incorrect shardchain blocks, a new block is committed into the "vertical blockchain", containing either the replacement for the invalid "horizontal blockchain" block, or a "block difference", containing only a description of those parts of the previous version of this block that need to be changed. This is a TON-specific mechanism to replace detected invalid blocks without making a true fork of all shardchains involved; it will be explained in more detail in [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks). For now, we just remark that each shardchain (and the masterchain) is not a conventional blockchain, but a *blockchain of blockchains*, or *2D-blockchain*, or just a *2-blockchain*. +- Each block in a shardchain (and in the masterchain) is actually not just a block, but a small blockchain. Normally, this "block blockchain" or "vertical blockchain" consists of exactly one block, and then we might think this is just the corresponding block of the shardchain (also called "horizontal blockchain" in this situation). However, if it becomes necessary to fix incorrect shardchain blocks, a new block is committed into the "vertical blockchain", containing either the replacement for the invalid "horizontal blockchain" block, or a "block difference", containing only a description of those parts of the previous version of this block that need to be changed. This is a TON-specific mechanism to replace detected [invalid blocks](#2-1-17-correcting-invalid-shardchain-blocks) without making a true fork of all shardchains involved. For now, we just remark that each shardchain (and the masterchain) is not a conventional blockchain, but a *blockchain of blockchains*, or *2D-blockchain*, or just a *2-blockchain*. #### 2.1.2. Infinite Sharding Paradigm @@ -84,7 +84,7 @@ We call this perspective the *Infinite Sharding Paradigm*. It explains many of t #### 2.1.3. Messages. Instant Hypercube Routing -The Infinite Sharding Paradigm instructs us to regard each account (or smart contract) as if it were in its own shardchain by itself. Then the only way one account might affect the state of another is by sending a *message* to it (this is a special instance of the so-called Actor model, with accounts as Actors; cf. [2.4.2](#2-4-2-accounts-as-processes-or-actors-actor-model)). Therefore, a system of messages between accounts (and shardchains, because the source and destination accounts are, generally speaking, located in different shardchains) is of paramount importance to a scalable system such as the TON Blockchain. In fact, a novel feature of the TON Blockchain, called *Instant Hypercube Routing* (cf. [2.4.20](#2-4-20-instant-hypercube-routing%3A-fast-path-for-messages)), enables it to deliver and process a message created in a block of one shardchain into the very next block of the destination shardchain, *regardless of the total number of shardchains in the system.* +The Infinite Sharding Paradigm instructs us to regard each account (or smart contract) as if it were in its own shardchain by itself. Then the only way one account might affect the state of another is by sending a *message* to it (this is a special instance of the so-called [Actor model](#2-4-2-accounts-as-processes-or-actors;-actor-model), with accounts as Actors). Therefore, a system of messages between accounts (and shardchains, because the source and destination accounts are, generally speaking, located in different shardchains) is of paramount importance to a scalable system such as the TON Blockchain. In fact, a novel feature of the TON Blockchain, called [Instant Hypercube Routing](#2-4-20-instant-hypercube-routing:-“fast-path”-for-messages), enables it to deliver and process a message created in a block of one shardchain into the very next block of the destination shardchain, *regardless of the total number of shardchains in the system.* #### 2.1.4. Quantity of masterchains, workchains and shardchains @@ -92,7 +92,7 @@ A TON Blockchain contains exactly one masterchain. However, the system can poten #### 2.1.5. Workchains can be virtual blockchains, not true blockchains -Because a workchain is usually subdivided into shardchains, the existence of the workchain is "virtual", meaning that it is not a true blockchain in the sense of the general definition provided in [2.2.1](#2-2-1-general-blockchain-definition) below, but just a collection of shardchains. When only one shardchain corresponds to a workchain, this unique shardchain may be identified with the workchain, which in this case becomes a "true" blockchain, at least for some time, thus gaining a superficial similarity to customary single-blockchain design. However, the Infinite Sharding Paradigm (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)) tells us that this similarity is indeed superficial: it is just a coincidence that the potentially huge number of "account-chains" can temporarily be grouped into one blockchain. +Because a workchain is usually subdivided into shardchains, the existence of the workchain is "virtual", meaning that it is not a true blockchain in the sense of the [general definition](#2-2-1-general-blockchain-definition) below, but just a collection of shardchains. When only one shardchain corresponds to a workchain, this unique shardchain may be identified with the workchain, which in this case becomes a "true" blockchain, at least for some time, thus gaining a superficial similarity to customary single-blockchain design. However, the [Infinite Sharding Paradigm](#2-1-2-infinite-sharding-paradigm) tells us that this similarity is indeed superficial: it is just a coincidence that the potentially huge number of "account-chains" can temporarily be grouped into one blockchain. #### 2.1.6. Identification of workchains @@ -108,7 +108,7 @@ Each shardchain is identified by a couple $(w,s)=(\mathit{workchain\_id}, \mathi #### 2.1.9. Identification of account-chains -Recall that account-chains have only a virtual existence (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)). However, they have a natural identifier—namely, $(\mathit{workchain\_id},\mathit{account\_id})$—because any account-chain contains information about the state and updates of exactly one account (either a simple account or smart contract—the distinction is unimportant here). +Recall that account-chains have only a [virtual existence](#2-1-2-infinite-sharding-paradigm). However, they have a natural identifier—namely, $(\mathit{workchain\_id},\mathit{account\_id})$—because any account-chain contains information about the state and updates of exactly one account (either a simple account or smart contract—the distinction is unimportant here). #### 2.1.10. Dynamic splitting and merging of shardchains; cf 2.7. @@ -120,7 +120,7 @@ Initially, only one shard $(w,\emptyset)$ is created for workchain $w$. Later, i #### 2.1.11. Basic workchain or Workchain Zero -While up to $2^{32}$ workchains can be defined with their specific rules and transactions, we initially define only one, with $\mathit{workchain\_id}=0$. This workchain, called Workchain Zero or the basic workchain, is the one used to work with *TON smart contracts* and transfer *TON coins*, also known as *Grams* (cf. [Appendix A](#appendix-the-ton-coin-or-the-gram)). Most applications are likely to require only Workchain Zero. Shardchains of the basic workchain will be called *basic shardchains*. +While up to $2^{32}$ workchains can be defined with their specific rules and transactions, we initially define only one, with $\mathit{workchain\_id}=0$. This workchain, called Workchain Zero or the basic workchain, is the one used to work with *TON smart contracts* and transfer *TON coins*, also known as [Grams](#a-the-ton-coin,-or-the-gram). Most applications are likely to require only Workchain Zero. Shardchains of the basic workchain will be called *basic shardchains*. #### 2.1.12. Block generation intervals @@ -130,7 +130,7 @@ We expect a new block to be generated in each shardchain and the masterchain app Once the hash of a block of a shardchain is incorporated into a block of the masterchain, that shardchain block and all its ancestors are considered "canonical", meaning that they can be referenced from the subsequent blocks of all shardchains as something fixed and immutable. In fact, each new shardchain block contains a hash of the most recent masterchain block, and all shardchain blocks referenced from that masterchain block are considered immutable by the new block. -Essentially, this means that a transaction or a message committed in a shardchain block may be safely used in the very next blocks of the other shardchains, without needing to wait for, say, twenty confirmations (i.e., twenty blocks generated after the original block in the same blockchain) before forwarding a message or taking other actions based on a previous transaction, as is common in most proposed "loosely-coupled" systems (cf. [2.8.14](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems)), such as EOS. This ability to use transactions and messages in other shardchains a mere five seconds after being committed is one of the reasons we believe our "tightly-coupled" system, the first of its kind, will be able to deliver unprecedented performance (cf. [2.8.12](#2-8-12-sharding-support) and [2.8.14](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems)). +Essentially, this means that a transaction or a message committed in a shardchain block may be safely used in the very next blocks of the other shardchains, without needing to wait for, say, twenty confirmations (i.e., twenty blocks generated after the original block in the same blockchain) before forwarding a message or taking other actions based on a previous transaction, as is common in most proposed [loosely-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems) systems, such as EOS. This ability to use transactions and messages in other shardchains a mere five seconds after being committed is one of the reasons we believe our "tightly-coupled" system, the first of its kind, will be able to deliver unprecedented performance ([2.8.12](#2-8-12-sharding-support) and [2.8.14](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems)). #### 2.1.14. Masterchain block hash as a global state @@ -142,13 +142,13 @@ The TON Blockchain uses a Proof-of-Stake (PoS) approach for generating new block Then a smaller subset of validators is assigned to each shard $(w,s)$ in a deterministic pseudorandom way, changing approximately every 1024 blocks. This subset of validators suggests and reaches consensus on what the next shardchain block would be, by collecting suitable proposed transactions from the clients into new valid block candidates. For each block, there is a pseudorandomly chosen order on the validators to determine whose block candidate has the highest priority to be committed at each turn. -Validators and other nodes check the validity of the proposed block candidates; if a validator signs an invalid block candidate, it may be automatically punished by losing part or all of its stake, or by being suspended from the set of validators for some time. After that, the validators should reach consensus on the choice of the next block, essentially by an efficient variant of the BFT (Byzantine Fault Tolerant; cf. [2.8.4](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft)) consensus protocol, similar to PBFT \[[4](#ref-4)] or Honey Badger BFT \[[11](#ref-11)]. If consensus is reached, a new block is created, and validators divide between themselves the transaction fees for the transactions included, plus some newly-created ("minted") coins. +Validators and other nodes check the validity of the proposed block candidates; if a validator signs an invalid block candidate, it may be automatically punished by losing part or all of its stake, or by being suspended from the set of validators for some time. After that, the validators should reach consensus on the choice of the next block, essentially by an efficient variant of the [BFT](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft) (Byzantine Fault Tolerant) consensus protocol, similar to PBFT[[4]](#references) or Honey Badger BFT[[11]](#references). If consensus is reached, a new block is created, and validators divide between themselves the transaction fees for the transactions included, plus some newly-created ("minted") coins. Each validator can be elected to participate in several validator subsets; in this case, it is expected to run all validation and consensus algorithms in parallel. -After all new shardchain blocks are generated or a timeout is passed, a new masterchain block is generated, including the hashes of the latest blocks of all shardchains. This is done by BFT consensus of *all* validators.[2](#fn2) +After all new shardchain blocks are generated or a timeout is passed, a new masterchain block is generated, including the hashes of the latest blocks of all shardchains. This is done by BFT consensus of *all* validators.[2](#fn2) -More detail on the TON PoS approach and its economical model is provided in section [2.6](#2-6-creating-and-validating-new-blocks). +More detail on the TON PoS approach and its economical model is provided in [2.6](#2-6-creating-and-validating-new-blocks). #### 2.1.16. Forks of the masterchain @@ -156,13 +156,13 @@ A complication that arises from our tightly-coupled approach is that switching t The general rule is that *if masterchain block $B'$ is a predecessor of $B$, $B'$ includes hash $\text{Hash}(B'_{w,s})$ of $(w,s)$-shardchain block $B'_{w,s}$, and $B$ includes hash $\text{Hash}(B_{w,s})$, then $B'_{w,s}$ **must** be a predecessor of $B_{w,s}$; otherwise, the masterchain block $B$ is invalid.* -We expect masterchain forks to be rare, next to non-existent, because in the BFT paradigm adopted by the TON Blockchain they can happen only in the case of incorrect behavior by a majority of validators (cf. [2.6.1](#2-6-1-validators) and [2.6.15](#2-6-15-generation-of-new-masterchain-blocks)), which would imply significant stake losses by the offenders. Therefore, no true forks in the shardchains should be expected. Instead, if an invalid shardchain block is detected, it will be corrected by means of the "vertical blockchain" mechanism of the 2-blockchain (cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)), which can achieve this goal without forking the "horizontal blockchain" (i.e., the shardchain). The same mechanism can be used to fix non-fatal mistakes in the masterchain blocks as well. +We expect masterchain forks to be rare, next to non-existent, because in the BFT paradigm adopted by the TON Blockchain they can happen only in the case of incorrect behavior by a majority of validators ([2.6.1](#2-6-1-validators) and [2.6.15](#2-6-15-generation-of-new-masterchain-blocks)), which would imply significant stake losses by the offenders. Therefore, no true forks in the shardchains should be expected. Instead, if an invalid shardchain block is detected, it will be corrected by means of the [vertical blockchain](#2-1-17-correcting-invalid-shardchain-blocks) mechanism of the 2-blockchain, which can achieve this goal without forking the "horizontal blockchain" (i.e., the shardchain). The same mechanism can be used to fix non-fatal mistakes in the masterchain blocks as well. #### 2.1.17. Correcting invalid shardchain blocks Normally, only valid shardchain blocks will be committed, because validators assigned to the shardchain must reach a two-thirds Byzantine consensus before a new block can be committed. However, the system must allow for detection of previously committed invalid blocks and their correction. -Of course, once an invalid shardchain block is found—either by a validator (not necessarily assigned to this shardchain) or by a "fisherman" (any node of the system that made a certain deposit to be able to raise questions about block validity; cf. [2.6.4](#2-6-4-fishermen%3A-obtaining-money-by-pointing-out-others%E2%80%99-mistakes))—the invalidity claim and its proof are committed into the masterchain, and the validators that have signed the invalid block are punished by losing part of their stake and/or being temporarily suspended from the set of validators (the latter measure is important for the case of an attacker stealing the private signing keys of an otherwise benign validator). +Of course, once an invalid shardchain block is found—either by a validator (not necessarily assigned to this shardchain) or by a [fisherman](#2-6-4-fishermen%3A-obtaining-money-by-pointing-out-others%E2%80%99-mistakes) (any node of the system that made a certain deposit to be able to raise questions about block validity)—the invalidity claim and its proof are committed into the masterchain, and the validators that have signed the invalid block are punished by losing part of their stake and/or being temporarily suspended from the set of validators (the latter measure is important for the case of an attacker stealing the private signing keys of an otherwise benign validator). However, this is not sufficient, because the overall state of the system (TON Blockchain) turns out to be invalid because of the invalid shardchain block previously committed. This invalid block must be replaced by a newer valid version. @@ -170,7 +170,7 @@ Most systems would achieve this by "rolling back" to the last block before the i The TON Blockchain solves this problem by making each "block" of each shardchain and of the masterchain ("horizontal blockchains") a small blockchain ("vertical blockchain") by itself, containing different versions of this "block", or their "differences". Normally, the vertical blockchain consists of exactly one block, and the shardchain looks like a classical blockchain. However, once the invalidity of a block is confirmed and committed into a masterchain block, the "vertical blockchain" of the invalid block is allowed to grow by a new block in the vertical direction, replacing or editing the invalid block. The new block is generated by the current validator subset for the shardchain in question. -The rules for a new "vertical" block to be valid are quite strict. In particular, if a virtual "account-chain block" (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)) contained in the invalid block is valid by itself, it must be left unchanged by the new vertical block. +The rules for a new "vertical" block to be valid are quite strict. In particular, if a virtual [account-chain block](#2-1-2-infinite-sharding-paradigm) contained in the invalid block is valid by itself, it must be left unchanged by the new vertical block. Once a new "vertical" block is committed on top of the invalid block, its hash is published in a new masterchain block (or rather in a new "vertical" block, lying above the original masterchain block where the hash of the invalid shardchain block was originally published), and the changes are propagated further to any shardchain blocks referring to the previous version of this block (e.g., those having received messages from the incorrect block). This is fixed by committing new "vertical" blocks in vertical blockchains for all blocks previously referring to the "incorrect" block; new vertical blocks will refer to the most recent (corrected) versions instead. Again, strict rules forbid changing account-chains that are not really affected (i.e., that receive the same messages as in the previous version). In this way, fixing an incorrect block generates "ripples" that are ultimately propagated towards the most recent blocks of all affected shardchains; these changes are reflected in new "vertical" masterchain blocks as well. @@ -182,7 +182,7 @@ The masterchain state implicitly defines a map transforming the hash of the firs The TON Blockchain supports up to $2^{32}$ different "cryptocurrencies", "coins", or "tokens", distinguished by a 32-bit $\mathit{currency\_id}$. New cryptocurrencies can be added by special transactions in the masterchain. Each workchain has a basic cryptocurrency, and can have several additional cryptocurrencies. -There is one special cryptocurrency with $\mathit{currency\_id}=0$, namely, the *TON coin*, also known as the *Gram* (cf. [Appendix A](#appendix-the-ton-coin-or-the-gram)). It is the basic cryptocurrency of Workchain Zero. It is also used for transaction fees and validator stakes. +There is one special cryptocurrency with $\mathit{currency\_id}=0$, namely, the *TON coin*, also known as the [Gram](#a-the-ton-coin%2C-or-the-gram). It is the basic cryptocurrency of Workchain Zero. It is also used for transaction fees and validator stakes. In principle, other workchains may collect transaction fees in other tokens. In this case, some smart contract for automated conversion of these transaction fees into Grams should be provided. @@ -198,11 +198,11 @@ The *TON Virtual Machine*, also abbreviated as *TON VM* or *TVM*, is the virtual Here we list some of its features. They are discussed further in [2.3.12](#2-3-12-peculiarities-of-ton-vm), [2.3.14](#2-3-14-tvm-cells) and elsewhere: -• TVM represents all data as a collection of *(TVM) cells* (cf. [2.3.14](#2-3-14-tvm-cells)). Each cell contains up to 128 data bytes and up to 4 references to other cells. As a consequence of the "everything is a bag of cells" philosophy (cf. [2.5.14](#2-5-14-everything-is-a-bag-of-cells-philosophy)), this enables TVM to work with all data related to the TON Blockchain, including blocks and blockchain global state if necessary. +• TVM represents all data as a collection of [TVM cells](#2-3-14-tvm-cells). Each cell contains up to 128 data bytes and up to 4 references to other cells. As a consequence of the [everything is a bag of cells](#2-5-14-“everything-is-a-bag-of-cells”-philosophy) philosophy, this enables TVM to work with all data related to the TON Blockchain, including blocks and blockchain global state if necessary. -• TVM can work with values of arbitrary algebraic data types (cf. [2.3.12](#2-3-12-peculiarities-of-ton-vm)), represented as trees or directed acyclic graphs of TVM cells. However, it is agnostic towards the existence of algebraic data types; it just works with cells. +• TVM can work with values of arbitrary [algebraic data types](#2-3-12-peculiarities-of-ton-vm), represented as trees or directed acyclic graphs of TVM cells. However, it is agnostic towards the existence of algebraic data types; it just works with cells. -• TVM has built-in support for hashmaps (cf. [2.3.7](#2-3-7-denition-of-hashmap-type-as-a-patricia-tree)). +• TVM has built-in support for [hashmaps](#2-3-7-definition-of-hashmap-type-as-a-patricia-tree). • TVM is a stack machine. Its stack keeps either 64-bit integers or cell references. @@ -222,13 +222,13 @@ Here we list some of its features. They are discussed further in [2.3.12](#2-3-1 • Support for popular hash functions, including SHA-256, is present. -• TVM can work with Merkle proofs (cf. [5.1.9](#5-1-9-ton-vm-support-for-smart-payment-channels)). +• TVM can work with [Merkle proofs](#5-1-9-ton-vm-support-for-“smart”-payment-channels). -• TVM offers support for "large" or "global" smart contracts. Such smart contracts must be aware of sharding (cf. [2.3.18](#2-3-18-local-and-global-smart-contracts%3B-smart-contract-instances) and [2.3.16](#2-3-16-support-for-sharding-in-ton-vm-data-structures)). Usual (local) smart contracts can be sharding-agnostic. +• TVM offers support for "large" or "global" smart contracts. Such smart contracts must be aware of sharding ([2.3.18](#2-3-18-local-and-global-smart-contracts%3B-smart-contract-instances) and [2.3.16](#2-3-16-support-for-sharding-in-ton-vm-data-structures)). Usual (local) smart contracts can be sharding-agnostic. • TVM supports closures. -• A "spineless tagless G-machine" [[13]](#ref-13) can be easily implemented inside TVM. +• A "spineless tagless G-machine" [[13]](#references) can be easily implemented inside TVM. Several high-level languages can be designed for TVM, in addition to the "TVM assembly". All these languages will have static types and will support algebraic data types. We envision the following possibilities: @@ -250,7 +250,7 @@ In general, any *(true) blockchain* is a sequence of *blocks*, each block $B$ co #### 2.2.2. Relevance for the TON Blockchain -Recall that the *TON Blockchain* is not a true blockchain, but a collection of 2-blockchains (i.e., of blockchains of blockchains; cf. [2.1.1](#2-1-1-list-of-blockchain-types)), so the above is not directly applicable to it. However, we start with these generalities on true blockchains to use them as building blocks for our more sophisticated constructions. +Recall that the *TON Blockchain* is not a true blockchain, but a collection of [2-blockchains](#2-1-1-list-of-blockchain-types) (i.e., of blockchains of blockchains), so the above is not directly applicable to it. However, we start with these generalities on true blockchains to use them as building blocks for our more sophisticated constructions. #### 2.2.3. Blockchain instance and blockchain type @@ -268,11 +268,11 @@ $$ \text{Blockchain} \equiv \Sigma_{(\mathbb{B} : \text{Block}^*)} \text{isValidBc}(\mathbb{B}) \tag{2} $$ -We use here the notation for dependent sums of types borrowed from \[[16](#ref-16)]. +We use here the notation for dependent sums of types borrowed from [[16]](#references). #### 2.2.4. Dependent type theory, Coq and TL -Note that we are using (Martin-Löf) dependent type theory here, similar to that used in the Coq[3](#fn3) proof assistant. A simplified version of dependent type theory is also used in *TL (Type Language)*,[4](#fn4) which will be used in the formal specification of the TON Blockchain to describe the serialization of all data structures and the layouts of blocks, transactions, and the like. +Note that we are using (Martin-Löf) dependent type theory here, similar to that used in the Coq[3](#fn3) proof assistant. A simplified version of dependent type theory is also used in *TL (Type Language)*,[4](#fn4) which will be used in the formal specification of the TON Blockchain to describe the serialization of all data structures and the layouts of blocks, transactions, and the like. In fact, dependent type theory gives a useful formalization of what a proof is, and such formal proofs (or their serializations) might become handy when one needs to provide proof of invalidity for some block, for example. @@ -286,7 +286,7 @@ A collection of constructor and type definitions is called a *TL-scheme*. It is An important feature of TL-schemes is that they determine an unambiguous way of serializing and deserializing values (or objects) of algebraic types defined. Namely, when a value needs to be serialized into a stream of bytes, first the name of the constructor used for this value is serialized. Recursively computed serializations of each field follow. -The description of a previous version of TL, suitable for serializing arbitrary objects into sequences of 32-bit integers, is available at https://core.telegram.org/mtproto/TL. A new version of TL, called *TL-B*, is being developed for the purpose of describing the serialization of objects used by the TON Project. This new version can serialize objects into streams of bytes and even bits (not just 32-bit integers), and offers support for serialization into a tree of TVM cells (cf. [2.3.14](#2-3-14-tvm-cells)). A description of TL-B will be a part of the formal specification of the TON Blockchain. +The description of a previous version of TL, suitable for serializing arbitrary objects into sequences of 32-bit integers, is available at https://core.telegram.org/mtproto/TL. A new version of TL, called *TL-B*, is being developed for the purpose of describing the serialization of objects used by the TON Project. This new version can serialize objects into streams of bytes and even bits (not just 32-bit integers), and offers support for serialization into a tree of [TVM cells](#2-3-14-tvm-cells). A description of TL-B will be a part of the formal specification of the TON Blockchain. #### 2.2.6. Blocks and transactions as state transformation operators @@ -349,7 +349,7 @@ Final remark: in order to make the probability statement of ([8](#2-2-9-hash-ass #### 2.2.10. Hash used for the TON Blockchain -We are using the 256-bit SHA-256 hash for the TON Blockchain for the time being. If it turns out to be weaker than expected, it can be replaced by another hash function in the future. The choice of the hash function is a configurable parameter of the protocol, so it can be changed without hard forks as explained in [2.1.21](#2-1-21-configurable-parameters). +We are using the 256-bit SHA-256 hash for the TON Blockchain for the time being. If it turns out to be weaker than expected, it can be replaced by another hash function in the future. The choice of the hash function is a [configurable parameter](#2-1-21-configurable-parameters) of the protocol, so it can be changed without hard forks. --- @@ -490,9 +490,9 @@ $$ \texttt{Hash}\bigl(\texttt{Root}(n,s_0,t)\bigr):=\texttt{Hash}\bigl(\texttt{Code}(n).\texttt{Code}(s_0).\texttt{Hash}(t)\bigr) \tag{22} $$ -Here $s.t$ denotes the concatenation of (bit) strings $s$ and $t$, and $\text{Code}(s)$ is a prefix code for all bit strings $s$. For example, one might encode 0 by 10, 1 by 11, and the end of the string by 0.[5](#fn5) +Here $s.t$ denotes the concatenation of (bit) strings $s$ and $t$, and $\text{Code}(s)$ is a prefix code for all bit strings $s$. For example, one might encode 0 by 10, 1 by 11, and the end of the string by 0.[5](#fn5) -We will see later (cf. [2.3.12](#2-3-12-peculiarities-of-ton-vm) and [2.3.14](#2-3-14-tvm-cells)) that this is a (slightly tweaked) version of recursively defined hashes for values of arbitrary (dependent) algebraic types. +We will see later ([2.3.12](#2-3-12-peculiarities-of-ton-vm) and [2.3.14](#2-3-14-tvm-cells)) that this is a (slightly tweaked) version of recursively defined hashes for values of arbitrary (dependent) algebraic types. #### 2.3.9. Recomputing Merkle tree hashes @@ -518,13 +518,13 @@ The TON VM or TVM (TON Virtual Machine), used to run smart contracts in the mast One might imagine first that the state of a TVM smart contract is not just a hashmap $\mathbf{2}^{256}\to \mathbf{2}^{256}$, or $\mathit{Hashmap}(256,\mathbf{2}^{256})$, but (as a first step) $\mathit{Hashmap}(256,X)$, where $X$ is a type with several constructors, enabling it to store, apart from 256-bit integers, other data structures, including other hashmaps $\mathit{Hashmap}(256,X)$ in particular. This would mean that a cell of TVM (persistent or temporary) storage—or a variable or an element of an array in a TVM smart-contract code—may contain not only an integer, but a whole new hashmap. Of course, this would mean that a cell contains not just 256 bits, but also, say, an 8-bit tag, describing how these 256 bits should be interpreted. -In fact, values do not need to be precisely 256-bit. The value format used by TVM consists of a sequence of raw bytes and references to other structures, mixed in arbitrary order, with some descriptor bytes inserted in suitable locations to be able to distinguish pointers from raw data (e.g., strings or integers); cf. [2.3.14](#2-3-14-tvm-cells). +In fact, values do not need to be precisely 256-bit. The value format used by TVM consists of a sequence of raw bytes and references to other structures, mixed in arbitrary order, with some [descriptor bytes](#2-3-14-tvm-cells) inserted in suitable locations to be able to distinguish pointers from raw data (e.g., strings or integers). -This raw value format may be used to implement arbitrary sum-product algebraic types. In this case, the value would contain a raw byte first, describing the constructor being used (from the perspective of a high-level language), and then other fields or constructor arguments, consisting of raw bytes and references to other structures depending on the constructor chosen (cf. [2.2.5](#2-2-5-tl%2C-or-the-type-language)). However, TVM does not know anything about the correspondence between constructors and their arguments; the mixture of bytes and references is explicitly described by certain descriptor bytes.[8](#fn8) +This raw value format may be used to implement arbitrary sum-product algebraic types. In this case, the value would contain a raw byte first, describing the constructor being used (from the perspective of a high-level language), and then other fields or constructor arguments, consisting of raw bytes and references to other structures depending on the [constructor](#2-2-5-tl%2C-or-the-type-language) chosen. However, TVM does not know anything about the correspondence between constructors and their arguments; the mixture of bytes and references is explicitly described by certain descriptor bytes.[8](#fn8) The Merkle tree hashing is extended to arbitrary such structures: to compute the hash of such a structure, all references are recursively replaced by hashes of objects referred to, and then the hash of the resulting byte string (descriptor bytes included) is computed. -In this way, the Merkle tree hashing for hashmaps, described in [2.3.8](#2-3-8-merkle-patricia-trees), is just a special case of hashing for arbitrary (dependent) algebraic data types, applied to type $Hashmap(n, X)$ with two constructors.[9](#fn9) +In this way, the [Merkle tree](#2-3-8-merkle-patricia-trees) hashing for hashmaps is just a special case of hashing for arbitrary (dependent) algebraic data types, applied to type $Hashmap(n, X)$ with two constructors.[9](#fn9) #### 2.3.13. Persistent storage of TON smart contracts @@ -534,11 +534,11 @@ Persistent storage of a TON smart contract essentially consists of its "global v Ultimately, the TON VM keeps all data in a collection of *(TVM) cells*. Each cell contains two descriptor bytes first, indicating how many bytes of raw data are present in this cell (up to 128) and how many references to other cells are present (up to four). Then these raw data bytes and references follow. Each cell is referenced exactly once, so we might have included in each cell a reference to its "parent" (the only cell referencing this one). However, this reference need not be explicit. -In this way, the persistent data storage cells of a TON smart contract are organized into a tree,[10](#fn10) with a reference to the root of this tree kept in the smart-contract description. If necessary, a Merkle tree hash of this entire persistent storage is recursively computed, starting from the leaves and then simply replacing all references in a cell with the recursively computed hashes of the referenced cells, and subsequently computing the hash of the byte string thus obtained. +In this way, the persistent data storage cells of a TON smart contract are organized into a tree,[10](#fn10) with a reference to the root of this tree kept in the smart-contract description. If necessary, a Merkle tree hash of this entire persistent storage is recursively computed, starting from the leaves and then simply replacing all references in a cell with the recursively computed hashes of the referenced cells, and subsequently computing the hash of the byte string thus obtained. #### 2.3.15. Generalized Merkle proofs for values of arbitrary algebraic types -Because the TON VM represents a value of arbitrary algebraic type by means of a tree consisting of (TVM) cells, and each cell has a well-defined (recursively computed) Merkle hash, depending in fact on the whole subtree rooted in this cell, we can provide "generalized Merkle proofs" for (parts of) values of arbitrary algebraic types, intended to prove that a certain subtree of a tree with a known Merkle hash takes a specific value or a value with a specific hash. This generalizes the approach of [2.3.10](#2-3-10-merkle-proofs), where only Merkle proofs for $x[i]=y$ have been considered. +Because the TON VM represents a value of arbitrary algebraic type by means of a tree consisting of (TVM) cells, and each cell has a well-defined (recursively computed) Merkle hash, depending in fact on the whole subtree rooted in this cell, we can provide "generalized Merkle proofs" for (parts of) values of arbitrary algebraic types, intended to prove that a certain subtree of a tree with a known Merkle hash takes a specific value or a value with a specific hash. This generalizes the approach where only [Merkle proofs](#2-3-10-merkle-proofs for $x[i]=y$ have been considered. #### 2.3.16. Support for sharding in TON VM data structures @@ -560,11 +560,11 @@ Payments collected for keeping persistent data are distributed among the validat #### 2.3.18. Local and global smart contracts; smart-contract instances -A smart contract normally resides just in one shard, selected according to the smart contract's $account\_id$, similarly to an "ordinary" account. This is usually sufficient for most applications. However, some "high-load" smart contracts may want to have an "instance" in each shardchain of some workchain. To achieve this, they must propagate their creating transaction into all shardchains, for instance, by committing this transaction into the "root" shardchain $(w,\emptyset)$[11](#fn11) of the workchain $w$ and paying a large commission.[12](#fn12) +A smart contract normally resides just in one shard, selected according to the smart contract's $account\_id$, similarly to an "ordinary" account. This is usually sufficient for most applications. However, some "high-load" smart contracts may want to have an "instance" in each shardchain of some workchain. To achieve this, they must propagate their creating transaction into all shardchains, for instance, by committing this transaction into the "root" shardchain $(w,\emptyset)$ [11](#fn11)of the workchain $w$ and paying a large commission.[12](#fn12) This action effectively creates instances of the smart contract in each shard, with separate balances. Originally, the balance transferred in the creating transaction is distributed simply by giving the instance in shard $(w,s)$ the $2^{-|s|}$ part of the total balance. When a shard splits into two child shards, balances of all instances of global smart contracts are split in half; when two shards merge, balances are added together. -In some cases, splitting/merging instances of global smart contracts may involve (delayed) execution of special methods of these smart contracts. By default, the balances are split and merged as described above, and some special "account-indexed" hashmaps are also automatically split and merged (cf. [2.3.16](#2-3-16-support-for-sharding-in-ton-vm-data-structures)). +In some cases, splitting/merging instances of global smart contracts may involve (delayed) execution of special methods of these smart contracts. By default, the balances are split and merged as described above, and some special "account-indexed" hashmaps are also automatically [split and merged](#2-3-16-support-for-sharding-in-ton-vm-data-structures). #### 2.3.19. Limiting splitting of smart contracts @@ -584,10 +584,10 @@ We can summarize all of the above to conclude that an account or smart-contract We also need to keep somewhere, either in the account state or in some other account-indexed hashmap, the following data: -- The output message queue of the account (cf. [2.4.17](#2-4-17-output-queues)) -- The collection of (hashes of) recently delivered messages (cf. [2.4.23](#2-4-23-preventing-double-delivery-of-messages)) +- The [output message queue](#2-4-17-output-queues) of the account +- The collection of (hashes of) recently [delivered messages](#2-4-23-preventing-double-delivery-of-messages) -Not all of these are really required for every account; for example, smart-contract code is needed only for smart contracts, but not for "simple" accounts. Furthermore, while any account must have a non-zero balance in the principal currency (e.g., Grams for the masterchain and shardchains of the basic workchain), it may have balances of zero in other currencies. In order to avoid keeping unused data, a sum-product type (depending on the workchain) is defined (during the workchain's creation), which uses different tag bytes (e.g., TL constructors; cf. [2.2.5](#2-2-5-tl-or-the-type-language)) to distinguish between different "constructors" used. Ultimately, the account state is itself kept as a collection of cells of the TVM persistent storage. +Not all of these are really required for every account; for example, smart-contract code is needed only for smart contracts, but not for "simple" accounts. Furthermore, while any account must have a non-zero balance in the principal currency (e.g., Grams for the masterchain and shardchains of the basic workchain), it may have balances of zero in other currencies. In order to avoid keeping unused data, a sum-product type (depending on the workchain) is defined (during the workchain's creation), which uses different tag bytes (e.g., [TL constructors](#2-2-5-tl,-or-the-type-language)) to distinguish between different "constructors" used. Ultimately, the account state is itself kept as a collection of cells of the TVM persistent storage. --- @@ -599,7 +599,7 @@ An important component of the TON Blockchain is the *messaging system* between b *Messages* are sent from one account to another. Each *transaction* consists of an account receiving one message, changing its state according to certain rules, and generating several (maybe one or zero) new messages to other accounts. Each message is generated and received (delivered) exactly once. -This means that messages play a fundamental role in the system, comparable to that of accounts (smart contracts). From the perspective of the Infinite Sharding Paradigm (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)), each account resides in its separate "account-chain", and the only way it can affect the state of some other account is by sending a message. +This means that messages play a fundamental role in the system, comparable to that of accounts (smart contracts). From the perspective of the [Infinite Sharding Paradigm](#2-1-2-infinite-sharding-paradigm), each account resides in its separate "account-chain", and the only way it can affect the state of some other account is by sending a message. #### 2.4.2. Accounts as processes or actors; Actor model @@ -643,21 +643,21 @@ These external input and output messages can be used for interacting with off-ch The *message body* is simply a sequence of bytes, the meaning of which is determined only by the receiving workchain and/or smart contract. For blockchains using TON VM, this could be the serialization of any TVM cell, generated automatically via the `Send()` operation. Such a serialization is obtained simply by recursively replacing all references in a TON VM cell with the cells referred to. Ultimately, a string of raw bytes appears, which is usually prepended by a 4-byte "message type" or "message constructor", used to select the correct method of the receiving smart contract. -Another option would be to use TL-serialized objects (cf. [2.2.5](#2-2-5-tl-or-the-type-language)) as message bodies. This might be especially useful for communication between different workchains, one or both of which are not necessarily using the TON VM. +Another option would be to use [TL-serialized objects](#2-2-5-tl,-or-the-type-language) as message bodies. This might be especially useful for communication between different workchains, one or both of which are not necessarily using the TON VM. #### 2.4.10. Gas limit and other workchain/VM-specific parameters -Sometimes a message needs to carry information about the gas limit, the gas price, transaction fees and similar values that depend on the receiving workchain and are relevant only for the receiving workchain, but not necessarily for the originating workchain. Such parameters are included in or before the message body, sometimes (depending on the workchain) with special 4-byte prefixes indicating their presence (which can be defined by a TL-scheme; cf. [2.2.5](#2-2-5-tl-or-the-type-language)). +Sometimes a message needs to carry information about the gas limit, the gas price, transaction fees and similar values that depend on the receiving workchain and are relevant only for the receiving workchain, but not necessarily for the originating workchain. Such parameters are included in or before the message body, sometimes (depending on the workchain) with special 4-byte prefixes indicating their presence (which can be defined by a [TL-scheme](#2-2-5-tl,-or-the-type-language). #### 2.4.11. Creating messages: smart contracts and transactions -There are two sources of new messages. Most messages are created during smart-contract execution (via the `Send()` operation in TON VM), when some smart contract is invoked to process an incoming message. Alternatively, messages may come from the outside as "external messages" or "messages from nowhere" (cf. [2.4.6](#2-4-6-external-messages-or-messages-from-nowhere)).[13](#fn13) +There are two sources of new messages. Most messages are created during smart-contract execution (via the `Send()` operation in TON VM), when some smart contract is invoked to process an incoming message. Alternatively, messages may come from the outside as [external messages](#2-4-6-external-messages%2C-or-“messages-from-nowhere”) or "messages from nowhere".[13](#fn13) #### 2.4.12. Delivering messages -When a message reaches the shardchain containing its destination account,[14](#fn14) it is "delivered" to its destination account. What happens next depends on the workchain; from an outside perspective, it is important that such a message can never be forwarded further from this shardchain. +When a message reaches the shardchain containing its destination account,[14](#fn14) it is "delivered" to its destination account. What happens next depends on the workchain; from an outside perspective, it is important that such a message can never be forwarded further from this shardchain. -For shardchains of the basic workchain, delivery consists in adding the message value (minus any gas payments) to the balance of the receiving account, and possibly in invoking a message-dependent method of the receiving smart contract afterwards, if the receiving account is a smart contract. In fact, a smart contract has only one entry point for processing all incoming messages, and it must distinguish between different types of messages by looking at their first few bytes (e.g., the first four bytes containing a TL constructor; cf. [2.2.5](#2-2-5-tl-or-the-type-language)). +For shardchains of the basic workchain, delivery consists in adding the message value (minus any gas payments) to the balance of the receiving account, and possibly in invoking a message-dependent method of the receiving smart contract afterwards, if the receiving account is a smart contract. In fact, a smart contract has only one entry point for processing all incoming messages, and it must distinguish between different types of messages by looking at their first few bytes (e.g., the first four bytes containing a [TL constructor](#2-2-5-tl,-or-the-type-language). #### 2.4.13. Delivery of a message is a transaction @@ -665,7 +665,7 @@ Because the delivery of a message changes the state of an account or smart contr #### 2.4.14. Messages between instances of the same smart contract -Recall that a smart contract may be *local* (i.e., residing in one shardchain as any ordinary account does) or *global* (i.e., having instances in all shards, or at least in all shards up to some known depth $d$; cf. [2.3.18](#2-3-18-local-and-global-smart-contracts-smart-contract-instances)). Instances of a global smart contract may exchange special messages to transfer information and value between each other if required. In this case, the (unforgeable) sender $account\_id$ becomes important (cf. [2.4.4](#2-4-4-message-sender)). +Recall that a smart contract may be [local](#2-3-18-local-and-global-smart-contracts%3B-smart-contract-instances) (i.e., residing in one shardchain as any ordinary account does) or [global](#2-3-18-local-and-global-smart-contracts%3B-smart-contract-instances) (i.e., having instances in all shards, or at least in all shards up to some known depth $d$). Instances of a global smart contract may exchange special messages to transfer information and value between each other if required. In this case, the (unforgeable) [sender](#2-4-4-message-sender) $account\_id$ becomes important. #### 2.4.15. Messages to any instance of a smart contract; wildcard addresses @@ -677,19 +677,19 @@ All messages received by a blockchain (usually a shardchain; sometimes the maste #### 2.4.17. Output queues -From the perspective of the Infinite Sharding Paradigm (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)), each account-chain (i.e., each account) has its own output queue, consisting of all messages it has generated, but not yet delivered to their recipients. Of course, account-chains have only a virtual existence; they are grouped into shardchains, and a shardchain has an output "queue", consisting of the union of the output queues of all accounts belonging to the shardchain. +From the perspective of the [Infinite Sharding Paradigm](#2-1-2-infinite-sharding-paradigm), each account-chain (i.e., each account) has its own output queue, consisting of all messages it has generated, but not yet delivered to their recipients. Of course, account-chains have only a virtual existence; they are grouped into shardchains, and a shardchain has an output "queue", consisting of the union of the output queues of all accounts belonging to the shardchain. This shardchain output "queue" imposes only partial order on its member messages. Namely, a message generated in a preceding block must be delivered before any message generated in a subsequent block, and any messages generated by the same account and having the same destination must be delivered in the order of their generation. #### 2.4.18. Reliable and fast inter-chain messaging -It is of paramount importance for a scalable multi-blockchain project such as TON to be able to forward and deliver messages between different shardchains (cf. [2.1.3](#2-1-3-messages-instant-hypercube-routing)), even if there are millions of them in the system. The messages should be delivered *reliably* (i.e., messages should not be lost or delivered more than once) and *quickly*. The TON Blockchain achieves this goal by using a combination of two "message routing" mechanisms. +It is of paramount importance for a scalable multi-blockchain project such as TON to be able to forward and [deliver messages](#2-1-3-messages-instant-hypercube-routing) between different shardchains, even if there are millions of them in the system. The messages should be delivered *reliably* (i.e., messages should not be lost or delivered more than once) and *quickly*. The TON Blockchain achieves this goal by using a combination of two "message routing" mechanisms. #### 2.4.19. Hypercube routing: "slow path" for messages with assured delivery -The TON Blockchain uses "hypercube routing" as a slow, but safe and reliable way of delivering messages from one shardchain to another, using several intermediate shardchains for transit if necessary. Otherwise, the validators of any given shardchain would need to keep track of the state of (the output queues of) all other shardchains, which would require prohibitive amounts of computing power and network bandwidth as the total quantity of shardchains grows, thus limiting the scalability of the system. Therefore, it is not possible to deliver messages directly from any shard to every other. Instead, each shard is "connected" only to shards differing in exactly one hexadecimal digit of their $(w,s)$ shard identifiers (cf. [2.1.8](#2-1-8-identification-of-shardchains)). In this way, all shardchains constitute a "hypercube" graph, and messages travel along the edges of this hypercube. +The TON Blockchain uses "hypercube routing" as a slow, but safe and reliable way of delivering messages from one shardchain to another, using several intermediate shardchains for transit if necessary. Otherwise, the validators of any given shardchain would need to keep track of the state of (the output queues of) all other shardchains, which would require prohibitive amounts of computing power and network bandwidth as the total quantity of shardchains grows, thus limiting the scalability of the system. Therefore, it is not possible to deliver messages directly from any shard to every other. Instead, each shard is "connected" only to shards differing in exactly one hexadecimal digit of their $(w,s)$ [shard identifiers](#2-1-8-identification-of-shardchains). In this way, all shardchains constitute a "hypercube" graph, and messages travel along the edges of this hypercube. -If a message is sent to a shard different from the current one, one of the hexadecimal digits (chosen deterministically) of the current shard identifier is replaced by the corresponding digit of the target shard, and the resulting identifier is used as the proximate target to forward the message to.[15](#fn15) +If a message is sent to a shard different from the current one, one of the hexadecimal digits (chosen deterministically) of the current shard identifier is replaced by the corresponding digit of the target shard, and the resulting identifier is used as the proximate target to forward the message to.[15](#fn15) The main advantage of hypercube routing is that the block validity conditions imply that validators creating blocks of a shardchain must collect and process messages from the output queues of "neighboring" shardchains, on pain of losing their stakes. In this way, any message can be expected to reach its final destination sooner or later; a message cannot be lost in transit or delivered twice. @@ -697,19 +697,19 @@ Notice that hypercube routing introduces some additional delays and expenses, be #### 2.4.20. Instant Hypercube Routing: "fast path" for messages -A novel feature of the TON Blockchain is that it introduces a "fast path" for forwarding messages from one shardchain to any other, allowing in most cases to bypass the "slow" hypercube routing of [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery) altogether and deliver the message into the very next block of the final destination shardchain. +A novel feature of the TON Blockchain is that it introduces a "fast path" for forwarding messages from one shardchain to any other, allowing in most cases to bypass the [slow hypercube routing](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery) altogether and deliver the message into the very next block of the final destination shardchain. The idea is as follows. During the "slow" hypercube routing, the message travels (in the network) along the edges of the hypercube, but it is delayed (for approximately five seconds) at each intermediate vertex to be committed into the corresponding shardchain before continuing its voyage. -To avoid unnecessary delays, one might instead relay the message along with a suitable Merkle proof along the edges of the hypercube, without waiting to commit it into the intermediate shardchains. In fact, the network message should be forwarded from the validators of the "task group" (cf. [2.6.8](#2-6-8-election-of-validator-task-groups)) of the original shard to the designated block producer (cf. [2.6.9](#2-6-9-rotating-priority-order-on-each-task-group)) of the "task group" of the destination shard; this might be done directly without going along the edges of the hypercube. When this message with the Merkle proof reaches the validators (more precisely, the collators; cf. [2.6.5](#2-6-5-collators-obtaining-money-by-suggesting-new-blocks-to-validators)) of the destination shardchain, they can commit it into a new block immediately, without waiting for the message to complete its travel along the "slow path". Then a confirmation of delivery along with a suitable Merkle proof is sent back along the hypercube edges, and it may be used to stop the travel of the message along the "slow path", by committing a special transaction. +To avoid unnecessary delays, one might instead relay the message along with a suitable Merkle proof along the edges of the hypercube, without waiting to commit it into the intermediate shardchains. In fact, the network message should be forwarded from the validators of the [task group](#2-6-8-election-of-validator-“task-groups”) of the original shard to the designated [block producer](#2-6-9-rotating-priority-order-on-each-task-group) of the "task group" of the destination shard; this might be done directly without going along the edges of the hypercube. When this message with the Merkle proof reaches the validators (more precisely, the [collators](#2-6-5-collators%3A-obtaining-money-by-suggesting-new-blocks-to-validators)) of the destination shardchain, they can commit it into a new block immediately, without waiting for the message to complete its travel along the "slow path". Then a confirmation of delivery along with a suitable Merkle proof is sent back along the hypercube edges, and it may be used to stop the travel of the message along the "slow path", by committing a special transaction. -Note that this "instant delivery" mechanism does not replace the "slow" but failproof mechanism described in [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery). The "slow path" is still needed because the validators cannot be punished for losing or simply deciding not to commit the "fast path" messages into new blocks of their blockchains.[16](#fn16) +Note that this "instant delivery" mechanism does not replace the [slow](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery) but failproof mechanism. The "slow path" is still needed because the validators cannot be punished for losing or simply deciding not to commit the "fast path" messages into new blocks of their blockchains.[16](#fn16) -Therefore, both message forwarding methods are run in parallel, and the "slow" mechanism is aborted only if a proof of success of the "fast" mechanism is committed into an intermediate shardchain.[17](#fn17) +Therefore, both message forwarding methods are run in parallel, and the "slow" mechanism is aborted only if a proof of success of the "fast" mechanism is committed into an intermediate shardchain.[17](#fn17) #### 2.4.21. Collecting input messages from output queues of neighboring shardchains -When a new block for a shardchain is proposed, some of the output messages of the neighboring (in the sense of the routing hypercube of [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery)) shardchains are included in the new block as "input" messages and immediately delivered (i.e., processed). There are certain rules as to the order in which these neighbors' output messages must be processed. Essentially, an "older" message (coming from a shardchain block referring to an older masterchain block) must be delivered before any "newer" message; and for messages coming from the same neighboring shardchain, the partial order of the output queue described in [2.4.17](#2-4-17-output-queues) must be observed. +When a new block for a shardchain is proposed, some of the output messages of the neighboring (in the sense of the [routing hypercube](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery)) shardchains are included in the new block as "input" messages and immediately delivered (i.e., processed). There are certain rules as to the order in which these neighbors' output messages must be processed. Essentially, an "older" message (coming from a shardchain block referring to an older masterchain block) must be delivered before any "newer" message; and for messages coming from the same neighboring shardchain, the partial order of the [output queue](#2-4-17-output-queues) must be observed. #### 2.4.22. Deleting messages from output queues @@ -717,11 +717,11 @@ Once an output queue message is observed as having been delivered by a neighbori #### 2.4.23. Preventing double delivery of messages -To prevent double delivery of messages taken from the output queues of the neighboring shardchains, each shardchain (more precisely, each account-chain inside it) keeps the collection of recently delivered messages (or just their hashes) as part of its state. When a delivered message is observed to be deleted from the output queue by its originating neighboring shardchain (cf. [2.4.22](#2-4-22-deleting-messages-from-output-queues)), it is deleted from the collection of recently delivered messages as well. +To prevent double delivery of messages taken from the output queues of the neighboring shardchains, each shardchain (more precisely, each account-chain inside it) keeps the collection of recently delivered messages (or just their hashes) as part of its state. When a delivered message is observed to be [deleted](#2-4-22-deleting-messages-from-output-queues) from the output queue by its originating neighboring shardchain, it is deleted from the collection of recently delivered messages as well. #### 2.4.24. Forwarding messages intended for other shardchains -Hypercube routing (cf. [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery)) means that sometimes outbound messages are delivered not to the shardchain containing the intended recipient, but to a neighboring shardchain lying on the hypercube path to the destination. In this case, "delivery" consists in moving the inbound message to the outbound queue. This is reflected explicitly in the block as a special *forwarding transaction*, containing the message itself. Essentially, this looks as if the message had been received by somebody inside the shardchain, and one identical message had been generated as result. +[Hypercube routing](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery) means that sometimes outbound messages are delivered not to the shardchain containing the intended recipient, but to a neighboring shardchain lying on the hypercube path to the destination. In this case, "delivery" consists in moving the inbound message to the outbound queue. This is reflected explicitly in the block as a special *forwarding transaction*, containing the message itself. Essentially, this looks as if the message had been received by somebody inside the shardchain, and one identical message had been generated as result. #### 2.4.25. Payment for forwarding and keeping a message @@ -753,17 +753,17 @@ We start with a "high-level" or "logical" description, which consists in saying #### 2.5.1. Shardchain state as a collection of account-chain states -According to the Infinite Sharding Paradigm (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)), any shardchain is just a (temporary) collection of virtual "account-chains", containing exactly one account each. This means that, essentially, the global shardchain state must be a hashmap +According to the [Infinite Sharding Paradigm](#2-1-2-infinite-sharding-paradigm), any shardchain is just a (temporary) collection of virtual "account-chains", containing exactly one account each. This means that, essentially, the global shardchain state must be a hashmap $$ \mathit{ShardchainState}:=(\mathit{Account}\dashrightarrow\mathit{AccountState}) \tag{23} $$ -where all $\mathit{account\_id}$ appearing as indices of this hashmap must begin with prefix $s$, if we are discussing the state of shard $(w,s)$ (cf. [2.1.8](#2-1-8-identification-of-shardchains)). +where all $\mathit{account\_id}$ appearing as indices of this hashmap must begin with prefix $s$, if we are discussing the [state of shard](#2-1-8-identification-of-shardchains) $(w,s)$. In practice, we might want to split $\mathit{AccountState}$ into several parts (e.g., keep the account output message queue separate to simplify its examination by the neighboring shardchains), and have several hashmaps $(\mathit{Account}\dashrightarrow\mathit{AccountStatePart}_i)$ inside the $\mathit{ShardchainState}$. We might also add a small number of "global" or "integral" parameters to the $\mathit{ShardchainState}$, (e.g., the total balance of all accounts belonging to this shard, or the total number of messages in all output queues). -However, equation ([23](#2-5-1-shardchain-state-as-a-collection-of-account-chain-states)) is a good first approximation of what the shardchain global state looks like, at least from a "logical" ("high-level") perspective. The formal description of algebraic types $\mathit{AccountState}$ and $\mathit{ShardchainState}$ can be done with the aid of a TL-scheme (cf. [2.2.5](#2-2-5-tl-or-the-type-language)), to be provided elsewhere. +However, equation ([23](#2-5-1-shardchain-state-as-a-collection-of-account-chain-states)) is a good first approximation of what the shardchain global state looks like, at least from a "logical" ("high-level") perspective. The formal description of algebraic types $\mathit{AccountState}$ and $\mathit{ShardchainState}$ can be done with the aid of a [TL-scheme](#2-2-5-tl,-or-the-type-language), to be provided elsewhere. #### 2.5.2. Splitting and merging shardchain states @@ -771,7 +771,7 @@ Notice that the Infinite Sharding Paradigm description of the shardchain state a #### 2.5.3. Account-chain state -The (virtual) account-chain state is just the state of one account, described by type $\mathit{AccountState}$. Usually it has all or some of the fields listed in [2.3.20](#2-3-20-account-smart-contract-state), depending on the specific constructor used. +The (virtual) account-chain state is just the state of one account, described by type $\mathit{AccountState}$. Usually it has all or some of the [fields](#2-3-20-account%2Fsmart-contract-state), depending on the specific constructor used. #### 2.5.4. Global workchain state @@ -783,9 +783,9 @@ Essentially, the global workchain state *must* be given by the same type $\mathi There is a "low-level" description of the account-chain or shardchain state as well, complementary to the "high-level" description given above. This description is quite important, because it turns out to be pretty universal, providing a common basis for representing, storing, serializing and transferring by network almost all data used by the TON Blockchain (blocks, shardchain states, smart-contract storage, Merkle proofs, etc.). At the same time, such a universal "low-level" description, once understood and implemented, allows us to concentrate our attention on the "high-level" considerations only. -Recall that the TVM represents values of arbitrary algebraic types (including, for instance, $\mathit{ShardchainState}$ of the shardchain state equation) by means of a tree of *TVM cells*, or *cells* for short (cf. [2.3.14](#2-3-14-tvm-cells) and [2.2.5](#2-2-5-tl-or-the-type-language)). +Recall that the TVM represents values of arbitrary algebraic types (including, for instance, $\mathit{ShardchainState}$ of the shardchain state equation) by means of a tree of *TVM cells*, or *cells* for short ([2.3.14](#2-3-14-tvm-cells) and [2.2.5](#2-2-5-tl,-or-the-type-language)). -Any such cell consists of two *descriptor bytes*, defining certain flags and values $0\leq b\leq 128$, the quantity of raw bytes, and $0\leq c\leq 4$, the quantity of references to other cells. Then $b$ raw bytes and $c$ cell references follow.[18](#fn18) +Any such cell consists of two *descriptor bytes*, defining certain flags and values $0\leq b\leq 128$, the quantity of raw bytes, and $0\leq c\leq 4$, the quantity of references to other cells. Then $b$ raw bytes and $c$ cell references follow.[18](#fn18) The exact format of cell references depends on the implementation and on whether the cell is located in RAM, on disk, in a network packet, in a block, and so on. A useful abstract model consists in imagining that all cells are kept in content-addressable memory, with the address of a cell equal to its (SHA-256) hash. Recall that the (Merkle) hash of a cell is computed exactly by replacing the references to its child cells by their (recursively computed) hashes and hashing the resulting byte string. @@ -793,7 +793,7 @@ In this way, if we use cell hashes to reference cells (e.g., inside descriptions Now we see that *any object representable by TVM, the global shardchain state included, can be represented as a "bag of cells"*—i.e., *a collection of cells along with a "root" reference to one of them* (e.g., by hash). Notice that duplicate cells are removed from this description (the "bag of cells" is a set of cells, not a multiset of cells), so the abstract tree representation might actually become a directed acyclic graph (dag) representation. -One might even keep this state on disk in a $B$- or $B+$-tree, containing all cells in question (maybe with some additional data, such as subtree height or reference counter), indexed by cell hash. However, a naive implementation of this idea would result in the state of one smart contract being scattered among distant parts of the disk file, something we would rather avoid.[19](#fn19) +One might even keep this state on disk in a $B$- or $B+$-tree, containing all cells in question (maybe with some additional data, such as subtree height or reference counter), indexed by cell hash. However, a naive implementation of this idea would result in the state of one smart contract being scattered among distant parts of the disk file, something we would rather avoid.[19](#fn19) Now we are going to explain in some detail how almost all objects used by the TON Blockchain can be represented as "bags of cells", thus demonstrating the universality of this approach. @@ -807,31 +807,31 @@ Imagine that we have an old version of some object represented as a "bag of cell #### 2.5.8. Updates to the state of an account -In particular, updates to the state of an account, or to the global state of a shardchain, or to any hashmap can be represented using the idea described in [2.5.7](#2-5-7-update-to-an-object-as-a-bag-of-cells). This means that when we receive a new shardchain block (which is a "bag of cells"), we interpret this "bag of cells" not just by itself, but by uniting it first with the "bag of cells" representing the previous state of the shardchain. In this sense each block may "contain" the whole state of the blockchain. +In particular, [updates](#2-5-7-update-to-an-object-as-a-“bag-of-cells”) to the state of an account, or to the global state of a shardchain, or to any hashmap can be represented. This means that when we receive a new shardchain block (which is a "bag of cells"), we interpret this "bag of cells" not just by itself, but by uniting it first with the "bag of cells" representing the previous state of the shardchain. In this sense each block may "contain" the whole state of the blockchain. #### 2.5.9. Updates to a block -Recall that a block itself is a "bag of cells", so, if it becomes necessary to edit a block, one can similarly define a "block update" as a "bag of cells", interpreted in the presence of the "bag of cells" which is the previous version of this block. This is roughly the idea behind the "vertical blocks" discussed in [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks). +Recall that a block itself is a "bag of cells", so, if it becomes necessary to edit a block, one can similarly define a "block update" as a "bag of cells", interpreted in the presence of the "bag of cells" which is the previous version of this block. This is roughly the idea behind the [vertical blocks](#2-1-17-correcting-invalid-shardchain-blocks). #### 2.5.10. Merkle proof as a "bag of cells" -Notice that a (generalized) Merkle proof—for example, one asserting that $x[i]=y$ starting from a known value of $\texttt{Hash}(x)=h$ (cf. [2.3.10](#2-3-10-merkle-proofs) and [2.3.15](#2-3-15-generalized-merkle-proofs-for-values-of-arbitrary-algebraic-types))—may also be represented as a "bag of cells". Namely, one simply needs to provide a subset of cells corresponding to a path from the root of $x:\mathit{Hashmap}(n,X)$ to its desired leaf with index $i:\mathbf{2}^n$ and value $y:X$. References to children of these cells not lying on this path will be left "unresolved" in this proof, represented by cell hashes. One can also provide a simultaneous Merkle proof of, say, $x[i]=y$ and $x[i']=y'$, by including in the "bag of cells" the cells lying on the union of the two paths from the root of $x$ to leaves corresponding to indices $i$ and $i'$. +Notice that a (generalized) Merkle proof—for example, one asserting that $x[i]=y$ starting from a known value of $\texttt{Hash}(x)=h$ ([2.3.10](#2-3-10-merkle-proofs) and [2.3.15](#2-3-15-generalized-merkle-proofs-for-values-of-arbitrary-algebraic-types))—may also be represented as a "bag of cells". Namely, one simply needs to provide a subset of cells corresponding to a path from the root of $x:\mathit{Hashmap}(n,X)$ to its desired leaf with index $i:\mathbf{2}^n$ and value $y:X$. References to children of these cells not lying on this path will be left "unresolved" in this proof, represented by cell hashes. One can also provide a simultaneous Merkle proof of, say, $x[i]=y$ and $x[i']=y'$, by including in the "bag of cells" the cells lying on the union of the two paths from the root of $x$ to leaves corresponding to indices $i$ and $i'$. #### 2.5.11. Merkle proofs as query responses from full nodes -In essence, a full node with a complete copy of a shardchain (or account-chain) state can provide a Merkle proof when requested by a light node (e.g., a network node running a light version of the TON Blockchain client), enabling the receiver to perform some simple queries without external help, using only the cells provided in this Merkle proof. The light node can send its queries in a serialized format to the full node, and receive the correct answers with Merkle proofs—or just the Merkle proofs, because the requester should be able to compute the answers using only the cells included in the Merkle proof. This Merkle proof would consist simply of a "bag of cells", containing only those cells belonging to the shardchain's state that have been accessed by the full node while executing the light node's query. This approach can be used in particular for executing "get queries" of smart contracts (cf. [4.3.12](#4-3-12-tentative-execution-of-get-methods-of-smart-contracts)). +In essence, a full node with a complete copy of a shardchain (or account-chain) state can provide a Merkle proof when requested by a light node (e.g., a network node running a light version of the TON Blockchain client), enabling the receiver to perform some simple queries without external help, using only the cells provided in this Merkle proof. The light node can send its queries in a serialized format to the full node, and receive the correct answers with Merkle proofs—or just the Merkle proofs, because the requester should be able to compute the answers using only the cells included in the Merkle proof. This Merkle proof would consist simply of a "bag of cells", containing only those cells belonging to the shardchain's state that have been accessed by the full node while executing the light node's query. This approach can be used in particular for executing [get queries](#4-3-12-tentative-execution-of-get-methods-of-smart-contracts) of smart contracts. #### 2.5.12. Augmented update, or state update with Merkle proof of validity -Recall (cf. [2.5.7](#2-5-7-update-to-an-object-as-a-bag-of-cells)) that we can describe the changes in an object state from an old value $x:X$ to a new value $x':X$ by means of an "update", which is simply a "bag of cells", containing those cells that lie in the subtree representing new value $x'$, but not in the subtree representing old value $x$, because the receiver is assumed to have a copy of the old value $x$ and all its cells. +Recall that we can describe the [changes](#2-5-7-update-to-an-object-as-a-“bag-of-cells”) in an object state from an old value $x:X$ to a new value $x':X$ by means of an "update", which is simply a "bag of cells", containing those cells that lie in the subtree representing new value $x'$, but not in the subtree representing old value $x$, because the receiver is assumed to have a copy of the old value $x$ and all its cells. However, if the receiver does not have a full copy of $x$, but knows only its (Merkle) hash $h=\text{Hash}(x)$, it will not be able to check the validity of the update (i.e., that all "dangling" cell references in the update do refer to cells present in the tree of $x$). One would like to have "verifiable" updates, augmented by Merkle proofs of existence of all referred cells in the old state. Then anybody knowing only $h=\text{Hash}(x)$ would be able to check the validity of the update and compute the new $h'=\text{Hash}(x')$ by itself. -Because our Merkle proofs are "bags of cells" themselves (cf. [2.5.10](#2-5-10-merkle-proof-as-a-bag-of-cells)), one can construct such an *augmented update* as a "bag of cells", containing the old root of $x$, some of its descendants along with paths from the root of $x$ to them, and the new root of $x'$ and all its descendants that are not part of $x$. +Because our [Merkle proofs](#2-5-10-merkle-proof-as-a-“bag-of-cells”) are bags of cellsthemselves, one can construct such an *augmented update* as a "bag of cells", containing the old root of $x$, some of its descendants along with paths from the root of $x$ to them, and the new root of $x'$ and all its descendants that are not part of $x$. #### 2.5.13. Account state updates in a shardchain block -In particular, account state updates in a shardchain block should be augmented as discussed in [2.5.12](#2-5-12-augmented-update-or-state-update-with-merkle-proof-of-validity). Otherwise, somebody might commit a block containing an invalid state update, referring to a cell absent in the old state; proving the invalidity of such a block would be problematic (how is the challenger to prove that a cell is *not* part of the previous state?). +In particular, account state updates in a shardchain block should be [augmented](#2-5-12-augmented-update,-or-state-update-with-merkle-proof-of-validity). Otherwise, somebody might commit a block containing an invalid state update, referring to a cell absent in the old state; proving the invalidity of such a block would be problematic (how is the challenger to prove that a cell is *not* part of the previous state?). Now, if all state updates included in a block are augmented, their validity is easily checked, and their invalidity is also easily shown as a violation of the recursive defining property of (generalized) Merkle hashes. @@ -839,7 +839,7 @@ Now, if all state updates included in a block are augmented, their validity is e Previous considerations show that everything we need to store or transfer, either in the TON Blockchain or in the network, is representable as a "bag of cells". This is an important part of the TON Blockchain design philosophy. Once the "bag of cells" approach is explained and some "low-level" serializations of "bags of cells" are defined, one can simply define everything (block format, shardchain and account state, etc.) on the high level of abstract (dependent) algebraic data types. -The unifying effect of the "everything is a bag of cells" philosophy considerably simplifies the implementation of seemingly unrelated services; cf. [5.1.9](#5-1-9-ton-vm-support-for-smart-payment-channels) for an example involving payment channels. +The unifying effect of the "everything is a bag of cells" philosophy considerably simplifies the implementation of seemingly unrelated services; for an example involving [payment channels](#5-1-9-ton-vm-support-for-“smart”-payment-channels). #### 2.5.15. Block "headers" for TON blockchains @@ -857,7 +857,7 @@ The TON Blockchain ultimately consists of shardchain and masterchain blocks. The #### 2.6.1. Validators -New blocks are created and validated by special designated nodes, called *validators*. Essentially, any node wishing to become a validator may become one, provided it can deposit a sufficiently large stake (in TON coins, i.e., Grams; cf. [Appendix A](#appendix-a-the-ton-coin-or-the-gram)) into the masterchain. Validators obtain some "rewards" for good work, namely, the transaction, storage and gas fees from all transactions (messages) committed into newly generated blocks, and some newly minted coins, reflecting the "gratitude" of the whole community to the validators for keeping the TON Blockchain working. This income is distributed among all participating validators proportionally to their stakes. +New blocks are created and validated by special designated nodes, called *validators*. Essentially, any node wishing to become a validator may become one, provided it can deposit a sufficiently large stake (in TON coins, i.e., [Grams](#a-the-ton-coin,-or-the-gram)) into the masterchain. Validators obtain some "rewards" for good work, namely, the transaction, storage and gas fees from all transactions (messages) committed into newly generated blocks, and some newly minted coins, reflecting the "gratitude" of the whole community to the validators for keeping the TON Blockchain working. This income is distributed among all participating validators proportionally to their stakes. However, being a validator is a high responsibility. If a validator signs an invalid block, it can be punished by losing part or all of its stake, and by being temporarily or permanently excluded from the set of validators. If a validator does not participate in creating a block, it does not receive its share of the reward associated with that block. If a validator abstains from creating new blocks for a long time, it may lose part of its stake and be suspended or permanently excluded from the set of validators. @@ -885,7 +885,7 @@ On the other hand, this nominating or lending system enables one to become a val #### 2.6.4. Fishermen: obtaining money by pointing out others' mistakes -Another way to obtain some rewards without being a validator is by becoming a *fisherman*. Essentially, any node can become a fisherman by making a small deposit in the masterchain. Then it can use special masterchain transactions to publish (Merkle) invalidity proofs of some (usually shardchain) blocks previously signed and published by validators. If other validators agree with this invalidity proof, the offending validators are punished (by losing part of their stake), and the fisherman obtains some reward (a fraction of coins confiscated from the offending validators). Afterwards, the invalid (shardchain) block must be corrected as outlined in [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks). Correcting invalid masterchain blocks may involve creating "vertical" blocks on top of previously committed masterchain blocks (cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)); there is no need to create a fork of the masterchain. +Another way to obtain some rewards without being a validator is by becoming a *fisherman*. Essentially, any node can become a fisherman by making a small deposit in the masterchain. Then it can use special masterchain transactions to publish (Merkle) invalidity proofs of some (usually shardchain) blocks previously signed and published by validators. If other validators agree with this invalidity proof, the offending validators are punished (by losing part of their stake), and the fisherman obtains some reward (a fraction of coins confiscated from the offending validators). Afterwards, the [invalid shardchain block](#2-1-17-correcting-invalid-shardchain-blocks) must be corrected. Correcting invalid masterchain blocks may involve creating [vertical blocks](#2-1-17-correcting-invalid-shardchain-blocks) on top of previously committed masterchain blocks; there is no need to create a fork of the masterchain. Normally, a fisherman would need to become a full node for at least some shardchains, and spend some computing resources by running the code of at least some smart contracts. While a fisherman does not need to have as much computing power as a validator, we think that a natural candidate to become a fisherman is a would-be validator that is ready to process new blocks, but has not yet been elected as a validator (e.g., because of a failure to deposit a sufficiently large stake). @@ -907,25 +907,26 @@ The "global" set of validators is elected once each month (actually, every $2^{1 In order to become a validator, a node must transfer some TON coins (Grams) into the masterchain, and then send them to a special smart contract as its suggested stake $s$. Another parameter, sent along with the stake, is $l\geq 1$, the maximum validating load this node is willing to accept relative to the minimal possible. There is also a global upper bound (another configurable parameter) $L$ on $l$, equal to, say, 10. -Then the global set of validators is elected by this smart contract, simply by selecting up to $T$ candidates with maximal suggested stakes and publishing their identities. Originally, the total number of validators is $T=100$; we expect it to grow to 1000 as the load increases. It is a configurable parameter (cf. [2.1.21](#2-1-21-configurable-parameters)). +Then the global set of validators is elected by this smart contract, simply by selecting up to $T$ candidates with maximal suggested stakes and publishing their identities. Originally, the total number of validators is $T=100$; we expect it to grow to 1000 as the load increases. It is a [configurable parameter] +(#2-1-21-configurable-parameters). The actual stake of each validator is computed as follows: If the top $T$ proposed stakes are $s_1\geq s_2\geq\cdots\geq s_T$, the actual stake of $i$-th validator is set to $s'_i:=\min(s_i,l_i\cdot s_T)$. In this way, $s'_i/s'_T\leq l_i$, so the $i$-th validator does not obtain more than $l_i\leq L$ times the load of the weakest validator (because the load is ultimately proportional to the stake). Then elected validators may withdraw the unused part of their stake, $s_i-s'_i$. Unsuccessful validator candidates may withdraw all of their proposed stake. -Each validator publishes its *public signing key*, not necessarily equal to the public key of the account the stake came from.[20](#fn20) +Each validator publishes its *public signing key*, not necessarily equal to the public key of the account the stake came from.[20](#fn20) The stakes of the validators are frozen until the end of the period for which they have been elected, and one month more, in case new disputes arise (i.e., an invalid block signed by one of these validators is found). After that, the stake is returned, along with the validator's share of coins minted and fees from transactions processed during this time. #### 2.6.8. Election of validator "task groups" -The whole global set of validators (where each validator is considered present with multiplicity equal to its stake—otherwise a validator might be tempted to assume several identities and split its stake among them) is used only to validate new masterchain blocks. The shardchain blocks are validated only by specially selected subsets of validators, taken from the global set of validators chosen as described in [2.6.7](#2-6-7-global-validator-set-election). +The whole global set of validators (where each validator is considered present with multiplicity equal to its stake—otherwise a validator might be tempted to assume several identities and split its stake among them) is used only to validate new masterchain blocks. The shardchain blocks are validated only by specially selected subsets of validators, taken from the [global set of validators](#2-6-7-global-validator-set-election) chosen. These validator "subsets" or "task groups", defined for every shard, are rotated each hour (actually, every $2^{10}$ masterchain blocks), and they are known one hour in advance, so that every validator knows which shards it will need to validate, and can prepare for that (e.g., by downloading missing shardchain data). The algorithm used to select validator task groups for each shard $(w,s)$ is deterministic pseudorandom. It uses pseudorandom numbers embedded by validators into each masterchain block (generated by a consensus using threshold signatures) to create a random seed, and then computes for example $\text{Hash}(\text{CODE}(w).\text{CODE}(s).\mathit{validator\_id}.\mathit{rand\_seed})$ for each validator. Then validators are sorted by the value of this hash, and the first several are selected, so as to have at least $20/T$ of the total validator stakes and consist of at least 5 validators. -This selection could be done by a special smart contract. In that case, the selection algorithm would easily be upgradable without hard forks by the voting mechanism mentioned in [2.1.21](#2-1-21-configurable-parameters). All other "constants" mentioned so far (such as $2^{19}$, $2^{10}$, $T$, 20, and 5) are also configurable parameters. +This selection could be done by a special smart contract. In that case, the selection algorithm would easily be upgradable without hard forks by the [voting mechanism](#2-1-21-configurable-parameters). All other "constants" mentioned so far (such as $2^{19}$, $2^{10}$, $T$, 20, and 5) are also configurable parameters. #### 2.6.9. Rotating priority order on each task group @@ -935,11 +936,11 @@ When a new shardchain block needs to be generated, the shard task group validato #### 2.6.10. Propagation of shardchain block candidates -Because shardchain task group membership is known one hour in advance, their members can use that time to build a dedicated "shard validators multicast overlay network", using the general mechanisms of the TON Network (cf. [3.3](#3-3-overlay-networks-and-multicasting-messages)). When a new shardchain block needs to be generated—normally one or two seconds after the most recent masterchain block has been propagated—everybody knows who has the highest priority to generate the next block (cf. [2.6.9](#2-6-9-rotating-priority-order-on-each-task-group)). This validator will create a new collated block candidate, either by itself or with the aid of a collator (cf. [2.6.5](#2-6-5-collators-obtaining-money-by-suggesting-new-blocks-to-validators)). The validator must check (validate) this block candidate (especially if it has been prepared by some collator) and sign it with its (validator) private key. Then the block candidate is propagated to the remainder of the task group using the prearranged multicast overlay network (the task group creates its own private overlay network as explained in [3.3](#3-3-overlay-networks-and-multicasting-messages), and then uses a version of the streaming multicast protocol described in [3.3.15](#3-3-15-streaming-broadcast-protocol) to propagate block candidates). +Because shardchain task group membership is known one hour in advance, their members can use that time to build a dedicated shard validators [multicast overlay network](#3-3-overlay-networks-and-multicasting-messages), using the general mechanisms of the TON Network. When a new shardchain block needs to be generated—normally one or two seconds after the most recent masterchain block has been propagated—everybody knows who has the highest [priority](#2-6-9-rotating-priority-order-on-each-task-group) to generate the next block. This validator will create a new collated block candidate, either by itself or with the aid of a [collator](#2-6-5-collators:-obtaining-money-by-suggesting-new-blocks-to-validators). The validator must check (validate) this block candidate (especially if it has been prepared by some collator) and sign it with its (validator) private key. Then the block candidate is propagated to the remainder of the task group using the prearranged multicast overlay network (the task group creates its own [private overlay network](#3-3-overlay-networks-and-multicasting-messages), and then uses a version of the [streaming multicast protocol](#3-3-15-streaming-broadcast-protocol) to propagate block candidates). -A truly BFT way of doing this would be to use a Byzantine multicast protocol, such as the one used in Honey Badger BFT [[11]](#ref-11): encode the block candidate by an $(N,2N/3)$-erasure code, send $1/N$ of the resulting data directly to each member of the group, and expect them to multicast directly their part of the data to all other members of the group. +A truly BFT way of doing this would be to use a Byzantine multicast protocol, such as the one used in Honey Badger BFT[[11]](#references): encode the block candidate by an $(N,2N/3)$-erasure code, send $1/N$ of the resulting data directly to each member of the group, and expect them to multicast directly their part of the data to all other members of the group. -However, a faster and more straightforward way of doing this (cf. also [3.3.15](#3-3-15-streaming-broadcast-protocol)) is to split the block candidate into a sequence of signed one-kilobyte blocks ("chunks"), augment their sequence by a Reed–Solomon or a fountain code (such as the RaptorQ code [[9]](#ref-9) [[14]](#ref-14)), and start transmitting chunks to the neighbors in the "multicast mesh" (i.e., the overlay network), expecting them to propagate these chunks further. Once a validator obtains enough chunks to reconstruct the block candidate from them, it signs a confirmation receipt and propagates it through its neighbors to the whole of the group. Then its neighbors stop sending new chunks to it, but may continue to send the (original) signatures of these chunks, believing that this node can generate the subsequent chunks by applying the Reed–Solomon or fountain code by itself (having all data necessary), combine them with signatures, and propagate to its neighbors that are not yet ready. +However, a faster and more straightforward way of doing [this](#3-3-15-streaming-broadcast-protocol) is to split the block candidate into a sequence of signed one-kilobyte blocks ("chunks"), augment their sequence by a Reed–Solomon or a fountain code (such as the RaptorQ code [[9]](#references) [[14]](#references), and start transmitting chunks to the neighbors in the "multicast mesh" (i.e., the overlay network), expecting them to propagate these chunks further. Once a validator obtains enough chunks to reconstruct the block candidate from them, it signs a confirmation receipt and propagates it through its neighbors to the whole of the group. Then its neighbors stop sending new chunks to it, but may continue to send the (original) signatures of these chunks, believing that this node can generate the subsequent chunks by applying the Reed–Solomon or fountain code by itself (having all data necessary), combine them with signatures, and propagate to its neighbors that are not yet ready. If the "multicast mesh" (overlay network) remains connected after removing all "bad" nodes (recall that up to one-third of nodes are allowed to be bad in a Byzantine way, i.e., behave in arbitrary malicious fashion), this algorithm will propagate the block candidate as quickly as possible. @@ -949,7 +950,7 @@ Not only the designated high-priority block creator may multicast its block cand Once a block candidate is received by a validator and the signature of its originating validator is checked, the receiving validator checks the validity of this block candidate, by performing all transactions in it and checking that their result coincides with the one claimed. All messages imported from other blockchains must be supported by suitable Merkle proofs in the collated data, otherwise the block candidate is deemed invalid (and, if a proof of this is committed to the masterchain, the validators having already signed this block candidate may be punished). On the other hand, if the block candidate is found to be valid, the receiving validator signs it and propagates its signature to other validators in the group, either through the "mesh multicast network", or by direct network messages. -We would like to emphasize that *a validator does not need access to the states of this or neighboring shardchains in order to check the validity of a (collated) block candidate*.[21](#fn21) This allows the validation to proceed very quickly (without disk accesses), and lightens the computational and storage burden on the validators (especially if they are willing to accept the services of outside collators for creating block candidates). +We would like to emphasize that *a validator does not need access to the states of this or neighboring shardchains in order to check the validity of a (collated) block candidate*.[21](#fn21) This allows the validation to proceed very quickly (without disk accesses), and lightens the computational and storage burden on the validators (especially if they are willing to accept the services of outside collators for creating block candidates). #### 2.6.12. Election of the next block candidate @@ -965,7 +966,7 @@ Validators propagate the headers and signatures of newly-generated shardchain bl #### 2.6.15. Generation of new masterchain blocks -After all (or almost all) new shardchain blocks have been generated, a new masterchain block may be generated. The procedure is essentially the same as for shardchain blocks (cf. [2.6.12](#2-6-12-election-of-the-next-block-candidate)), with the difference that *all* validators (or at least two-thirds of them) must participate in this process. Because the headers and signatures of new shardchain blocks are propagated to all validators, hashes of the newest blocks in each shardchain can and must be included in the new masterchain block. Once these hashes are committed into the masterchain block, outside observers and other shardchains may consider the new shardchain blocks committed and immutable (cf. [2.1.13](#2-1-13-using-the-masterchain-to-make-workchains-and-shardchains-tightly-coupled)). +After all (or almost all) new shardchain blocks have been generated, a new masterchain block may be generated. The procedure is essentially the same as for [shardchain blocks](#2-6-12-election-of-the-next-block-candidate), with the difference that *all* validators (or at least two-thirds of them) must participate in this process. Because the headers and signatures of new shardchain blocks are propagated to all validators, hashes of the newest blocks in each shardchain can and must be included in the new masterchain block. Once these hashes are committed into the masterchain block, outside observers and other shardchains may consider the new shardchain blocks committed and [immutable](#2-1-13-using-the-masterchain-to-make-workchains-and-shardchains-tightly-coupled). #### 2.6.16. Validators must keep the state of masterchain @@ -997,17 +998,17 @@ However, if a validator fails to sign a block before it is committed, its signat Normally, when a validator signs a block, the signature testifies only to the *relative validity* of a block: this block is valid provided all previous blocks in this and other shardchains are valid. The validator cannot be punished for taking for granted invalid data committed into previous blocks. -However, the validator signature of a block has an integer parameter called "depth". If it is non-zero, it means that the validator asserts the (relative) validity of the specified number of previous blocks as well. This is a way for "slow" or "temporarily offline" validators to catch up and sign some of the blocks that have been committed without their signatures. Then some part of the block reward will still be given to them (cf. [2.6.20](#2-6-20-slow-validators-may-receive-lower-rewards)). +However, the validator signature of a block has an integer parameter called "depth". If it is non-zero, it means that the validator asserts the (relative) validity of the specified number of previous blocks as well. This is a way for "slow" or "temporarily offline" validators to catch up and sign some of the blocks that have been committed without their signatures. Then some part of the [block reward](#2-6-20-slow-validators-may-receive-lower-rewards) will still be given to them. #### 2.6.22. Validators are responsible for *relative* validity of signed shardchain blocks; absolute validity follows -We would like to emphasize once again that a validator's signature on a shardchain block $B$ testifies to only the *relative* validity of that block (or maybe of $d$ previous blocks as well, if the signature has "depth" $d$, cf. [2.6.21](#2-6-21-depth-of-validator-signatures); but this does not affect the following discussion much). In other words, the validator asserts that the next state $s'$ of the shardchain is obtained from the previous state $s$ by applying the block evaluation function $\mathit{ev\_block}$ described in [2.2.6](#2-2-6-blocks-and-transactions-as-state-transformation-operators): +We would like to emphasize once again that a validator's signature on a shardchain block $B$ testifies to only the *relative* validity of that block (or maybe of $d$ previous blocks as well, if the [signature](#2-6-21-“depth”-of-validator-signatures) has "depth" $d$; but this does not affect the following discussion much). In other words, the validator asserts that the next state $s'$ of the shardchain is obtained from the previous state $s$ by applying the block [evaluation function](#2-2-6-blocks-and-transactions-as-state-transformation-operators) $\mathit{ev\_block}$: $$ s' = \mathit{ev\_block}(B)(s) \tag{24} $$ -In this way, the validator that signed block $B$ cannot be punished if the original state $s$ turns out to be "incorrect" (e.g., because of the invalidity of one of the previous blocks). A fisherman (cf. [2.6.4](#2-6-4-fishermen-obtaining-money-by-pointing-out-others-mistakes)) should complain only if it finds a block that is *relatively* invalid. The PoS system as a whole endeavors to make every block *relatively* valid, not *recursively (or absolutely)* valid. Notice, however, that *if all blocks in a blockchain are relatively valid, then all of them and the blockchain as a whole are absolutely valid*; this statement is easily shown using mathematical induction on the length of the blockchain. In this way, easily verifiable assertions of *relative* validity of blocks together demonstrate the much stronger *absolute validity* of the whole blockchain. +In this way, the validator that signed block $B$ cannot be punished if the original state $s$ turns out to be "incorrect" (e.g., because of the invalidity of one of the previous blocks). A [fisherman](#2-6-4-fishermen%3A-obtaining-money-by-pointing-out-others’-mistakes) should complain only if it finds a block that is *relatively* invalid. The PoS system as a whole endeavors to make every block *relatively* valid, not *recursively (or absolutely)* valid. Notice, however, that *if all blocks in a blockchain are relatively valid, then all of them and the blockchain as a whole are absolutely valid*; this statement is easily shown using mathematical induction on the length of the blockchain. In this way, easily verifiable assertions of *relative* validity of blocks together demonstrate the much stronger *absolute validity* of the whole blockchain. Note that by signing a block $B$ the validator asserts that the block is valid given the original state $s$ (i.e., that the result of ([24](#2-6-22-validators-are-responsible-for-relative-validity-of-signed-shardchain-blocks%3B-absolute-validity-follows)) is not the value $\bot$ indicating that the next state cannot be computed). In this way, the validator must perform minimal formal checks of the cells of the original state that are accessed during the evaluation of the ([24](#2-6-22-validators-are-responsible-for-relative-validity-of-signed-shardchain-blocks%3B-absolute-validity-follows)). @@ -1019,7 +1020,7 @@ The situation with the masterchain blocks is somewhat different: by signing a ma #### 2.6.24. The total number of validators -The upper limit $T$ for the total number of validators to be elected (cf. [2.6.7](#2-6-7-global-validator-set-election)) cannot become, in the system described so far, more than, say, several hundred or a thousand, because all validators are expected to participate in a BFT consensus protocol to create each new masterchain block, and it is not clear whether such protocols can scale to thousands of participants. Even more importantly, masterchain blocks must collect the signatures of at least two-thirds of all the validators (by stake), and these signatures must be included in the new block (otherwise all other nodes in the system would have no reason to trust the new block without validating it by themselves). If more than, say, one thousand validator signatures would have to be included in each masterchain block, this would imply more data in each masterchain block, to be stored by all full nodes and propagated through the network, and more processing power spent to check these signatures (in a PoS system, full nodes do not need to validate blocks by themselves, but they need to check the validators' signatures instead). +The upper limit $T$ for the total number of validators to be [elected](#2-6-7-global-validator-set-election)) cannot become, in the system described so far, more than, say, several hundred or a thousand, because all validators are expected to participate in a BFT consensus protocol to create each new masterchain block, and it is not clear whether such protocols can scale to thousands of participants. Even more importantly, masterchain blocks must collect the signatures of at least two-thirds of all the validators (by stake), and these signatures must be included in the new block (otherwise all other nodes in the system would have no reason to trust the new block without validating it by themselves). If more than, say, one thousand validator signatures would have to be included in each masterchain block, this would imply more data in each masterchain block, to be stored by all full nodes and propagated through the network, and more processing power spent to check these signatures (in a PoS system, full nodes do not need to validate blocks by themselves, but they need to check the validators' signatures instead). While limiting $T$ to a thousand validators seems more than sufficient for the first phase of the deployment of the TON Blockchain, a provision must be made for future growth, when the total number of shardchains becomes so large that several hundred validators will not suffice to process all of them. To this end, we introduce an additional configurable parameter $T'\leq T$ (originally equal to $T$), and only the top $T'$ elected validators (by stake) are expected to create and sign new masterchain blocks. @@ -1031,25 +1032,25 @@ However, popular Proof-of-Work blockchains, such as Bitcoin and Ethereum, curren Therefore, as of 2017, more than 75% of new Ethereum or Bitcoin blocks are produced by less than ten miners. In fact, the two largest Ethereum mining pools produce together more than half of all new blocks! Clearly, such a system is much more centralized than one relying on $T\approx1000$ nodes to produce new blocks. -One might also note that the investment required to become a TON Blockchain validator—i.e., to buy the hardware (say, several high-performance servers) and the stake (which can be easily collected through a pool of nominators if necessary; cf. [2.6.3](#2-6-3-nominators-and-mining-pools))—is much lower than that required to become a successful stand-alone Bitcoin or Ethereum miner. In fact, the parameter $L$ of [2.6.7](#2-6-7-global-validator-set-election) will force nominators not to join the largest "mining pool" (i.e., the validator that has amassed the largest stake), but rather to look for smaller validators currently accepting funds from nominators, or even to create new validators, because this would allow a higher proportion $s'_i/s_i$ of the validator's—and by extension also the nominator's—stake to be used, hence yielding larger rewards from mining. In this way, the TON Proof-of-Stake system actually *encourages* decentralization (creating and using more validators) and *punishes* centralization. +One might also note that the investment required to become a TON Blockchain validator—i.e., to buy the hardware (say, several high-performance servers) and the stake (which can be easily collected through a [pool of nominators](#2-6-3-nominators-and-“mining-pools”) if necessary—is much lower than that required to become a successful stand-alone Bitcoin or Ethereum miner. In fact, the [parameter](#2-6-7-global-validator-set-election) $L$ will force nominators not to join the largest "mining pool" (i.e., the validator that has amassed the largest stake), but rather to look for smaller validators currently accepting funds from nominators, or even to create new validators, because this would allow a higher proportion $s'_i/s_i$ of the validator's—and by extension also the nominator's—stake to be used, hence yielding larger rewards from mining. In this way, the TON Proof-of-Stake system actually *encourages* decentralization (creating and using more validators) and *punishes* centralization. #### 2.6.26. Relative reliability of a block The *(relative) reliability* of a block is simply the total stake of all validators that have signed this block. In other words, this is the amount of money certain actors would lose if this block turns out to be invalid. If one is concerned with transactions transferring value lower than the reliability of the block, one can consider them to be safe enough. In this sense, the relative reliability is a measure of trust an outside observer can have in a particular block. -Note that we speak of the *relative* reliability of a block, because it is a guarantee that the block is valid *provided the previous block and all other shardchains' blocks referred to are valid* (cf. [2.6.22](#2-6-22-validators-are-responsible-for-relative-validity-of-signed-shardchain-blocks-absolute-validity-follows)). +Note that we speak of the [relative](#2-6-22-validators-are-responsible-for-relative-validity-of-signed-shardchain-blocks;-absolute-validity-follows) reliability of a block, because it is a guarantee that the block is valid *provided the previous block and all other shardchains' blocks referred to are valid*. -The relative reliability of a block can grow after it is committed—for example, when belated validators' signatures are added (cf. [2.6.21](#2-6-21-depth-of-validator-signatures)). On the other hand, if one of these validators loses part or all of its stake because of its misbehavior related to some other blocks, the relative reliability of a block may *decrease*. +The relative reliability of a block can grow after it is committed—for example, when belated [validators' signatures](#2-6-21-“depth”-of-validator-signatures) are added. On the other hand, if one of these validators loses part or all of its stake because of its misbehavior related to some other blocks, the relative reliability of a block may *decrease*. #### 2.6.27. "Strengthening" the blockchain -It is important to provide incentives for validators to increase the relative reliability of blocks as much as possible. One way of doing this is by allocating a small reward to validators for adding signatures to blocks of other shardchains. Even "would-be" validators, who have deposited a stake insufficient to get into the top $T$ validators by stake and to be included in the global set of validators (cf. [2.6.7](#2-6-7-global-validator-set-election)), might participate in this activity (if they agree to keep their stake frozen instead of withdrawing it after having lost the election). Such would-be validators might double as fishermen (cf. [2.6.4](#2-6-4-fishermen-obtaining-money-by-pointing-out-others-mistakes)): if they have to check the validity of certain blocks anyway, they might as well opt to report invalid blocks and collect the associated rewards. +It is important to provide incentives for validators to increase the relative reliability of blocks as much as possible. One way of doing this is by allocating a small reward to validators for adding signatures to blocks of other shardchains. Even "would-be" validators, who have deposited a stake insufficient to get into the top $T$ validators by stake and to be included in the [global set of validators](#2-6-7-global-validator-set-election), might participate in this activity (if they agree to keep their stake frozen instead of withdrawing it after having lost the election). Such would-be validators might double as [fishermen](#2-6-4-fishermen%3A-obtaining-money-by-pointing-out-others’-mistakes): if they have to check the validity of certain blocks anyway, they might as well opt to report invalid blocks and collect the associated rewards. #### 2.6.28. Recursive reliability of a block One can also define the *recursive reliability* of a block to be the minimum of its relative reliability and the recursive reliabilities of all blocks it refers to (i.e., the masterchain block, the previous shardchain block, and some blocks of neighboring shardchains). In other words, if the block turns out to be invalid, either because it is invalid by itself or because one of the blocks it depends on is invalid, then at least this amount of money would be lost by someone. If one is truly unsure whether to trust a specific transaction in a block, one should compute the *recursive* reliability of this block, not just the *relative* one. -It does not make sense to go too far back when computing recursive reliability, because, if we look too far back, we will see blocks signed by validators whose stakes have already been unfrozen and withdrawn. In any case, we do not allow the validators to automatically reconsider blocks that are that old (i.e., created more than two months ago, if current values of configurable parameters are used), and create forks starting from them or correct them with the aid of "vertical blockchains" (cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)), even if they turn out to be invalid. We assume that a period of two months provides ample opportunities for detecting and reporting any invalid blocks, so that if a block is not challenged during this period, it is unlikely to be challenged at all. +It does not make sense to go too far back when computing recursive reliability, because, if we look too far back, we will see blocks signed by validators whose stakes have already been unfrozen and withdrawn. In any case, we do not allow the validators to automatically reconsider blocks that are that old (i.e., created more than two months ago, if current values of configurable parameters are used), and create forks starting from them or correct them with the aid of [vertical blockchains](#2-1-17-correcting-invalid-shardchain-blocks), even if they turn out to be invalid. We assume that a period of two months provides ample opportunities for detecting and reporting any invalid blocks, so that if a block is not challenged during this period, it is unlikely to be challenged at all. #### 2.6.29. Consequence of Proof-of-Stake for light nodes @@ -1061,13 +1062,13 @@ Indeed, because the most recent shardchain block hashes are included in the mast ### 2.7 Splitting and Merging Shardchains -One of the most characteristic and unique features of the TON Blockchain is its ability to automatically split a shardchain in two when the load becomes too high, and merge them back if the load subsides (cf. [2.1.10](#2-1-10-dynamic-splitting-and-merging-of-shardchains)). We must discuss it in some detail because of its uniqueness and its importance to the scalability of the whole project. +One of the most characteristic and unique features of the TON Blockchain is its ability to automatically split a shardchain in two when the load becomes too high, and merge them back if the [load subsides](#2-1-10-dynamic-splitting-and-merging-of-shardchains;-cf-2-7). We must discuss it in some detail because of its uniqueness and its importance to the scalability of the whole project. #### 2.7.1. Shard configuration -Recall that, at any given moment of time, each workchain $w$ is split into one or several shardchains $(w,s)$ (cf. [2.1.8](#2-1-8-identification-of-shardchains)). These shardchains may be represented by leaves of a binary tree, with root $(w,\emptyset)$, and each non-leaf node $(w,s)$ having children $(w,s.0)$ and $(w,s.1)$. In this way, every account belonging to workchain $w$ is assigned to exactly one shard, and everybody who knows the current shardchain configuration can determine the shard $(w,s)$ containing account $\mathit{account\_id}$: it is the only shard with binary string $s$ being a prefix of $\mathit{account\_id}$. +Recall that, at any given moment of time, each workchain $w$ is split into one or several [shardchains](#2-1-8-identification-of-shardchains) $(w,s)$. These shardchains may be represented by leaves of a binary tree, with root $(w,\emptyset)$, and each non-leaf node $(w,s)$ having children $(w,s.0)$ and $(w,s.1)$. In this way, every account belonging to workchain $w$ is assigned to exactly one shard, and everybody who knows the current shardchain configuration can determine the shard $(w,s)$ containing account $\mathit{account\_id}$: it is the only shard with binary string $s$ being a prefix of $\mathit{account\_id}$. -The shard configuration—i.e., this *shard binary tree*, or the collection of all active $(w,s)$ for a given $w$ (corresponding to the leaves of the shard binary tree)—is part of the masterchain state and is available to everybody who keeps track of the masterchain.[22](#fn22) +The shard configuration—i.e., this *shard binary tree*, or the collection of all active $(w,s)$ for a given $w$ (corresponding to the leaves of the shard binary tree)—is part of the masterchain state and is available to everybody who keeps track of the masterchain.[22](#fn22) #### 2.7.2. Most recent shard configuration and state @@ -1077,11 +1078,11 @@ Recall that hashes of the most recent shardchain blocks are included in each mas The shard configuration may be changed in two ways: either a shard $(w,s)$ can be *split* into two shards $(w,s.0)$ and $(w,s.1)$, or two "sibling" shards $(w,s.0)$ and $(w,s.1)$ can be *merged* into one shard $(w,s)$. -These split/merge operations are announced several (e.g., $2^6$; this is a configurable parameter) blocks in advance, first in the "headers" of the corresponding shardchain blocks, and then in the masterchain block that refers to these shardchain blocks. This advance announcement is needed for all parties concerned to prepare for the planned change (e.g., build an overlay multicast network to distribute new blocks of the newly-created shardchains, as discussed in [3.3](#3-3-overlay-networks-and-multicasting-messages)). Then the change is committed, first into the (header of the) shardchain block (in case of a split; for a merge, blocks of both shardchains should commit the change), and then propagated to the masterchain block. In this way, the masterchain block defines not only the most recent shard configuration *before* its creation, but also the next immediate shard configuration. +These split/merge operations are announced several (e.g., $2^6$; this is a configurable parameter) blocks in advance, first in the "headers" of the corresponding shardchain blocks, and then in the masterchain block that refers to these shardchain blocks. This advance announcement is needed for all parties concerned to prepare for the planned change (e.g., build an [overlay multicast network](#3-3-overlay-networks-and-multicasting-messages) to distribute new blocks of the newly-created shardchains). Then the change is committed, first into the (header of the) shardchain block (in case of a split; for a merge, blocks of both shardchains should commit the change), and then propagated to the masterchain block. In this way, the masterchain block defines not only the most recent shard configuration *before* its creation, but also the next immediate shard configuration. #### 2.7.4. Validator task groups for new shardchains -Recall that each shard, i.e., each shardchain, normally is assigned a subset of validators (a validator task group) dedicated to creating and validating new blocks in the corresponding shardchain (cf. [2.6.8](#2-6-8-election-of-validator-task-groups)). These task groups are elected for some period of time (approximately one hour) and are known some time in advance (also approximately one hour), and are immutable during this period.[23](#fn23) +Recall that each shard, i.e., each shardchain, normally is assigned a [subset of validators](#2-6-8-election-of-validator-“task-groups”) (a validator task group) dedicated to creating and validating new blocks in the corresponding shardchain. These task groups are elected for some period of time (approximately one hour) and are known some time in advance (also approximately one hour), and are immutable during this period.[23](#fn23) However, the actual shard configuration may change during this period because of split/merge operations. One must assign task groups to newly created shards. This is done as follows: @@ -1107,13 +1108,13 @@ The split operation for a shardchain is triggered by certain formal conditions ( After the "split commit" flag is included in a block $B$ of shardchain $(w,s)$, there cannot be a subsequent block $B'$ in that shardchain. Instead, two blocks $B'_0$ and $B'_1$ of shardchains $(w,s.0)$ and $(w,s.1)$, respectively, will be created, both referring to block $B$ as their previous block (and both of them will indicate by a flag in the header that the shard has been just split). The next masterchain block will contain hashes of blocks $B'_0$ and $B'_1$ of the new shardchains; it is not allowed to contain the hash of a new block $B'$ of shardchain $(w,s)$, because a "split commit" event has already been committed into the previous masterchain block. -Notice that both new shardchains will be validated by the same validator task group as the old one, so they will automatically have a copy of their state. The state splitting operation itself is quite simple from the perspective of the Infinite Sharding Paradigm (cf. [2.5.2](#2-5-2-splitting-and-merging-shardchain-states)). +Notice that both new shardchains will be validated by the same validator task group as the old one, so they will automatically have a copy of their state. The state [splitting](#2-5-2-splitting-and-merging-shardchain-states) operation itself is quite simple from the perspective of the Infinite Sharding Paradigm. #### 2.7.8. Determining the necessity of merge operations The necessity of shard merge operations is also detected by certain formal conditions (e.g., if for 64 consecutive blocks the sum of the sizes of the two blocks of sibling shardchains does not exceed 60% of maximal block size). These formal conditions should also take into account the total gas spent by these blocks and compare it to the current block gas limit, otherwise the blocks may happen to be small because there are some computation-intensive transactions that prevent the inclusion of more transactions. -These conditions are monitored by validator task groups of both sibling shards $(w,s.0)$ and $(w,s.1)$. Notice that siblings are necessarily neighbors with respect to hypercube routing (cf. [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery)), so validators from the task group of any shard will be monitoring the sibling shard to some extent anyways. +These conditions are monitored by validator task groups of both sibling shards $(w,s.0)$ and $(w,s.1)$. Notice that siblings are necessarily neighbors with respect to [hypercube routing](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery), so validators from the task group of any shard will be monitoring the sibling shard to some extent anyways. When these conditions are met, either one of the validator subgroups can suggest to the other that they merge by sending a special message. Then they combine into a provisional "merged task group", with combined membership, capable of running BFT consensus algorithms and of propagating block updates and block candidates if necessary. @@ -1135,23 +1136,25 @@ As a first step, we suggest some classification criteria for blockchains (i.e., The list of criteria we consider is the following: -- Single-blockchain vs. multi-blockchain architecture (cf. [2.8.2](#2-8-2-single-blockchain-vs-multi-blockchain-projects)) -- Consensus algorithm: Proof-of-Stake vs. Proof-of-Work (cf. [2.8.3](#2-8-3-creating-and-validating-blocks-proof-of-work-vs-proof-of-stake)) -- For Proof-of-Stake systems, the exact block generation, validation and consensus algorithm used (the two principal options are DPOS vs. BFT; cf. [2.8.4](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft)) -- Support for "arbitrary" (Turing-complete) smart contracts (cf. [2.8.6](#2-8-6-support-for-turing-complete-code-in-transactions)) +The list of criteria we consider is the following: + +- [Single-blockchain vs. multi-blockchain](#2-8-2-single-blockchain-vs-multi-blockchain-projects) architecture +- Consensus algorithm: [Proof-of-Stake vs. Proof-of-Work](#2-8-3-creating-and-validating-blocks%3A-proof-of-work-vs-proof-of-stake) +- For Proof-of-Stake systems, the exact block generation, validation and consensus algorithm used (the two principal options are [DPOS vs. BFT](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft)) +- Support for ["arbitrary" (Turing-complete) smart contracts](#2-8-6-support-for-turing-complete-code-in-transactions%2C-i-e-%2C-essentially-arbitrary-smart-contracts) -Multi-blockchain systems have additional classification criteria (cf. [2.8.7](#2-8-7-classification-of-multichain-systems)): +Multi-blockchain systems have additional [classification criteria](#2-8-7-classification-of-multichain-systems): -- Type and rules of member blockchains: homogeneous, heterogeneous (cf. [2.8.8](#2-8-8-blockchain-types-homogeneous-and-heterogeneous-systems)), mixed (cf. [2.8.9](#2-8-9-mixed-heterogeneous-homogeneous-systems)). Confederations (cf. [2.8.10](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations)). -- Absence or presence of a *masterchain*, internal or external (cf. [2.8.11](#2-8-11-presence-of-a-masterchain-external-or-internal)) -- Native support for sharding (cf. [2.8.12](#2-8-12-sharding-support)). Static or dynamic sharding (cf. [2.8.13](#2-8-13-dynamic-and-static-sharding)). -- Interaction between member blockchains: loosely-coupled and tightly-coupled systems (cf. [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)) +- Type and rules of member blockchains: [homogeneous, heterogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems), [mixed](#2-8-9-mixed-heterogeneous-homogeneous-systems). [Confederations](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations). +- Absence or presence of a *masterchain*, [internal or external](#2-8-11-presence-of-a-masterchain%2C-external-or-internal) +- Native support for [sharding](#2-8-12-sharding-support). [Static or dynamic sharding](#2-8-13-dynamic-and-static-sharding). +- Interaction between member blockchains: [loosely-coupled and tightly-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems) systems #### 2.8.2. Single-blockchain vs. multi-blockchain projects The first classification criterion is the quantity of blockchains in the system. The oldest and simplest projects consist of a *single blockchain* ("singlechain projects" for short); more sophisticated projects use (or, rather, plan to use) *multiple blockchains* ("multichain projects"). -Singlechain projects are generally simpler and better tested; they have withstood the test of time. Their main drawback is low performance, or at least transaction throughput, which is on the level of ten (Bitcoin) to less than one hundred[24](#fn24) (Ethereum) transactions per second for general-purpose systems. Some specialized systems (such as Bitshares) are capable of processing tens of thousands of specialized transactions per second, at the expense of requiring the blockchain state to fit into memory, and limiting the processing to a predefined special set of transactions, which are then executed by highly-optimized code written in languages like C++ (no VMs here). +Singlechain projects are generally simpler and better tested; they have withstood the test of time. Their main drawback is low performance, or at least transaction throughput, which is on the level of ten (Bitcoin) to less than one hundred[24](#fn24)(Ethereum) transactions per second for general-purpose systems. Some specialized systems (such as Bitshares) are capable of processing tens of thousands of specialized transactions per second, at the expense of requiring the blockchain state to fit into memory, and limiting the processing to a predefined special set of transactions, which are then executed by highly-optimized code written in languages like C++ (no VMs here). Multichain projects promise the scalability everybody craves. They may support larger total states and more transactions per second, at the expense of making the project much more complex, and its implementation more challenging. As a result, there are few multichain projects already running, but most proposed projects are multichain. We believe that the future belongs to multichain projects. @@ -1191,17 +1194,17 @@ As to the two last questions, their answers turn out to be highly correlated, le The BFT approach has the advantage that a newly-produced block has *from the very beginning* the signatures of a majority of validators testifying to its validity. Another advantage is that, if a majority of validators executes the BFT consensus protocol correctly, no forks can appear at all. On the other hand, BFT algorithms tend to be quite convoluted and require more time for the subset of validators to reach consensus. Therefore, blocks cannot be generated too often. This is why we expect the TON Blockchain (which is a BFT project from the perspective of this classification) to produce a block only once every five seconds. In practice, this interval might be decreased to 2–3 seconds (though we do not promise this), but not further, if validators are spread across the globe. -The DPOS algorithm has the advantage of being quite simple and straightforward. It can generate new blocks quite often—say, once every two seconds, or maybe even once every second,[25](#fn25) because of its reliance on designated block producers known in advance. +The DPOS algorithm has the advantage of being quite simple and straightforward. It can generate new blocks quite often—say, once every two seconds, or maybe even once every second,[25](#fn25) because of its reliance on designated block producers known in advance. However, DPOS requires all nodes—or at least all validators—to validate all blocks received, because a validator producing and signing a new block confirms not only the *relative* validity of this block, but also the validity of the previous block it refers to, and all the blocks further back in the chain (maybe up to the beginning of the period of responsibility of the current subset of validators). There is a predetermined order on the current subset of validators, so that for each block there is a designated producer (i.e., validator expected to generate that block); these designated producers are rotated in a round-robin fashion. In this way, a block is at first signed only by its producing validator; then, when the next block is mined, and its producer chooses to refer to this block and not to one of its predecessors (otherwise its block would lie in a shorter chain, which might lose the "longest fork" competition in the future), the signature of the next block is essentially an additional signature on the previous block as well. In this way, a new block gradually collects the signatures of more validators—say, twenty signatures in the time needed to generate the next twenty blocks. A full node will either need to wait for these twenty signatures, or validate the block by itself, starting from a sufficiently confirmed block (say, twenty blocks back), which might be not so easy. -The obvious disadvantage of the DPOS algorithm is that a new block (and transactions committed into it) achieves the same level of trust ("recursive reliability" as discussed in [2.6.28](#2-6-28-recursive-reliability-of-a-block)) only after twenty more blocks are mined, compared to the BFT algorithms, which deliver this level of trust (say, twenty signatures) immediately. Another disadvantage is that DPOS uses the "longest fork wins" approach for switching to other forks; this makes forks quite probable if at least some producers fail to produce subsequent blocks after the one we are interested in (or we fail to observe these blocks because of a network partition or a sophisticated attack). +The obvious disadvantage of the DPOS algorithm is that a new block (and transactions committed into it) achieves the same level of trust ([recursive reliability](#2-6-28-recursive-reliability-of-a-block)) only after twenty more blocks are mined, compared to the BFT algorithms, which deliver this level of trust (say, twenty signatures) immediately. Another disadvantage is that DPOS uses the "longest fork wins" approach for switching to other forks; this makes forks quite probable if at least some producers fail to produce subsequent blocks after the one we are interested in (or we fail to observe these blocks because of a network partition or a sophisticated attack). -We believe that the BFT approach, while more sophisticated to implement and requiring longer time intervals between blocks than DPOS, is better adapted to "tightly-coupled" (cf. [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)) multichain systems, because other blockchains can start acting almost immediately after seeing a committed transaction (e.g., generating a message intended for them) in a new block, without waiting for twenty confirmations of validity (i.e., the next twenty blocks), or waiting for the next six blocks to be sure that no forks appear and verifying the new block by themselves (verifying blocks of other blockchains may become prohibitive in a scalable multi-chain system). Thus they can achieve scalability while preserving high reliability and availability (cf. [2.8.12](#2-8-12-sharding-support)). +We believe that the BFT approach, while more sophisticated to implement and requiring longer time intervals between blocks than DPOS, is better adapted to [tightly-coupled](#2-8-14-interaction-between-blockchains:-loosely-coupled-and-tightly-coupled-systems) multichain systems, because other blockchains can start acting almost immediately after seeing a committed transaction (e.g., generating a message intended for them) in a new block, without waiting for twenty confirmations of validity (i.e., the next twenty blocks), or waiting for the next six blocks to be sure that no forks appear and verifying the new block by themselves (verifying blocks of other blockchains may become prohibitive in a scalable multi-chain system). Thus they can achieve [scalability](#2-8-12-sharding-support) while preserving high reliability and availability. On the other hand, DPOS might be a good choice for a "loosely-coupled" multi-chain system, where fast interaction between blockchains is not required—e.g., if each blockchain ("workchain") represents a separate distributed exchange, and inter-blockchain interaction is limited to rare transfers of tokens from one workchain into another (or, rather, trading one altcoin residing in one workchain for another at a rate approaching $1:1$). This is what is actually done in the BitShares project, which uses DPOS quite successfully. -To summarize, while DPOS can *generate* new blocks and *include transactions* into them *faster* (with smaller intervals between blocks), these transactions reach the level of trust required to use them in other blockchains and off-chain applications as "committed" and "immutable" *much more slowly* than in the BFT systems—say, in thirty seconds[26](#fn26) instead of five. Faster transaction *inclusion* does not mean faster transaction *commitment*. This could become a huge problem if fast inter-blockchain interaction is required. In that case, one must abandon DPOS and opt for BFT PoS instead. +To summarize, while DPOS can *generate* new blocks and *include transactions* into them *faster* (with smaller intervals between blocks), these transactions reach the level of trust required to use them in other blockchains and off-chain applications as "committed" and "immutable" *much more slowly* than in the BFT systems—say, in thirty seconds[26](#fn26)instead of five. Faster transaction *inclusion* does not mean faster transaction *commitment*. This could become a huge problem if fast inter-blockchain interaction is required. In that case, one must abandon DPOS and opt for BFT PoS instead. #### 2.8.6. Support for Turing-complete code in transactions, i.e., essentially arbitrary smart contracts @@ -1211,7 +1214,7 @@ Of course, support for arbitrary smart contracts makes the system truly flexible Ultimately, support for Turing-complete smart contracts seems to be desirable in any general-purpose blockchain project; otherwise, the designers of the blockchain project must decide in advance which applications their blockchain will be used for. In fact, the lack of support for smart contracts in the Bitcoin blockchain was the principal reason why a new blockchain project, Ethereum, had to be created. -In a (heterogeneous; cf. [2.8.8](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems)) multi-chain system, one might have "the best of both worlds" by supporting Turing-complete smart contracts in some blockchains (i.e., workchains), and a small predefined set of highly-optimized transactions in others. +In a [heterogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems) multi-chain system, one might have "the best of both worlds" by supporting Turing-complete smart contracts in some blockchains (i.e., workchains), and a small predefined set of highly-optimized transactions in others. #### 2.8.7. Classification of multichain systems @@ -1239,17 +1242,17 @@ In some cases, the masterchain is *external*, meaning that it is not a part of t #### 2.8.12. Sharding support -Some blockchain projects (or systems) have native support for *sharding*, meaning that several (necessarily homogeneous; cf. [2.8.8](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems)) blockchains are thought of as *shards* of a single (from a high-level perspective) virtual blockchain. For example, one can create 256 shard blockchains ("shardchains") with the same rules, and keep the state of an account in exactly one shard selected depending on the first byte of its $\mathit{account\_id}$. +Some blockchain projects (or systems) have native support for *sharding*, meaning that several (necessarily [homogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems)) blockchains are thought of as *shards* of a single (from a high-level perspective) virtual blockchain. For example, one can create 256 shard blockchains ("shardchains") with the same rules, and keep the state of an account in exactly one shard selected depending on the first byte of its $\mathit{account\_id}$. Sharding is a natural approach to scaling blockchain systems, because, if it is properly implemented, users and smart contracts in the system need not be aware of the existence of sharding at all. In fact, one often wants to add sharding to an existing single-chain project (such as Ethereum) when the load becomes too high. -An alternative approach to scaling would be to use a "confederation" of heterogeneous workchains as described in [2.8.10](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations), allowing each user to keep her account in one or several workchains of her choice, and transfer funds from her account in one workchain to another workchain when necessary, essentially performing a $1:1$ altcoin exchange operation. The drawbacks of this approach have already been discussed in [2.8.10](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations). +An alternative approach to scaling would be to use a "confederation" of [heterogeneous](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations) workchains, allowing each user to keep her account in one or several workchains of her choice, and transfer funds from her account in one workchain to another workchain when necessary, essentially performing a $1:1$ altcoin exchange operation. The drawbacks of this approach have already been discussed in [2.8.10](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations). -However, sharding is not so easy to implement in a fast and reliable fashion, because it implies a lot of messages between different shardchains. For example, if accounts are evenly distributed between $N$ shards, and the only transactions are simple fund transfers from one account to another, then only a small fraction ($1/N$) of all transactions will be performed within a single blockchain; almost all ($1-1/N$) transactions will involve two blockchains, requiring inter-blockchain communication. If we want these transactions to be fast, we need a fast system for transferring messages between shardchains. In other words, the blockchain project needs to be "tightly-coupled" in the sense described in [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems). +However, sharding is not so easy to implement in a fast and reliable fashion, because it implies a lot of messages between different shardchains. For example, if accounts are evenly distributed between $N$ shards, and the only transactions are simple fund transfers from one account to another, then only a small fraction ($1/N$) of all transactions will be performed within a single blockchain; almost all ($1-1/N$) transactions will involve two blockchains, requiring inter-blockchain communication. If we want these transactions to be fast, we need a fast system for transferring messages between shardchains. In other words, the blockchain project needs to be [tightly-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems). #### 2.8.13. Dynamic and static sharding -Sharding might be *dynamic* (if additional shards are automatically created when necessary) or *static* (when there is a predefined number of shards, which is changeable only through a hard fork at best). Most sharding proposals are static; the TON Blockchain uses dynamic sharding (cf. [2.7](#2-7-splitting-and-merging-shardchains)). +Sharding might be *dynamic* (if additional shards are automatically created when necessary) or *static* (when there is a predefined number of shards, which is changeable only through a hard fork at best). Most sharding proposals are static; the TON Blockchain uses [dynamic sharding](#2-7-splitting-and-merging-shardchains). #### 2.8.14. Interaction between blockchains: loosely-coupled and tightly-coupled systems @@ -1263,9 +1266,9 @@ If one does not wait long enough before transferring the message, or if a fork h Sometimes partial support for messaging is added, by standardizing the format of messages and the location of input and output message queues in the blocks of all workchains (this is especially useful in heterogeneous systems). While this facilitates messaging to a certain extent, it is conceptually not too different from the previous case, so such systems are still "loosely-coupled". -By contrast, "tightly-coupled" systems include special mechanisms to provide fast messaging between all blockchains. The desired behavior is to be able to deliver a message into another workchain immediately after it has been generated in a block of the originating blockchain. On the other hand, "tightly-coupled" systems are also expected to maintain overall consistency in the case of forks. While these two requirements appear to be contradictory at first glance, we believe that the mechanisms used by the TON Blockchain (the inclusion of shardchain block hashes into masterchain blocks; the use of "vertical" blockchains for fixing invalid blocks, cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks); hypercube routing, cf. [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery); Instant Hypercube Routing, cf. [2.4.20](#2-4-20-instant-hypercube-routing%3A-“fast-path”-for-messages)) enable it to be a "tightly-coupled" system, perhaps the only one so far. +By contrast, "tightly-coupled" systems include special mechanisms to provide fast messaging between all blockchains. The desired behavior is to be able to deliver a message into another workchain immediately after it has been generated in a block of the originating blockchain. On the other hand, "tightly-coupled" systems are also expected to maintain overall consistency in the case of forks. While these two requirements appear to be contradictory at first glance, we believe that the mechanisms used by the TON Blockchain (the inclusion of shardchain block hashes into masterchain blocks; the use of "vertical" blockchains for fixing [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks); [2.4.19](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery); [2.4.20](#2-4-20-instant-hypercube-routing%3A-“fast-path”-for-messages)), enable it to be a "tightly-coupled" system, perhaps the only one so far. -Of course, building a "loosely-coupled" system is much simpler; however, fast and efficient sharding (cf. [2.8.12](#2-8-12-sharding-support)) requires the system to be "tightly-coupled". +Of course, building a "loosely-coupled" system is much simpler; however, fast and efficient [sharding](#2-8-12-sharding-support) requires the system to be "tightly-coupled". #### 2.8.15. Simplified classification. Generations of blockchain projects @@ -1282,13 +1285,13 @@ While not all blockchain projects fall precisely into one of these categories, m #### 2.8.16. Complications of changing the "genome" of a blockchain project -The above classification defines the "genome" of a blockchain project. This genome is quite "rigid": it is almost impossible to change it once the project is deployed and is used by a lot of people. One would need a series of hard forks (which would require the approval of the majority of the community), and even then the changes would need to be very conservative in order to preserve backward compatibility (e.g., changing the semantics of the virtual machine might break existing smart contracts). An alternative would be to create new "sidechains" with their different rules, and bind them somehow to the blockchain (or the blockchains) of the original project. One might use the blockchain of the existing single-blockchain project as an external masterchain for an essentially new and separate project.[27](#fn27) +The above classification defines the "genome" of a blockchain project. This genome is quite "rigid": it is almost impossible to change it once the project is deployed and is used by a lot of people. One would need a series of hard forks (which would require the approval of the majority of the community), and even then the changes would need to be very conservative in order to preserve backward compatibility (e.g., changing the semantics of the virtual machine might break existing smart contracts). An alternative would be to create new "sidechains" with their different rules, and bind them somehow to the blockchain (or the blockchains) of the original project. One might use the blockchain of the existing single-blockchain project as an external masterchain for an essentially new and separate project.[27](#fn27) -Our conclusion is that the genome of a project is very hard to change once it has been deployed. Even starting with PoW and planning to replace it with PoS in the future is quite complicated.[28](#fn28) Adding shards to a project originally designed without support for them seems almost impossible.[29](#fn29) In fact, adding support for smart contracts into a project (namely, Bitcoin) originally designed without support for such features has been deemed impossible (or at least undesirable by the majority of the Bitcoin community) and eventually led to the creation of a new blockchain project, Ethereum. +Our conclusion is that the genome of a project is very hard to change once it has been deployed. Even starting with PoW and planning to replace it with PoS in the future is quite complicated.[28](#fn28) Adding shards to a project originally designed without support for them seems almost impossible.[29](#fn29)In fact, adding support for smart contracts into a project (namely, Bitcoin) originally designed without support for such features has been deemed impossible (or at least undesirable by the majority of the Bitcoin community) and eventually led to the creation of a new blockchain project, Ethereum. #### 2.8.17. Genome of the TON Blockchain -Therefore, if one wants to build a scalable blockchain system, one must choose its genome carefully from the very beginning. If the system is meant to support some additional specific functionality in the future not known at the time of its deployment, it should support "heterogeneous" workchains (having potentially different rules) from the start. For the system to be truly scalable, it must support sharding from the very beginning; sharding makes sense only if the system is "tightly-coupled" (cf. [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)), so this in turn implies the existence of a masterchain, a fast system of inter-blockchain messaging, usage of BFT PoS, and so on. +Therefore, if one wants to build a scalable blockchain system, one must choose its genome carefully from the very beginning. If the system is meant to support some additional specific functionality in the future not known at the time of its deployment, it should support "heterogeneous" workchains (having potentially different rules) from the start. For the system to be truly scalable, it must support sharding from the very beginning; sharding makes sense only if the system is [tightly-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems), so this in turn implies the existence of a masterchain, a fast system of inter-blockchain messaging, usage of BFT PoS, and so on. When one takes into account all these implications, most of the design choices made for the TON Blockchain project appear natural, and almost the only ones possible. @@ -1296,7 +1299,7 @@ When one takes into account all these implications, most of the design choices m ### 2.9 Comparison to Other Blockchain Projects -We conclude our brief discussion of the TON Blockchain and its most important and unique features by trying to find a place for it on a map containing existing and proposed blockchain projects. We use the classification criteria described in [2.8](#2-8-classification-of-blockchain-projects) to discuss different blockchain projects in a uniform way and construct such a "map of blockchain projects". We represent this map as [Table 1](#table-1), and then briefly discuss a few projects separately to point out their peculiarities that may not fit into the general scheme. +We conclude our brief discussion of the TON Blockchain and its most important and unique features by trying to find a place for it on a map containing existing and proposed blockchain projects. We use the [classification](#2-8-classification-of-blockchain-projects) criteria to discuss different blockchain projects in a uniform way and construct such a "map of blockchain projects". We represent this map as [Table 1](#table-1), and then briefly discuss a few projects separately to point out their peculiarities that may not fit into the general scheme.
@@ -1313,13 +1316,13 @@ We conclude our brief discussion of the TON Blockchain and its most important an | Cosmos | 2017, ? | 4 | PoS BFT | yes | m | ht. | no | L | | TON | 2017, (2018) | 5 | PoS BFT | yes | m | mix | dyn. | T | -**Table 1:** A summary of some notable blockchain projects. The columns are: *Project* — project name; *Year* — year announced and year deployed; *G.* — generation (cf. [2.8.15](#2-8-15-simplified-classification-generations-of-blockchain-projects)); *Cons.* — consensus algorithm (cf. [2.8.3](#2-8-3-creating-and-validating-blocks-proof-of-work-vs-proof-of-stake) and [2.8.4](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft)); *Sm.* — support for arbitrary code (smart contracts; cf. [2.8.6](#2-8-6-support-for-turing-complete-code-in-transactions%2C-i-e-%2C-essentially-arbitrary-smart-contracts)); *Ch.* — single/multiple blockchain system (cf. [2.8.2](#2-8-2-single-blockchain-vs-multi-blockchain-projects)); *R.* — heterogeneous/homogeneous multichain systems (cf. [2.8.8](#2-8-8-blockchain-types-homogeneous-and-heterogeneous-systems)); *Sh.* — sharding support (cf. [2.8.12](#2-8-12-sharding-support)); *Int.* — interaction between blockchains, (L)oose or (T)ight (cf. [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)). +**Table 1:** A summary of some notable blockchain projects. The columns are: *Project* — project name; *Year* — year announced and year deployed; *G.* — [generation](#2-8-15-simplified-classification-generations-of-blockchain-projects); *Cons.* — consensus algorithm ([2.8.3](#2-8-3-creating-and-validating-blocks%3A-proof-of-work-vs-proof-of-stake) and [2.8.4](#2-8-4-variants-of-proof-of-stake-dpos-vs-bft)); *Sm.* — support for [arbitrary code](#2-8-6-support-for-turing-complete-code-in-transactions%2C-i-e-%2C-essentially-arbitrary-smart-contracts) (smart contracts); *Ch.* — [single/multiple blockchain](#2-8-2-single-blockchain-vs-multi-blockchain-projects) system; *R.* — [heterogeneous/homogeneous](#2-8-8-blockchain-types%3A-homogeneous-and-heterogeneous-systems) multichain systems; *Sh.* — [sharding support](#2-8-12-sharding-support); *Int.* — interaction between blockchains, [(L)oose or (T)ight](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems). -#### 2.9.1. Bitcoin [[12]](#ref-12); https://bitcoin.org/ +#### 2.9.1. Bitcoin[[12]](#references); https://bitcoin.org/ *Bitcoin* (2009) is the first and the most famous blockchain project. It is a typical *first-generation* blockchain project: it is single-chain, it uses Proof-of-Work with a "longest-fork-wins" fork selection algorithm, and it does not have a Turing-complete scripting language (however, simple scripts without loops are supported). The Bitcoin blockchain has no notion of an account; it uses the UTXO (Unspent Transaction Output) model instead. -#### 2.9.2. Ethereum [[2]](#ref-2); https://ethereum.org/ +#### 2.9.2. Ethereum[[2]](#references); https://ethereum.org/ *Ethereum* (2015) is the first blockchain with support for Turing-complete smart contracts. As such, it is a typical *second-generation* project, and the most popular among them. It uses Proof-of-Work on a single blockchain, but has smart contracts and accounts. @@ -1329,29 +1332,29 @@ We conclude our brief discussion of the TON Blockchain and its most important an #### 2.9.4. Tezos; https://www.tezos.com/ -*Tezos* (2018 or later) is a proposed PoS-based single-blockchain project. We mention it here because of its unique feature: its block interpretation function $\mathit{ev\_block}$ (cf. [2.2.6](#2-2-6-blocks-and-transactions-as-state-transformation-operators)) is not fixed, but is determined by an OCaml module, which can be upgraded by committing a new version into the blockchain (and collecting some votes for the proposed change). In this way, one will be able to create custom single-chain projects by first deploying a "vanilla" Tezos blockchain, and then gradually changing the block interpretation function in the desired direction, without any need for hard forks. +*Tezos* (2018 or later) is a proposed PoS-based single-blockchain project. We mention it here because of its unique feature: its block [interpretation function](#2-2-6-blocks-and-transactions-as-state-transformation-operators) $\mathit{ev\_block}$ is not fixed, but is determined by an OCaml module, which can be upgraded by committing a new version into the blockchain (and collecting some votes for the proposed change). In this way, one will be able to create custom single-chain projects by first deploying a "vanilla" Tezos blockchain, and then gradually changing the block interpretation function in the desired direction, without any need for hard forks. This idea, while intriguing, has the obvious drawback that it forbids any optimized implementations in other languages like C++, so a Tezos-based blockchain is destined to have lower performance. We think that a similar result might have been obtained by publishing a formal *specification* of the proposed block interpretation function $\mathit{ev\_trans}$, without fixing a particular *implementation*. -#### 2.9.5. Casper[30](#fn30) +#### 2.9.5. Casper[30](#fn30) *Casper* is an upcoming PoS algorithm for Ethereum; its gradual deployment in 2017 (or 2018), if successful, will change Ethereum into a single-chain PoS or mixed PoW+PoS system with smart contract support, transforming Ethereum into a *third-generation* project. -#### 2.9.6. BitShares [[8]](#ref-8); https://bitshares.org +#### 2.9.6. BitShares[[8]](#references); https://bitshares.org *BitShares* (2014) is a platform for distributed blockchain-based exchanges. It is a heterogeneous multi-blockchain DPoS system without smart contracts; it achieves its high performance by allowing only a small set of predefined specialized transaction types, which can be efficiently implemented in C++, assuming the blockchain state fits into memory. It is also the first blockchain project to use Delegated Proof-of-Stake (DPoS), demonstrating its viability at least for some specialized purposes. -#### 2.9.7. EOS [[5]](#ref-5); https://eos.io +#### 2.9.7. EOS[[5]](#references); https://eos.io -*EOS* (2018 or later) is a proposed heterogeneous multi-blockchain DPoS system *with* smart contract support and with some minimal support for messaging (still loosely-coupled in the sense described in [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)). It is an attempt by the same team that has previously successfully created the BitShares and SteemIt projects, demonstrating the strong points of the DPoS consensus algorithm. Scalability will be achieved by creating specialized workchains for projects that need it (e.g., a distributed exchange might use a workchain supporting a special set of optimized transactions, similarly to what BitShares did) and by creating multiple workchains with the same rules (*confederations* in the sense described in [2.8.10](#2-8-11-heterogeneous-systems-with-several-workchains-having-the-same-rules-or-confederations)). The drawbacks and limitations of this approach to scalability have been discussed in *loc. cit.* Cf. also [2.8.5](#2-8-5-comparison-of-dpos-and-bft-pos), [2.8.12](#2-8-13-sharding-support), and [2.8.14](#2-8-15-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems) for a more detailed discussion of DPoS, sharding, interaction between workchains and their implications for the scalability of a blockchain system. +*EOS* (2018 or later) is a proposed heterogeneous multi-blockchain DPoS system *with* smart contract support and with some minimal support for messaging (still [loosely-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems)). It is an attempt by the same team that has previously successfully created the BitShares and SteemIt projects, demonstrating the strong points of the DPoS consensus algorithm. Scalability will be achieved by creating specialized workchains for projects that need it (e.g., a distributed exchange might use a workchain supporting a special set of optimized transactions, similarly to what BitShares did) and by creating multiple workchains with the same rules [confederations](#2-8-10-heterogeneous-systems-with-several-workchains-having-the-same-rules%2C-or-confederations)). The drawbacks and limitations of this approach to scalability have been discussed in *loc. cit.* also [2.8.5](#2-8-5-comparison-of-dpos-and-bft-pos), [2.8.12](#2-8-12-sharding-support), and [2.8.14](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems) for a more detailed discussion of DPoS, sharding, interaction between workchains and their implications for the scalability of a blockchain system. -At the same time, even if one will not be able to "create a Facebook inside a blockchain" (cf. [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain)), EOS or otherwise, we think that EOS might become a convenient platform for some highly-specialized weakly interacting distributed applications, similar to BitShares (decentralized exchange) and SteemIt (decentralized blog platform). +At the same time, even if one will not be able to "create a [Facebook](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”) inside a blockchain", EOS or otherwise, we think that EOS might become a convenient platform for some highly-specialized weakly interacting distributed applications, similar to BitShares (decentralized exchange) and SteemIt (decentralized blog platform). -#### 2.9.8. PolkaDot [[17]](#ref-17); https://polkadot.io/ +#### 2.9.8. PolkaDot[[17]](#references); https://polkadot.io/ *PolkaDot* (2019 or later) is one of the best thought-out and most detailed proposed multichain Proof-of-Stake projects; its development is led by one of the Ethereum co-founders. This project is one of the closest projects to the TON Blockchain on our map. (In fact, we are indebted for our terminology for "fishermen" and "nominators" to the PolkaDot project.) -PolkaDot is a heterogeneous loosely-coupled multichain Proof-of-Stake project, with Byzantine Fault Tolerant (BFT) consensus for generation of new blocks and a masterchain (which might be external—e.g., the Ethereum blockchain). It also uses hypercube routing, somewhat like (the slow version of) TON's as described in [2.4.19](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery). +PolkaDot is a heterogeneous loosely-coupled multichain Proof-of-Stake project, with Byzantine Fault Tolerant (BFT) consensus for generation of new blocks and a masterchain (which might be external—e.g., the Ethereum blockchain). It also uses [hypercube routing](#2-4-19-hypercube-routing:-“slow-path”-for-messages-with-assured-delivery), somewhat like (the slow version of) TON's. Its unique feature is its ability to create not only *public*, but also *private* blockchains. These private blockchains would also be able to interact with other public blockchains, PolkaDot or otherwise. @@ -1361,7 +1364,7 @@ However, PolkaDot has no sharding support and is not tightly-coupled. This somew #### 2.9.9. Universa; https://universa.io -The only reason we mention this unusual blockchain project here is because it is the only project so far to make in passing an explicit reference to something similar to our Infinite Sharding Paradigm (cf. [2.1.2](#2-1-2-infinite-sharding-paradigm)). Its other peculiarity is that it bypasses all complications related to Byzantine Fault Tolerance by promising that only trusted and licensed partners of the project will be admitted as validators, hence they will never commit invalid blocks. This is an interesting decision; however, it essentially makes a blockchain project deliberately *centralized*, something blockchain projects usually want to avoid (why does one need a blockchain at all to work in a trusted centralized environment?). +The only reason we mention this unusual blockchain project here is because it is the only project so far to make in passing an explicit reference to something similar to our [Infinite Sharding Paradigm](#2-1-2-infinite-sharding-paradigm). Its other peculiarity is that it bypasses all complications related to Byzantine Fault Tolerance by promising that only trusted and licensed partners of the project will be admitted as validators, hence they will never commit invalid blocks. This is an interesting decision; however, it essentially makes a blockchain project deliberately *centralized*, something blockchain projects usually want to avoid (why does one need a blockchain at all to work in a trusted centralized environment?). #### 2.9.10. Plasma; https://plasma.io @@ -1375,7 +1378,7 @@ There are also some specialized blockchain projects, such as FileCoin (a system #### 2.9.12. The TON Blockchain -The TON (Telegram Open Network) Blockchain (planned 2018) is the project we are describing in this document. It is designed to be the first fifth-generation blockchain project—that is, a BFT PoS-multichain project, mixed homogeneous/heterogeneous, with support for (shardable) custom workchains, with native sharding support, and tightly-coupled (in particular, capable of forwarding messages between shards almost instantly while preserving a consistent state of all shardchains). As such, it will be a truly scalable general-purpose blockchain project, capable of accommodating essentially any applications that can be implemented in a blockchain at all. When augmented by the other components of the TON Project (cf. [1](#1-brief-description-of-ton-components)), its possibilities expand even further. +The TON (Telegram Open Network) Blockchain (planned 2018) is the project we are describing in this document. It is designed to be the first fifth-generation blockchain project—that is, a BFT PoS-multichain project, mixed homogeneous/heterogeneous, with support for (shardable) custom workchains, with native sharding support, and tightly-coupled (in particular, capable of forwarding messages between shards almost instantly while preserving a consistent state of all shardchains). As such, it will be a truly scalable general-purpose blockchain project, capable of accommodating essentially any applications that can be implemented in a blockchain at all. When augmented by the other [components](#1-brief-description-of-ton-components) of the TON Project, its possibilities expand even further. #### 2.9.13. Is it possible to "upload Facebook into a blockchain"? @@ -1393,8 +1396,6 @@ Therefore, a conservative estimate is that one would need 100 times more servers We believe that *it is not true that everything should be uploaded into the blockchain*. For example, it is not necessary to keep user photographs in the blockchain; registering the hashes of these photographs in the blockchain and keeping the photographs in a distributed off-chain storage (such as FileCoin or TON Storage) would be a better idea. This is the reason why TON is not just a blockchain project, but a collection of several components (TON P2P Network, TON Storage, TON Services) centered around the TON Blockchain as outlined in Chapters [1](#1-brief-description-of-ton-components) and [4](#4-ton-services-and-applications). ---- - ## 3. TON Networking Any blockchain project requires not only a specification of block format and blockchain validation rules, but also a network protocol used to propagate new blocks, send and collect transaction candidates and so on. In other words, a specialized peer-to-peer network must be set up by every blockchain project. This network must be peer-to-peer, because blockchain projects are normally expected to be decentralized, so one cannot rely on a centralized group of servers and use conventional client-server architecture, as, for instance, classical online banking applications do. Even light clients (e.g., light cryptocurrency wallet smartphone applications), which must connect to full nodes in a client-server–like fashion, are actually free to connect to another full node if their previous peer goes down, provided the protocol used to connect to full nodes is standardized enough. @@ -1411,7 +1412,7 @@ The cornerstone in building the TON networking protocols is the *(TON) Abstract An *abstract network address*, or an *abstract address*, or just *address* for short, is a 256-bit integer, essentially equal to a 256-bit ECC public key. This public key can be generated arbitrarily, thus creating as many different network identities as the node likes. However, one must know the corresponding *private* key in order to receive (and decrypt) messages intended for such an address. -In fact, the address is *not* the public key itself; instead, it is a 256-bit hash ($\text{Hash}=\text{Sha256}$) of a serialized TL-object (cf. [2.2.5](#2-2-5-tl-or-the-type-language)) that can describe several types of public keys and addresses depending on its constructor (first four bytes). In the simplest case, this serialized TL-object consists just of a 4-byte magic number and a 256-bit elliptic curve cryptography (ECC) public key; in this case, the address will equal the hash of this 36-byte structure. One might use, however, 2048-bit RSA keys, or any other scheme of public-key cryptography instead. +In fact, the address is *not* the public key itself; instead, it is a 256-bit hash ($\text{Hash}=\text{Sha256}$) of a [serialized TL-object](#2-2-5-tl,-or-the-type-language) that can describe several types of public keys and addresses depending on its constructor (first four bytes). In the simplest case, this serialized TL-object consists just of a 4-byte magic number and a 256-bit elliptic curve cryptography (ECC) public key; in this case, the address will equal the hash of this 36-byte structure. One might use, however, 2048-bit RSA keys, or any other scheme of public-key cryptography instead. When a node learns another node's abstract address, it must also receive its "preimage" (i.e., the serialized TL-object, the hash of which equals that abstract address) or else it will not be able to encrypt and send datagrams to that address. @@ -1435,17 +1436,17 @@ This approach has the advantage that, if more than one datagram is expected to b #### 3.1.5. Channels and channel identifiers -In the simplest case, the first 256 bits of a UDP datagram carrying an embedded TON ADNL datagram will be equal to the recipient's address. However, in general they constitute a *channel identifier*. There are different types of channels. Some of them are point-to-point; they are created by two parties who wish to exchange a lot of data in the future and generate a shared secret by exchanging several packets encrypted as described in [3.1.3](#3-1-3-simplest-case-of-adnl-over-udp) or [3.1.4](#3-1-4-less-secure-way-with-the-senders-address-in-plaintext), by running classical or elliptic curve Diffie–Hellman (if extra security is required), or simply by one party generating a random shared secret and sending it to the other party. +In the simplest case, the first 256 bits of a UDP datagram carrying an embedded TON ADNL datagram will be equal to the recipient's address. However, in general they constitute a *channel identifier*. There are different types of channels. Some of them are point-to-point; they are created by two parties who wish to exchange a lot of data in the future and generate a shared secret by exchanging several packets encrypted as described in [3.1.3](#3-1-3-simplest-case-of-adnl-over-udp) or [3.1.4](#3-1-4-less-secure-way%2C-with-the-sender’s-address-in-plaintext), by running classical or elliptic curve Diffie–Hellman (if extra security is required), or simply by one party generating a random shared secret and sending it to the other party. After that, a channel identifier is derived from the shared secret combined with some additional data (such as the sender's and recipient's addresses), for instance by hashing, and that identifier is used as the first 256 bits of UDP datagrams carrying data encrypted with the aid of that shared secret. #### 3.1.6. Channel as a tunnel identifier -In general, a "channel", or "channel identifier" simply selects a way of processing an inbound UDP datagram, known to the receiver. If the channel is the receiver's abstract address, the processing is done as outlined in [3.1.3](#3-1-3-simplest-case-of-adnl-over-udp) or [3.1.4](#3-1-4-less-secure-way-with-the-senders-address-in-plaintext); if the channel is an established point-to-point channel discussed in [3.1.5](#3-1-5-channels-and-channel-identifiers), the processing consists in decrypting the datagram with the aid of the shared secret as explained in *loc. cit.*, and so on. +In general, a "channel", or "channel identifier" simply selects a way of processing an inbound UDP datagram, known to the receiver. If the channel is the receiver's abstract address, the processing is done as outlined in [3.1.3](#3-1-3-simplest-case-of-adnl-over-udp) or [3.1.4](#3-1-4-less-secure-way%2C-with-the-sender’s-address-in-plaintext); if the channel is an established point-to-point [channel](#3-1-5-channels-and-channel-identifiers), the processing consists in decrypting the datagram with the aid of the shared secret as explained in *loc. cit.*, and so on. -In particular, a channel identifier can actually select a "tunnel", when the immediate recipient simply forwards the received message to somebody else—the actual recipient or another proxy. Some encryption or decryption steps (reminiscent of "onion routing" [[6]](#ref-6) or even "garlic routing"[31](#fn31)) might be done along the way, and another channel identifier might be used for re-encrypted forwarded packets (for example, a peer-to-peer channel could be employed to forward the packet to the next recipient on the path). +In particular, a channel identifier can actually select a "tunnel", when the immediate recipient simply forwards the received message to somebody else—the actual recipient or another proxy. Some encryption or decryption steps (reminiscent of "onion routing" [[6]](#references) or even "garlic routing"[31](#fn31)) might be done along the way, and another channel identifier might be used for re-encrypted forwarded packets (for example, a peer-to-peer channel could be employed to forward the packet to the next recipient on the path). -In this way, some support for "tunneling" and "proxying"—somewhat similar to that provided by the TOR or $I^2P$ projects—can be added on the level of the TON Abstract Datagram Network Layer, without affecting the functionality of all higher-level TON network protocols, which would be agnostic of such an addition. This opportunity is exploited by the *TON Proxy* service (cf. [4.1.11](#4-1-11-example%3A-ton-proxy-is-a-fog-service)). +In this way, some support for "tunneling" and "proxying"—somewhat similar to that provided by the TOR or $I^2P$ projects—can be added on the level of the TON Abstract Datagram Network Layer, without affecting the functionality of all higher-level TON network protocols, which would be agnostic of such an addition. This opportunity is exploited by the [TON Proxy](#4-1-11-example%3A-ton-proxy-is-a-fog-service) service. #### 3.1.7. Zero channel and the bootstrap problem @@ -1465,17 +1466,17 @@ The ADNL, being an unreliable (small-size) datagram protocol based on 256-bit ab #### 3.1.9. RLDP, or Reliable Large Datagram Protocol over ADNL -A reliable arbitrary-size datagram protocol built upon the ADNL, called RLDP, is used instead of a TCP-like protocol. This reliable datagram protocol can be employed, for instance, to send RPC queries to remote hosts and receive answers from them (cf. [4.1.5](#4-1-5-pure-network-services-ton-sites-and-ton-services)). +A reliable arbitrary-size datagram protocol built upon the ADNL, called RLDP, is used instead of a TCP-like protocol. This reliable datagram protocol can be employed, for instance, to send [RPC queries](#4-1-5-pure-network-services%3A-“ton-sites”-and-“ton-services”) to remote hosts and receive answers from them. ### 3.2 TON DHT: Kademlia-like Distributed Hash Table -The *TON Distributed Hash Table (DHT)* plays a crucial role in the networking part of the TON Project, being used to locate other nodes in the network. For example, a client wanting to commit a transaction into a shardchain might want to find a validator or a collator of that shardchain, or at least some node that might relay the client's transaction to a collator. This can be done by looking up a special key in the TON DHT. Another important application of the TON DHT is that it can be used to quickly populate a new node's neighbor table (cf. [3.1.7](#3-1-7-zero-channel-and-the-bootstrap-problem)), simply by looking up a random key, or the new node's address. If a node uses proxying and tunneling for its inbound datagrams, it publishes the tunnel identifier and its entry point (e.g., IP address and UDP port) in the TON DHT; then all nodes wishing to send datagrams to that node will obtain this contact information from the DHT first. +The *TON Distributed Hash Table (DHT)* plays a crucial role in the networking part of the TON Project, being used to locate other nodes in the network. For example, a client wanting to commit a transaction into a shardchain might want to find a validator or a collator of that shardchain, or at least some node that might relay the client's transaction to a collator. This can be done by looking up a special key in the TON DHT. Another important application of the TON DHT is that it can be used to quickly populate a new node's [neighbor table](#3-1-7-zero-channel-and-the-bootstrap-problem), simply by looking up a random key, or the new node's address. If a node uses proxying and tunneling for its inbound datagrams, it publishes the tunnel identifier and its entry point (e.g., IP address and UDP port) in the TON DHT; then all nodes wishing to send datagrams to that node will obtain this contact information from the DHT first. -The TON DHT is a member of the family of *Kademlia-like distributed hash tables* [[10]](#ref-10). +The TON DHT is a member of the family of *Kademlia-like distributed hash tables* [[10]](#references). #### 3.2.1. Keys of the TON DHT -The *keys* of the TON DHT are simply 256-bit integers. In most cases, they are computed as SHA-256 of a TL-serialized object (cf. [2.2.5](#2-2-5-tl-or-the-type-language)), called *preimage* of the key, or *key description*. In some cases, the abstract addresses of the TON Network nodes (cf. [3.1.1](#3-1-1-abstract-network-addresses)) can also be used as keys of the TON DHT, because they are also 256-bit, and they are also hashes of TL-serialized objects. For example, if a node is not afraid of publishing its IP address, it can be found by anybody who knows its abstract address by simply looking up that address as a key in the DHT. +The *keys* of the TON DHT are simply 256-bit integers. In most cases, they are computed as SHA-256 of a [TL-serialized object](#2-2-5-tl%2C-or-the-type-language), called *preimage* of the key, or *key description*. In some cases, the [abstract addresses](#3-1-1-abstract-network-addresses) of the TON Network nodes can also be used as keys of the TON DHT, because they are also 256-bit, and they are also hashes of TL-serialized objects. For example, if a node is not afraid of publishing its IP address, it can be found by anybody who knows its abstract address by simply looking up that address as a key in the DHT. #### 3.2.2. Values of the DHT @@ -1483,7 +1484,7 @@ The *values* assigned to these 256-bit keys are essentially arbitrary byte strin #### 3.2.3. Nodes of the DHT. Semi-permanent network identities -The key-value mapping of the TON DHT is kept on the *nodes* of the DHT—essentially, all members of the TON Network. To this end, any node of the TON Network (perhaps with the exception of some very light nodes), apart from any number of ephemeral and permanent abstract addresses described in [3.1.1](#3-1-1-abstract-network-addresses), has at least one "semi-permanent address", which identifies it as a member of the TON DHT. This *semi-permanent* or *DHT address* should not to be changed too often, otherwise other nodes would be unable to locate the keys they are looking for. If a node does not want to reveal its "true" identity, it generates a separate abstract address to be used only for the purpose of participating in the DHT. However, this abstract address must be public, because it will be associated with the node's IP address and port. +The key-value mapping of the TON DHT is kept on the *nodes* of the DHT—essentially, all members of the TON Network. To this end, any node of the TON Network (perhaps with the exception of some very light nodes), apart from any number of ephemeral and permanent [abstract addresses](#3-1-1-abstract-network-addresses), has at least one "semi-permanent address", which identifies it as a member of the TON DHT. This *semi-permanent* or *DHT address* should not to be changed too often, otherwise other nodes would be unable to locate the keys they are looking for. If a node does not want to reveal its "true" identity, it generates a separate abstract address to be used only for the purpose of participating in the DHT. However, this abstract address must be public, because it will be associated with the node's IP address and port. #### 3.2.4. Kademlia distance @@ -1503,15 +1504,15 @@ We say that a distributed hash table (DHT) with 256-bit keys and 256-bit node ad Here $s$ is a small parameter, say, $s=7$, needed to improve reliability of the DHT (if we would keep the key only on one node, the nearest one to $K$, the value of that key would be lost if that only node goes offline). -The TON DHT is a Kademlia-like DHT, according to this definition. It is implemented over the ADNL protocol described in [3.1](#3-1-abstract-datagram-network-layer). +The TON DHT is a Kademlia-like DHT, according to this definition. It is implemented over the [ADNL protocol](#3-1-abstract-datagram-network-layer). #### 3.2.6. Kademlia routing table -Any node participating in a Kademlia-like DHT usually maintains a *Kademlia routing table*. In the case of TON DHT, it consists of $n=256$ buckets, numbered from $0$ to $n-1$. The $i$-th bucket will contain information about some known nodes (a fixed number $t$ of "best" nodes, and maybe some extra candidates) that lie at a Kademlia distance from $2^i$ to $2^{i+1}-1$ from the node's address $a$.[32](#fn32) This information includes their (semi-permanent) addresses, IP addresses and UDP ports, and some availability information such as the time and the delay of the last ping. +Any node participating in a Kademlia-like DHT usually maintains a *Kademlia routing table*. In the case of TON DHT, it consists of $n=256$ buckets, numbered from $0$ to $n-1$. The $i$-th bucket will contain information about some known nodes (a fixed number $t$ of "best" nodes, and maybe some extra candidates) that lie at a Kademlia distance from $2^i$ to $2^{i+1}-1$ from the node's address $a$.[32](#fn32) This information includes their (semi-permanent) addresses, IP addresses and UDP ports, and some availability information such as the time and the delay of the last ping. When a Kademlia node learns about any other Kademlia node as a result of some query, it includes it into a suitable bucket of its routing table, first as a candidate. Then, if some of the "best" nodes in that bucket fail (e.g., do not respond to ping queries for a long time), they can be replaced by some of the candidates. In this way the Kademlia routing table stays populated. -New nodes from the Kademlia routing table are also included in the ADNL neighbor table described in [3.1.7](#3-1-7-zero-channel-and-the-bootstrap-problem). If a "best" node from a bucket of the Kademlia routing table is used often, a channel in the sense described in [3.1.5](#3-1-5-channels-and-channel-identifiers) can be established to facilitate the encryption of datagrams. +New nodes from the Kademlia routing table are also included in the ADNL [neighbor table](#3-1-7-zero-channel-and-the-bootstrap-problem). If a "best" node from a bucket of the Kademlia routing table is used often, a [channel](#3-1-5-channels-and-channel-identifiers) in the sense can be established to facilitate the encryption of datagrams. A special feature of the TON DHT is that it tries to select nodes with the smallest round-trip delays as the "best" nodes for the buckets of the Kademlia routing table. @@ -1520,7 +1521,7 @@ A special feature of the TON DHT is that it tries to select nodes with the small A Kademlia node usually supports the following network queries: - $\text{Ping}$ — Checks node availability. -- $\text{Store}(\text{key},\text{value})$ — Asks the node to keep *value* as a value for key *key*. For TON DHT, the $\text{Store}$ queries are slightly more complicated (cf. [3.2.9](#3-2-9-storing-values-in-ton-dht)). +- $\text{Store}(\text{key},\text{value})$ — Asks the node to keep *value* as a value for key *key*. For TON DHT, the $\text{Store}$ queries are slightly more [complicated](#3-2-9-storing-values-in-ton-dht). - $\text{Find\_Node}(\text{key},l)$ — Asks the node to return $l$ Kademlia-nearest known nodes (from its Kademlia routing table) to *key*. - $\text{Find\_Value}(\text{key},l)$ — The same as above, but if the node knows the value corresponding to key *key*, it just returns that value. @@ -1538,7 +1539,7 @@ When a Kademlia node goes online, it first populates its Kademlia routing table Storing values in TON DHT is slightly different from a general Kademlia-like DHT. When someone wishes to store a value, she must provide not only the key $K$ itself to the $\text{Store}$ query, but also its *preimage*—i.e., a TL-serialized string (with one of several predefined TL-constructors at the beginning) containing a "description" of the key. This key description is later kept by the node, along with the key and the value. -The key description describes the "type" of the object being stored, its "owner", and its "update rules" in case of future updates. The owner is usually identified by a public key included in the key description. If it is included, normally only updates signed by the corresponding private key will be accepted. The "type" of the stored object is normally just a byte string. However, in some cases it can be more sophisticated—for example, an input tunnel description (cf. [3.1.6](#3-1-6-channel-as-a-tunnel-identifier)), or a collection of node addresses. +The key description describes the "type" of the object being stored, its "owner", and its "update rules" in case of future updates. The owner is usually identified by a public key included in the key description. If it is included, normally only updates signed by the corresponding private key will be accepted. The "type" of the stored object is normally just a byte string. However, in some cases it can be more sophisticated—for example, an [input tunnel](#3-1-6-channel-as-a-tunnel-identifier) description, or a collection of node addresses. The "update rules" can also be different. In some cases, they simply permit replacing the old value with the new value, provided the new value is signed by the owner (the signature must be kept as part of the value, to be checked later by any other nodes after they obtain the value of this key). In other cases, the old value somehow affects the new value. For example, it can contain a sequence number, and the old value is overwritten only if the new sequence number is larger (to prevent replay attacks). @@ -1548,7 +1549,7 @@ Yet another interesting case is when the value contains a list of nodes—perhap This mechanism might be used to create a distributed "torrent tracker", where all nodes interested in a certain "torrent" (i.e., a certain file) can find other nodes that are interested in the same torrent, or already have a copy. -*TON Storage* (cf. [4.1.8](#4-1-8-example%3A-keeping-files-off-chain%3B-ton-storage)) uses this technology to find the nodes that have a copy of a required file (e.g., a snapshot of the state of a shardchain, or an old block). However, its more important use is to create "overlay multicast subnetworks" and "network interest groups" (cf. [3.3](#3-3-overlay-networks-and-multicasting-messages)). The idea is that only some nodes are interested in the updates of a specific shardchain. If the number of shardchains becomes very large, finding even one node interested in the same shard may become complicated. This "distributed torrent tracker" provides a convenient way to find some of these nodes. Another option would be to request them from a validator, but this would not be a scalable approach, and validators might choose not to respond to such queries coming from arbitrary unknown nodes. +[TON Storage](#4-1-8-example%3A-keeping-files-off-chain%3B-ton-storage) uses this technology to find the nodes that have a copy of a required file (e.g., a snapshot of the state of a shardchain, or an old block). However, its more important use is to create [overlay multicast subnetworks](#3-3-overlay-networks-and-multicasting-messages) and "network interest groups". The idea is that only some nodes are interested in the updates of a specific shardchain. If the number of shardchains becomes very large, finding even one node interested in the same shard may become complicated. This "distributed torrent tracker" provides a convenient way to find some of these nodes. Another option would be to request them from a validator, but this would not be a scalable approach, and validators might choose not to respond to such queries coming from arbitrary unknown nodes. #### 3.2.11. Fall-back keys @@ -1556,21 +1557,21 @@ Most of the "key types" described so far have an extra 32-bit integer field in t #### 3.2.12. Locating services -Some services, located in the TON Network and available through the (higher-level protocols built upon the) TON ADNL described in [3.1](#3-1-abstract-datagram-network-layer), may want to publish their abstract addresses somewhere, so that their clients would know where to find them. +Some services, located in the TON Network and available through the (higher-level protocols built upon the) [TON ADNL](#3-1-abstract-datagram-network-layer), may want to publish their abstract addresses somewhere, so that their clients would know where to find them. However, publishing the service's abstract address in the TON Blockchain may not be the best approach, because the abstract address might need to be changed quite often, and because it could make sense to provide several addresses, for reliability or load balancing purposes. -An alternative is to publish a public key into the TON Blockchain, and use a special DHT key indicating that public key as its "owner" in the TL description string (cf. [2.2.5](#2-2-5-tl-or-the-type-language)) to publish an up-to-date list of the service's abstract addresses. This is one of the approaches exploited by TON Services. +An alternative is to publish a public key into the TON Blockchain, and use a special DHT key indicating that public key as its "owner" in the [TL description](#2-2-5-tl%2C-or-the-type-language) string to publish an up-to-date list of the service's abstract addresses. This is one of the approaches exploited by TON Services. #### 3.2.13. Locating owners of TON blockchain accounts In most cases, owners of TON blockchain accounts would not like to be associated with abstract network addresses, and especially IP addresses, because this can violate their privacy. In some cases, however, the owner of a TON blockchain account may want to publish one or several abstract addresses where she could be contacted. -A typical case is that of a node in the TON Payments "lightning network" (cf. [5.2](#5-2-payment-channel-network-or-lightning-network)), the platform for instant cryptocurrency transfers. A public TON Payments node may want not only to establish payment channels with other peers, but also to publish an abstract network address that could be used to contact it at a later time for transferring payments along the already-established channels. +A typical case is that of a node in the TON Payments [lightning network](#5-2-payment-channel-network%2C-or-“lightning-network”), the platform for instant cryptocurrency transfers. A public TON Payments node may want not only to establish payment channels with other peers, but also to publish an abstract network address that could be used to contact it at a later time for transferring payments along the already-established channels. -One option would be to include an abstract network address in the smart contract creating the payment channel. A more flexible option is to include a public key in the smart contract, and then use DHT as explained in [3.2.12](#3-2-12-locating-services). +One option would be to include an abstract network address in the smart contract creating the payment channel. A more flexible option is to [include](#3-2-12-locating-services) a public key in the smart contract, and then use DHT. -The most natural way would be to use the same private key that controls the account in the TON Blockchain to sign and publish updates in the TON DHT about the abstract addresses associated with that account. This is done almost in the same way as described in [3.2.12](#3-2-12-locating-services); however, the DHT key employed would require a special key description, containing only the $\mathit{account\_id}$ itself, equal to SHA-256 of the "account description", which contains the public key of the account. The signature, included in the value of this DHT key, would contain the account description as well. +The most natural way would be to use the same private key that controls the account in the TON Blockchain to sign and publish updates in the TON DHT about the abstract addresses associated with that account. This is done almost in the [same way](#3-2-12-locating-services); however, the DHT key employed would require a special key description, containing only the $\mathit{account\_id}$ itself, equal to SHA-256 of the "account description", which contains the public key of the account. The signature, included in the value of this DHT key, would contain the account description as well. In this way, a mechanism for locating abstract network addresses of some owners of the TON Blockchain accounts becomes available. @@ -1588,7 +1589,7 @@ Notice that in order to make an abstract address "public" (reachable from any no ### 3.3 Overlay Networks and Multicasting Messages -In a multi-blockchain system like the TON Blockchain, even full nodes would normally be interested in obtaining updates (i.e., new blocks) only about some shardchains. To this end, a special overlay (sub)network must be built inside the TON Network, on top of the ADNL protocol discussed in [3.1](#3-1-abstract-datagram-network-layer), one for each shardchain. +In a multi-blockchain system like the TON Blockchain, even full nodes would normally be interested in obtaining updates (i.e., new blocks) only about some shardchains. To this end, a special overlay (sub)network must be built inside the TON Network, on top of the [ADNL protocol](#3-1-abstract-datagram-network-layer), one for each shardchain. Therefore, the need to build arbitrary overlay subnetworks, open to any nodes willing to participate, arises. Special gossip protocols, built upon ADNL, will be run in these overlay networks. In particular, these gossip protocols may be used to propagate (broadcast) arbitrary data inside such a subnetwork. @@ -1602,11 +1603,11 @@ In most cases, the overlay network is implemented using some protocol built upon #### 3.3.2. Overlay networks in TON -Overlay networks in TON are built upon the ADNL protocol discussed in [3.1](#3-1-abstract-datagram-network-layer); they use 256-bit ADNL abstract addresses as addresses in the overlay networks as well. Each node usually selects one of its abstract addresses to double as its address in the overlay network. +Overlay networks in TON are built upon the [ADNL protocol](#3-1-abstract-datagram-network-layer); they use 256-bit ADNL abstract addresses as addresses in the overlay networks as well. Each node usually selects one of its abstract addresses to double as its address in the overlay network. In contrast to ADNL, the TON overlay networks usually do not support sending datagrams to arbitrary other nodes. Instead, some "semipermanent links" are established between some nodes (called "neighbors" with respect to the overlay network under consideration), and messages are usually forwarded along these links (i.e., from a node to one of its neighbors). In this way, a TON overlay network is a (usually not full) subgraph inside the (full) graph of the ADNL network. -Links to neighbors in TON overlay networks can be implemented using dedicated peer-to-peer ADNL channels (cf. [3.1.5](#3-1-5-channels-and-channel-identifiers)). +Links to neighbors in TON overlay networks can be implemented using dedicated peer-to-peer ADNL [channels](#3-1-5-channels-and-channel-identifiers). Each node of an overlay network maintains a list of neighbors (with respect to the overlay network), containing their abstract addresses (which they use to identify them in the overlay network) and some link data (e.g., the ADNL channel used to communicate with them). @@ -1620,7 +1621,7 @@ Some overlay networks are *centrally controlled*, by one or several nodes, or by #### 3.3.5. Joining an overlay network -When a node wants to join an overlay network, it first must learn its 256-bit *network identifier*, usually equal to SHA-256 of the *description* of the overlay network—a TL-serialized object (cf. [2.2.5](#2-2-5-tl-or-the-type-language)) which may contain, for instance, the central authority of the overlay network (i.e., its public key and perhaps its abstract address,[33](#fn33)) a string with the name of the overlay network, a TON Blockchain shard identifier if this is an overlay network related to that shard, and so on. +When a node wants to join an overlay network, it first must learn its 256-bit *network identifier*, usually equal to SHA-256 of the *description* of the overlay network—a [TL-serialized object](#2-2-5-tl%2C-or-the-type-language) which may contain, for instance, the central authority of the overlay network (i.e., its public key and perhaps its abstract address,[33](#fn33)) a string with the name of the overlay network, a TON Blockchain shard identifier if this is an overlay network related to that shard, and so on. Sometimes it is possible to recover the overlay network description starting from the network identifier, simply by looking it up in the TON DHT. In other cases (e.g., for private overlay networks), one must obtain the network description along with the network identifier. @@ -1632,13 +1633,13 @@ This is also needed for nodes that do not want to join the overlay network, but The method used for locating members of an overlay network is defined in the description of that network. Sometimes (especially for private networks) one must already know a member node to be able to join. In other cases, the abstract addresses of some nodes are contained in the network description. A more flexible approach is to indicate in the network description only the central authority responsible for the network, and then the abstract addresses will be available through values of certain DHT keys, signed by that central authority. -Finally, truly decentralized public overlay networks can use the "distributed torrent-tracker" mechanism described in [3.2.10](#3-2-10-distributed-torrent-trackers-and-network-interest-groups-in-ton-dht), also implemented with the aid of the TON DHT. +Finally, truly decentralized public overlay networks can use the [distributed torrent-tracker](#3-2-10-distributed-“torrent-trackers”-and-“network-interest-groups”-in-ton-dht) mechanism, also implemented with the aid of the TON DHT. #### 3.3.7. Locating more members of the overlay network. Creating links Once one node of the overlay network is found, a special query may be sent to that node requesting a list of other members, for instance, neighbors of the node being queried, or a random selection thereof. -This enables the joining member to populate her "adjacency" or "neighbor list" with respect to the overlay network, by selecting some newly-learned network nodes and establishing links to them (i.e., dedicated ADNL point-to-point channels, as outlined in [3.3.2](#3-3-2-overlay-networks-in-ton)). After that, special messages are sent to all neighbors indicating that the new member is ready to work in the overlay network. The neighbors include their links to the new member in their neighbor lists. +This enables the joining member to populate her "adjacency" or "neighbor list" with respect to the [overlay network](#3-3-2-overlay-networks-in-ton), by selecting some newly-learned network nodes and establishing links to them (i.e., dedicated ADNL point-to-point channels). After that, special messages are sent to all neighbors indicating that the new member is ready to work in the overlay network. The neighbors include their links to the new member in their neighbor lists. #### 3.3.8. Maintaining the neighbor list @@ -1658,7 +1659,7 @@ In this way, the graph of an overlay network still maintains enough randomness t #### 3.3.11. Gossip protocols in an overlay network -An overlay network is often used to run one of the so-called *gossip protocols*, which achieve some global goal while letting every node interact only with its neighbors. For example, there are gossip protocols to construct an approximate list of all members of a (not too large) overlay network, or to compute an estimate of the number of members of an (arbitrarily large) overlay network, using only a bounded amount of memory at each node (cf. [[15](#ref-15), 4.4.3] or [[1]](#ref-1) for details). +An overlay network is often used to run one of the so-called *gossip protocols*, which achieve some global goal while letting every node interact only with its neighbors. For example, there are gossip protocols to construct an approximate list of all members of a (not too large) overlay network, or to compute an estimate of the number of members of an (arbitrarily large) overlay network, using only a bounded amount of memory at each node [[15]](#references) or [[1]](#references) for details). #### 3.3.12. Overlay network as a broadcast domain @@ -1668,7 +1669,7 @@ There are in fact several broadcast protocols, optimized for different use cases #### 3.3.13. More sophisticated broadcast protocols -Some applications may warrant more sophisticated broadcast protocols. For instance, for broadcasting messages of substantial size, it makes sense to send to the neighbors not the newly-received message itself, but its hash (or a collection of hashes of new messages). The neighbor may request the message itself after learning a previously unseen message hash, to be transferred, say, using the reliable large datagram protocol (RLDP) discussed in [3.1.9](#3-1-9-rldp-or-reliable-large-datagram-protocol-over-adnl). In this way, the new message will be downloaded from one neighbor only. +Some applications may warrant more sophisticated broadcast protocols. For instance, for broadcasting messages of substantial size, it makes sense to send to the neighbors not the newly-received message itself, but its hash (or a collection of hashes of new messages). The neighbor may request the message itself after learning a previously unseen message hash, to be transferred, say, using the reliable large datagram protocol [RLDP](#3-1-9-rldp%2C-or-reliable-large-datagram-protocol-over-adnl). In this way, the new message will be downloaded from one neighbor only. #### 3.3.14. Checking the connectivity of an overlay network @@ -1680,15 +1681,15 @@ In the case of an overlay network used for propagating new blocks (or just new b Finally, there is a *streaming broadcast protocol* for TON overlay networks, used, for example, to propagate block candidates among validators of some shardchain ("shardchain task group"), who, of course, create a private overlay network for that purpose. The same protocol can be used to propagate new shardchain blocks to all full nodes for that shardchain. -This protocol has already been outlined in [2.6.10](#2-6-10-propagation-of-shardchain-block-candidates): the new (large) broadcast message is split into, say, $N$ one-kilobyte chunks; the sequence of these chunks is augmented to $M\geq N$ chunks by means of an erasure code such as the Reed–Solomon or a fountain code (e.g., the RaptorQ code [[9]](#ref-9) [[14]](#ref-14)), and these $M$ chunks are streamed to all neighbors in ascending chunk number order. The participating nodes collect these chunks until they can recover the original large message (one would have to successfully receive at least $N$ of the chunks for this), and then instruct their neighbors to stop sending new chunks of the stream, because now these nodes can generate the subsequent chunks on their own, having a copy of the original message. Such nodes continue to generate the subsequent chunks of the stream and send them to their neighbors, unless the neighbors in turn indicate that this is no longer necessary. +With this [protocol](#2-6-10-propagation-of-shardchain-block-candidates) the new (large) broadcast message is split into, say, $N$ one-kilobyte chunks; the sequence of these chunks is augmented to $M\geq N$ chunks by means of an erasure code such as the Reed–Solomon or a fountain code (e.g., the RaptorQ code [[9]](#references) [[14]](#references), and these $M$ chunks are streamed to all neighbors in ascending chunk number order. The participating nodes collect these chunks until they can recover the original large message (one would have to successfully receive at least $N$ of the chunks for this), and then instruct their neighbors to stop sending new chunks of the stream, because now these nodes can generate the subsequent chunks on their own, having a copy of the original message. Such nodes continue to generate the subsequent chunks of the stream and send them to their neighbors, unless the neighbors in turn indicate that this is no longer necessary. -In this way, a node does not need to download a large message in its entirety before propagating it further. This minimizes broadcast latency, especially when combined with the optimizations described in [3.3.10](#3-3-10-ton-overlay-networks-are-optimized-for-lower-latency). +In this way, a node does not need to download a large message in its entirety before propagating it further. This minimizes broadcast latency, especially when combined with the [optimizations](#3-3-10-ton-overlay-networks-are-optimized-for-lower-latency). #### 3.3.16. Constructing new overlay networks based on existing ones Sometimes one does not want to construct an overlay network from scratch. Instead, one or several previously existing overlay networks are known, and the membership of the new overlay network is expected to overlap significantly with the combined membership of these overlay networks. -An important example arises when a TON shardchain is split in two, or two sibling shardchains are merged into one (cf. [2.7](#2-7-splitting-and-merging-shardchains)). In the first case, the overlay networks used for propagating new blocks to full nodes must be constructed for each of the new shardchains; however, each of these new overlay networks can be expected to be contained in the block propagation network of the original shardchain (and comprise approximately half its members). In the second case, the overlay network for propagating new blocks of the merged shardchain will consist approximately of the union of members of the two overlay networks related to the two sibling shardchains being merged. +An important example arises when a TON shardchain is split in two, or two sibling shardchains are merged into [one](#2-7-splitting-and-merging-shardchains). In the first case, the overlay networks used for propagating new blocks to full nodes must be constructed for each of the new shardchains; however, each of these new overlay networks can be expected to be contained in the block propagation network of the original shardchain (and comprise approximately half its members). In the second case, the overlay network for propagating new blocks of the merged shardchain will consist approximately of the union of members of the two overlay networks related to the two sibling shardchains being merged. In such cases, the description of the new overlay network may contain an explicit or implicit reference to a list of related existing overlay networks. A node wishing to join the new overlay network may check whether it is already a member of one of these existing networks, and query its neighbors in these networks whether they are interested in the new network as well. In case of a positive answer, new point-to-point channels can be established to such neighbors, and they can be included in the neighbor list for the new overlay network. @@ -1696,7 +1697,7 @@ This mechanism does not totally supplant the general mechanism described in [3.3 #### 3.3.17. Overlay networks within overlay networks -Another interesting case arises in the implementation of *TON Payments* (a "lightning network" for instant off-chain value transfers; cf. [5.2](#5-2-payment-channel-network-or-lightning-network)). In this case, first an overlay network containing all transit nodes of the "lightning network" is constructed. However, some of these nodes have established payment channels in the blockchain; they must always be neighbors in this overlay network, in addition to any "random" neighbors selected by the general overlay network algorithms described in [3.3.6](#3-3-6-locating-one-member-of-the-overlay-network), [3.3.7](#3-3-7-locating-more-members-of-the-overlay-network-creating-links) and [3.3.8](#3-3-8-maintaining-the-neighbor-list). These "permanent links" to the neighbors with established payment channels are used to run specific lightning network protocols, thus effectively creating an overlay subnetwork (not necessarily connected, if things go awry) inside the encompassing (almost always connected) overlay network. +Another interesting case arises in the implementation of *TON Payments* (a [lightning network](#5-2-payment-channel-network%2C-or-“lightning-network”) for instant off-chain value transfers). In this case, first an overlay network containing all transit nodes of the "lightning network" is constructed. However, some of these nodes have established payment channels in the blockchain; they must always be neighbors in this overlay network, in addition to any "random" neighbors selected by the general overlay network algorithms described in [3.3.6](#3-3-6-locating-one-member-of-the-overlay-network), [3.3.7](#3-3-7-locating-more-members-of-the-overlay-network-creating-links) and [3.3.8](#3-3-8-maintaining-the-neighbor-list). These "permanent links" to the neighbors with established payment channels are used to run specific lightning network protocols, thus effectively creating an overlay subnetwork (not necessarily connected, if things go awry) inside the encompassing (almost always connected) overlay network. --- @@ -1710,43 +1711,43 @@ We start with a discussion of how different blockchain and network-related appli #### 4.1.1. Applications and services -We will use the words "application" and "service" interchangeably. However, there is a subtle and somewhat vague distinction: an *application* usually provides some services directly to human users, while a *service* is usually exploited by other applications and services. For example, TON Storage is a service, because it is designed to keep files on behalf of other applications and services, even though a human user might use it directly as well. A hypothetical "Facebook in a blockchain" (cf. [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain)) or Telegram messenger, if made available through the TON Network (i.e., implemented as a "ton-service" c.f [4.1.6](#4-1-6-telegram-messenger-as-a-ton-service%3B-mtproto-over-rldp)), would rather be an *application*, even though some "bots" might access it automatically without human intervention. +We will use the words "application" and "service" interchangeably. However, there is a subtle and somewhat vague distinction: an *application* usually provides some services directly to human users, while a *service* is usually exploited by other applications and services. For example, TON Storage is a service, because it is designed to keep files on behalf of other applications and services, even though a human user might use it directly as well. A hypothetical [2.9.13](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”) or [4.1.6](#4-1-6-telegram-messenger-as-a-ton-service%3B-mtproto-over-rldp), if made available through the TON Network (i.e., implemented as a "ton-service"), would rather be an *application*, even though some "bots" might access it automatically without human intervention. #### 4.1.2. Location of the application: on-chain, off-chain or mixed A service or an application designed for the TON ecosystem needs to keep its data and process that data somewhere. This leads to the following classification of applications (and services): -- *On-chain* applications (cf. [4.1.4](#4-1-4-pure-on-chain-applications%3A-distributed-applications%2C-or-dapps%2C-residing-in-the-blockchain)): All data and processing are in the TON Blockchain. -- *Off-chain* applications (cf. [4.1.5](#4-1-5-pure-network-services%3A-ton-sites-and-ton-services)): All data and processing are outside the TON Blockchain, on servers available through the TON Network. -- *Mixed* applications (cf. [4.1.7](#4-1-7-mixed-services%3A-partly-off-chain%2C-partly-on-chain)): Some, but not all, data and processing are in the TON Blockchain; the rest are on off-chain servers available through the TON Network. - +- [On-chain](#4-1-4-pure-“on-chain”-applications:-distributed-applications,-or-“dapps”,-residing-in-the-blockchain) applications: All data and processing are in the TON Blockchain. +- [Off-chain](#4-1-5-pure-network-services:-“ton-sites”-and-“ton-services”) applications: All data and processing are outside the TON Blockchain, on servers available through the TON Network. +- [Mixed](#4-1-7-mixed-services%3A-partly-off-chain%2C-partly-on-chain) applications: Some, but not all, data and processing are in the TON Blockchain; the rest are on off-chain servers available through the TON Network. +- #### 4.1.3. Centralization: centralized and decentralized, or distributed, applications -Another classification criterion is whether the application (or service) relies on a centralized server cluster, or is really "distributed" (cf. [4.1.9](#4-1-9-decentralized-mixed-services%2C-or-“fog-services”)). All on-chain applications are automatically decentralized and distributed. Off-chain and mixed applications may exhibit different degrees of centralization. +Another classification criterion is whether the application (or service) relies on a centralized server cluster, or is really [distributed](#4-1-9-decentralized-mixed-services%2C-or-“fog-services”). All on-chain applications are automatically decentralized and distributed. Off-chain and mixed applications may exhibit different degrees of centralization. Now let us consider the above possibilities in more detail. #### 4.1.4. Pure "on-chain" applications: distributed applications, or "dapps", residing in the blockchain -One of the possible approaches, mentioned in [4.1.2](#4-1-2-location-of-the-application-on-chain-off-chain-or-mixed), is to deploy a "distributed application" (commonly abbreviated as "dapp") completely in the TON Blockchain, as one smart contract or a collection of smart contracts. All data will be kept as part of the permanent state of these smart contracts, and all interaction with the project will be done by means of (TON Blockchain) messages sent to or received from these smart contracts. +One of the possible [approaches](#4-1-2-location-of-the-application%3A-on-chain%2C-off-chain-or-mixed), is to deploy a "distributed application" (commonly abbreviated as "dapp") completely in the TON Blockchain, as one smart contract or a collection of smart contracts. All data will be kept as part of the permanent state of these smart contracts, and all interaction with the project will be done by means of (TON Blockchain) messages sent to or received from these smart contracts. -We have already discussed in [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain) that this approach has its drawbacks and limitations. It has its advantages, too: such a distributed application needs no servers to run on or to store its data (it runs "in the blockchain"—i.e., on the validators' hardware), and enjoys the blockchain's extremely high (Byzantine) reliability and accessibility. The developer of such a distributed application does not need to buy or rent any hardware; all she needs to do is develop some software (i.e., the code for the smart contracts). After that, she will effectively rent the computing power from the validators, and will pay for it in Grams, either herself or by putting this burden on the shoulders of her users. +We have already [discussed](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”) that this approach has its drawbacks and limitations. It has its advantages, too: such a distributed application needs no servers to run on or to store its data (it runs "in the blockchain"—i.e., on the validators' hardware), and enjoys the blockchain's extremely high (Byzantine) reliability and accessibility. The developer of such a distributed application does not need to buy or rent any hardware; all she needs to do is develop some software (i.e., the code for the smart contracts). After that, she will effectively rent the computing power from the validators, and will pay for it in Grams, either herself or by putting this burden on the shoulders of her users. #### 4.1.5. Pure network services: "ton-sites" and "ton-services" -Another extreme option is to deploy the service on some servers and make it available to the users through the ADNL protocol described in [3.1](#3-1-abstract-datagram-network-layer), and maybe some higher level protocol such as the RLDP discussed in [3.1.9](#3-1-9-rldp-or-reliable-large-datagram-protocol-over-adnl), which can be used to send RPC queries to the service in any custom format and obtain answers to these queries. In this way, the service will be totally off-chain, and will reside in the TON Network, almost without using the TON Blockchain. +Another extreme option is to deploy the service on some servers and make it available to the users through the [ADNL protocol](#3-1-abstract-datagram-network-layer), and maybe some higher level protocol such as the [RLDP](#3-1-9-rldp%2C-or-reliable-large-datagram-protocol-over-adnl), which can be used to send RPC queries to the service in any custom format and obtain answers to these queries. In this way, the service will be totally off-chain, and will reside in the TON Network, almost without using the TON Blockchain. -The TON Blockchain might be used only to locate the abstract address or addresses of the service, as outlined in [3.2.12](#3-2-12-locating-services), perhaps with the aid of a service such as the TON DNS (cf. [4.3.1](#4-3-1-ton-dns-a-mostly-on-chain-hierarchical-domain-name-service)) to facilitate translation of domain-like human-readable strings into abstract addresses. +The TON Blockchain might be used only to [locate](#3-2-12-locating-services) the abstract address or addresses of the service, perhaps with the aid of a service such as the [TON DNS](#4-3-1-ton-dns%3A-a-mostly-on-chain-hierarchical-domain-name-service) to facilitate translation of domain-like human-readable strings into abstract addresses. To the extent the ADNL network (i.e., the TON Network) is similar to the Invisible Internet Project ($I^2P$), such (almost) purely network services are analogous to the so-called "eep-services" (i.e., services that have an $I^2P$-address as their entry point, and are available to clients through the $I^2P$ network). We will say that such purely network services residing in the TON Network are "ton-services". -An "eep-service" may implement HTTP as its client-server protocol; in the TON Network context, a "ton-service" might simply use RLDP (cf. [3.1.9](#3-1-9-rldp-or-reliable-large-datagram-protocol-over-adnl)) datagrams to transfer HTTP queries and responses to them. If it uses the TON DNS to allow its abstract address to be looked up by a human-readable domain name, the analogy to a web site becomes almost perfect. One might even write a specialized browser, or a special proxy ("ton-proxy") that is run locally on a user's machine, accepts arbitrary HTTP queries from an ordinary web browser the user employs (once the local IP address and the TCP port of the proxy are entered into the browser's configuration), and forwards these queries through the TON Network to the abstract address of the service. Then the user would have a browsing experience similar to that of the World Wide Web (WWW). +An "eep-service" may implement HTTP as its client-server protocol; in the TON Network context, a "ton-service" might simply use [RLDP] (#3-1-9-rldp%2C-or-reliable-large-datagram-protocol-over-adnl)) datagrams to transfer HTTP queries and responses to them. If it uses the TON DNS to allow its abstract address to be looked up by a human-readable domain name, the analogy to a web site becomes almost perfect. One might even write a specialized browser, or a special proxy ("ton-proxy") that is run locally on a user's machine, accepts arbitrary HTTP queries from an ordinary web browser the user employs (once the local IP address and the TCP port of the proxy are entered into the browser's configuration), and forwards these queries through the TON Network to the abstract address of the service. Then the user would have a browsing experience similar to that of the World Wide Web (WWW). In the $I^2P$ ecosystem, such "eep-services" are called "eep-sites". One can easily create "ton-sites" in the TON ecosystem as well. This is facilitated somewhat by the existence of services such as the TON DNS, which exploit the TON Blockchain and the TON DHT to translate (TON) domain names into abstract addresses. #### 4.1.6. Telegram Messenger as a ton-service; MTProto over RLDP -We would like to mention in passing that the MTProto protocol,[34](#fn34) used by Telegram Messenger[35](#fn35) for client-server interaction, can be easily embedded into the RLDP protocol discussed in [3.1.9](#3-1-9-rldp-or-reliable-large-datagram-protocol-over-adnl), thus effectively transforming Telegram into a ton-service. Because the TON Proxy technology can be switched on transparently for the end user of a ton-site or a ton-service, being implemented on a lower level than the RLDP and ADNL protocols (cf. [3.1.6](#3-1-6-channel-as-a-tunnel-identifier)), this would make Telegram effectively unblockable. Of course, other messaging and social networking services might benefit from this technology as well. +We would like to mention in passing that the MTProto protocol,[34](#fn34) used by Telegram Messenger[35](#fn35) for client-server interaction, can be easily embedded into the [RLDP](#3-1-9-rldp%2C-or-reliable-large-datagram-protocol-over-adnl) protocol , thus effectively transforming Telegram into a ton-service. Because the [TON Proxy](#3-1-6-channel-as-a-tunnel-identifier) technology can be switched on transparently for the end user of a ton-site or a ton-service, being implemented on a lower level than the RLDP and ADNL protocols, this would make Telegram effectively unblockable. Of course, other messaging and social networking services might benefit from this technology as well. #### 4.1.7. Mixed services: partly off-chain, partly on-chain @@ -1756,7 +1757,7 @@ Some services might use a mixed approach: do most of the processing off-chain, b An example of such a service is given by *TON Storage*. In its simplest form, it allows users to store files off-chain, by keeping on-chain only a hash of the file to be stored, and possibly a smart contract where some other parties agree to keep the file in question for a given period of time for a pre-negotiated fee. In fact, the file may be subdivided into chunks of some small size (e.g., 1 kilobyte), augmented by an erasure code such as a Reed–Solomon or a fountain code, a Merkle tree hash may be constructed for the augmented sequence of chunks, and this Merkle tree hash might be published in the smart contract instead of or along with the usual hash of the file. This is somewhat reminiscent of the way files are stored in a torrent. -An even simpler form of storing files is completely off-chain: one might essentially create a "torrent" for a new file, and use TON DHT as a "distributed torrent tracker" for this torrent (cf. [3.2.10](#3-2-10-distributed-torrent-trackers-and-network-interest-groups-in-ton-dht)). This might actually work pretty well for popular files. However, one does not get any availability guarantees. For example, a hypothetical "blockchain Facebook" (cf. [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain)), which would opt to keep the profile photographs of its users completely off-chain in such "torrents", might risk losing photographs of ordinary (not especially popular) users, or at least risk being unable to present these photographs for prolonged periods. The TON Storage technology, which is mostly off-chain, but uses an on-chain smart contract to enforce availability of the stored files, might be a better match for this task. +An even simpler form of storing files is completely off-chain: one might essentially create a "torrent" for a new file, and use TON DHT as a [distributed torrent tracker](#3-2-10-distributed-“torrent-trackers”-and-“network-interest-groups”-in-ton-dht) for this torrent. This might actually work pretty well for popular files. However, one does not get any availability guarantees. For example, a hypothetical [blockchain Facebook](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”), which would opt to keep the profile photographs of its users completely off-chain in such "torrents", might risk losing photographs of ordinary (not especially popular) users, or at least risk being unable to present these photographs for prolonged periods. The TON Storage technology, which is mostly off-chain, but uses an on-chain smart contract to enforce availability of the stored files, might be a better match for this task. #### 4.1.9. Decentralized mixed services, or "fog services" @@ -1776,17 +1777,17 @@ Another example of such a decentralized mixed application arises when one wants #### 4.1.12. Example: TON Payments is a fog service -The TON Payments platform (cf. [5](#5-ton-payments)) is also an example of such a decentralized mixed application. +The [TON Payments](#5-ton-payments) platform is also an example of such a decentralized mixed application. ### 4.2 Connecting Users and Service Providers -We have seen in [4.1.9](#4-1-9-decentralized-mixed-services%2C-or-“fog-services”) that "fog services" (i.e., mixed decentralized services) will usually need some *markets*, *exchanges* or *registries*, where those needing specific services might meet those providing them. +We have seen that [fog services](#4-1-9-decentralized-mixed-services%2C-or-“fog-services”) (i.e., mixed decentralized services) will usually need some *markets*, *exchanges* or *registries*, where those needing specific services might meet those providing them. Such markets are likely to be implemented as on-chain, off-chain or mixed services themselves, centralized or distributed. #### 4.2.1. Example: connecting to TON Payments -For example, if one wants to use TON Payments (cf. [5](#5-ton-payments)), the first step would be to find at least some existing transit nodes of the "lightning network" (cf. [5.2](#5-2-payment-channel-network-or-lightning-network)), and establish payment channels with them, if they are willing. Some nodes can be found with the aid of the "encompassing" overlay network, which is supposed to contain all transit lightning network nodes (cf. [3.3.17](#3-3-17-overlay-networks-within-overlay-networks)). However, it is not clear whether these nodes will be willing to create new payment channels. Therefore, a registry is needed where nodes ready to create new links can publish their contact information (e.g., their abstract addresses). +For example, if one wants to use [TON Payments](#5-ton-payments), the first step would be to find at least some existing transit nodes of the [lightning network](#5-2-payment-channel-network%2C-or-“lightning-network”), and establish payment channels with them, if they are willing. Some nodes can be found with the aid of the "encompassing" [overlay network](#3-3-17-overlay-networks-within-overlay-networks), which is supposed to contain all transit lightning network nodes. However, it is not clear whether these nodes will be willing to create new payment channels. Therefore, a registry is needed where nodes ready to create new links can publish their contact information (e.g., their abstract addresses). #### 4.2.2. Example: uploading a file into TON Storage @@ -1794,21 +1795,21 @@ Similarly, if one wants to upload a file into the TON Storage, she must locate s #### 4.2.3. On-chain, mixed and off-chain registries -Such a registry of service providers might be implemented completely on-chain, with the aid of a smart contract which would keep the registry in its permanent storage. However, this would be quite slow and expensive. A mixed approach is more efficient, where the relatively small and rarely changed on-chain registry is used only to point out some nodes (by their abstract addresses, or by their public keys, which can be used to locate actual abstract addresses as described in [3.2.12](#3-2-12-locating-services)), which provide off-chain (centralized) registry services. +Such a registry of service providers might be implemented completely on-chain, with the aid of a smart contract which would keep the registry in its permanent storage. However, this would be quite slow and expensive. A mixed approach is more efficient, where the relatively small and rarely changed on-chain registry is used only to point out some nodes (by their abstract addresses, or by their public keys, which can be used to [locate](#3-2-12-locating-services) actual abstract addresses, which provide off-chain (centralized) registry services. -Finally, a decentralized, purely off-chain approach might consist of a public overlay network (cf. [3.3](#3-3-overlay-networks-and-multicasting-messages)), where those willing to offer their services, or those looking to buy somebody's services, simply broadcast their offers, signed by their private keys. If the service to be provided is very simple, even broadcasting the offers might be not necessary: the approximate membership of the overlay network itself might be used as a "registry" of those willing to provide a particular service. Then a client requiring this service might locate (cf. [3.3.7](#3-3-7-locating-more-members-of-the-overlay-network-creating-links)) and query some nodes of this overlay network, and then query their neighbors, if the nodes already known are not ready to satisfy its needs. +Finally, a decentralized, purely off-chain approach might consist of a public [overlay network](#3-3-overlay-networks-and-multicasting-messages), where those willing to offer their services, or those looking to buy somebody's services, simply broadcast their offers, signed by their private keys. If the service to be provided is very simple, even broadcasting the offers might be not necessary: the approximate membership of the overlay network itself might be used as a "registry" of those willing to provide a particular service. Then a client requiring this service might [locate](#3-3-7-locating-more-members-of-the-overlay-network-creating-links) and query some nodes of this overlay network, and then query their neighbors, if the nodes already known are not ready to satisfy its needs. #### 4.2.4. Registry or exchange in a side-chain -Another approach to implementing decentralized mixed registries consists in creating an independent specialized blockchain ("side-chain"), maintained by its own set of self-proclaimed validators, who publish their identities in an on-chain smart contract and provide network access to all interested parties to this specialized blockchain, collecting transaction candidates and broadcasting block updates through dedicated overlay networks (cf. [3.3](#3-3-overlay-networks-and-multicasting-messages)). Then any full node for this sidechain can maintain its own copy of the shared registry (essentially equal to the global state of this side-chain), and process arbitrary queries related to this registry. +Another approach to implementing decentralized mixed registries consists in creating an independent specialized blockchain ("side-chain"), maintained by its own set of self-proclaimed validators, who publish their identities in an on-chain smart contract and provide network access to all interested parties to this specialized blockchain, collecting transaction candidates and broadcasting block updates through dedicated [overlay networks](#3-3-overlay-networks-and-multicasting-messages). Then any full node for this sidechain can maintain its own copy of the shared registry (essentially equal to the global state of this side-chain), and process arbitrary queries related to this registry. #### 4.2.5. Registry or exchange in a workchain -Another option is to create a dedicated workchain inside the TON Blockchain, specialized for creating registries, markets and exchanges. This might be more efficient and less expensive than using smart contracts residing in the basic workchain (cf. [2.1.11](#2-1-11-basic-workchain-or-workchain-zero)). However, this would still be more expensive than maintaining registries in side-chains (cf. [4.2.4](#4-2-4-registry-or-exchange-in-a-side-chain)). +Another option is to create a dedicated workchain inside the TON Blockchain, specialized for creating registries, markets and exchanges. This might be more efficient and less expensive than using smart contracts residing in the [basic workchain](#2-1-11-basic-workchain-or-workchain-zero). However, this would still be more expensive than maintaining [registries](#4-2-4-registry-or-exchange-in-a-side-chain) in side-chains. ### 4.3 Accessing TON Services -We have discussed in [4.1](#4-1-ton-service-implementation-strategies) the different approaches one might employ for creating new services and applications residing in the TON ecosystem. Now we discuss how these services might be accessed, and some of the "helper services" that will be provided by TON, including *TON DNS* and *TON Storage*. +We have discussed the different [approaches](#4-1-ton-service-implementation-strategies) one might employ for creating new services and applications residing in the TON ecosystem. Now we discuss how these services might be accessed, and some of the "helper services" that will be provided by TON, including *TON DNS* and *TON Storage*. #### 4.3.1. TON DNS: a mostly on-chain hierarchical domain name service @@ -1820,17 +1821,17 @@ While anybody might in principle implement such a service using the TON Blockcha For example, a user looking to transfer some cryptocurrency to another user or to a merchant may prefer to remember a TON DNS domain name of the account of that user or merchant, instead of keeping their 256-bit account identifiers at hand and copy-pasting them into the recipient field in their light wallet client. -Similarly, TON DNS may be used to locate account identifiers of smart contracts or entry points of ton-services and ton-sites (cf. [4.1.5](#4-1-5-pure-network-services-ton-sites-and-ton-services)), enabling a specialized client ("ton-browser"), or a usual internet browser combined with a specialized ton-proxy extension or stand-alone application, to deliver a WWW-like browsing experience to the user. +Similarly, TON DNS may be used to locate account identifiers of smart contracts or entry points of [ton-services and ton-sites](#4-1-5-pure-network-services%3A-“ton-sites”-and-“ton-services”), enabling a specialized client ("ton-browser"), or a usual internet browser combined with a specialized ton-proxy extension or stand-alone application, to deliver a WWW-like browsing experience to the user. #### 4.3.3. TON DNS smart contracts The TON DNS is implemented by means of a tree of special (DNS) smart contracts. Each DNS smart contract is responsible for registering subdomains of some fixed domain. The "root" DNS smart contract, where level one domains of the TON DNS system will be kept, is located in the masterchain. Its account identifier must be hardcoded into all software that wishes to access the TON DNS database directly. -Any DNS smart contract contains a hashmap, mapping variable-length null-terminated UTF-8 strings into their "values". This hashmap is implemented as a binary Patricia tree, similar to that described in [2.3.7](#2-3-7-definition-of-hashmap-type-as-a-patricia-tree) but supporting variable-length bitstrings as keys. +Any DNS smart contract contains a hashmap, mapping variable-length null-terminated UTF-8 strings into their "values". This hashmap is implemented as a binary [Patricia tree](#2-3-7-definition-of-hashmap-type-as-a-patricia-tree) but supporting variable-length bitstrings as keys. #### 4.3.4. Values of the DNS hashmap, or TON DNS records -As to the values, they are "TON DNS records" described by a TL-scheme (cf. [2.2.5](#2-2-5-tl-or-the-type-language)). They consist of a "magic number", selecting one of the options supported, and then either an account identifier, or a smart-contract identifier, or an abstract network address (cf. [3.1](#3-1-abstract-datagram-network-layer)), or a public key used to locate abstract addresses of a service (cf. [3.2.12](#3-2-12-locating-services)), or a description of an overlay network, and so on. An important case is that of another DNS smart contract: in such a case, that smart contract is used to resolve subdomains of its domain. In this way, one can create separate registries for different domains, controlled by the owners of those domains. +As to the values, they are "TON DNS records" described by a [TL-scheme](#2-2-5-tl%2C-or-the-type-language). They consist of a "magic number", selecting one of the options supported, and then either an account identifier, or a smart-contract identifier, or an [abstract network address](#3-1-abstract-datagram-network-layer), or a public key used to [locate](#3-2-12-locating-services) abstract addresses of a service, or a description of an overlay network, and so on. An important case is that of another DNS smart contract: in such a case, that smart contract is used to resolve subdomains of its domain. In this way, one can create separate registries for different domains, controlled by the owners of those domains. These records may also contain an expiration time, a caching time (usually very large, because updating values in a blockchain too often is expensive), and in most cases a reference to the owner of the subdomain in question. The owner has the right to change this record (in particular, the owner field, thus transferring the domain to somebody else's control), and to prolong it. @@ -1846,7 +1847,7 @@ In principle, any full node for the masterchain or shardchain containing a DNS s However, this approach would work only for certain DNS smart contracts. It would fail miserably if a non-standard DNS smart contract were used. -Instead, an approach based on *general smart contract interfaces and get methods* ([cf. 4.3.11](#4-3-11-get-methods-of-smart-contracts)) is used. Any DNS smart contract must define a get method with a known signature, which is invoked to look up a key. Since this approach makes sense for other smart contracts as well, especially those providing on-chain and mixed services, we explain it in some detail in [4.3.11](#4-3-11-get-methods-of-smart-contracts). +Instead, an approach based on general smart contract interfaces and [get methods](#4-3-11-“get-methods”-of-smart-contracts) is used. Any DNS smart contract must define a get method with a known signature, which is invoked to look up a key. Since this [approach](#4-3-11-“get-methods”-of-smart-contracts) makes sense for other smart contracts as well, especially those providing on-chain and mixed services. #### 4.3.7. Translating a TON DNS domain @@ -1856,15 +1857,15 @@ For example, if one wants to translate `A.B.C`, one looks up keys `.C`, `.B.C`, #### 4.3.8. Translating TON DNS domains for light nodes -In this way, a full node for the masterchain—and also for all shardchains involved in the domain look-up process—might translate any domain name into its current value without external help. A light node might request a full node to do this on its behalf and return the value, along with a Merkle proof (cf. [2.5.11](#2-5-11-merkle-proofs-as-query-responses-from-full-nodes)). This Merkle proof would enable the light node to verify that the answer is correct, so such TON DNS responses cannot be "spoofed" by a malicious interceptor, in contrast to the usual DNS protocol. +In this way, a full node for the masterchain—and also for all shardchains involved in the domain look-up process—might translate any domain name into its current value without external help. A light node might request a full node to do this on its behalf and return the value, along with a [Merkle proof](#2-5-11-merkle-proofs-as-query-responses-from-full-nodes). This Merkle proof would enable the light node to verify that the answer is correct, so such TON DNS responses cannot be "spoofed" by a malicious interceptor, in contrast to the usual DNS protocol. Because no node can be expected to be a full node with respect to all shardchains, actual TON DNS domain translation would involve a combination of these two strategies. #### 4.3.9. Dedicated "TON DNS servers" -One might provide a simple "TON DNS server", which would receive RPC "DNS" queries (e.g., via the ADNL or RLDP protocols described in [3.1](#3-1-abstract-datagram-network-layer)), requesting that the server translate a given domain, process these queries by forwarding some subqueries to other (full) nodes if necessary, and return answers to the original queries, augmented by Merkle proofs if required. +One might provide a simple "TON DNS server", which would receive RPC "DNS" queries (e.g., via the [ADNL or RLDP](#3-1-abstract-datagram-network-layer) protocols, requesting that the server translate a given domain, process these queries by forwarding some subqueries to other (full) nodes if necessary, and return answers to the original queries, augmented by Merkle proofs if required. -Such "DNS servers" might offer their services (for free or not) to any other nodes and especially light clients, using one of the methods described in [4.2](#4-2-connecting-users-and-service-providers). Notice that these servers, if considered part of the TON DNS service, would effectively transform it from a distributed on-chain service into a distributed mixed service (i.e., a "fog service"). +Such "DNS servers" might offer their [services](#4-2-connecting-users-and-service-providers) (for free or not) to any other nodes and especially light clients, using one of the methods. Notice that these servers, if considered part of the TON DNS service, would effectively transform it from a distributed on-chain service into a distributed mixed service (i.e., a "fog service"). This concludes our brief overview of the TON DNS service, a scalable on-chain registry for human-readable domain names of TON Blockchain and TON Network entities. @@ -1882,21 +1883,21 @@ This way is much more elegant and in line with object oriented programming (OOP) #### 4.3.12. Tentative execution of get methods of smart contracts -We have already remarked (cf. [2.4.6](#2-4-6-external-messages-or-messages-from-nowhere)) that any full node can tentatively execute any method of any smart contract (i.e., deliver any message to a smart contract), starting from a given state of the smart contract, without actually committing the corresponding transaction. The full node can simply load the code of the smart contract under consideration into the TON VM, initialize its persistent storage from the global state of the shardchain (known to all full nodes of the shardchain), and execute the smart-contract code with the inbound message as its input parameter. The output messages created will yield the result of this computation. +We have already [remarked](#2-4-6-external-messages%2C-or-“messages-from-nowhere”) that any full node can tentatively execute any method of any smart contract (i.e., deliver any message to a smart contract), starting from a given state of the smart contract, without actually committing the corresponding transaction. The full node can simply load the code of the smart contract under consideration into the TON VM, initialize its persistent storage from the global state of the shardchain (known to all full nodes of the shardchain), and execute the smart-contract code with the inbound message as its input parameter. The output messages created will yield the result of this computation. -In this way, any full node can evaluate arbitrary get methods of arbitrary smart contracts, provided their signature (i.e., the format of inbound and outbound messages) is known. The node may keep track of the cells of the shardchain state accessed during this evaluation, and create a Merkle proof of the validity of the computation performed, for the benefit of a light node that might have asked the full node to do so (cf. [2.5.11](#2-5-11-merkle-proofs-as-query-responses-from-full-nodes)). +In this way, any full node can evaluate arbitrary get methods of arbitrary smart contracts, provided their signature (i.e., the format of inbound and outbound messages) is known. The node may keep track of the cells of the shardchain state accessed during this evaluation, and create a Merkle proof of the validity of the computation performed, for the benefit of a light node that might have asked the [full node](#2-5-11-merkle-proofs-as-query-responses-from-full-nodes) to do so. #### 4.3.13. Smart-contract interfaces in TL-schemes -Recall that the methods implemented by a smart contract (i.e., the input messages accepted by it) are essentially some TL-serialized objects, which can be described by a TL-scheme (cf. [2.2.5](#2-2-5-tl-or-the-type-language)). The resulting output messages can be described by the same TL-scheme as well. In this way, the interface provided by a smart contract to other accounts and smart contracts may be formalized by means of a TL-scheme. +Recall that the methods implemented by a smart contract (i.e., the input messages accepted by it) are essentially some TL-serialized objects, which can be described by a [TL-scheme](#2-2-5-tl%2C-or-the-type-language). The resulting output messages can be described by the same TL-scheme as well. In this way, the interface provided by a smart contract to other accounts and smart contracts may be formalized by means of a TL-scheme. In particular, (a subset of) get methods supported by a smart contract can be described by such a formalized smart-contract interface. #### 4.3.14. Public interfaces of a smart contract -Notice that a formalized smart-contract interface, either in form of a TL-scheme (represented as a TL source file; cf. [2.2.5](#2-2-5-tl-or-the-type-language)) or in serialized form,[36](#fn36) can be published—for example, in a special field in the smart-contract account description, stored in the blockchain, or separately, if this interface will be referred to many times. In the latter case a hash of the supported public interface may be incorporated into the smart-contract description instead of the interface description itself. +Notice that a formalized smart-contract interface, either in form of a TL-scheme (represented as a [TL source](#2-2-5-tl%2C-or-the-type-language) file) or in serialized form,[36](#fn36)can be published—for example, in a special field in the smart-contract account description, stored in the blockchain, or separately, if this interface will be referred to many times. In the latter case a hash of the supported public interface may be incorporated into the smart-contract description instead of the interface description itself. -An example of such a public interface is that of a DNS smart contract, which is supposed to implement at least one standard get method for looking up subdomains (cf. [4.3.6](#4-3-6-retrieving-data-from-a-dns-smart-contract)). A standard method for registering new subdomains can be also included in the standard public interface of DNS smart contracts. +An example of such a public interface is that of a [DNS smart contract](#4-3-6-retrieving-data-from-a-dns-smart-contract), which is supposed to implement at least one standard get method for looking up subdomains. A standard method for registering new subdomains can be also included in the standard public interface of DNS smart contracts. #### 4.3.15. User interface of a smart contract @@ -1906,7 +1907,7 @@ In this way, the user will be able to interact with arbitrary smart contracts fr #### 4.3.16. User interface of a "ton-service" -It turns out that ton-services (i.e., services residing in the TON Network and accepting queries through the ADNL and RLDP protocols of [3](#3-ton-networking); [cf. 4.1.5](#4-1-5-pure-network-services%3A-ton-sites-and-ton-services)) may also profit from having public interfaces, described by TL-schemes ([cf. 2.2.5](#2-2-5-tl%2C-or-the-type-language)). A client application, such as a light wallet or a ton-browser, might prompt the user to select one of the methods and to fill in a form with parameters defined by the interface, similarly to what has just been discussed in [4.3.15](#4-3-15-user-interface-of-a-smart-contract). The only difference is that the resulting TL-serialized message is not submitted as a transaction in the blockchain; instead, it is sent as an RPC query to the abstract address of the ton-service in question, and the response to this query is parsed and displayed according to the formal interface (i.e., a TL-scheme). +It turns out that [ton-services](#4-1-5-pure-network-services:-“ton-sites”-and-“ton-services”) (i.e., services residing in the [TON Network](#3-ton-networking) and accepting queries through the ADNL and RLDP protocols may also profit from having public interfaces, described by TL-schemes. A client application, such as a light wallet or a ton-browser, might prompt the user to select one of the methods and to fill in a form with parameters defined by the [interface](#4-3-15-user-interface-of-a-smart-contract). The only difference is that the resulting TL-serialized message is not submitted as a transaction in the blockchain; instead, it is sent as an RPC query to the abstract address of the ton-service in question, and the response to this query is parsed and displayed according to the formal interface (i.e., a TL-scheme). #### 4.3.17. Locating user interfaces via TON DNS @@ -1914,11 +1915,11 @@ The TON DNS record containing an abstract address of a ton-service or a smart-co #### 4.3.18. Blurring the distinction between on-chain and off-chain services -In this way, the distinction between on-chain, off-chain and mixed services (cf. [4.1.2](#4-1-2-location-of-the-application-on-chain-off-chain-or-mixed)) is blurred for the end user: she simply enters the domain name of the desired service into the address line of her ton-browser or wallet, and the rest is handled seamlessly by the client application. +In this way, the distinction between [on-chain, off-chain and mixed](#4-1-2-location-of-the-application%3A-on-chain%2C-off-chain-or-mixed) services is blurred for the end user: she simply enters the domain name of the desired service into the address line of her ton-browser or wallet, and the rest is handled seamlessly by the client application. #### 4.3.19. A light wallet and TON entity explorer can be built into Telegram Messenger clients -An interesting opportunity arises at this point. A light wallet and TON entity explorer, implementing the above functionality, can be embedded into the Telegram Messenger smartphone client application, thus bringing the technology to more than 200 million people. Users would be able to send hyperlinks to TON entities and resources by including TON URIs (cf. [4.3.22](#4-3-22-hyperlink-urls-may-specify-some-parameters)) in messages; such hyperlinks, if selected, will be opened internally by the Telegram client application of the receiving party, and interaction with the chosen entity will begin. +An interesting opportunity arises at this point. A light wallet and TON entity explorer, implementing the above functionality, can be embedded into the Telegram Messenger smartphone client application, thus bringing the technology to more than 200 million people. Users would be able to send [hyperlinks](#4-3-22-hyperlink-urls-may-specify-some-parameters) to TON entities and resources by including TON URIs in messages; such hyperlinks, if selected, will be opened internally by the Telegram client application of the receiving party, and interaction with the chosen entity will begin. #### 4.3.20. "ton-sites" as ton-services supporting an HTTP interface @@ -1926,7 +1927,7 @@ A *ton-site* is simply a ton-service that supports an HTTP interface, perhaps al #### 4.3.21. Hyperlinks -Notice that the HTML pages returned by ton-sites may contain *ton-hyperlinks*—that is, references to other ton-sites, smart contracts and accounts by means of specially crafted URI schemes (cf. [4.3.22](#4-3-22-hyperlink-urls-may-specify-some-parameters))—containing either abstract network addresses, account identifiers, or human-readable TON DNS domains. Then a "ton-browser" might follow such a hyperlink when the user selects it, detect the interface to be used, and display a user interface form as outlined in [4.3.15](#4-3-15-user-interface-of-a-smart-contract) and [4.3.16](#4-3-16-user-interface-of-a-ton-service). +Notice that the HTML pages returned by ton-sites may contain [ton-hyperlinks](#4-3-22-hyperlink-urls-may-specify-some-parameters)—that is, references to other ton-sites, smart contracts and accounts by means of specially crafted URI schemes—containing either abstract network addresses, account identifiers, or human-readable TON DNS domains. Then a "ton-browser" might follow such a hyperlink when the user selects it, detect the interface to be used, and display a user interface form as outlined in [4.3.15](#4-3-15-user-interface-of-a-smart-contract) and [4.3.16](#4-3-16-user-interface-of-a-“ton-service”). #### 4.3.22. Hyperlink URLs may specify some parameters @@ -1950,13 +1951,14 @@ All of the above will lead to the creation of a whole web of cross-referencing e This "TON WWW" of on-chain and off-chain services has some advantages over its conventional counterpart. For example, payments are inherently integrated in the system. User identity can be always presented to the services (by means of automatically generated signatures on the transactions and RPC requests generated), or hidden at will. Services would not need to check and re-check user credentials; these credentials can be published in the blockchain once and for all. User network anonymity can be easily preserved by means of TON Proxy, and all services will be effectively unblockable. Micropayments are also possible and easy, because ton-browsers can be integrated with the TON Payments system. --- + ## 5 TON Payments The last component of the TON Project we will briefly discuss in this text is *TON Payments*, the platform for (micro)payment channels and "lightning network" value transfers. It would enable "instant" payments, without the need to commit all transactions into the blockchain, pay the associated transaction fees (e.g., for the gas consumed), and wait five seconds until the block containing the transactions in question is confirmed. The overall overhead of such instant payments is so small that one can use them for micropayments. For example, a TON file-storing service might charge the user for every 128 KiB of downloaded data, or a paid TON Proxy might require some tiny micropayment for every 128 KiB of traffic relayed. -While *TON Payments* is likely to be released later than the core components of the TON Project, some considerations need to be made at the very beginning. For example, the TON Virtual Machine (TON VM; cf. [2.1.20](#2-1-20-ton-virtual-machine)), used to execute the code of TON Blockchain smart contracts, must support some special operations with Merkle proofs. If such support is not present in the original design, adding it at a later stage might become problematic (cf. [2.8.16](#2-8-16-complications-of-changing-the-genome-of-a-blockchain-project)). We will see, however, that the TON VM comes with natural support for "smart" payment channels (cf. [5.1.9](#5-1-9-ton-vm-support-for-smart-payment-channels)) out of the box. +While *TON Payments* is likely to be released later than the core components of the TON Project, some considerations need to be made at the very beginning. For example, the [TON Virtual Machine](#2-1-20-ton-virtual-machine) (TON VM), used to execute the code of TON Blockchain smart contracts, must support some special operations with Merkle proofs. If such support is not present in the original design, adding it at a later stage might become [problematic](#2-8-16-complications-of-changing-the-“genome”-of-a-blockchain-project). We will see, however, that the TON VM comes with natural support for [smart](#5-1-9-ton-vm-support-for-“smart”-payment-channels) payment channels out of the box. ### 5.1 Payment Channels @@ -1970,7 +1972,7 @@ Before creating the "money pool", the two sides agree to a certain protocol. The All this updating of balances inside the pool is done completely off-chain. When the two parties decide to withdraw their due funds from the pool, they do so according to the final state of the pool. This is achieved by sending a special message to the smart contract, containing the agreed-upon final state $(a^*,b^*)$ along with the signatures of both $A$ and $B$. Then the smart contract sends $a^*$ coins to $A$, $b^*$ coins to $B$ and self-destructs. -This smart contract, along with the network protocol used by $A$ and $B$ to update the state of the pool, is a simple *payment channel between $A$ and $B$.* According to the classification described in [4.1.2](#4-1-2-location-of-the-application-on-chain-off-chain-or-mixed), it is a *mixed* service: part of its state resides in the blockchain (the smart contract), but most of its state updates are performed off-chain (by the network protocol). If everything goes well, the two parties will be able to perform as many payments to each other as they want (with the only restriction being that the "capacity" of the channel is not overrun—i.e., their balances in the payment channel both remain non-negative), committing only two transactions into the blockchain: one to open (create) the payment channel (smart contract), and another to close (destroy) it. +This smart contract, along with the network protocol used by $A$ and $B$ to update the state of the pool, is a simple *payment channel between $A$ and $B$.* According to the [classification](#4-1-2-location-of-the-application%3A-on-chain%2C-off-chain-or-mixed), it is a *mixed* service: part of its state resides in the blockchain (the smart contract), but most of its state updates are performed off-chain (by the network protocol). If everything goes well, the two parties will be able to perform as many payments to each other as they want (with the only restriction being that the "capacity" of the channel is not overrun—i.e., their balances in the payment channel both remain non-negative), committing only two transactions into the blockchain: one to open (create) the payment channel (smart contract), and another to close (destroy) it. #### 5.1.2. Trustless payment channels @@ -2006,7 +2008,7 @@ If a validator signs an invalid block, or creates a fork, or signs two different #### 5.1.5. Asynchronous payment channel as a virtual blockchain with two workchains -The synchronous payment channel discussed in [5.1.3](#5-1-3-simple-bidirectional-synchronous-trustless-payment-channel) has a certain disadvantage: one cannot begin the next transaction (money transfer inside the payment channel) before the previous one is confirmed by the other party. This can be fixed by replacing the single virtual blockchain discussed in [5.1.4](#5-1-4-synchronous-payment-channel-as-a-simple-virtual-blockchain-with-two-validators) by a system of two interacting virtual workchains (or rather shardchains). +The [synchronous payment channel](#5-1-3-simple-bidirectional-synchronous-trustless-payment-channel) has a certain disadvantage: one cannot begin the next transaction (money transfer inside the payment channel) before the previous one is confirmed by the other party. This can be fixed by replacing the single [virtual blockchain](#5-1-4-synchronous-payment-channel-as-a-simple-virtual-blockchain-with-two-validators) by a system of two interacting virtual workchains (or rather shardchains). The first of these workchains contains only transactions by $A$, and its blocks can be generated only by $A$; its states are $S_i=(i,\phi_i,j,\psi_j)$, where $i$ is the block sequence number (i.e., the count of transactions, or money transfers, performed by $A$ so far), $\phi_i$ is the total amount transferred from $A$ to $B$ so far, $j$ is the sequence number of the most recent valid block in $B$'s blockchain that $A$ is aware of, and $\psi_j$ is the amount of money transferred from $B$ to $A$ in its $j$ transactions. A signature of $B$ put onto its $j$-th block should also be a part of this state. Hashes of the previous block of this workchain and of the $j$-th block of the other workchain may be also included. Validity conditions for $S_i$ include $\phi_i\geq 0$, $\phi_i\geq\phi_{i-1}$ if $i>0$, $\psi_j\geq0$, and $-a\leq\psi_j-\phi_i\leq b$. @@ -2018,29 +2020,29 @@ The payment channel is finalized by $A$ signing (its version of) the final state #### 5.1.6. Unidirectional payment channels -If only $A$ needs to make payments to $B$ (e.g., $B$ is a service provider, and $A$ its client), then a unilateral payment channel can be created. Essentially, it is just the first workchain described in [5.1.5](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) without the second one. Conversely, one can say that the asynchronous payment channel described in [5.1.5](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) consists of two unidirectional payment channels, or "half-channels", managed by the same smart contract. +If only $A$ needs to make payments to $B$ (e.g., $B$ is a service provider, and $A$ its client), then a unilateral payment channel can be created. Essentially, it is just the [first workchain](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) without the second one. Conversely, one can say that the [asynchronous payment channel](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) consists of two unidirectional payment channels, or "half-channels", managed by the same smart contract. #### 5.1.7. More sophisticated payment channels. Promises -We will see later in [5.2.4](#5-2-4-chain-money-transfers) that the "lightning network" (cf. [5.2](#5-2-payment-channel-network-or-lightning-network)), which enables instant money transfers through chains of several payment channels, requires higher degrees of sophistication from the payment channels involved. +We will see later in [5.2.4](#5-2-4-chain-money-transfers) that the [lightning network](#5-2-payment-channel-network%2C-or-“lightning-network”), which enables instant money transfers through chains of several payment channels, requires higher degrees of sophistication from the payment channels involved. In particular, we want to be able to commit "promises", or "conditional money transfers": $A$ agrees to send $c$ coins to $B$, but $B$ will get the money only if a certain condition is fulfilled, for instance, if $B$ can present some string $u$ with $\text{Hash}(u)=v$ for a known value of $v$. Otherwise, $A$ can get the money back after a certain period of time. -Such a promise could easily be implemented on-chain by a simple smart contract. However, we want promises and other kinds of conditional money transfers to be possible off-chain, in the payment channel, because they considerably simplify money transfers along a chain of payment channels existing in the "lightning network" (cf. [5.2.4](#5-2-4-chain-money-transfers)). +Such a promise could easily be implemented on-chain by a simple smart contract. However, we want promises and other kinds of conditional money transfers to be possible off-chain, in the payment channel, because they considerably simplify [money transfers](#5-2-4-chain-money-transfers) along a chain of payment channels existing in the "lightning network". -The "payment channel as a simple blockchain" picture outlined in [5.1.4](#5-1-4-synchronous-payment-channel-as-a-simple-virtual-blockchain-with-two-validators) and [5.1.5](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) becomes convenient here. Now we consider a more complicated virtual blockchain, the state of which contains a set of such unfulfilled "promises", and the amount of funds locked in such promises. This blockchain—or the two workchains in the asynchronous case—will have to refer explicitly to the previous blocks by their hashes. Nevertheless, the general mechanism remains the same. +The payment channel as a simple blockchain" picture outlined in [5.1.4](#5-1-4-synchronous-payment-channel-as-a-simple-virtual-blockchain-with-two-validators) and [5.1.5](#5-1-5-asynchronous-payment-channel-as-a-virtual-blockchain-with-two-workchains) becomes convenient here. Now we consider a more complicated virtual blockchain, the state of which contains a set of such unfulfilled "promises", and the amount of funds locked in such promises. This blockchain—or the two workchains in the asynchronous case—will have to refer explicitly to the previous blocks by their hashes. Nevertheless, the general mechanism remains the same. #### 5.1.8. Challenges for the sophisticated payment channel smart contracts Notice that, while the final state of a sophisticated payment channel is still small, and the "clean" finalization is simple (if the two sides have agreed on their amounts due, and both have signed their agreement, nothing else remains to be done), the unilateral finalization method and the method for punishing fraudulent behavior need to be more complex. Indeed, they must be able to accept Merkle proofs of misbehavior, and to check whether the more sophisticated transactions of the payment channel blockchain have been processed correctly. -In other words, the payment channel smart contract must be able to work with Merkle proofs, to check their "hash validity", and must contain an implementation of $\mathit{ev\_trans}$ and $\mathit{ev\_block}$ functions (cf. [2.2.6](#2-2-6-blocks-and-transactions-as-state-transformation-operators)) for the payment channel (virtual) blockchain. +In other words, the payment channel smart contract must be able to work with Merkle proofs, to check their "hash validity", and must contain an implementation of $\mathit{ev\_trans}$ and $\mathit{ev\_block}$ [functions](#2-2-6-blocks-and-transactions-as-state-transformation-operators) for the payment channel (virtual) blockchain. #### 5.1.9. TON VM support for "smart" payment channels -The TON VM, used to run the code of TON Blockchain smart contracts, is up to the challenge of executing the smart contracts required for "smart", or sophisticated, payment channels (cf. [5.1.8](#5-1-8-challenges-for-the-sophisticated-payment-channel-smart-contracts)). +The TON VM, used to run the code of TON Blockchain smart contracts, is up to the challenge of executing the smart contracts required for "smart", or sophisticated, [payment channels](#5-1-8-challenges-for-the-sophisticated-payment-channel-smart-contracts). -At this point the "everything is a bag of cells" paradigm ([cf. 2.5.14](#2-5-14-everything-is-a-bag-of-cells-philosophy)) becomes extremely convenient. Since all blocks (including the blocks of the ephemeral payment channel blockchain) are represented as bags of cells (and described by some algebraic data types), and the same holds for messages and Merkle proofs as well, a Merkle proof can easily be embedded into an inbound message sent to the payment channel smart contract. The "hash condition" of the Merkle proof will be checked automatically, and when the smart contract accesses the "Merkle proof" presented, it will work with it as if it were a value of the corresponding algebraic data type—albeit incomplete, with some subtrees of the tree replaced by special nodes containing the Merkle hash of the omitted subtree. Then the smart contract will work with that value, which might represent, for instance, a block of the payment channel (virtual) blockchain along with its state, and will evaluate the $\mathit{ev\_block}$ function (cf. [2.2.6](#2-2-6-blocks-and-transactions-as-state-transformation-operators)) of that blockchain on this block and the previous state. Then either the computation finishes, and the final state can be compared with that asserted in the block, or an "absent node" exception is thrown while attempting to access an absent subtree, indicating that the Merkle proof is invalid. +At this point the [everything is a bag of cells](#2-5-14-“everything-is-a-bag-of-cells”-philosophy) paradigm becomes extremely convenient. Since all blocks (including the blocks of the ephemeral payment channel blockchain) are represented as bags of cells (and described by some algebraic data types), and the same holds for messages and Merkle proofs as well, a Merkle proof can easily be embedded into an inbound message sent to the payment channel smart contract. The "hash condition" of the Merkle proof will be checked automatically, and when the smart contract accesses the "Merkle proof" presented, it will work with it as if it were a value of the corresponding algebraic data type—albeit incomplete, with some subtrees of the tree replaced by special nodes containing the Merkle hash of the omitted subtree. Then the smart contract will work with that value, which might represent, for instance, a block of the payment channel (virtual) blockchain along with its state, and will evaluate the $\mathit{ev\_block}$ [function](#2-2-6-blocks-and-transactions-as-state-transformation-operators) of that blockchain on this block and the previous state. Then either the computation finishes, and the final state can be compared with that asserted in the block, or an "absent node" exception is thrown while attempting to access an absent subtree, indicating that the Merkle proof is invalid. In this way, the implementation of the verification code for smart payment channel blockchains turns out to be quite straightforward using TON Blockchain smart contracts. One might say that *the TON Virtual Machine comes with built-in support for checking the validity of other simple blockchains.* The only limiting factor is the size of the Merkle proof to be incorporated into the inbound message to the smart contract (i.e., into the transaction). @@ -2048,11 +2050,11 @@ In this way, the implementation of the verification code for smart payment chann We would like to discuss the possibility of creating a simple (synchronous or asynchronous) payment channel inside an existing payment channel. -While this may seem somewhat convoluted, it is not much harder to understand and implement than the "promises" discussed in [5.1.7](#5-1-7-more-sophisticated-payment-channels-promises). Essentially, instead of promising to pay $c$ coins to the other party if a solution to some hash problem is presented, $A$ promises to pay up to $c$ coins to $B$ according to the final settlement of some other (virtual) payment channel blockchain. Generally speaking, this other payment channel blockchain need not even be between $A$ and $B$; it might involve some other parties, say, $C$ and $D$, willing to commit $c$ and $d$ coins into their simple payment channel, respectively. (This possibility is exploited later in [5.2.5](#5-2-5-virtual-payment-channels-inside-a-chain-of-payment-channels).) +While this may seem somewhat convoluted, it is not much harder to understand and implement than the [promises](#5-1-7-more-sophisticated-payment-channels-promises). Essentially, instead of promising to pay $c$ coins to the other party if a solution to some hash problem is presented, $A$ promises to pay up to $c$ coins to $B$ according to the final settlement of some other (virtual) payment channel blockchain. Generally speaking, this other payment channel blockchain need not even be between $A$ and $B$; it might involve some other parties, say, $C$ and $D$, willing to commit $c$ and $d$ coins into their simple [payment channel](#5-2-5-virtual-payment-channels-inside-a-chain-of-payment-channels), respectively. If the encompassing payment channel is asymmetric, two promises need to be committed into the two workchains: $A$ will promise to pay $-\delta$ coins to $B$ if the final settlement of the "internal" simple payment channel yields a negative final imbalance $\delta$ with $0\leq-\delta\leq c$; and $B$ will have to promise to pay $\delta$ to $A$ if $\delta$ is positive. On the other hand, if the encompassing payment channel is symmetric, this can be done by committing a single "simple payment channel creation" transaction with parameters $(c,d)$ into the single payment channel blockchain by $A$ (which would freeze $c$ coins belonging to $A$), and then committing a special "confirmation transaction" by $B$ (which would freeze $d$ coins of $B$). -We expect the internal payment channel to be extremely simple (e.g., the simple synchronous payment channel discussed in [5.1.3](#5-1-3-simple-bidirectional-synchronous-trustless-payment-channel)), to minimize the size of Merkle proofs to be submitted. The external payment channel will have to be "smart" in the sense described in [5.1.7](#5-1-7-more-sophisticated-payment-channels-promises). +We expect the internal payment channel to be extremely simple (e.g., the simple [synchronous payment channel](#5-1-3-simple-bidirectional-synchronous-trustless-payment-channel), to minimize the size of Merkle proofs to be submitted. The external payment channel will have to be [smart](#5-1-7-more-sophisticated-payment-channels-promises). ### 5.2 Payment Channel Network, or "Lightning Network" @@ -2068,7 +2070,7 @@ Payment channel networks overcome the limitations of payment channels by enablin #### 5.2.3. Overview of payment channel networks -Recall that a *payment channel network*, known also as a "lightning network", consists of a collection of participating nodes, some of which have established long-lived payment channels between them. We will see in a moment that these payment channels will have to be "smart" in the sense of [5.1.7](#5-1-7-more-sophisticated-payment-channels-promises). When a participating node $A$ wants to transfer money to any other participating node $E$, she tries to find a path linking $A$ to $E$ inside the payment channel network. When such a path is found, she performs a "chain money transfer" along this path. +Recall that a *payment channel network*, known also as a "lightning network", consists of a collection of participating nodes, some of which have established long-lived payment channels between them. We will see in a moment that these payment channels will have to be [smart](#5-1-7-more-sophisticated-payment-channels-promises). When a participating node $A$ wants to transfer money to any other participating node $E$, she tries to find a path linking $A$ to $E$ inside the payment channel network. When such a path is found, she performs a "chain money transfer" along this path. #### 5.2.4. Chain money transfers @@ -2076,7 +2078,7 @@ Suppose that there is a chain of payment channels from $A$ to $B$, from $B$ to $ A simplistic approach would be to transfer $x$ coins to $B$ along the existing payment channel, and ask him to forward the money further to $C$. However, it is not evident why $B$ would not simply take the money for himself. Therefore, one must employ a more sophisticated approach, not requiring all parties involved to trust each other. -This can be achieved as follows. $A$ generates a large random number $u$ and computes its hash $v=\text{Hash}(u)$. Then she creates a promise to pay $x$ coins to $B$ if a number $u$ with hash $v$ is presented (cf. [5.1.7](#5-1-7-more-sophisticated-payment-channels-promises)), inside her payment channel with $B$. This promise contains $v$, but not $u$, which is still kept secret. +This can be achieved as follows. $A$ generates a large random number $u$ and computes its hash $v=\text{Hash}(u)$. Then she creates a [promise](#5-1-7-more-sophisticated-payment-channels-promises) to pay $x$ coins to $B$ if a number $u$ with hash $v$ is presented, inside her payment channel with $B$. This promise contains $v$, but not $u$, which is still kept secret. After that, $B$ creates a similar promise to $C$ in their payment channel. He is not afraid to give such a promise, because he is aware of the existence of a similar promise given to him by $A$. If $C$ ever presents a solution of the hash problem to collect $x$ coins promised by $B$, then $B$ will immediately submit this solution to $A$ to collect $x$ coins from $A$. @@ -2086,21 +2088,21 @@ Some minor details are omitted in this description. For example, these promises #### 5.2.5. Virtual payment channels inside a chain of payment channels -Now suppose that $A$ and $E$ expect to make a lot of payments to each other. They might create a new payment channel between them in the blockchain, but this would still be quite expensive, because some funds would be locked in this payment channel. Another option would be to use chain money transfers described in [5.2.4](#5-2-4-chain-money-transfers) for each payment. However, this would involve a lot of network activity and a lot of transactions in the virtual blockchains of all payment channels involved. +Now suppose that $A$ and $E$ expect to make a lot of payments to each other. They might create a new payment channel between them in the blockchain, but this would still be quite expensive, because some funds would be locked in this payment channel. Another option would be to use [chain money transfers](#5-2-4-chain-money-transfers) for each payment. However, this would involve a lot of network activity and a lot of transactions in the virtual blockchains of all payment channels involved. -An alternative is to create a virtual payment channel inside the chain linking $A$ to $E$ in the payment channel network. For this, $A$ and $E$ create a (virtual) blockchain for their payments, as if they were going to create a payment channel in the blockchain. However, instead of creating a payment channel smart contract in the blockchain, they ask all intermediate payment channels—those linking $A$ to $B$, $B$ to $C$, etc.—to create simple payment channels inside them, bound to the virtual blockchain created by $A$ and $E$ (cf. [5.1.10](#5-1-10-simple-payment-channel-within-a-smart-payment-channel)). In other words, now a promise to transfer money according to the final settlement between $A$ and $E$ exists inside every intermediate payment channel. +An alternative is to create a virtual payment channel inside the chain linking $A$ to $E$ in the payment channel network. For this, $A$ and $E$ create a (virtual) blockchain for their payments, as if they were going to create a payment channel in the blockchain. However, instead of creating a payment channel smart contract in the blockchain, they ask all intermediate payment channels—those linking $A$ to $B$, $B$ to $C$, etc.—to create simple [payment channels](#5-1-10-simple-payment-channel-within-a-smart-payment-channel) inside them, bound to the virtual blockchain created by $A$ and $E$. In other words, now a promise to transfer money according to the final settlement between $A$ and $E$ exists inside every intermediate payment channel. -If the virtual payment channel is unidirectional, such promises can be implemented quite easily, because the final imbalance $\delta$ is going to be non-positive, so simple payment channels can be created inside intermediate payment channels in the same order as described in [5.2.4](#5-2-4-chain-money-transfers). Their expiration times can also be set in the same way. +If the virtual payment channel is unidirectional, such promises can be implemented quite easily, because the final imbalance $\delta$ is going to be non-positive, so simple payment channels can be created inside intermediate [payment channels](#5-2-4-chain-money-transfers) in the same order . Their expiration times can also be set in the same way. -If the virtual payment channel is bidirectional, the situation is slightly more complicated. In that case, one should split the promise to transfer $\delta$ coins according to the final settlement into two half-promises, as explained in [5.1.10](#5-1-10-simple-payment-channel-within-a-smart-payment-channel): to transfer $\delta^-=\max(0,-\delta)$ coins in the forward direction, and to transfer $\delta^+=\max(0,\delta)$ in the backward direction. These half-promises can be created in the intermediate payment channels independently, one chain of half-promises in the direction from $A$ to $E$, and the other chain in the opposite direction. +If the virtual payment channel is bidirectional, the situation is slightly more complicated. In that case, one should split the promise to transfer $\delta$ coins according to the final settlement into two [half-promises](#5-1-10-simple-payment-channel-within-a-smart-payment-channel): to transfer $\delta^-=\max(0,-\delta)$ coins in the forward direction, and to transfer $\delta^+=\max(0,\delta)$ in the backward direction. These half-promises can be created in the intermediate payment channels independently, one chain of half-promises in the direction from $A$ to $E$, and the other chain in the opposite direction. #### 5.2.6. Finding paths in the lightning network -One point remains undiscussed so far: how will $A$ and $E$ find a path connecting them in the payment network? If the payment network is not too large, an OSPF-like protocol can be used: all nodes of the payment network create an overlay network (cf. [3.3.17](#3-3-17-overlay-networks-within-overlay-networks)), and then every node propagates all available link (i.e., participating payment channel) information to its neighbors by a gossip protocol. Ultimately, all nodes will have a complete list of all payment channels participating in the payment network, and will be able to find the shortest paths by themselves—for example, by applying a version of Dijkstra's algorithm modified to take into account the "capacities" of the payment channels involved (i.e., the maximal amounts that can be transferred along them). Once a candidate path is found, it can be probed by a special ADNL datagram containing the full path, and asking each intermediate node to confirm the existence of the payment channel in question, and to forward this datagram further according to the path. After that, a chain can be constructed, and a protocol for chain transfers (cf. [5.2.4](#5-2-4-chain-money-transfers)), or for creating a virtual payment channel inside a chain of payment channels (cf. [5.2.5](#5-2-5-virtual-payment-channels-inside-a-chain-of-payment-channels)), can be run. +One point remains undiscussed so far: how will $A$ and $E$ find a path connecting them in the payment network? If the payment network is not too large, an OSPF-like protocol can be used: all nodes of the payment network create an [overlay network](#3-3-17-overlay-networks-within-overlay-networks), and then every node propagates all available link (i.e., participating payment channel) information to its neighbors by a gossip protocol. Ultimately, all nodes will have a complete list of all payment channels participating in the payment network, and will be able to find the shortest paths by themselves—for example, by applying a version of Dijkstra's algorithm modified to take into account the "capacities" of the payment channels involved (i.e., the maximal amounts that can be transferred along them). Once a candidate path is found, it can be probed by a special ADNL datagram containing the full path, and asking each intermediate node to confirm the existence of the payment channel in question, and to forward this datagram further according to the path. After that, a chain can be constructed, and a protocol for [chain transfers](#5-2-4-chain-money-transfers), or for creating a virtual payment channel inside a chain of [payment channels](#5-2-5-virtual-payment-channels-inside-a-chain-of-payment-channels), can be run. #### 5.2.7. Optimizations -Some optimizations might be done here. For example, only transit nodes of the lightning network need to participate in the OSPF-like protocol discussed in [5.2.6](#5-2-6-finding-paths-in-the-lightning-network). Two "leaf" nodes wishing to connect through the lightning network would communicate to each other the lists of transit nodes they are connected to (i.e., with which they have established payment channels). Then paths connecting transit nodes from one list to transit nodes from the other list can be inspected as outlined above in [5.2.6](#5-2-6-finding-paths-in-the-lightning-network). +Some optimizations might be done here. For example, only transit nodes of the [lightning network](#5-2-6-finding-paths-in-the-lightning-network) need to participate in the OSPF-like protocol. Two "leaf" nodes wishing to connect through the lightning network would communicate to each other the lists of transit nodes they are connected to (i.e., with which they have established payment channels). Then paths connecting transit nodes from one list to transit nodes from the other list can be [inspected](#5-2-6-finding-paths-in-the-lightning-network). #### 5.2.8. Conclusion @@ -2110,11 +2112,11 @@ We have outlined how the blockchain and network technologies of the TON project We have proposed a scalable multi-blockchain architecture capable of supporting a massively popular cryptocurrency and decentralized applications with user-friendly interfaces. -To achieve the necessary scalability, we proposed the *TON Blockchain*, a "tightly-coupled" multi-blockchain system (cf. [2.8.14](#2-8-14-interaction-between-blockchains-loosely-coupled-and-tightly-coupled-systems)) with bottom-up approach to sharding (cf. [2.8.12](#2-8-12-sharding-support) and [2.1.2](#2-1-2-infinite-sharding-paradigm)). To further increase potential performance, we introduced the 2-blockchain mechanism for replacing invalid blocks (cf. [2.1.17](#2-1-17-correcting-invalid-shardchain-blocks)) and Instant Hypercube Routing for faster communication between shards (cf. [2.4.20](#2-4-20-instant-hypercube-routing-fast-path-for-messages)). A brief comparison of the TON Blockchain to existing and proposed blockchain projects (cf. [2.8](#2-8-classification-of-blockchain-projects) and [2.9](#2-9-comparison-to-other-blockchain-projects)) highlights the benefits of this approach for systems that seek to handle millions of transactions per second. +To achieve the necessary scalability, we proposed the *TON Blockchain*, a [tightly-coupled](#2-8-14-interaction-between-blockchains%3A-loosely-coupled-and-tightly-coupled-systems) multi-blockchain system with bottom-up approach to sharding ([2.8.12](#2-8-12-sharding-support) or [2.1.2](#2-1-2-infinite-sharding-paradigm)). To further increase potential performance, we introduced the 2-blockchain mechanism for replacing [invalid](#2-1-17-correcting-invalid-shardchain-blocks) blocks and [Instant Hypercube Routing](#2-4-20-instant-hypercube-routing:-“fast-path”-for-messages) for faster communication between shards. A brief comparison of the TON Blockchain to existing and proposed blockchain projects ([2.8](#2-8-classification-of-blockchain-projects) and [2.9](#2-9-comparison-to-other-blockchain-projects)) highlights the benefits of this approach for systems that seek to handle millions of transactions per second. -The *TON Network*, described in Chapter [3](#3-ton-networking), covers the networking demands of the proposed multi-blockchain infrastructure. This network component may also be used in combination with the blockchain to create a wide spectrum of applications and services, impossible using the blockchain alone (cf. [2.9.13](#2-9-13-is-it-possible-to-upload-facebook-into-a-blockchain)). These services, discussed in Chapter [4](#4-ton-services-and-applications), include *TON DNS*, a service for translating human-readable object identifiers into their addresses; *TON Storage*, a distributed platform for storing arbitrary files; *TON Proxy*, a service for anonymizing network access and accessing TON-powered services; and *TON Payments* (cf. Chapter [5](#5-ton-payments)), a platform for instant off-chain money transfers across the TON ecosystem that applications may use for micropayments. +The [TON Network](#3-ton-networking), covers the networking demands of the proposed multi-blockchain infrastructure. This network component may also be used in combination with the blockchain to create a wide spectrum of [applications and services](#2-9-13-is-it-possible-to-“upload-facebook-into-a-blockchain”), impossible using the blockchain alone. These [services](#4-ton-services-and-applications), include *TON DNS*, a service for translating human-readable object identifiers into their addresses; *TON Storage*, a distributed platform for storing arbitrary files; *TON Proxy*, a service for anonymizing network access and accessing TON-powered services; and [TON Payments](#5-ton-payments), a platform for instant off-chain money transfers across the TON ecosystem that applications may use for micropayments. -The TON infrastructure allows for specialized light client wallet and "ton-browser" desktop and smartphone applications that enable a browser-like experience for the end user (cf. [4.3.24](#4-3-24-ton-www)), making cryptocurrency payments and interaction with smart contracts and other services on the TON Platform accessible to the mass user. Such a light client can be integrated into the Telegram Messenger client (cf. [4.3.19](#4-3-19-a-light-wallet-and-ton-entity-explorer-can-be-built-into-telegram-messenger-clients)), thus eventually bringing a wealth of blockchain-based applications to hundreds of millions of users. +The TON infrastructure allows for specialized light client wallet and "ton-browser" desktop and smartphone applications that enable a [browser-like experience](#4-3-24-ton-www) for the end user, making cryptocurrency payments and interaction with smart contracts and other services on the TON Platform accessible to the mass user. Such a light client can be integrated into the [Telegram Messenger client](#4-3-19-a-light-wallet-and-ton-entity-explorer-can-be-built-into-telegram-messenger-clients), thus eventually bringing a wealth of blockchain-based applications to hundreds of millions of users. --- @@ -2146,7 +2148,7 @@ The total supply of Grams is originally limited to $5$ Gigagrams (i.e., five bil This supply will increase very slowly, as rewards to validators for mining new masterchain and shardchain blocks accumulate. These rewards would amount to approximately $20\%$ (the exact number may be adjusted in future) of the validator's stake per year, provided the validator diligently performs its duties, signs all blocks, never goes offline and never signs invalid blocks. In this way, the validators will have enough profit to invest into better and faster hardware needed to process the ever growing quantity of users' transactions. -We expect that at most $10\%$[37](#fn37) of the total supply of Grams, on average, will be bound in validator stakes at any given moment. This will produce an inflation rate of $2\%$ per year, and as a result, will double the total supply of Grams (to ten Gigagrams) in 35 years. Essentially, this inflation represents a payment made by all members of the community to the validators for keeping the system up and running. +We expect that at most $10\%$[37](#fn37)of the total supply of Grams, on average, will be bound in validator stakes at any given moment. This will produce an inflation rate of $2\%$ per year, and as a result, will double the total supply of Grams (to ten Gigagrams) in 35 years. Essentially, this inflation represents a payment made by all members of the community to the validators for keeping the system up and running. On the other hand, if a validator is caught misbehaving, a part or all of its stake will be taken away as a punishment, and a larger portion of it will subsequently be "burned", decreasing the total supply of Grams. This would lead to deflation. A smaller portion of the fine may be redistributed to the validator or the "fisherman" who committed a proof of the guilty validator's misbehavior. @@ -2229,11 +2231,11 @@ While this is necessary for the quick start of the project, the ultimate goal is #### A.5.3. Taking a pre-arranged amount from the Reserve -The TON Foundation will transfer to its account a small part of the TON Reserve—say, 10% of all coins (i.e. 500 Megagrams) after the end of the initial sale of Grams—to be used for its own purposes as outlined in [A.5.2](#a-5-2-the-ton-foundation-needs-grams-for-operational-purposes). This is best done simultaneously with the transfer of the funds intended for TON developers, as mentioned in [A.5.1](#a-5-1-some-unallocated-grams-will-be-given-to-developers). +The TON Foundation will transfer to its account a [small part](#a-5-2-the-ton-foundation-needs-grams-for-operational-purposes) of the TON Reserve—say, 10% of all coins (i.e. 500 Megagrams) after the end of the initial sale of Grams—to be used for its own purposes. This is best done simultaneously with the transfer of the funds intended for [TON developers](#a-5-1-some-unallocated-grams-will-be-given-to-developers). After the transfers to the TON Foundation and the TON developers, the TON Reserve price $p(n)$ of the Gram will immediately rise by a certain amount, known in advance. For example, if 10% of all coins are transferred for the purposes of the TON Foundation, and 4% are transferred for the encouragement of the developers, then the total quantity $n$ of coins in circulation will immediately increase by $\Delta n=7\cdot10^8$, with the price of the Gram multiplying by $e^{\alpha\,\Delta n}=e^{0.7}\approx 2$ (i.e, doubling). -The remaining "unallocated" Grams will be used by the TON Reserve as explained above in [A.5](#a-5-using-unallocated-grams). If the TON Foundation needs any more Grams thereafter, it will simply convert into Grams some of the funds it had previously obtained during the sale of the coins, either on the free market or by buying Grams from the TON Reserve. To prevent excessive centralization, the TON Foundation will never endeavour to have more than 10% of the total amount of Grams (i.e., 500 Megagrams) on its account. +The remaining [unallocated](#a-5-using-unallocated-grams) Grams will be used by the TON Reserve. If the TON Foundation needs any more Grams thereafter, it will simply convert into Grams some of the funds it had previously obtained during the sale of the coins, either on the free market or by buying Grams from the TON Reserve. To prevent excessive centralization, the TON Foundation will never endeavour to have more than 10% of the total amount of Grams (i.e., 500 Megagrams) on its account. ### A.6. Bulk sales of Grams @@ -2251,115 +2253,114 @@ The "bulk sales" mechanism will probably be used extensively during the initial ## References -[1] K. Birman, *Reliable Distributed Systems: Technologies, Web Services and Applications*, Springer, 2005. +1. K. Birman, *Reliable Distributed Systems: Technologies, Web Services and Applications*, Springer, 2005. -[2] V. Buterin, *Ethereum: A next-generation smart contract and decentralized application platform*, https://github.com/ethereum/wiki/wiki/White-Paper, 2013. +2. V. Buterin, *Ethereum: A next-generation smart contract and decentralized application platform*, https://github.com/ethereum/wiki/wiki/White-Paper, 2013. -[3] M. Ben-Or, B. Kelmer, T. Rabin, *Asynchronous secure computations with optimal resilience*, in *Proceedings of the thirteenth annual ACM symposium on Principles of distributed computing*, p. 183–192. ACM, 1994. +3. M. Ben-Or, B. Kelmer, T. Rabin, *Asynchronous secure computations with optimal resilience*, in *Proceedings of the thirteenth annual ACM symposium on Principles of distributed computing*, p. 183–192. ACM, 1994. -[4] M. Castro, B. Liskov, et al., *Practical byzantine fault tolerance*, Proceedings of the Third Symposium on Operating Systems Design and Implementation (1999), p. 173–186, available at http://pmg.csail.mit.edu/papers/osdi99.pdf. +4. M. Castro, B. Liskov, et al., *Practical byzantine fault tolerance*, Proceedings of the Third Symposium on Operating Systems Design and Implementation (1999), p. 173–186, available at http://pmg.csail.mit.edu/papers/osdi99.pdf. -[5] EOS.IO, *EOS.IO technical white paper*, https://github.com/EOSIO/Documentation/blob/master/TechnicalWhitePaper.md, 2017. +5. EOS.IO, *EOS.IO technical white paper*, https://github.com/EOSIO/Documentation/blob/master/TechnicalWhitePaper.md, 2017. -[6] D. Goldschlag, M. Reed, P. Syverson, *Onion Routing for Anonymous and Private Internet Connections*, Communications of the ACM, **42**, num. 2 (1999), http://www.onion-router.net/Publications/CACM-1999.pdf. +6. D. Goldschlag, M. Reed, P. Syverson, *Onion Routing for Anonymous and Private Internet Connections*, Communications of the ACM, **42**, num. 2 (1999), http://www.onion-router.net/Publications/CACM-1999.pdf. -[7] L. Lamport, R. Shostak, M. Pease, *The byzantine generals problem*, ACM Transactions on Programming Languages and Systems, **4/3** (1982), p. 382–401. +7. L. Lamport, R. Shostak, M. Pease, *The byzantine generals problem*, ACM Transactions on Programming Languages and Systems, **4/3** (1982), p. 382–401. -[8] S. Larimer, *The history of BitShares*, https://docs.bitshares.org/bitshares/history.html, 2013. +8. S. Larimer, *The history of BitShares*, https://docs.bitshares.org/bitshares/history.html, 2013. -[9] M. Luby, A. Shokrollahi, et al., *RaptorQ forward error correction scheme for object delivery*, IETF RFC 6330, https://tools.ietf.org/html/rfc6330, 2011. +9. M. Luby, A. Shokrollahi, et al., *RaptorQ forward error correction scheme for object delivery*, IETF RFC 6330, https://tools.ietf.org/html/rfc6330, 2011. -[10] P. Maymounkov, D. Mazières, *Kademlia: A peer-to-peer information system based on the XOR metric*, in *IPTPS '01 revised papers from the First International Workshop on Peer-to-Peer Systems*, p. 53–65, available at http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf, 2002. +10. P. Maymounkov, D. Mazières, *Kademlia: A peer-to-peer information system based on the XOR metric*, in *IPTPS '01 revised papers from the First International Workshop on Peer-to-Peer Systems*, p. 53–65, available at http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf, 2002. -[11] A. Miller, Yu Xia, et al., *The honey badger of BFT protocols*, Cryptology e-print archive 2016/99, https://eprint.iacr.org/2016/199.pdf, 2016. +11. A. Miller, Yu Xia, et al., *The honey badger of BFT protocols*, Cryptology e-print archive 2016/99, https://eprint.iacr.org/2016/199.pdf, 2016. -[12] S. Nakamoto, *Bitcoin: A peer-to-peer electronic cash system*, https://bitcoin.org/bitcoin.pdf, 2008. +12. S. Nakamoto, *Bitcoin: A peer-to-peer electronic cash system*, https://bitcoin.org/bitcoin.pdf, 2008. -[13] S. Peyton Jones, *Implementing lazy functional languages on stock hardware: the Spineless Tagless G-machine*, Journal of Functional Programming **2** (2), p. 127–202, 1992. +13. S. Peyton Jones, *Implementing lazy functional languages on stock hardware: the Spineless Tagless G-machine*, Journal of Functional Programming **2** (2), p. 127–202, 1992. -[14] A. Shokrollahi, M. Luby, *Raptor Codes*, IEEE Transactions on Information Theory **6**, no. 3–4 (2006), p. 212–322. +14. A. Shokrollahi, M. Luby, *Raptor Codes*, IEEE Transactions on Information Theory **6**, no. 3–4 (2006), p. 212–322. -[15] M. van Steen, A. Tanenbaum, *Distributed Systems, 3rd ed.*, 2017. +15. M. van Steen, A. Tanenbaum, *Distributed Systems, 3rd ed.*, 2017. -[16] The Univalent Foundations Program, *Homotopy Type Theory: Univalent Foundations of Mathematics*, Institute for Advanced Study, 2013, available at https://homotopytypetheory.org/book. +16. The Univalent Foundations Program, *Homotopy Type Theory: Univalent Foundations of Mathematics*, Institute for Advanced Study, 2013, available at https://homotopytypetheory.org/book. -[17] G. Wood, *PolkaDot: vision for a heterogeneous multi-chain framework*, draft 1, https://github.com/w3f/polkadot-white-paper/raw/master/PolkaDotPaper.pdf, 2016. +17. G. Wood, *PolkaDot: vision for a heterogeneous multi-chain framework*, draft 1, https://github.com/w3f/polkadot-white-paper/raw/master/PolkaDotPaper.pdf, 2016. ----- +--- ## Footnotes -1. https://github.com/ethereum/wiki/wiki/Sharding-FAQ [Back ↑](#2-1-1-list-of-blockchain-types) - -2. Actually, two-thirds by stake is enough to achieve consensus, but an effort is made to collect as many signatures as possible. [Back ↑](#2-1-15-generation-of-new-blocks-by-validators) +**1.** https://github.com/ethereum/wiki/wiki/Sharding-FAQ [Back ↑](#ref-fn1) -3. https://coq.inria.fr [Back ↑](#2-2-4-dependent-type-theory-coq-and-tl) +**2.** Actually, two-thirds by stake is enough to achieve consensus, but an effort is made to collect as many signatures as possible. [Back ↑](#ref-fn2) -4. https://core.telegram.org/mtproto/TL [Back ↑](#2-2-4-dependent-type-theory-coq-and-tl) +**3.** https://coq.inria.fr [Back ↑](#ref-fn3) -5. One can show that this encoding is optimal for approximately half of all edge labels of a Patricia tree with random or consecutive indices. Remaining edge labels are likely to be long (i.e., almost 256 bits long). Therefore, a nearly optimal encoding for edge labels is to use the above code with prefix 0 for "short" bit strings, and encode 1, then nine bits containing length $l=|s|$ of bitstring $s$, and then the $l$ bits of $s$ for "long" bitstrings (with $l\geq10$). [Back ↑](#2-3-8-merkle-patricia-trees) +**4.** https://core.telegram.org/mtproto/TL [Back ↑](#ref-fn4) -6. A *light node* is a node that does not keep track of the full state of a shardchain; instead, it keeps minimal information such as the hashes of the several most recent blocks, and relies on information obtained from full nodes when it becomes necessary to inspect some parts of the full state. [Back ↑](#2-3-10-merkle-proofs) +**5.** One can show that this encoding is optimal for approximately half of all edge labels of a Patricia tree with random or consecutive indices. Remaining edge labels are likely to be long (i.e., almost 256 bits long). Therefore, a nearly optimal encoding for edge labels is to use the above code with prefix 0 for "short" bit strings, and encode 1, then nine bits containing length $l=|s|$ of bitstring $s$, and then the $l$ bits of $s$ for "long" bitstrings (with $l\geq10$). [Back ↑](#ref-fn5) -7. A *full node* is a node keeping track of the complete up-to-date state of the shardchain in question. [Back ↑](#2-3-10-merkle-proofs) +**6.** A light node is a node that does not keep track of the full state of a shardchain; instead, it keeps minimal information such as the hashes of the several most recent blocks, and relies on information obtained from full nodes when it becomes necessary to inspect some parts of the full state. [Back ↑](#ref-fn6) -8. These two descriptor bytes, present in any TVM cell, describe only the total number of references and the total number of raw bytes; references are kept together either before or after all raw bytes. [Back ↑](#2-3-12-peculiarities-of-ton-vm) +**7.** A full node is a node keeping track of the complete up-to-date state of the shardchain in question. [Back ↑](#ref-fn7) -9. Actually, $\text{Leaf}$ and $\text{Node}$ are constructors of an auxiliary type, $\text{HashmapAux}(n,X)$. Type $\text{Hashmap}(n,X)$ has constructors $\text{Root}$ and $\text{EmptyRoot}$, with $\text{Root}$ containing a value of type $\text{HashmapAux}(n,X)$. [Back ↑](#2-3-12-peculiarities-of-ton-vm) +**8.** These two descriptor bytes, present in any TVM cell, describe only the total number of references and the total number of raw bytes; references are kept together either before or after all raw bytes. [Back ↑](#ref-fn8) -10. Logically; the "bag of cells" representation described in [2.5.5](#2-5-5-low-level-perspective%3A-bag-of-cells) identifies all duplicate cells, transforming this tree into a directed acyclic graph (dag) when serialized. [Back ↑](#2-3-14-tvm-cells) +**9.** Actually, $\text{Leaf}$ and $\text{Node}$ are constructors of an auxiliary type, $\text{HashmapAux}(n,X)$. Type $\text{Hashmap}(n,X)$ has constructors $\text{Root}$ and $\text{EmptyRoot}$, with $\text{Root}$ containing a value of type $\text{HashmapAux}(n,X)$. [Back ↑](#ref-fn9) -11. A more expensive alternative is to publish such a "global" smart contract in the masterchain. [Back ↑](#2-3-18-local-and-global-smart-contracts-smart-contract-instances) +**10.** Logically; the "bag of cells" representation described in 2.5.5 identifies all duplicate cells, transforming this tree into a directed acyclic graph (dag) when serialized. [Back ↑](#ref-fn10) -12. This is a sort of "broadcast" feature for all shards, and as such, it must be quite expensive. [Back ↑](#2-3-18-local-and-global-smart-contracts-smart-contract-instances) +**11.** A more expensive alternative is to publish such a "global" smart contract in the masterchain. [Back ↑](#ref-fn11) -13. The above needs to be literally true only for the basic workchain and its shardchains; other workchains may provide other ways of creating messages. [Back ↑](#2-4-11-creating-messages-smart-contracts-and-transactions) +**12.** This is a sort of "broadcast" feature for all shards, and as such, it must be quite expensive. [Back ↑](#ref-fn12) -14. As a degenerate case, this shardchain may coincide with the originating shardchain—for example, if we are working inside a workchain which has not yet been split. [Back ↑](#2-4-12-delivering-messages) +**13.** The above needs to be literally true only for the basic workchain and its shardchains; other workchains may provide other ways of creating messages. [Back ↑](#ref-fn13) -15. This is not necessarily the final version of the algorithm used to compute the next hop for hypercube routing. In particular, hexadecimal digits may be replaced by $r$-bit groups, with $r$ a configurable parameter, not necessarily equal to four. [Back ↑](#2-4-19-hypercube-routing-slow-path-for-messages-with-assured-delivery) +**14.** As a degenerate case, this shardchain may coincide with the originating shardchain—for example, if we are working inside a workchain which has not yet been split. [Back ↑](#ref-fn14) -16. However, the validators have some incentive to do so as soon as possible, because they will be able to collect all forwarding fees associated with the message that have not yet been consumed along the slow path. [Back ↑](#2-4-20-instant-hypercube-routing-fast-path-for-messages) +**15.** This is not necessarily the final version of the algorithm used to compute the next hop for hypercube routing. In particular, hexadecimal digits may be replaced by $r$-bit groups, with $r$ a configurable parameter, not necessarily equal to four. [Back ↑](#ref-fn15) -17. In fact, one might temporarily or permanently disable the "instant delivery" mechanism altogether, and the system would continue working, albeit more slowly. [Back ↑](#2-4-20-instant-hypercube-routing-fast-path-for-messages) +**16.** However, the validators have some incentive to do so as soon as possible, because they will be able to collect all forwarding fees associated with the message that have not yet been consumed along the slow path. [Back ↑](#ref-fn16) -18. One can show that, if Merkle proofs for all data stored in a tree of cells are needed equally often, one should use cells with $b+ch\approx 2(h+r)$ to minimize average Merkle proof size, where $h=32$ is the hash size in bytes, and $r\approx4$ is the "byte size" of a cell reference. In other words, a cell should contain either two references and a few raw bytes, or one reference and about 36 raw bytes, or no references at all with 72 raw bytes. [Back ↑](#2-5-5-low-level-perspective-bag-of-cells) +**17.** In fact, one might temporarily or permanently disable the "instant delivery" mechanism altogether, and the system would continue working, albeit more slowly. [Back ↑](#ref-fn17) -19. A better implementation would be to keep the state of the smart contract as a serialized string, if it is small, or in a separate $B$-tree, if it is large; then the top-level structure representing the state of a blockchain would be a $B$-tree, whose leaves are allowed to contain references to other $B$-trees. [Back ↑](#2-5-5-low-level-perspective-bag-of-cells) +**18.** One can show that, if Merkle proofs for all data stored in a tree of cells are needed equally often, one should use cells with $b+ch\approx2(h+r)$ to minimize average Merkle proof size, where $h=32$ is the hash size in bytes, and $r\approx4$ is the "byte size" of a cell reference. In other words, a cell should contain either two references and a few raw bytes, or one reference and about 36 raw bytes, or no references at all with 72 raw bytes. [Back ↑](#ref-fn18) -20. It makes sense to generate and use a new key pair for every validator election. [Back ↑](#2-6-7-global-validator-set-election) +**19.** A better implementation would be to keep the state of the smart contract as a serialized string, if it is small, or in a separate $B$-tree, if it is large; then the top-level structure representing the state of a blockchain would be a $B$-tree, whose leaves are allowed to contain references to other $B$-trees. [Back ↑](#ref-fn19) -21. A possible exception is the state of output queues of the neighboring shardchains, needed to guarantee the message ordering requirements described in [2.4.21](#2-4-21-collecting-input-messages-from-output-queues-of-neighboring-shardchains), because the size of Merkle proofs might become prohibitive in this case. [Back ↑](#2-6-11-validation-of-block-candidates) +**20.** It makes sense to generate and use a new key pair for every validator election. [Back ↑](#ref-fn20) -22. Actually, the shard configuration is completely determined by the last masterchain block; this simplifies getting access to the shard configuration. [Back ↑](#2-7-1-shard-configuration) +**21.** A possible exception is the state of output queues of the neighboring shardchains, needed to guarantee the message ordering requirements described in 2.4.21, because the size of Merkle proofs might become prohibitive in this case. [Back ↑](#ref-fn21) -23. Unless some validators are temporarily or permanently banned because of signing invalid blocks—then they are automatically excluded from all task groups. [Back ↑](#2-7-4-validator-task-groups-for-new-shardchains) +**22.** Actually, the shard configuration is completely determined by the last masterchain block; this simplifies getting access to the shard configuration. [Back ↑](#ref-fn22) -24. More like 15, for the time being. However, some upgrades are being planned to make Ethereum transaction throughput several times larger. [Back ↑](#2-8-2-single-blockchain-vs-multi-blockchain-projects) +**23.** Unless some validators are temporarily or permanently banned because of signing invalid blocks—then they are automatically excluded from all task groups. [Back ↑](#ref-fn23) -25. Some people even claim DPOS block generation times of half a second, which does not seem realistic if validators are scattered across several continents. [Back ↑](#2-8-5-comparison-of-dpos-and-bft-pos) +**24.** More like 15, for the time being. However, some upgrades are being planned to make Ethereum transaction throughput several times larger. [Back ↑](#ref-fn24) -26. For instance, EOS, one of the best DPOS projects proposed up to this date, promises a 45-second confirmation and inter-blockchain interaction delay (cf. EOS White Paper, "Transaction Confirmation" and "Latency of Interchain Communication" sections). [Back ↑](#2-8-5-comparison-of-dpos-and-bft-pos) +**25.** Some people even claim DPOS block generation times of half a second, which does not seem realistic if validators are scattered across several continents. [Back ↑](#ref-fn25) -27. For example, the Plasma project plans to use the Ethereum blockchain as its (external) masterchain; it does not interact much with Ethereum otherwise, and it could have been suggested and implemented by a team unrelated to the Ethereum project. [Back ↑](#2-8-16-complications-of-changing-the-genome-of-a-blockchain-project) +**26.** For instance, EOS, one of the best DPOS projects proposed up to this date, promises a 45-second confirmation and inter-blockchain interaction delay (cf. EOS White Paper, "Transaction Confirmation" and "Latency of Interchain Communication" sections). [Back ↑](#ref-fn26) -28. As of 2017, Ethereum is still struggling to transition from PoW to a combined PoW+PoS system; we hope it will become a truly PoS system someday. [Back ↑](#2-8-16-complications-of-changing-the-genome-of-a-blockchain-project) +**27.** For example, the Plasma project plans to use the Ethereum blockchain as its (external) masterchain; it does not interact much with Ethereum otherwise, and it could have been suggested and implemented by a team unrelated to the Ethereum project. [Back ↑](#ref-fn27) -29. There are sharding proposals for Ethereum dating back to 2015; it is unclear how they might be implemented and deployed without disrupting Ethereum or creating an essentially independent parallel project. [Back ↑](#2-8-16-complications-of-changing-the-genome-of-a-blockchain-project) +**28.** As of 2017, Ethereum is still struggling to transition from PoW to a combined PoW+PoS system; we hope it will become a truly PoS system someday. [Back ↑](#ref-fn28) -30. https://blog.ethereum.org/2015/08/01/introducing-casper-friendly-ghost/ [Back ↑](#2-9-5-casper) +**29.** There are sharding proposals for Ethereum dating back to 2015; it is unclear how they might be implemented and deployed without disrupting Ethereum or creating an essentially independent parallel project. [Back ↑](#ref-fn29) -31. https://geti2p.net/en/docs/how/garlic-routing [Back ↑](#3-1-6-channel-as-a-tunnel-identifier) +**30.** https://blog.ethereum.org/2015/08/01/introducing-casper-friendly-ghost/ [Back ↑](#ref-fn30) -32. If there are sufficiently many nodes in a bucket, it can be subdivided further into, say, eight sub-buckets depending on the top four bits of the Kademlia distance. This would speed up DHT lookups. [Back ↑](#3-2-6-kademlia-routing-table) +**31.** https://geti2p.net/en/docs/how/garlic-routing [Back ↑](#ref-fn31) -33. Alternatively, the abstract address might be stored in the DHT as explained in [3.2.12](#3-2-12-locating-services). [Back ↑](#3-3-5-joining-an-overlay-network) +**32.** If there are sufficiently many nodes in a bucket, it can be subdivided further into, say, eight sub-buckets depending on the top four bits of the Kademlia distance. This would speed up DHT lookups. [Back ↑](#ref-fn32) -34. https://core.telegram.org/mtproto [Back ↑](#4-1-4-telegram-messenger-as-a-ton-service-mtproto-over-rldp) +**33.** Alternatively, the abstract address might be stored in the DHT as explained in 3.2.12. [Back ↑](#ref-fn33) -35. https://telegram.org/ [Back ↑](#4-1-4-telegram-messenger-as-a-ton-service-mtproto-over-rldp) +**34.** https://core.telegram.org/mtproto [Back ↑](#ref-fn34) -36. TL-schemes can be TL-serialized themselves; cf. https://core.telegram.org/mtproto/TL-tl. [Back ↑](#4-3-14-public-interfaces-of-a-smart-contract) +**35.** https://telegram.org/ [Back ↑](#ref-fn35) -37. The maximum total amount of validator stakes is a configurable parameter of the blockchain, so this restriction can be enforced by the protocol if necessary. [Back ↑](#a-3-original-supply-mining-rewards-and-inflation) +**36.** TL-schemes can be TL-serialized themselves; cf. https://core.telegram.org/mtproto/TL-tl. [Back ↑](#ref-fn36) +**37.** The maximum total amount of validator stakes is a configurable parameter of the blockchain, so this restriction can be enforced by the protocol if necessary. [Back ↑](#ref-fn37) \ No newline at end of file