1. Document Description

1.1 Targeted Readers

This document is applicable to the enterprises and individuals with the demands of high security, high privacy, data storage of intellectual property and others, distribution and commercialization. This document guides the developers to use ThunderChain File System, carry out the acts of data uploading, downloading, copying, deletion and authorization, etc., carry out secondary development based on this document so as to enable the product to own the features of distributed storage of big data and to load information on the chain, etc., and rapidly integrate the capacities of huge volume storage, permanent prevention from data falsification and forgery, traceability, never lost, security encryption and authorized transfer, etc.

1.2 Release Notes

v3.0.2(2019/1/7)

2. Terms

2.1 ThunderChain File System

ThunderChain File System (TCFS for short) is the open file system of data cloud storage and authorized distribution especially developed by Xunleifor Blockchain with innovative distributed technology based on the million-level shared computing nodes. The developers can enable the products to own the features of distributed storage of file data and to load information on the chain, and thus rapidly integrate the capacities of openness and transparency, permanent prevention from data falsification and forgery, traceability, high reliability, security encryption, huge volume storage and authorized transfer, etc.

There are mainly four features:

1. Openness and transparency: Data is indexed with a secure cryptographic hash, which will become a new file after modification and generate a new hash to ensure the data to be tamper-proof; with the unique file management technology based on merkle DAG structure, all change history of the files is saved and capable of being inquired and traced.
2. High security: Data are stored by security slice using the public and private key signature technique to verify the holding relationship between the user and the file. A proprietary token authorization mechanism allows only authorized users to access data.
3. High reliability: After redundant data are encoded by FEC (Forward Erasure Code), they are stored on each shared computing node. At the system level, the automatic file repair mechanism is adopted, with datareliability of more than 15x9.Use smart contracts to implement the storage incentive mechanism to ensure stable participating nodes and more reliable file storage.
4. Huge volume data storage: The distributed storage of file data is on million-level shared computing nodes, with hundreds of PB and ever-expanding storage space, which can save tens of billions of files and sufficiently meet the business demand.

3. Terms

3.1 APP ID

APP ID refers to the number created to indicate the attributed app during the developer's registration on an open platform.

3.2 APP Key

App Key refers to the token created during the developer registration on an open platform. As the token of APP ID, it cannot be exposed. It is encrypted by MD5 as the token of APP ID (marked as APP Sign, details explained in Section 3.3).

3.3 APP Sign

APP Sign refers to the token generated by APP Key through MD5 encryption. When calling SDK interface, APP ID and APP Sign need to be passed to SDK for authentication. APP Sign generation mode is MD5 (APP ID+APP Key+ts), where ts refers to the expiration time of signature (unix timestamp), in seconds. Calculation example refers to Section 5.2.3.

3.4 Public Key/Private Key/Address

Public Key and PrivateKey are the key pair (namely a public key and a private key) gained from asymmetric cryptographic algorithm. Address is the coded value of public key calculated from one-way encryption Hash algorithm. In the system, address is the owner information that identifies the users and indicates the file. This information is to be used by User Sign. Public Key/Private Key/Address is stored in keyjson format. Refer to section 6.8.2 for the creation method.

3.5 User Sign

User Sign is used to prove that the current operation is authorized by a specific user. The signature is normally gained in the following modes:

1. Call LinkToken Pocket for signature
2. Signature on server SDK
3. Signature from signature function (refer to Section 5.7 signature description for detailed creation algorithm)

3.6 TCFS Path

TCFS Path is content addressing specification based on authorization, and a path format combining user authorization with directory structure merkle DAG. The format definition is as follows: /tcfs/[owner]:[visitor]:[ts]:[sign]:[pathlength]/[newest/hash]/[path]/[filename]
For details, refer to Section 5.8 TCFS Path Detailed Description.

4. Start to Use

4.1 Developer Registration and Login

The individual or enterprise developers must be registered as the user of open platform before using ThunderChain File System. The mobile No. is the sole identity of the user. After successful registration, the operations including email, individual/enterprise authentication, and app creation, etc., can be linked. Refer to ThunderChainBeginner's Guide

4.2 How to Create APP ID and APP Key

After the developer completes app creation and passes verification, APP ID and App Key will be assigned to the developer, which can be checked in Administration Center – App Center – App Details. APP ID refers to the No. created to indicate the attributed app during the developer's registration on cloud storage. App Key refers to encrypted key corresponding to developer APP ID. As the token of APP ID, the developer should keep it properly.

4.3 How to Calculate the Rates

Specification	12-month purchase	Current promotion
100GB	discounted price 99 LinkToken	99 LinkToken refunded after 12 months

Note: LinkToken purchased each time can only be integer multiple of 100GB.

5. Process Description

5.1 Use Process of ThunderChain File System

5.2 APP ID Verification Process

5.2.1 APP ID Description

The enterprise and individual developers need to register. After passing verification, an APP ID and corresponding APP Key will be assigned. During interface call, APP ID and md5 values of some parameters should be input. ThunderChain File System will internally verify whether the two match and are legitimate so as to verify the identity.

5.2.2 APPID Verification Process Sequence Diagram

The verification logic of APPID/APPKey should be used in all the interfaces of ThunderChain File System. APP Sign calculation refers to Section 3.3.

5.2.3 APP Sign Calculation Examples

Examples:

APPID=de61d625adef267d69d63ca711939d685ffc8007
APPKey= 4daa50d08a107185af2c015cba126575781f7b20
ts = 1529114930
md5_value=md5sum("de61d625adef267d69d63ca711939d685ffc80074daa50d08a107185af2c015cba126575781f7b201529114930")=2b2cface59cc85b0916a6c46b68f2f20

ts refers to the expiration time of signature (unix timestamp).

In 6.1 Parameter Structure Definition, the parameter input is as follows

TCFSAppCertification ac{
appid=”de61d625adef267d69d63ca711939d685ffc8007”
timestamp=1529114930
appSign=”2b2cface59cc85b0916a6c46b68f2f20”
}

5.3 File Upload Process

5.3.1 Upload Description

AddFile interface of ThunderChain File System is used for file uploading, which requires APPID/APPKey verification and Address identity signature authentication. After passing verification, the local file will be read, coded and sliced before being distributed to the shared nodes, and the file information will be written in ThunderChain File System.

5.3.2 Upload Process Sequence Diagram

5.3.3 User Guide

Third-party App only needs to complete the following steps for file uploading:

1. Get md5 value of APPID+APPKey+ timestamp from third-party background
2. Get sign in the mode of Section 5.7
3. Call AddFile interface
4. Inquire task status through GetTaskStatus

5.4 File Downloading

5.4.1 Downloading Description

When ThunderChain File System is used to download the file, it can only download the uploaded file. The files failing to be uploaded or authorized cannot be downloaded. The file downloading requires APPID/APPKey verification and Address identity signature verification.

5.4.2 Downloading Process Sequence Diagram

5.4.3 User Guide

Third-party App only needs to complete the following steps for file downloading:

1. Get md5 value of APPID+APPKey+ timestamp from third-party background
2. Get sign in the mode of Section 5.7
3. Call GetFile interface
4. Inquire task status through GetTaskStatus

5.5 File Copying

5.5.1 Copying Description

The copying function of ThunderChain File System is to copy FA the file owned by user A (AddressA) to user B (AddressB) to enable storage space of user B to own file FA.

5.5.2 Copying Process Sequence Diagram

5.5.3 User Guide

Third-party App only needs to complete the following steps for file copying:

1. Get md5 value of APPID+APPKey+ timestamp from third-party background
2. Get sign in the mode of Section 5.7. Be noted that user A (AddressA) and user B (AddressB) both need to sign and the signature needs to add filehash field.
3. Call CopyFile interface

5.5.4 Examples of Copying Function

Because CopyFile will copy the file from user A to user B, user A and user B both need to sign.

5.6 File Deletion

5.6.1 Deletion Description

Deletion function of ThunderChain File System is to delete the file of user A (addressA) from the latest directory tree, and the file will not participate in rate calculation any more. The system regularly cleans unpaid files, so they may be successfully accessed for some time after deletion. But the file operation history will remain permanently.

5.6.2 Deletion Process Sequence Diagram

5.6.3 User Guide

Third-party App only needs to complete the following steps for file deletion:

1. Get md5 value of APPID+APPKey+ timestamp from third-party background
2. Get sign in the mode of Section 5.7.
3. Call DeleteFile interface

5.7 User Signature

5.7.1 Signature Description

Signature refers to the verification information to verify whether Address identity is legitimate. The purpose of designing sign is to simplify private key used for multiple times since user private key is used when user wallet signs with private key. The signature with limited duration is designed as one-time signature that can be saved. Every time when SDK interface of ThunderChain File System is called, the signature is directly input so as to fulfill the objective of one-time signature for the use of multiple times.

5.7.2 Signature Generation Mode

Normally, TCFS_UserSign (keyjson, passwd, data，sign, signLen). keyjson is corresponding keystore file of the user wallet. passwd is user's password.

Signature is normally gained in the following three modes:

1. Call LinkToken Pocket for signature Flow of calling chain pocket is as detailed in document center-> chain link access->3.7.
2. Signature on server SDK
3. Call SDK user signature interface TCFS_UserSign

5.8 TCFS Path Detailed Description

5.8.1 TCFS Path Format Definition

/tcfs/[owner]:[visitor]:[ts]:[sign]:[pathlength]/[newest/hash]/[pathname]/[filename]
Among these:
/ is reserved character
tcfs is key word and cannot be changed
owner is the owner address of the path
visitor is the visitor address of the path
ts refers to the expiration time of signature (unix timestamp) sign refers to the authorized signature given by the file owner to the visitor. sign=UserSign(owner+visitor+ts+path[0:pathlength]). ThunderChain File System will verify the signature. Within the duration of valid time stamp, if the signature matches, it will be deemed as successful authorization. pathlength refers to the path length of the file to be authorized in file path
[newest/hash]/[pathname]/[filename] are 2 representation formats of file path. newest/[pathname]/[filename] indicates the latest version of the file; hash/[pathname]/[filename] refers to the file in corresponding hash path for the file.

5.8.2 Directory Structure Description

The format of file path is based on the directory structure, which needs to support 2 modes of newest and hash access. On directory nodes, all the nodes contain version information, corresponding hash value, and the nodes in latest version of current directory structure attributed to newest.

In the above picture, newest points at blank initially. After f1 node is added, newest points at version1.

In the above picture, after d2/f2 file is added, newest pointer points at version2, and all the nodes contain hash information, which provides hash directory inquiry. For example, for access to f1, the hash mode is /hash_v1/f1 or /hash_v2/f1; in newest mode, it is /newest/f1.

5.8.3 Format Examples

The explanation of detailed format examples is as follows:
Format 1, newest format with signature
/tcfs/owner:visitor:ts:sign:pathlength/newest/[pathname]/[filename]
/tcfs/0x7eff122b94897ea5b0e2a9abf47b86337fafebdc:0xd5e51811fd8d4525aaa247e5d96800462b057db3:1535709360:0x052cb64baeeb12f49dd6c5d6db7ec93f0c5d45dc1bc1f728d0a2170bdd52af4f75fb02b4ed09825756dccdc55b8b09c6ab0ad11ccab441c9c9d0b7cb25e002b500:12/newest/abc/lib/
owner:0x7eff122b94897ea5b0e2a9abf47b86337fafebdc indicates that the owner's ownership for directory /newest/abc/lib/
visitor: 0xd5e51811fd8d4525aaa247e5d96800462b057db3 indicates that the visitor needs to access directory /newest/abc/lib/
ts: 1535709360 refers to the valid expiry time stamp of the authorization given by owner to visitor access to all owner's directory /newest/abc/lib/ (unix timestamp).
sign: 0x052cb64baeeb12f49dd6c5d6db7ec93f0c5d45dc1bc1f728d0a2170bdd52af4f75fb02b4ed09825756dccdc55b8b09c6ab0ad11ccab441c9c9d0b7cb25e002b500 refers to the authorization signature of the owner to the visitor's access to all owner's directory /newest/abc/lib/.
pathlength:12. refers to authorized path length of the directory. 12 refers to the signature of directory /newest/abc/, and it is stipulated that after the signature of parent path is given, the access to subpath is also granted.
path:/newest/abc/lib/ refers to the directory the visitor needs to access.

Format 2, newest format without signature
/tcfs/::::/newest/[pathname]/[filename]
/tcfs/::::/newest/Chinese directory3/abc/ddd
path: /newest/ Chinese directory3/abc/ddd refers to the directory the owner needs to access. Be noted that when there is no signature field sign, the owner can only access his/her own directory. In Section 6 Interface List, other parameters will transmit owner.

Format 3, hash format with signature
/tcfs/0x7eff122b94897ea5b0e2a9abf47b86337fafebdc:0xd5e51811fd8d4525aaa247e5d96800462b057db3:1536148761:0xde2baffe959990d641be0aff1864fa27149212b2ccd774614c8a8bf63552d44c5bc9b334ded33722b52c7e53d91e68480574fe9342c2ad5e49f9f025af2348ce00:46/5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/
owner:0x7eff122b94897ea5b0e2a9abf47b86337fafebdc, refers to the ownership of the owner for the directory /5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/
visitor: 0xd5e51811fd8d4525aaa247e5d96800462b057db3, means that the visitor needs to access directory /5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/
ts: 1536148761 refers to valid expiry time stamp of authorized signature of the owner given to visitor to access all owner's directory (unix timestamp).
sign: 0xde2baffe959990d641be0aff1864fa27149212b2ccd774614c8a8bf63552d44c5bc9b334ded33722b52c7e53d91e68480574fe9342c2ad5e49f9f025af2348ce00 refers to the owner's authorized signature for the visitor to access all owner's directory.
pathlength:46 refers to authorized path length of the directory. 46 is the signature for directory /5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/ and stipulates that with the signature of parent path, the access to subpath is also granted.
path: /5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/, refers to the directory the visitor needs to access.

Format 4, hash format without signature
/tcfs/::::/5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/
path: /5415dc5c31c6e4849fb6063ba09ff18365c25a21/abc/ refers to the directory the owner needs to access. Be noted that when there is no signature field sign, the owner can only access his/her own directory.

6. Interface List

So far, TCFS SDK only supports Ubuntu 16.04 (x64) system.

6.1 Parameter Structure Definition

//Error code
enum{
    TCFS_ERRCODE_OK = 0, 
    TCFS_ERRCODE_INITIALIZE_ERROR,
    TCFS_ERRCODE_ALREADY_INITIALIZED,
    TCFS_ERRCODE_UNINITIALIZED,
    TCFS_ERRCODE_PATH_ERROR,
    TCFS_ERRCODE_TCFS_PATH_ERROR,
    TCFS_ERRCODE_APPID_VERIFY_FAILED,
    TCFS_ERRCODE_ADDRESS_VERIFY_FAILED,
    TCFS_ERRCODE_NO_ENOUGH_SPACE,
    TCFS_ERRCODE_SIGN_FAILED,
    TCFS_ERRCODE_EXCCED_USER_MEM,
    TCFS_ERRCODE_ACCESS_SERVER_FAILED,
    TCFS_ERRCODE_ACCESS_FILE_FAILED,
    TCFS_ERRCODE_EXCCED_MAX_FILE_NUM
};

// information for file or folder
struct TreeNodeT
{
    char path[MAX_PATH_LEN + 1]; // path of file or folder
    char hash[HASH_LEN + 1];    // HASH in directory tree
    uint64_t ctime;     // creation time
    uint64_t size;  // file size, not involved in folder
    int type; // indicates whether it is a file or a folder, 0 indicates a file, 1 indicates a folder 
};


struct TCFSAppCertification { // certificate issued by developer
char appid[64]; // developer identification
uint64_t    timestamp; // sign expiry time (s)
char appSign[64];  // verification value of developer identity 
};

struct TCFSUserCertification { // certificate issued by user
char address[64]; // user identification
uint64_t timestamp; // signature expiry time (s)
char userSign[128]; // signature for structure members for self-verification of user identification
};

6.2 Initialization Interface

int TCFS_Init()

• Function description: For uploading and storing in ThunderChain File System by slicing after file encryption
• Parameter: None
• Returned value

Parameter name	type	description
ret	int	If upload task addition succeeds, return 0. If it fails, return ErrorCode

6.3 Quitting Interface

int TCFS_Uninit()

• Function description: For releasing TCFS SDK
• Parameter: None
• Returned value

Parameter name	type	description
ret	int	If upload task addition succeeds, return 0. If it fails, return ErrorCode

6.4 Uploading Interface

int TCFS_Upload(TCFSAppCertification appCert, TCFSUserCertification userCert, const char srcPath, const char tcfsPath, bool isDir);

• Function description: For uploading and storing in ThunderChain File System by slicing after file encryption
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
srcPath	const char*	refers to file path to be uploaded. Only available for local file
tcfsPath	const char*	refers to target TCFS directory, newest format without sign
isDir	bool	indicates whether it is a file or a folder

• Returned value

Parameter name	type	description
ret	int	If upload task addition succeeds, return 0. If it fails, return ErrorCode

6.5 Downloading Interface

int TCFS_Download(TCFSAppCertification appCert, TCFSUserCertification userCert, const char dstPath, const char tcfsPath, bool isDir)

• Function description: Drag the file from ThunderChain File System and recover as local file
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
dstPath	const char*	refers to file path to be downloaded
tcfsPath	const char*	refers to TCFS path of the file to be collected. When accessing the directory of others, support directory
isDir	bool	indicates whether it is a file or a folder

• Returned value

Parameter name	type	description
ret	int	If upload task addition succeeds, return 0. If it fails, return ErrorCode

6.6 Deletion Interface

int TCFS_Delete(TCFSAppCertification appCert, TCFSUserCertification userCert, const char *tcfsPath, int isDir)

• Function description: Delete the file from directory tree in ThunderChain File System
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
tcfsPath	const char*	refers to TCFS path of the file to be deleted, support directory

• Returned value

Parameter name	type	description
ret	int	If upload task addition succeeds, return 0. If it fails, return ErrorCode

6.7 Copying Interface

int TCFS_Copy(TCFSAppCertification appCert, TCFSUserCertification userCert, const char signTcfsPath, const char tcfsPath)

• Function description: To copy the file from tcfsPath to destPath in ThunderChain File System
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
signTcfsPath	const char*	refers to path of the file to be copied. Support directory must be newest/hash with sign in TCFS path format
tcfsPath	const char*	refers to target TCFS directory, newest format without sign

• Returned value

Parameter name	type	description
ret	int	If it succeeds, return 0. If it fails, return ErrorCode

6.8 User Signature Interface

int TCFS_UserSign(const char keyjson, const char password, const char data, char signData, int signLen)

• Function description: sign data
• Parameter

Parameter name	type	description
keyjson	const char*	user's private key string file keyjson, detailed in Section 6.8.2
password	const char*	corresponding password of user's private key file
data	const char*	data for signature
signData	char*	for the return of signed data array
signLen	int	sign array length

• Returned value

Parameter name	type	description
ret	int	If it succeeds, return 0. If it fails, return ErrorCode

6.8.1 Signature Examples

const keyJson = `{
  "address":"7eff122b94897ea5b0e2a9abf47b86337fafebdc",
  "id":"f86a62b4-0621-4616-99af-c4b7f38fcc48","version":3,
  "crypto":{
    "cipher":"aes-128-ctr","ciphertext":"19de8a919e2f4cbdde2b7352ebd0be8ead2c87db35fc8e4c9acaf74aaaa57dad",
    "cipherparams":{"iv":"ba2bd370d6c9d5845e92fbc6f951c792"},
    "kdf":"scrypt","kdfparams":{"dklen":32,"n":262144,"p":1,"r":8,"salt":"c7cc2380a96adc9eb31d20bd8d8a7827199e8b16889582c0b9089da6a9f58e84"},
    "mac":"ff2c0caf051ca15d8c43b6f321ec10bd99bd654ddcf12dd1a28f730cc3c13730"
  }
}`
pwd := "1234"

appid := "de61d625adef267d69d63ca711939d685ffc8007"
address := "0x7eff122b94897ea5b0e2a9abf47b86337fafebdc"
ts := 1529216849
filehash := ""
sign_str := appid + address + fmt.Sprintf("%d", ts)  + nonce + filehash
sign, err := UserSign (keyJson, pwd, sign_str)

Including：
ts refers to the expiration time of signature (unix timestamp)
sign_str= de61d625adef267d69d63ca711939d685ffc80070x7eff122b94897ea5b0e2a9abf47b86337fafebdc1529216849abcdefgh
sign= a54a4c369f608d3966dc531324795073585e288fd908e47c017302e514f8790a7bd41a2cb020230f14a48fb4d9d6f88eb33019b3b1f6ef097c486938324f593601

6.8.2 Examples of keyjson Creation

The steps of creating keyjson with LinkToken Pocket are as follows:

1. Create account Click Create Account in upper right corner and enter password in the new page.

2. Export keystore file Export the backup and then get keystore file. The content of keystore file is keyjson.

6.9 Directory Browsing Interface

int TCFS_ListDir(TCFSAppCertification appCert, TCFSUserCertification userCert, const char tcfsPath, int start, int len, TreeNodeT node[], int maxNodeNum, int realNodeNum)

• Function: browse thunderbolt chain file system directory structure
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
tcfsPath	const char*	ar*
start	int	starting node to return
length	int	number of nodes to return
node	TreeNodeT*[]	for the return of file or folder array
maxNodeNum	int	maximum number of storage files or folder entries
realNodeNum	int	real number of storage files or folder entries

• Returned value

Parameter name	type	description
ret	int	If it succeeds, return 0. If it fails, return ErrorCode

6.10 Directory Tree Browsing Interface

int TCFS_ListTree(TCFSAppCertification appCert, TCFSUserCertification userCert, const char tcfsPath, TreeNodeT node[], int maxNodeNum, int realNodeNum)

• Function：Query task status
• Parameter

Parameter name	type	description
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
tcfsPath	const char*	refers to target TCFS directory, newest/hash format without sign
start	int	starting node to return
length	int	umber of nodes to return
node	TreeNodeT*[]	for the return of file or folder array
maxNodeNum	int	maximum number of storage files or folder entries
realNodeNum	int	real number of storage files or folder entries

• Returned value

Parameter name	type	description
ret	int	If it succeeds, return 0. If it fails, return ErrorCode

6.11 Getting User Space Interface

int TCFS_GetSpace(TCFSAppCertification appCert, TCFSUserCertification userCert, uint64_t totalSize, uint64_t usedSize)

• Function：get user space
• Parameter

Parameter name	type	descripti
appCert	TCFSAppCertification*	for the input of developer authentication information
userCert	TCFSUserCertification*	for the input of file owner authentication information, where userSign= TCFS_UserSign(keyjson, passwd, addr+appid+timestamp)
totalSize	uint64_t*	for the return of user total space
usedSize	uint64_t*	for the return of user used space

• Returned value

Parameter name	type	description
ret	int	If it succeeds, return 0. If it fails, return ErrorCode