TON payment processor

Microservice for accepting payments and making withdrawals to wallets in TON blockchain.
Supports TON coins and Jettons (conforming certain criteria)
Provides REST API for integration. Service is ADNL based and interacts directly with node and do not use any third party API.

Warning The project is in the testing and proof-of-concepts phase. Use with caution. Suggestions are welcome.

• How it works
  • Features
• Glossary
• Prerequisites
  • Criteria for valid Jettons
• Deployment
  • Configurable parameters
  • Service deploy
• Payment notifications
• Binary comment support
• REST API
• Technical notes
• Threat model
• Manual migrations
• TODO list

How it works

The service provides the following functionality:

• generate new deposit address of wallet (TON or Jetton) for TON blockchain. This address you provide to customer for payment. Payments are accumulated in the hot-wallet.
• make withdrawal of TONs or Jettons from hot-wallet to customer wallet at TON blockchain.

Features

• Deposit addresses can be reused for multiple payments
• Sends withdrawals with comment
• Sends withdrawals in batches using highload wallet
• Aggregates part of TONs or Jettons at cold-wallet
• Supports authorization by Bearer token
• Service withdrawals (cancellation of incorrect payments)

Glossary

• deposit-address - address generated by the service to which users send payments.
• deposit - blockchain account with deposit-address
• hot-wallet - wallet for aggregation all incoming TONs and Jettons from deposit-addresses.
• cold-wallet - wallet to which part of the funds from the hot wallet is sent for security. Cold-wallet seed phrase is not used by the service.
• user_id - unique text value to identify deposit-addresses or withdrawal request for a specific user.
• query_id - unique text value to identify withdrawal request for a specific user to prevent double spending.
• basic unit - minimum indivisible unit for TON (e.g. for TON basic unit = nanoTONs) or Jetton.
• hot_wallet_minimum_balance - minimum TON balance in nanoTONs at hot wallet to start service.
• hot_wallet_maximum_balance - maximum balance (of TONs or Jettons) in basic units at hot wallet. Anything more than this amount will be withdrawn to a cold wallet.
• minimum_withdrawal_amount - minimum balance (of TONs or Jettons) in basic units at deposit account to make withdrawal to hot wallet. It is necessary to prevent the case when the withdrawal fee will be close to the balance on the deposit.

Prerequisites

• Need minimum (configured) amount of TONs at HighloadV2 wallet address correlated with seed phrase or already deployed HighloadV2 wallet.
• To ensure the reliability and security of the service, you need to provide your own TON node (with lite server) on the same machine as the service. If you want to use an untrusted node (such as a rented node), you need to set PROOF_CHECK_ENABLED=true and specify NETWORK_CONFIG_URL.
• Jettons used must meet certain criteria

Criteria for valid Jettons

• conforming to the standard TEP-74
• the Jetton wallet should not spontaneously change its balance, only with transfer.
• fee for the withdrawal of Jettons from the wallet should not be too high and meet the internal setting of the service

For more information on Jettons compatibility, see Jettons compatibility

Deployment

Configurable parameters

ENV variable	Description
LITESERVER	IP and port of lite server, example: 185.86.76.183:5815
LITESERVER_KEY	public key of lite server 5v2dHtSclsGsZVbNVwTj4hQDso5xvQjzL/yPEHJevHk=. Be careful with base64 encoding and ENV var. Use ''
SEED	seed phrase for main hot wallet. 24 words compatible with standard TON wallets
DB_URI	URI for DB connection, example: postgresql://db_user:db_password@localhost:5432/payment_processor
POSTGRES_DB	name of database for storing payments data
POSTGRES_READONLY_PASSWORD	password for grafana readonly db user
API_PORT	port for REST API, example 8081
API_TOKEN	Bearer token for REST API, example 123
IS_TESTNET	true if service works in TESTNET, false - for MAINNET. Default: true.
JETTONS	list of Jettons, processed by service in format: JETTON_SYMBOL_1:MASTER_CONTRACT_ADDR_1:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance, JETTON_SYMBOL_2:MASTER_CONTRACT_ADDR_2:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance, example: TGR:kQCKt2WPGX-fh0cIAz38Ljd_OKQjoZE_cqk7QrYGsNP6wfP0:1000000:100000
TON_CUTOFFS	cutoffs in nanoTONs in format: hot_wallet_min_balance:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance, example 1000000000:100000000000:1000000000:95000000000
COLD_WALLET	cold-wallet address, example kQCdyiS-fIV9UVfI9Phswo4l2MA-hm8YseH3XZ_YiH9Y1ufw. If cold wallet is not active - use non-bounceable address (use https://ton.org/address for convert)
DEPOSIT_SIDE_BALANCE	true - service calculates total income for user by deposit incoming, false - by hot wallet incoming. Default: true.
QUEUE_ENABLED	true - service sends incoming notifications to queue, false - sending disabled. Default: false.
QUEUE_URI	URI for queue client connection, example amqp://guest:guest@payment_rabbitmq:5672/
QUEUE_NAME	name of exchange
WEBHOOK_ENDPOINT	endpoint to send webhooks, example: http://hostname:3333/webhook. If the value is not set, then webhooks are not sent.
WEBHOOK_TOKEN	Bearer token for webhook request. If not set then not used.
ALLOWABLE_LAG	allowable time lag between service time and last block time in seconds, default: 15
FORWARD_TON_AMOUNT	forward ton amount for jetton withdrawals, default: 1 nanoton
PROOF_CHECK_ENABLED	enable verification of all proofs to securely connect to an untrusted node, default: false. Also you need to define NETWORK_CONFIG_URL.
NETWORK_CONFIG_URL	the path to load the network configuration to get the trusted key block from it. This is necessary for proof verification, example: https://ton.org/global.config.json

! Be careful with IS_TESTNET variable. This does not guarantee that a testnet node is being used. It is only for address checking purposes.

There are also internal service settings (fees and timeouts) that are specified in the source code in the Config package. Calibration parameters recommendations in Technical notes.

hot_wallet_residual_balance and hot_wallet_max_balance

In order to avoid triggering a withdrawal to a cold wallet with each receipt of funds, a hysteresis is introduced. hot_wallet_max_balance - this is the amount at which the withdrawal from the hot wallet to the cold one will be triggered hot_wallet_residual_balance is the amount that will remain on the hot wallet after the withdrawal

hot_wallet_max_balance must be greater than hot_wallet_residual_balance

If the hot_wallet_residual_balance is not set, then it is calculated using the formula: hot_wallet_residual_balance = hot_wallet_max_balance * hysteresis, where hysteresis is a hardcoded value (at the time of writing this is 0.95)

Service deploy

Do not use same .env file for payment-processor and other services!

1. Build docker images from makefile

make -f Makefile

2. Prepare .env file for payment-postgres service or fill environment variables in docker-compose.yml file. Database scheme automatically init.

docker-compose -f docker-compose.yml up -d payment-postgres

3. Prepare .env file for payment-processor service or fill environment variables in docker-compose.yml file.

docker-compose -f docker-compose.yml up -d payment-processor

4. Optionally you can start Grafana for service monitoring. Prepare .env file for payment-grafana service or fill environment variables in docker-compose.yml file.

docker-compose -f docker-compose.yml up -d payment-grafana

5. Optionally you can start RabbitMQ to collect payment notifications (if QUEUE_ENABLED env var is true for payment-processor). Prepare .env file for payment-rabbitmq service or fill environment variables in docker-compose.yml file.

docker-compose -f docker-compose.yml up -d payment-rabbitmq

Payment notifications

ATTENTION! Sending notifications does not guarantee that all notifications will be sent. If the service is restarted after the data is saved to the database and before the notification data is sent, these notifications will not be sent after restart.

The service has several mechanisms for notification of payments. These are webhooks and a AMQP (to RabbitMQ). Depending on the DEPOSIT_SIDE_BALANCE setting, a notification is received either about the payment to the deposit address, or about the withdrawal from the deposit to the hot wallet. Source address and comment returned if known.

Message format when DEPOSIT_SIDE_BALANCE == true:

{
	"deposit_address":"0QCdsj-u39qVlfYdpPKuAY0hTe5VIsiJcpB5Rx4tOUOyBFhL",
	"time": 12345678,
	"amount":"100", 
	"source_address":"0QAOp2OZwWdkF5HhJ0WVDspgh6HhpmHyQ3cBuBmfJ4q_AIVe",
	"comment":"hello",
    "tx_hash": "f9b9e7efd3a38da318a894576499f0b6af5ca2da97ccd15c5f1d291a808a0ebf",
    "user_id": "123"
}

Message format when DEPOSIT_SIDE_BALANCE == false (there is fewer data, because the accumulated amount is withdrawn from the deposit):

{
	"deposit_address":"0QCdsj-u39qVlfYdpPKuAY0hTe5VIsiJcpB5Rx4tOUOyBFhL",
	"time": 12345678,
	"amount":"200",
    "tx_hash": "f9b9e7efd3a38da318a894576499f0b6af5ca2da97ccd15c5f1d291a808a0ebf",
    "user_id": "123"
}

Using RabbitMQ

1. Set QUEUE_ENABLED = true env variable
2. Set QUEUE_URI as described at Configurable parameters
3. Set QUEUE_NAME env variable. Be careful, this is not the name of a specific queue in the rabbit, but the name of the exchange.
4. Start RabbitMQ as described at Service deploy

Using webhooks

1. Set WEBHOOK_ENDPOINT env variable
2. Optionally set WEBHOOK_TOKEN env variable

When the payment-processor is running, it will send a POST request to the webhook endpoint with each payment and wait for a response with a 200 code and an empty body. If a successful delivery response is not received after several attempts, the service will stop with an error. If the variable WEBHOOK_TOKEN is set, it will also add header Authorization: Bearer {token}.

Binary comment support

Method /v1/withdrawal/send also supports binary_comment. The comment is written in a hex form. If the bits qty is not a multiple of a byte, then the record form with a flip bit is supported, for example 9fe7_. A binary_comment is writing directly to the body of the message (for the TON transfer) and to the forward_payload (for the Jetton transfer) with its opcode according to the following TLB scheme:

binary_comment#b3ddcf7d {n:#} data:(SnakeData ~n) = InternalMsgBody;

crc32('binary_comment n:# data:SnakeData ~n = InternalMsgBody') = 0xb3ddcf7d

This comment will not be displayed by the explorer as text and can be useful for transmitting metadata that will be read by indexers.

The documentation contains a standard way of writing a binary comment, but due to the fact that it is not supported by services, an alternative recording method was chosen.