Helpers and monitoring
Health checks in Taurus-PROTECT
Any of our components are exposing Health checks through the same API endpoint: /api/rest/v1/health/all?tenantId=1
List of Taurus-PROTECT component names that produce health checks:
- validatord
- vaultd
- hsm
- blocksd
- gated
Jobs perform component health checks and produce health reports accessible at /api/rest/v1/health/[component]/[check]. This is accessible without authentication so we must be careful with what we expose.
[component] can be replaced with values that you get from the “all” health checks.
For any of these endpoints, you can get those common values:
- “tenantId”: is the tenant where health check was run against
- “componentName”: name of the component
- “componentId”: uinque ID for the component (can be retrieved from the logs)
- “group”: group to whom the healthcheck belongs to
- “healthCheck”: type of health check
- “status”: status of the executed call
- “report”:
- “name”: group name
- “status”: status of the executed call
- “duration”: duration of the call
- “lastUpdateDate”: when was the health check executed
- “validUntilDate”: until when this health check will be valid.
How to setup
Make sure baseUrl is a variable pointing to your validator instance, example:
export baseUrl='https://your-protect-instance.example.com'
Get Explorers health checks
Description : Performs an health check on the connectivity between validatord and blocksd.
curl --location "$baseUrl/api/rest/v1/health/blocksd/connectivity" \
Get health checks list by tenant
Description : Performs a complete health check on validatord.
curl --location "$baseUrl/api/rest/v1/health/all?tenantId=1" \
Get health checks for a specific group
Description: Performs a health check on a specific group of monitored activities.
Example with gated group:
curl --location "$baseUrl/api/rest/v1/health/gated?tenantId=1" \
Get Explorers Balances health checks by tenant
Description : Performs a health check by getting balances from blocksd. Results will give you the list of currencies where a balance has been returned.
curl --location "$baseUrl/api/rest/v1/health/blocksd/getbalance?tenantId=1" \
Get Explorers Fees health checks by tenant
Description : Performs a health check by getting Blockchains current fees from blocksd. Results will give you the list of currencies where a fee has been returned.
curl --location "$baseUrl/api/rest/v1/health/blocksd/getfees?tenantId=1" \
Get Database health checks by tenant
Description : Performs a health check by checking database connectivity between validatord and the database engine you are currently using.
curl --location "$baseUrl/api/rest/v1/health/database?tenantId=1" \
Get Prices health checks by tenant
Description : Performs a health check between validatord and prices providers you are currently using. Example here with Cryptocompare.
curl --location "$baseUrl/api/rest/v1/health/database?tenantId=1" \
Get Score Providers health checks by tenant
Description : Performs a health check between validatord and KYC/KYT scoring providers you are currently using. Example here with Chainalysis and Elliptic.
curl --location "$baseUrl/api/rest/v1/health/scoreproviders?tenantId=1" \
Get Gated health checks by tenant
Description : Performs a health check on the connectivity between validatord and gated (proxy).
curl --location "$baseUrl/api/rest/v1/health/gated?tenantId=1" \
Get HSM health checks by tenant
Description : Performs a health check on the connectivity between validatord and hsm.
curl --location "$baseUrl/api/rest/v1/health/hsm?tenantId=1" \
Get vaultd health checks by tenant
curl --location "$baseUrl/api/rest/v1/health/hsm?tenantId=1" \
Jobs API specific health checks
Get Jobs health checks by tenant
curl --location "$baseUrl/api/rest/v1/health/jobs?tenantId=1" \
Some job names examples:
- Actions:
action_service_running-actions-handler,
action_service_sleeping-actions-handler
- Audit
audit_service_trails-ingestor
- Balance:
balance_service_balances-desync-checker balance_service_balances-garbage-collector balance_service_balances-history-garbage-collector balance_service_balances-history-refresher balance_service_balances-refresher balance_service_existential-balance-updates-processor balance_service_funds-releaser balance_service_rebase-token-balances-refresher
- Change service:
change_service_email-event-task-handler
- Events:
event_service_auto-transfer-events-task-handler event_service_initialized-auto-transfer-request-handler event_service_pending-auto-transfer-request-handler event_service_triggered-auto-transfer-events-handler eventbus_service_event-dispatcher eventbus_service_event-sequencer
- Exchanges:
exchange_service_exchanges-balances-handler
- Prices:
price_service_price-history-refresher price_service_price-refresher
- Requests:
request_service_action-auto-prepared-2-requests-handler request_service_action-prepared-requests-handler request_service_approved-2-requests-handler request_service_approved-request-bundles-handler request_service_approved-requests-handler request_service_broadcasting-2-requests-handler request_service_broadcasting-request-bundles-handler request_service_broadcasting-requests-handler request_service_bundle-approved-requests-handler request_service_canceled-signed-requests-handler request_service_created-request-bundles-handler request_service_creating-request-bundles-handler request_service_creating-requests-handler request_service_diem-burn-broadcasted-requests-handler request_service_diem-mbs-approved-requests-handler request_service_diem-mbs-completed-requests-handler request_service_diem-mbs-pending-requests-handler request_service_dot-extrinsics-garbage-collector request_service_dot-extrinsics-refresher request_service_expired-requests-handler request_service_expired-to-cancel-signed-requests-handler request_service_failed-request-bundles-handler request_service_fast-approved-2-requests-handler request_service_fast-approved-requests-handler request_service_funds-releaser request_service_hsmfailed-2-requests-handler request_service_hsmfailed-requests-handler request_service_hsmready-2-requests-handler request_service_hsmready-request-bundles-handler request_service_hsmready-requests-handler request_service_hsmsigned-2-requests-handler request_service_hsmsigned-request-bundles-handler request_service_hsmsigned-requests-handler request_service_invalid-signed-requests-handler request_service_new-signed-requests-handler request_service_opt-ins-releaser request_service_pending-requests-handler request_service_pending-signed-requests-handler request_service_ready-requests-handler request_service_replaced-signed-requests-handler request_service_timeout-signed-requests-handler request_service_verified-request-batch-signatures-handler
- Staking:
staking_service_active-icp-neurons-refresher staking_service_ada-staking-addresses-refresher staking_service_ada-staking-rewards-refresher staking_service_cosmoshub-staking-rewards-refresher staking_service_cosmoskichain-staking-rewards-refresher staking_service_disbursing-icp-neurons-handler staking_service_dissolved-icp-neurons-handler staking_service_dot-staking-withdraw-unbonded-handler staking_service_ftm-staking-addresses-refresher staking_service_icp-balance-activity-handler staking_service_icp-neurons-creator staking_service_icp-staking-activity-handler staking_service_merging-icp-neurons-handler staking_service_near-staking-rewards-refresher staking_service_new-icp-neurons-handler staking_service_observing-icp-neurons-handler staking_service_sol-staking-info-refresher staking_service_spawning-icp-neurons-handler staking_service_staking-states-collection-refresher staking_service_xtz-delegate-addresses-refresher
- Transactions:
transaction_service_mig-hasanytransactionattribute transaction_service_network-migrater transaction_service_observed-transactions-handler transaction_service_populate-transaction-balance transaction_service_token-balances-garbage-collector transaction_service_token-balances-refresher transaction_service_token-holders-populator transaction_service_transactions-exporter
- Wallets:
wallet_service_ada-staking-addresses-refresher wallet_service_addresses-nft-migrater wallet_service_algo-optin-status-refresher wallet_service_balances-history-refresher wallet_service_balances-refresher wallet_service_created-addresses-handler wallet_service_currency-wallets-businessrules-creator wallet_service_network-migrater wallet_service_nonces-refresher wallet_service_observed-addresses-handler wallet_service_signed-addresses-handler wallet_service_token-wallets-syncer wallet_service_xlm-address-flags-refresher wallet_service_xlm-existentialbalance-refresher wallet_service_xlm-trustline-refresher wallet_service_xtz-delegate-addresses-refresher wallet_service_xtz-initializing-addresses-handler wallet_service_xtz-ready-for-init-addresses-handler wallet_service_xtz-revelation-status-refresher wallet_service_xtz-uninitialized-addresses-handler
- Webhooks:
webhook_service_webhook_caller webhook_service_webhook_event_handler
- Whitelists:
whitelist_service_network-migrater whitelist_service_verified-wladdress-batch-signatures-handler
Get Job health checks by Job name and by tenant
Description : Returns computed statistics for a single job.
curl --location "$baseUrl/api/rest/v1/health/jobs/scorerefresh?tenantId=1" \
Get Jobs list
This endpoint requires authentication as the technical user.
Description : Collects data from the job_statuses table and computes simple statistics grouped by job name.
- Number of successes.
- Number of failures.
- Number of pending.
job_statusesvalues for the last success.job_statusesvalues for the last failure.- Average duration of successful runs.
- Maximum duration of successful runs.
- Minimum duration of successful runs.
curl --location "$baseUrl/api/rest/v1/jobs" \
Get Job data from database table
This endpoint requires authentication as the technical user.
Description : Returns values from a row in the job_statuses table. The job_uuid refers to a single job run. This can be found in logs.
curl --location "$baseUrl/api/rest/v1/jobs/request_service_approved-requests-handler" \
Get Job data by Job UUID
This endpoint requires authentication as the technical user.
Description : Returns values from a row in the job_statuses table. The job_uuid refers to a single job run. This can be found in logs.
curl --location "$baseUrl/api/rest/v1/jobs/request_service_approved-requests-handler/status/f8e36233-115c-438e-8893-a404b7a47f95" \
HSM and vaultd specific health checks
Get HSM status
Description : perform a health check through vaultd and returns HSM status.
curl --location "$baseUrl/api/rest/v1/hsm/status" \
Get HSM version status
Description : perform an health check through vaultd and returns HSM version.
curl --location "$baseUrl/api/rest/v1/hsm/versions" \
Usage of API and authentication
Authenticate
Get BTC nostro wallet
Get all addresses from a currencies
Get rules and decode the proto
Health checks in Taurus-EXPLORERS
Taurus-EXPLORERS is exposing Health checks through the same API endpoint:
https://explorers.t-dx.com/api/rest/v1/health/all
You will get a list of different checks per blockchain:
- height, giving you node head and what is currently stored in Explorers
- indexing
- collections:
- balance
- collection_erc_token_balance
- staking
- opt-in.
Taurus-EXPLORERS Health checks description
Height
The height health check ensures that the latest block processed by blocksd, and stored in blocksd internal database, is not more than acceptableLateBlock behind the current blockchain1 height in terms of blocks2. can be set in blocksd configuration yml file as such:
1 Note: the blockchain node connected to the blocksd instance is used as source of truth for querying the blockchain state.
2 Note: only for BTC, if indexing the testnet network, the check additionally verifies that, if behind more than acceptableLateBlock blocks, blocksd is still not more than acceptableLateBlockDelaylate with respect to the timestamp of the latest block of the blockchain.
Indexing
Collections
Taurus-Explorer Health checks list
Here is a non-exhaustive list of available checks for each enabled coin:
Jobs perform component health checks and produce health reports. For any of these endpoints, you can get those common values:
- “componentName”: name of the component
- “componentId”: unique ID for the component (can be retrieved from the logs)
- “group”: group to whom the healthcheck belongs to, generally a blockchain
- “healthCheck”: type of health check
- “status”: status of the executed call
- “report”:
- “name”: group name
- “status”: status of the executed call
- “msg”: an output message of the health check call
- “duration”: duration of the call
- “lastUpdateDate”: when was the health check executed
- “validUntilDate”: until when this health check will be valid.
Get Explorers health check list
Description : Performs a complete health check on Explorers.
curl --location "https://explorers.t-dx.com/api/rest/v1/health/all"
Get Explorers health checks for a specific group
Description: Performs a health check on a specific group of monitored activities.
Example with FANTOM group:
curl --location "https://explorers.t-dx.com/api/rest/v1/health/ftm" \
Another example with Polkadot group:
Description: Performs a health check on a specific blockchain of monitored activities:
- block height
- collection balance
- staking collection balance
- indexing: when was the last inserted block
curl --location "https://explorers.t-dx.com/api/rest/v1/health/dot" \
Get Explorers health checks for a specific chain
Description: Performs a specific health check on a chosen chain.
Example with Cardano:
curl --location "https://explorers.t-dx.com/api/rest/v1/health/ada/height" \