Cryptocurrency Data Dictionary

Centralized Exchanges

For all centralized exchange's in our collection, we poll their market data REST and WebSocket APIs made publicly available on an exchange’s API documentation. We poll every exchange at regular intervals to ensure that we are collecting every trade data point and for redundancy, also collect through WebSocket. Immediately after receiving these trades, we de-duplicate and normalize the data into our own schema, to ensure consistency across exchanges.  


Decentralized Exchanges

For all Ethereum-based decentralized exchanges in our coverage, we collect market data directly from the Ethereum blockchain. Every transaction on a DEX is recorded on the blockchain, so we have full block-by-block historical and live trade data. This data is stored, normalized, and standardized across all DEX protocols, and redistributed through our REST API, streaming service, and in CSV files.

Kaiko provides trade data for both centralized and decentralized exchanges through the same API endpoints. For DEXs, the data becomes available as it is recorded on the Ethereum blockchain and for CEXs, the data is available as Kaiko receives it from an exchange's API/ WebSocket.

The trade data output format for DEX/CEXs is identical. The only difference is that DEX instruments are identified by their smart contract address, corresponding to the liquidity pool address. For each available pair of a decentralized exchange, the liquidity pool’s address is made available through the trade_id field in the trades output. The difference in the trade_id is the only difference between CEXs and DEXs output.

ID: this is the unique identifier for each trade, given by the exchange. The format of this ID can vary across exchanges. For centralized exchanges that do not include an ID for their trades, we will create our own ID. We generate the IDs by taking all the values of a trade (timestamp, amount, price, side) and hash those into a new unique value. For decentralized exchanges, the trade ID is composed of the (i) liquidity pool’s address on which the trade happened, (2) the transaction hash corresponding to the trade (3) and the log index that is an even more granular element that helps identify the trade in a given block and transaction. 

Exchange: our internal two-letter symbol for the exchange. For each .CSV that you receive, this symbol corresponds to the exchange name from the filename (so for Binance_BTCUSDT_*.csv, this is 'bn'). The full list of exchanges and their corresponding codes can be found under the field “Kaiko_legacy_slug”. Through our API, the exchange symbols are from our list of codes (“code”) and can be found here.

Symbol: the pair trading on the exchange

Date: We take the timestamp of the trade from the exchange. Our timestamps are not when Kaiko collected the data, but when the trade was executed as recorded by the exchange or as recorded on the blockchain for DEX trades.

Price: The price of the asset displayed in the quote asset. For example, the price of a trade of BTC/USD would be $60,750.

Amount: The amount traded of the base asset. For example, the amount of a trade of BTC/USD would be .3, meaning that .3 BTC were traded at a given price.

isSell / Sell/ Taker_Side_Sell: This variable is the same across our Data Feed, API, and WebSocket, but is given different names due to legacy formatting issues. This is the indicator of whether the trade was a sell or a buy order, from the perspective of the taker. If ‘isSell: true': a taker’s sell order was filled by a maker’s buy order, if ‘isSell: false': a taker’s buy order was filled by a maker’s sell order. You can read more about this field on our documentation here.

Centralized Exchanges

We strive to provide the most complete and qualitative data on the market. However, we partially rely upon the exchanges’ API to ensure this. Due to downtime of these API's (for example, due to maintenance) we sometimes experience gaps in our data and cannot always guarantee 100% coverage.


Decentralized Exchanges 
We provide gapless DEX data because all trades are permanently recorded on the blockchain. 

We do not create an aggregation when there is not data for a given timespan. If you believe this is not the case for a certain pair, you can send us the request and we will check to see if there is in fact no trade data recorded for the timespan.

Twice per minute, we take a snapshot of the state of the order book at that particular instant. This means that in between snapshots, orders can be put and filled without us recording them. In practice, we take two snapshots per minute, in order to ensure that there is at least 1 state of the order book at any given minute with never more than 60 seconds difference between 2 snapshots.

In our current system, the snapshots are never taken at exactly 1 minute intervals due to timestamp inaccuracies across exchanges. Because there is a slight millisecond inaccuracy, if you look at timestamps from different days or even hours, it might seem random. However, we will always have at least 1 snapshot for every minute that passes. 

Our default order book product is 10%. This means that we include all bids/asks placed within 10% of the mid-price at the time the order book snapshot was taken. We poll each exchange's REST API for order books, and determine the mid-price for that snapshot. Then, we calculate 10% +/- from the midprice. Finally, we remove all bids and asks from the snapshot that fall outside of this 10% range. 

For most top exchanges, we are able to collect the full order book snapshot from the exchange's REST API. For other exchanges, we are only able to collect a certain number of "levels" (such as the best 20, 50, 100, 1,000 levels). For exchanges where we are able to collect the full order book snapshot, we cut the snapshot down into a "10% Snapshot" which includes all bids and asks within 10% of the midprice for that specific snapshot. For exchanges where we are unable to collect the full order book snapshot, we will still cut the snapshot down into a 10% snapshot if the bids and asks are greater than 10% from the midprice. For snapshots with bids/asks less than 10% of the midprice, we leave the snapshot as is.

On some exchanges where we are unable to collect the full snapshot, we are still able to collect the full 10% snapshot because the exchange is low volume.

We have documented each exchange's available market depth here. 

Order books contain bids and asks for a currency pair, essentially giving an "under the hood" look at all orders placed by makers on an exchange before they are executed as trades (which occurs once a taker fills a maker’s bid or ask). Trades are executed transactions and result when a buy and a sell order match on an exchange. Unfortunately, exchanges do not provide order books and trades as one data set--there are no common identifying variables that match a specific trade with a specific bid or ask on the order book as there are no shared ID's. Thus, the two datasets are always collected separately as it is impossible to package them into one dataset.

Unfortunately, exchanges do not store their own order book data. Thus, we are unable to backfill any missing order book data.

A snapshot includes all bids and asks on an order book at a moment in time. Kaiko's current order book snapshots are taken twice per minute. In between snapshots, we do not capture any bids or asks placed, thus we miss out on information that could be important to traders. 

Tick-level order books contain all updated, changed, and removed bids and asks on an order book. This data is gapless and provides an extremely detailed look at order book flows. Each update is known as an event, and enables traders to rebuild an internal order book by adding or subtracting bids and asks.

Full order book data is the most granular data available in cryptocurrency markets, and can be used for detailed research or simulations of trading strategies. With this data, traders and researchers can project market demand and support, determine price levels trades will be filled at, simulate entry and exits, train machine learning algorithms, search for arbitrage opportunities, and more. Order book ticks can be applied to order book snapshots to reconstruct historical market states and replay changes to the order book. 

Upon making an initial connection through Kaiko Stream, we will send the most recent order book snapshot for each instrument in your subscription. This provides a basis upon which to build and maintain an internal order book. Incoming events can be added/subtracted from the initial snapshot, which would reflect exactly how the order book looks on the exchange. 

We will soon begin storing this data and will make it available in CSV files early 2022, and will also provide 5 weeks of rolling history through Kaiko Stream. We unfortunately cannot backfill full order book data as exchanges do not store this data type. 

Yes! We provide full order books for top-tier exchanges today, and will soon expand our coverage to all in Kaiko's collection.

Full order book data is delivered through Kaiko Stream. Stream is our advanced live data distribution service built using gRPC technology, one of the most efficient messaging protocols. Clients can subscribe to live order book feeds and filter on the exchange and currency pair. You can view the product page for Stream here. 

Once we begin storing this data, we will deliver CSV files via Amazon Web Services or Google Cloud Platform.

Financial market order book data can be divided into three categories: Level 1, Level 2, and Level 3. In cryptocurrency markets, these categories are often blurred and differ slightly depending on the data formatting provided by the exchange. Only 3 exchanges in the industry provide non-aggregated, L3 data. All other exchanges provide L2 order book data and this is what Kaiko provides. 

Level 1: L1 data refers to Top of Book, or the best bid / best ask of a trading pair's order book. This data is commonly referred to as "quote" data, and is accessed in real-time. Traders subscribing to L1 quote data receive real-time changes to the best bid / best ask of an order book, which includes the price level and order amount.

Level 2: L2 data is more granular than L1 data, and includes bids and asks at all price levels of a trading pair's order book. This data is commonly referred to as "depth of book" data. In cryptocurrency markets, market makers place limit orders at price levels that differ from the current price of an asset. L2 data includes all bids and asks placed by these market makers, aggregated by price level. For example, if 10 traders placed bids of various sizes at $8,000 while the current price of BTC was $8,500, this would appear as an L2 quote of [price: $8,000, amount: 23.4]. The example amount of 23.4 would be the sum of all orders placed by the 10 traders. Because L2 data is aggregated by price level, it can also be referred to as "market by price level" data.

Level 3: L3 data provides even deeper information than L2 data. L3 data refers to non-aggregated bids and asks placed by individual market makers. An L3 data feed would include every individual bid and ask, whereas an L2 data feed would include these bids and asks aggregated by price level. For example, L3 data for the quotes placed by the 10 traders above would result in 10 different data points with each individual order amount, rather than a single data point aggregated by price level. Thus, L3 data can be referred to as "market by order" data.

Each consecutive level of data provides an increased level of granularity. Some traders may find L1 data sufficient for their trading strategies, while others require L2 data. L3 data is the most granular level of order book data, frequently used by the most sophisticated traders.

Collecting derivatives data is uniquely complex due to the variations in formatting and endpoints across exchanges. We poll each exchange's REST API to receive the data, yet most exchanges provide derivatives data through multiple endpoints. Thus, for a single "derivatives data snapshot", we poll multiple endpoints and combine the data into a single data packet. We query each unique derivatives data endpoint once per minute, thus it is possible that the data points returned through our API were not collected at the same exact moment. 
 
This product is in alpha. Changes to the format and update frequency might occur at any time without prior notice. 

August 2020. Binance Futures and Bybit began collection in June 2021.

This data is available through our REST API through our 'Derivatives Data' endpoints, both live and historical data. You can view our API documentation here. We also provide this data in CSV files with history since August 2020, when we began collection for all exchanges. 

Please reach out to us if you would like a free trial. 

Currently, this data is updated on average once per minute. Not every derivatives measure updates at this frequency. For example, funding rates are only recalculated every few hours, so we will not have this data as frequently as others.

Right now, we do not have a dedicated endpoint to get option chains. However, by concurrently querying all option contract data for an instrument, users can construct an option chain (or option matrix) themselves.

Yes! We are planning to add lending rates, liquidation data, more data on indices and aggregated derivatives data in the future.

 You can read our full calculation methodology here. 

A composite price is calculated by leveraging three parameters: asset reference prices, asset weights and a floor value. Its computation starts with the conversion of asset weights into absolute weights (ie. a quantity of asset) by applying the asset allocation to the initial floor value (eg. 100$). Asset contributions are obtained by multiplying reference prices with their corresponding absolute weights and then summed to derive the composite price. The percent change from the initial floor value tracks the performance of the composite at each fixing frequency. Asset level performance can also be measured by comparing current reference price from their initial floor value.

This data is available through our REST API live and with up to 7 days history. To use the endpoint, you would enter the parameters listed above for asset weighting, fixing time, outlier management, etc., and modify the parameters according to your use case. You can view our API documentation here.

We also provide this data in historical CSV files when given the parameters for the required price feed. 

This data service is optimized for NAV calculations and can be leveraged by crypto asset funds, asset managers and any financial professional requiring single-asset or multi-asset valuation. The asset weighting parameter of our pricing and valuation tool is a powerful feature that enables any fund manager to input their portfolio allocation to determine a reference price, which can be used to value shares. Single-asset and multi-asset composite prices can also provide a statistical measure for price performance over time.