Profile picture

Co-founder @ RMOTR

Crypto Analysis Using Python and Cryptowatch API

Last updated: May 5th, 20192019-05-05Project preview

In this post I want to show you how simple it is to start doing analysis of different cryptos (Bitcoin, Litecoin, Ether, etc) using a little bit of Python programming. In this post, you'll learn how to:

  • Pull Crypto data from Cryptowatch API (what's an API anyway? šŸ˜• I'll explain that too)
  • Building pandas DataFrames with that data
  • Create transformations to extract moving averages and standard deviations
  • Plot your results including Bollinger Bands and rolling statistics.
  • Create other beautiful visualizations like candlesticks using Bokeh.

You can copy this project and work on your own Jupyter Notebook using the button on the right šŸ‘‰


Cryptowatch APIĀ¶

(or how I Learned to Stop Worrying and learn about APIs)

To perform any analysis, we must first gather data. Thankfully, we can use Cryptowatch: a free service that offers an API to access crypto data. The API is really well documented and the free hourly quota is generous.

First steps using Cryptowatch APIĀ¶

I'll show you what is possibly the simplest example to consult Cryptowatch's API. An API is composed of multiple "endpoints"; each endpoint serves a particular purpose, exposes a particular set of data. Example of endpoints can be for example Exchanges (information about Exchanges) and Markets (information about prices).

In particular, this is a REST API, and we'll use requests to consult it (already installed on RMOTR Notebooks).

InĀ [1]:
import requests

Let's start using the markets endpoint to get a summary of a given crypto at a given exchange, for example, let's start first with:

Bitcoin on BitfinexĀ¶
InĀ [2]:
resp = requests.get('https://api.cryptowat.ch/markets/bitfinex/btcusd/summary')
InĀ [3]:
resp.ok  # Was successful?
Out[3]:
True

The data from cryptowatch's API is returned using JSON format:

InĀ [4]:
resp.json()
Out[4]:
{'result': {'price': {'last': 4219.5464,
   'high': 4267.9,
   'low': 3738.2,
   'change': {'percentage': 0.09958474, 'absolute': 382.14648}},
  'volume': 48714.93085452,
  'volumeQuote': 193435864.90488192},
 'allowance': {'cost': 3551149, 'remaining': 7996448851}}

You can see there the current price of Bitcoin, along with it's volume and some more data:

InĀ [5]:
doc = resp.json()
InĀ [6]:
print(f"Price: {doc['result']['price']['last']}")
print(f"Volume: {doc['result']['volume']:.2f}")
Price: 4219.5464
Volume: 48714.93
Bitcoin on Coinbase ProĀ¶
InĀ [7]:
resp = requests.get('https://api.cryptowat.ch/markets/coinbase-pro/btcusd/summary')
InĀ [8]:
resp.ok
Out[8]:
True
InĀ [9]:
doc = resp.json()
doc
Out[9]:
{'result': {'price': {'last': 4123.01,
   'high': 4151.65,
   'low': 3636.01,
   'change': {'percentage': 0.10543864, 'absolute': 393.25977}},
  'volume': 29651.66242737,
  'volumeQuote': 114336823.69250095},
 'allowance': {'cost': 8026779, 'remaining': 7988422072}}
InĀ [10]:
print(f"Price: {doc['result']['price']['last']}")
print(f"Volume: {doc['result']['volume']:.2f}")
Price: 4123.01
Volume: 29651.66
Bitcoin on KrakenĀ¶
InĀ [11]:
resp = requests.get('https://api.cryptowat.ch/markets/kraken/btcusd/summary')
InĀ [12]:
resp.ok
Out[12]:
True
InĀ [13]:
doc = resp.json()
doc
Out[13]:
{'result': {'price': {'last': 4120,
   'high': 4149.7,
   'low': 3636.1,
   'change': {'percentage': 0.10408401, 'absolute': 388.3999}},
  'volume': 12982.26155886,
  'volumeQuote': 49990530.05673022},
 'allowance': {'cost': 3691911, 'remaining': 7984730161}}
InĀ [14]:
print(f"Price: {doc['result']['price']['last']}")
print(f"Volume: {doc['result']['volume']:.2f}")
Price: 4120
Volume: 12982.26
Other crypto example: Ether on BitfinexĀ¶
InĀ [15]:
resp = requests.get('https://api.cryptowat.ch/markets/bitfinex/ethusd/summary')
InĀ [16]:
resp.ok
Out[16]:
True
InĀ [17]:
doc = resp.json()
doc
Out[17]:
{'result': {'price': {'last': 118.48,
   'high': 119.865486,
   'low': 105.34,
   'change': {'percentage': 0.0928881, 'absolute': 10.07}},
  'volume': 642712.99879173,
  'volumeQuote': 72562792.94281721},
 'allowance': {'cost': 8138410, 'remaining': 7976591751}}
InĀ [18]:
print(f"Price: {doc['result']['price']['last']}")
print(f"Volume: {doc['result']['volume']:.2f}")
Price: 118.48
Volume: 642713.00

Other endpointsĀ¶

How many "exchanges" does Cryptowatch support? That's an easy answer! We can use the exchanges endpoint to consult all the exchanged supported:

InĀ [19]:
resp = requests.get('https://api.cryptowat.ch/exchanges')
InĀ [20]:
resp.ok
Out[20]:
True
InĀ [21]:
doc = resp.json()

The json doc returned by this endpoint is large, here's a sample of the results:

InĀ [22]:
doc['result'][:3]
Out[22]:
[{'id': 11,
  'symbol': 'bitflyer',
  'name': 'bitFlyer',
  'route': 'https://api.cryptowat.ch/exchanges/bitflyer',
  'active': True},
 {'id': 24,
  'symbol': 'bithumb',
  'name': 'Bithumb',
  'route': 'https://api.cryptowat.ch/exchanges/bithumb',
  'active': True},
 {'id': 4,
  'symbol': 'kraken',
  'name': 'Kraken',
  'route': 'https://api.cryptowat.ch/exchanges/kraken',
  'active': True}]

And we can use a simple loop to list them all:

InĀ [23]:
for exchange in doc['result']:
    print(f"* {exchange['name']}")
* bitFlyer
* Bithumb
* Kraken
* meXBT
* Poloniex
* Mt. Gox
* Bitsquare
* BitBay
* Okex
* HitBTC
* CEX.IO
* 796
* BitMEX
* Huobi
* QuadrigaCX
* Vault of Satoshi
* BTC China
* Quoine
* CryptoFacilities
* WEX
* BitVC
* Luno
* Bitstamp
* Gemini
* Coinone
* Coinbase Pro
* Bitfinex
* Binance
* Gate.io
* Bittrex
* Qryptos
* Bit-Z
* Cryptsy
* OKCoin

Getting pricesĀ¶

The whole point of our analysis is to get the historic prices cryptos and compare them. The endpoint that returns that data is ohlc (Open High Low Close). Example of Bitcoin on Bitfinex:

InĀ [24]:
resp = requests.get('https://api.cryptowat.ch/markets/bitfinex/btcusd/ohlc')
InĀ [25]:
resp.ok
Out[25]:
True
InĀ [26]:
doc = resp.json()

The returned json document is huge and it includes historic prices with different periodicity; for example, every minute, every 5 minutes, every hour or every day. Here's a list of all the periods:

InĀ [27]:
periods = {
    '60': '1m',  # 1 Minute
    '180': '3m', # 3 Minutes
    '300': '5m',
    '900': '15m',
    '1800': '30m',
    '3600': '1h', # 1 Hour
    '7200': '2h',
    '14400': '4h',
    '21600': '6h',
    '43200': '12h',
    '86400': '1d', # 1 Day
    '259200': '3d',
    '604800': '1w', # 1 Week
}

So, for example, in the 1 day period I can get a list of prices and volumes, here are the first three values:

InĀ [28]:
doc['result']['86400'][:3]
Out[28]:
[[1500508800, 2302.9, 2400, 2213.9, 2253.4, 35714.965, 0.0],
 [1500595200, 2253.3, 2922, 2253.3, 2865.2, 83576.836, 0.0],
 [1500681600, 2865.2, 2890, 2615, 2659, 57424.004, 0.0]]

Each value is a list that contains:

  1. Timestamp (expressed as unix epoch)
  2. Open Price
  3. High Price
  4. Low Price
  5. Close Price
  6. Volume
  7. Not used (I don't know what this is)

We can use the datetime module to parse the timestamp and print the price:

InĀ [29]:
from datetime import datetime
InĀ [30]:
price = doc['result']['86400'][0]
price
Out[30]:
[1500508800, 2302.9, 2400, 2213.9, 2253.4, 35714.965, 0.0]
InĀ [31]:
datetime.fromtimestamp(price[0])
Out[31]:
datetime.datetime(2017, 7, 20, 0, 0)
InĀ [32]:
print(f"{datetime.fromtimestamp(price[0])} - Close Price: ${price[-3]:.2f}, Volume: {price[-2]:.2f}")
2017-07-20 00:00:00 - Close Price: $2253.40, Volume: 35714.96

I'll display now only the last 3 prices of each period:

InĀ [33]:
for period, label in periods.items():
    if period in doc['result']:
        prices = doc['result'][period]
        print(f"Period {label}:")
        print('-' * 70)
        for price in prices[:3]:
            dt = datetime.fromtimestamp(price[0])#.strftime('%Y-%m-%d')
            print(f"{dt} | Price: {price[-2]:<15.2f} - Volume: {price[-1]:<15.2f}")
        print("")
Period 1m:
----------------------------------------------------------------------
2018-11-28 04:53:00 | Price: 160.04          - Volume: 653852.71      
2018-11-28 04:54:00 | Price: 67.79           - Volume: 277640.28      
2018-11-28 04:55:00 | Price: 64.14           - Volume: 262366.79      

Period 3m:
----------------------------------------------------------------------
2018-11-27 12:15:00 | Price: 37.45           - Volume: 140952.45      
2018-11-27 12:18:00 | Price: 82.15           - Volume: 309370.01      
2018-11-27 12:21:00 | Price: 8.10            - Volume: 30466.84       

Period 5m:
----------------------------------------------------------------------
2018-11-26 19:40:00 | Price: 100.67          - Volume: 397234.22      
2018-11-26 19:45:00 | Price: 34.79           - Volume: 136995.91      
2018-11-26 19:50:00 | Price: 40.80           - Volume: 160947.40      

Period 15m:
----------------------------------------------------------------------
2018-11-23 08:30:00 | Price: 136.18          - Volume: 595306.88      
2018-11-23 08:45:00 | Price: 584.13          - Volume: 2557537.11     
2018-11-23 09:00:00 | Price: 685.39          - Volume: 3022980.15     

Period 30m:
----------------------------------------------------------------------
2018-11-18 04:00:00 | Price: 85.66           - Volume: 485714.87      
2018-11-18 04:30:00 | Price: 57.73           - Volume: 327339.28      
2018-11-18 05:00:00 | Price: 125.25          - Volume: 708914.62      

Period 1h:
----------------------------------------------------------------------
2018-11-07 19:00:00 | Price: 276.50          - Volume: 1818234.45     
2018-11-07 20:00:00 | Price: 80.88           - Volume: 531698.56      
2018-11-07 21:00:00 | Price: 80.70           - Volume: 530307.82      

Period 2h:
----------------------------------------------------------------------
2018-10-18 00:00:00 | Price: 230.61          - Volume: 1553981.00     
2018-10-18 02:00:00 | Price: 767.30          - Volume: 5166481.50     
2018-10-18 04:00:00 | Price: 1332.13         - Volume: 8993934.00     

Period 4h:
----------------------------------------------------------------------
2018-09-06 12:00:00 | Price: 5754.03         - Volume: 36786556.00    
2018-09-06 16:00:00 | Price: 6185.95         - Volume: 39968588.00    
2018-09-06 20:00:00 | Price: 3448.17         - Volume: 22174288.00    

Period 6h:
----------------------------------------------------------------------
2018-07-27 00:00:00 | Price: 19256.20        - Volume: 155030450.00   
2018-07-27 06:00:00 | Price: 5499.81         - Volume: 43484264.00    
2018-07-27 12:00:00 | Price: 4576.01         - Volume: 36344900.00    

Period 12h:
----------------------------------------------------------------------
2018-03-26 12:00:00 | Price: 21910.78        - Volume: 181429570.00   
2018-03-27 00:00:00 | Price: 31614.97        - Volume: 253897810.00   
2018-03-27 12:00:00 | Price: 26101.18        - Volume: 206840370.00   

Period 1d:
----------------------------------------------------------------------
2017-07-20 00:00:00 | Price: 35714.96        - Volume: 0.00           
2017-07-21 00:00:00 | Price: 83576.84        - Volume: 0.00           
2017-07-22 00:00:00 | Price: 57424.00        - Volume: 0.00           

Period 3d:
----------------------------------------------------------------------
2014-10-31 00:00:00 | Price: 88087.88        - Volume: 0.00           
2014-11-03 00:00:00 | Price: 54802.82        - Volume: 0.00           
2014-11-06 00:00:00 | Price: 61005.13        - Volume: 0.00           

Period 1w:
----------------------------------------------------------------------
2013-10-17 00:00:00 | Price: 26052.07        - Volume: 0.00           
2013-10-24 00:00:00 | Price: 82086.86        - Volume: 0.00           
2013-10-31 00:00:00 | Price: 98314.09        - Volume: 0.00           

Getting OHLC with PandasĀ¶

We'll now focus on just one period (I'll use hourly, feel free to change it) and we'll transform the records to a pandas DataFrame, so we can analyze it:

InĀ [34]:
import pandas as pd
InĀ [35]:
periods = '3600'
InĀ [36]:
resp = requests.get('https://api.cryptowat.ch/markets/bitfinex/btcusd/ohlc', params={
    'periods': periods
})
InĀ [37]:
resp.ok
Out[37]:
True
InĀ [38]:
data = resp.json()
InĀ [39]:
df = pd.DataFrame(data['result'][periods], columns=[
    'CloseTime', 'OpenPrice', 'HighPrice', 'LowPrice', 'ClosePrice', 'Volume', 'NA'
])
InĀ [40]:
df.head()
Out[40]:
CloseTime OpenPrice HighPrice LowPrice ClosePrice Volume NA
0 1541617200 6586.9000 6587.0 6567.0000 6575.7974 276.496865 1.818234e+06
1 1541620800 6575.7974 6577.8 6567.1000 6567.4000 80.882415 5.316986e+05
2 1541624400 6567.3000 6575.8 6566.0356 6573.9000 80.704764 5.303078e+05
3 1541628000 6573.9126 6574.0 6568.3000 6573.0000 100.871970 6.626737e+05
4 1541631600 6573.0000 6573.0 6569.4000 6570.5000 105.118426 6.907457e+05

This is a pandas DataFrame, and it's the cornerstone of Python Data Analysis. We'll do a few things to improve our DataFrame and simplify the analysis. First, we'll get rid of the NA column:

InĀ [41]:
df.drop(columns=['NA'], inplace=True)

Then we'll transform CloseTime into a real Timestamp object. If you check closely, it's a number now:

InĀ [42]:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 500 entries, 0 to 499
Data columns (total 6 columns):
CloseTime     500 non-null int64
OpenPrice     500 non-null float64
HighPrice     500 non-null float64
LowPrice      500 non-null float64
ClosePrice    500 non-null float64
Volume        500 non-null float64
dtypes: float64(5), int64(1)
memory usage: 23.5 KB
InĀ [43]:
df['CloseTime'] = pd.to_datetime(df['CloseTime'], unit='s')
InĀ [44]:
df.head()
Out[44]:
CloseTime OpenPrice HighPrice LowPrice ClosePrice Volume
0 2018-11-07 19:00:00 6586.9000 6587.0 6567.0000 6575.7974 276.496865
1 2018-11-07 20:00:00 6575.7974 6577.8 6567.1000 6567.4000 80.882415
2 2018-11-07 21:00:00 6567.3000 6575.8 6566.0356 6573.9000 80.704764
3 2018-11-07 22:00:00 6573.9126 6574.0 6568.3000 6573.0000 100.871970
4 2018-11-07 23:00:00 6573.0000 6573.0 6569.4000 6570.5000 105.118426

Now CloseTime it's a proper Timestamp type, which will simplify our process. The next step is making it the index of the DataFrame:

InĀ [45]:
df.set_index('CloseTime', inplace=True)
InĀ [46]:
df.head()
Out[46]:
OpenPrice HighPrice LowPrice ClosePrice Volume
CloseTime
2018-11-07 19:00:00 6586.9000 6587.0 6567.0000 6575.7974 276.496865
2018-11-07 20:00:00 6575.7974 6577.8 6567.1000 6567.4000 80.882415
2018-11-07 21:00:00 6567.3000 6575.8 6566.0356 6573.9000 80.704764
2018-11-07 22:00:00 6573.9126 6574.0 6568.3000 6573.0000 100.871970
2018-11-07 23:00:00 6573.0000 6573.0 6569.4000 6570.5000 105.118426

Let's visualize our Close Price using matplotlib:

InĀ [47]:
import matplotlib.pyplot as plt

%matplotlib inline
InĀ [48]:
df['ClosePrice'].plot(figsize=(14, 7))
Out[48]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f05448d76a0>

As anticipated, prices have gone down significantly since November 07, but seems to be a subtle recovery. One common method to analyze the variation and volatility of prices is with Bollinger Bands; let's replicate it using Pandas.

First, I'll "clean" my dataset and keep only ClosePrice, as "Price":

InĀ [50]:
price = df[['ClosePrice']].copy()
price.head()
Out[50]:
ClosePrice
CloseTime
2018-11-07 19:00:00 6575.7974
2018-11-07 20:00:00 6567.4000
2018-11-07 21:00:00 6573.9000
2018-11-07 22:00:00 6573.0000
2018-11-07 23:00:00 6570.5000
InĀ [52]:
price.columns = ['Price']
price.head()
Out[52]:
Price
CloseTime
2018-11-07 19:00:00 6575.7974
2018-11-07 20:00:00 6567.4000
2018-11-07 21:00:00 6573.9000
2018-11-07 22:00:00 6573.0000
2018-11-07 23:00:00 6570.5000

Second step is to compute the "rolling mean" (or "moving average"), it's extremely simple using pandas:

InĀ [57]:
price['Moving Mean'] = price['Price'].rolling(90).mean()
price[90:95]
Out[57]:
Price Moving Mean
CloseTime
2018-11-11 13:00:00 6424.3677 6472.339263
2018-11-11 14:00:00 6402.2764 6470.504557
2018-11-11 15:00:00 6417.9000 6468.771223
2018-11-11 16:00:00 6383.3430 6466.663923
2018-11-11 17:00:00 6387.9000 6464.635034

I chose arbitrarily 90 periods, (90 hours, as we have hourly data), you could have chosen a different window. Let's take a look at it using matplotlib again:

InĀ [58]:
price.plot(figsize=(14, 7))
Out[58]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f053e837e48>

What a Moving Mean does is just to smooth the process and remove noise; with it, you can see a clear downwards trend.

Our third step will now be adding the actual Bollinger bands; there are two bands (above and below) and they're computed just as a number of standard deviations above/below the rolling mean. Let's start by computing the "rolling standard deviation":

InĀ [59]:
price['Moving Std'] = price['Price'].rolling(90).std()
price[90:95]
Out[59]:
Price Moving Mean Moving Std
CloseTime
2018-11-11 13:00:00 6424.3677 6472.339263 47.890704
2018-11-11 14:00:00 6402.2764 6470.504557 47.368095
2018-11-11 15:00:00 6417.9000 6468.771223 46.386113
2018-11-11 16:00:00 6383.3430 6466.663923 45.903348
2018-11-11 17:00:00 6387.9000 6464.635034 45.293648

We can now add the two bands, as 2 stds above and below the moving mean:

InĀ [60]:
price['Upper Band'] = price['Moving Mean'] + (2 * price['Moving Std'])
price['Lower Band'] = price['Moving Mean'] - (2 * price['Moving Std'])
InĀ [61]:
price[['Price', 'Moving Mean', 'Upper Band', 'Lower Band']].plot(figsize=(14, 7))
Out[61]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f053e7eaf28>

As you can see, there are periods of extreme volatility and dispersion, where the price is going down quickly. At those periods, the bands widen (std is larger).

InĀ [64]:
df.head()
Out[64]:
OpenPrice HighPrice LowPrice ClosePrice Volume
CloseTime
2018-11-07 19:00:00 6586.9000 6587.0 6567.0000 6575.7974 276.496865
2018-11-07 20:00:00 6575.7974 6577.8 6567.1000 6567.4000 80.882415
2018-11-07 21:00:00 6567.3000 6575.8 6566.0356 6573.9000 80.704764
2018-11-07 22:00:00 6573.9126 6574.0 6568.3000 6573.0000 100.871970
2018-11-07 23:00:00 6573.0000 6573.0 6569.4000 6570.5000 105.118426

More VisualizationsĀ¶

When working with Time Series, it's really useful to have good visualizations, like candlestick charts and others with subplots for volume. They're not difficult to create with libraries like matplotlib and Bokeh (more on this in a second).

First, let's start with a simple matplotlib chart displaying prices + volume:

InĀ [118]:
fig, (ax1, ax2) = plt.subplots(nrows=2, sharex=True, figsize=(14, 7), gridspec_kw = {'height_ratios':[3, 1]})
df['ClosePrice'].plot(ax=ax1)
ax1.set_xlabel('Close Time')
ax1.set_ylabel('Price')

df['Volume'].plot(ax=ax2, color='purple')
ax2.set_ylabel('Volume')

fig.tight_layout()

Hello BokehĀ¶

We're going to use another viz library, Bokeh, to create a candlestick chart. Bokeh's great advantage is that it creates interactive charts (using a javascript library) that lets you zoom in/out, drag the charts, etc. Let me show you how it works:

InĀ [78]:
from bokeh.plotting import figure, output_file, show
from bokeh.io import output_notebook
InĀ [79]:
output_notebook()
Loading BokehJS ...
InĀ [143]:
TOOLS = "pan,wheel_zoom,box_zoom,reset,save"

p = figure(x_axis_type="datetime", tools=TOOLS, plot_width=700, title = "BTC Candlestick", x_range=(df.index[-250], df.index[-1]))
p.grid.grid_line_alpha=0.3

inc = df['ClosePrice'] > df['OpenPrice']
dec = df['OpenPrice'] > df['ClosePrice']
w = 12 * 60 * 60 * 80

p.segment(df.index, df['HighPrice'], df.index, df['LowPrice'], color="black")
p.vbar(df.index[inc], w, df.OpenPrice[inc], df.ClosePrice[inc], fill_color="#00af50", line_color="black")
p.vbar(df.index[dec], w, df.OpenPrice[dec], df.ClosePrice[dec], fill_color="#F2583E", line_color="black")

show(p)

Try zooming in and out or dragging the chart to the left to see previous days.

Wrapping upĀ¶

As you can see, getting started with Python for Crypto analysis is straightforward. Mostly thanks to Cryptowatch's API and the simplicity of Pandas + matplotlib.

Now, it's your turn!, copy this project and start playing with it.

Notebooks AI
Notebooks AI Profile20060