Newer
Older
This project contains a preprocessor Pipeline for Polkadot, Moonbeam and Interlay.
It requires a full node per chain with WS RPC enabled. The `producer_js` module will
connect to the node and store all raw block data as JSON in a Postgres db. The `preprocessor`
will use this data to create new tables in a seperate db. Data aquired contains historical balance data, transfer data, staking and reward data and specific chain only data for the Parachains.
## Setup
### Setup nodes
.sh scripts are provided to setup a node for each chain. The scripts will install the node and setup the WS RPC. They are found in `service_files/nodes`.
### Setup producer
See README in `producer_js` folder.
.sh scripts to run after building the container are provided in `service_files/producer`.
### Setup preprocessor
See README in `preprocessor/db` folder.
.sh scripts to run after building the container are provided in `service_files/preprocessor`.
## Other
The code used for the visualizations in the thesis can be found in `visualizations`.