By Vivek Teegalapally in Ansible — Jul 24, 2024

Running BlockBook using Docker on AWS EC2 with Terraform, Ansible

This guide provides detailed steps to run BlockBook using Docker on a Virtual Machine (VM). It covers the installation of Docker, pulling the necessary Docker images, configuring the containers, setting up a bridge network, and setting up a reverse proxy using Nginx and Certbot for SSL. By the end of this guide, you will have a fully functional BlockBook instance running in Docker containers on your VM.

In addition, we will provide steps to automate the provisioning and setup of your VM using Terraform and Ansible. This will ensure that you can replicate the environment quickly and efficiently.

Prerequisites
Setting Up the Environment
1. Install Docker
2. Creating a Bridge Network
3. Managing Docker Volumes
4. Pulling Docker Images
Understanding Docker-Related Files for BlockBook
1. Dockerfile
2. Entrypoint Bash Script (entrypoint.sh)
3. Docker Compose File
4. .env File
Running BlockBook Docker Containers
1. Starting Services with Docker Compose
2. Running Containers Manually
3. Checking if Services are Running
Provisioning with Terraform and Ansible

Prerequisites

A VM running Ubuntu 18.04 or higher.
Docker installed (see Docker installation guide)
Basic knowledge of Docker and networking concepts.

Setting Up the Environment

Install Docker

First, make sure Docker is installed and running on your VM. For installation instructions, refer to the official Docker documentation.

Creating a Bridge Network

To enable communication between your BlockBook and backend containers, create a custom Docker bridge network:

docker network create blockbook-net

A bridge network allows Docker containers to communicate with each other and with the host system while remaining isolated from other networks. This setup is ideal for scenarios where multiple containers need to work together, such as with BlockBook and its backend services.

Managing Docker Volumes

Docker volumes persist data generated by and used by Docker containers. To ensure data persistence and separation between your Docker containers and the host system, use Docker volumes:

docker volume create blockbook

The blockbook volume stores data for both the BlockBook backend and frontend services. Using a single volume ensures that data remains consistent and accessible across the services.

Pulling Docker Images

The docker image to be used is available on DockerHub.

docker pull vivekteega/blockbook:1.0.0

In this section, we’ll delve into the key Docker-related files that are essential for setting up and running BlockBook. These files include the Dockerfile, entrypoint.sh, .env, and docker-compose.yml. Understanding these components will give you insight into how BlockBook is containerized and configured.

Dockerfile

The Dockerfile defines how to build the Docker image for BlockBook. Here’s a breakdown:

FROM ubuntu:22.04

# Backend
COPY ./deb-files/backend-flo_0.15.1.1-satoshilabs-1_amd64.deb /opt/backend.deb
COPY ./deb-files/blockbook-flo_0.4.0_amd64.deb /opt/blockbook.deb

RUN apt update && apt install -y /opt/backend.deb /opt/blockbook.deb curl && \
    sed -i 's/daemon=1/daemon=0/' /opt/coins/nodes/flo/flo.conf && \
    sed -i '/rpcport=8066/a rpcallowip=0.0.0.0/0' /opt/coins/nodes/flo/flo.conf && \
    echo "addnode=ramanujam.ranchimall.net" >> /opt/coins/nodes/flo/flo.conf && \
    echo "addnode=turing.ranchimall.net" >> /opt/coins/nodes/flo/flo.conf && \
    echo "addnode=stevejobs.ranchimall.net" >> /opt/coins/nodes/flo/flo.conf && \
    echo "addnode=brahmagupta.ranchimall.net" >> /opt/coins/nodes/flo/flo.conf && \
    echo "addnode=feynman.ranchimall.net" >> /opt/coins/nodes/flo/flo.conf

WORKDIR /opt/coins/blockbook/flo

# Execution
COPY ./entrypoint.sh /opt/entrypoint.sh
RUN chmod +x /opt/entrypoint.sh
ENTRYPOINT ["/opt/entrypoint.sh"]

Base Image: Uses Ubuntu 22.04.
Dependencies: Installs BlockBook and backend dependencies.
Configuration: Modifies configuration files to set up the BlockBook environment.
Entrypoint: Sets the script to run when the container starts.

Entrypoint Bash Script (`entrypoint.sh`)

The entrypoint.sh script is crucial for initialising the BlockBook containers. It performs several key operations, including handling bootstrap files, updating configuration files, and starting the BlockBook services. Here's a summary of its main components:

Bootstrap Handling

The script manages bootstrap files for both frontend and backend components:

Bootstrap from URL: If a URL for the bootstrap file is provided, it downloads the file, extracts it, and stores it in a specified directory.
Bootstrap from File: If a local bootstrap file is provided, it verifies the file's hash to ensure it hasn't been processed before, then extracts it.

Configuration Updates

For both frontend and backend components, the script updates configuration files with the appropriate IP addresses if necessary. This ensures that the services have the correct settings to connect with each other.

Running Services

Based on the provided argument (frontend or backend), the script:

Frontend: Sets up and runs the BlockBook frontend service, configuring it with the necessary bootstrap files and updating configuration settings.
Backend: Sets up and runs the BlockBook backend service, similarly managing bootstrap files and configuration.

Here's an overview of the script:

#!/bin/sh
set -e

# Bootstrap file paths and URLs for frontend and backend
# Function definitions for calculating file hash, updating configuration, and handling bootstrap

case "$1" in
  "frontend")
    # Handle frontend bootstrap and start BlockBook frontend service
    ;;
  "backend")
    # Handle backend bootstrap and start BlockBook backend service
    ;;
  *)
    echo "Invalid option: $1"
    exit 1
    ;;
esac

This script ensures that the BlockBook containers are properly initialised with the required data and configuration before they start running.

Docker Compose file

The docker-compose.yml file defines and runs BlockBook containers:

version: '3.8'

services:
  blockbook-backend:
    image: ${BACKEND_IMAGE}
    container_name: ${BACKEND_CONTAINER_NAME}
    networks:
      - blockbook
    volumes:
      - blockbook:/opt
      - ${BACKEND_BOOTSTRAP_FILE:-/dev/null}:/local-bootstrap/bootstrap-blockbook-flo-mainnet-backend.tar.gz  # Use /dev/null if no file
    ports:
      - "${BACKEND_PORT_1}:38366"
      - "${BACKEND_PORT_2}:8066"
    command: backend
    environment:
      - BOOTSTRAP_FILE=/local-bootstrap/bootstrap-blockbook-flo-mainnet-backend.tar.gz  # Set to empty string if not provided
      - BOOTSTRAP_URL=${BACKEND_BOOTSTRAP_URL:-}  # Set to empty string if not provided

  blockbook-frontend:
    image: ${FRONTEND_IMAGE}
    container_name: ${FRONTEND_CONTAINER_NAME}
    networks:
      - blockbook
    volumes:
      - blockbook:/opt
      - ${FRONTEND_BOOTSTRAP_FILE:-/dev/null}:/local-bootstrap/bootstrap-blockbook-flo-mainnet-frontend.tar.gz  # Use /dev/null if no file
    ports:
      - "${FRONTEND_PORT_1}:9166"
      - "${FRONTEND_PORT_2}:9066"
    depends_on:
      - blockbook-backend
    entrypoint: /bin/sh -c
    command: >
      "/opt/entrypoint.sh frontend $(getent hosts blockbook-backend | awk '{ print $1 }')"
    environment:
      - BOOTSTRAP_FILE=/local-bootstrap/bootstrap-blockbook-flo-mainnet-frontend.tar.gz  # Set to empty string if not provided
      - BOOTSTRAP_URL=${FRONTEND_BOOTSTRAP_URL:-}  # Set to empty string if not provided

volumes:
  blockbook:

networks:
  blockbook:
    driver: bridge

Images and Containers: Specifies images and container names for backend and frontend.
Volumes: Mounts volumes for persistent data and bootstrap files.
Ports: Maps container ports to host ports.
Environment Variables: Configures bootstrap file paths and URLs.
Network: Uses a custom bridge network for container communication.

.env file

The .env file configures environment variables for Docker Compose:

BACKEND_IMAGE=vivekteega/blockbook:1.0.0
FRONTEND_IMAGE=vivekteega/blockbook:1.0.0
BACKEND_CONTAINER_NAME=blockbook-backend
FRONTEND_CONTAINER_NAME=blockbook-frontend
BACKEND_BOOTSTRAP_FILE=/path/to/your/bootstrap-blockbook-flo-mainnet-backend.tar.gz  # Set to empty string if not provided
FRONTEND_BOOTSTRAP_FILE=/path/to/your/bootstrap-blockbook-flo-mainnet-frontend.tar.gz  # Set to empty string if not provided
BACKEND_BOOTSTRAP_URL=/path/to/your/bootstrap-blockbook-flo-mainnet-backend.tar.gz  # Set to empty string if not provided
FRONTEND_BOOTSTRAP_URL=/path/to/your/bootstrap-blockbook-flo-mainnet-frontend.tar.gz  # Set to empty string if not provided
BACKEND_PORT_1=38366
BACKEND_PORT_2=8066
FRONTEND_PORT_1=9166
FRONTEND_PORT_2=9066

Running BlockBook Docker Containers

Starting Services with Docker Compose

To start the services defined in your docker-compose.yml file:

docker-compose up -d

This command will launch the BlockBook backend and frontend services as defined in your docker-compose.yml file. The services will start processing and indexing blockchain data.

Running Containers Manually

Running Backend Container

To run the BlockBook backend container manually, set up the network and volume, then execute:

docker run -d --name blockbook-backend \
  -p 38366:38366 -p 8066:8066 \
  -v blockbook:/opt \
  -v ${BACKEND_BOOTSTRAP_FILE:-/dev/null}:/local-bootstrap/bootstrap-blockbook-flo-mainnet-backend.tar.gz \
  vivekteega/blockbook:1.0.0 backend

Running BlockBook Container

Provide instructions to run the BlockBook container:

docker run -d --name blockbook-frontend \
  -p 9166:9166 \
  -v blockbook:/opt \
  -v ${FRONTEND_BOOTSTRAP_FILE:-/dev/null}:/local-bootstrap/bootstrap-blockbook-flo-mainnet-frontend.tar.gz \
  vivekteega/blockbook:1.0.0 frontend

Checking if Services are Running

Logs

Backend Logs

sudo docker logs -f blockbook-backend

Frontend Logs

sudo docker logs -f blockbook-frontend

Network Status

To check the network status for both services:

netstat -plnt | grep 8066 # for backend mainnet netstat -plnt | grep 18066 # for backend testnet netstat -plnt | grep 9166 # for BlockBook mainnet netstat -plnt | grep 19166 # for BlockBook testnet

Provisioning with Terraform and Ansible

To automate VM provisioning and BlockBook setup:

Terraform: Provision EC2 instances on AWS.
Ansible: Configure and deploy BlockBook on the VM.

Refer to the DevOps Kaizen repository for Terraform code and Ansible playbooks.

If you have any questions or need assistance, feel free to reach out via email or connect on social media. Happy deploying!

Table of Contents