GKEに構築する方法をまとめる。フルノード、特にlightning nodeを含む場合は、クラウドサービス上に構築するのが現実的だと思う。

前提

GCPプロジェクト準備済
GCLOUDをinstall済
dockerをinstall済
kubectlをinstall済

k8s

まずはクラスタを作る。対象のプロジェクトが存在するconfigurationを指定する。

gcloud config configurations activate CONFIGURATION_NAME

続いて、空のクラスタを作成する。

gcloud container clusters get-credentials test-cluster --zone asia-northeast1-a --project <projectID>  
gcloud container clusters describe test-cluster

続いて、今後のローカルでのkubectlとの連携のために、credentialを渡す。

$ gcloud container clusters get-credentials test-cluster
Fetching cluster endpoint and auth data.  
kubeconfig entry generated for test-cluster.  
$ kubectl config get-contexts

対象のコンテキストが指定されていることを確認する。

docker

LND、BTCDともに、最新のgithubのソースを利用する。

LND : https://github.com/lightningnetwork/lnd.git
BTCD : https://github.com/btcsuite/btcd.git

docker fileは下記の通り。基本的にはLNDに上がっているものをすこしだけ変えている。手動でのデプロイプロセスをまとめているが、今後cloud buildを利用した形に変える。

LND

FROM golang:1.13-alpine as builder

MAINTAINER bruwbird

# Install build dependencies such as git and glide.
RUN apk add --no-cache git gcc musl-dev

WORKDIR $GOPATH/src/github.com/lightningnetwork

# Grab and install the latest version of of lnd and all related dependencies.
RUN git clone https://github.com/lightningnetwork/lnd.git

# Force Go to use the cgo based DNS resolver. This is required to ensure DNS
# queries required to connect to linked containers succeed.
ENV GODEBUG netdns=cgo

# Install dependencies and install/build lnd.
RUN apk add --no-cache --update alpine-sdk \  
    make \
    &&  cd /go/src/github.com/lightningnetwork/lnd \
    &&  make \
    &&  make install tags="signrpc walletrpc chainrpc invoicesrpc routerrpc"

# Start a new, final image to reduce size.
FROM alpine as final

# Expose lnd ports (server, rpc).
EXPOSE 9735 10009

# Copy the binaries and entrypoint from the builder image.
COPY --from=builder /go/bin/lncli /bin/  
COPY --from=builder /go/bin/lnd /bin/

# Add bash.
RUN apk add --no-cache \  
    bash

# Copy the entrypoint script.
COPY "start-lnd.sh" .  
RUN chmod +x start-lnd.sh

start-lnd.sh

#!/usr/bin/env bash

# exit from script if error was raised.
set -e

# error function is used within a bash function in order to send the error
# message directly to the stderr output and exit.
error() {  
    echo "$1" > /dev/stderr
    exit 0
}

# return is used within bash function in order to return the value.
return() {  
    echo "$1"
}

# set_default function gives the ability to move the setting of default
# env variable from docker file to the script thereby giving the ability to the
# user override it durin container start.
set_default() {  
    # docker initialized env variables with blank string and we can't just
    # use -z flag as usually.
    BLANK_STRING='""'

    VARIABLE="$1"
    DEFAULT="$2"

    if [[ -z "$VARIABLE" || "$VARIABLE" == "$BLANK_STRING" ]]; then

        if [ -z "$DEFAULT" ]; then
            error "You should specify default variable"
        else
            VARIABLE="$DEFAULT"
        fi
    fi

   return "$VARIABLE"
}

# Set default variables if needed.
RPCUSER=$(set_default "$RPCUSER" "devuser")  
RPCPASS=$(set_default "$RPCPASS" "devpass")  
DEBUG=$(set_default "$DEBUG" "debug")  
NETWORK=$(set_default "$NETWORK" "simnet")  
CHAIN=$(set_default "$CHAIN" "bitcoin")  
BACKEND="btcd"  
if [[ "$CHAIN" == "litecoin" ]]; then  
    BACKEND="ltcd"
fi

exec lnd \  
    --logdir="/data" \
    "--$CHAIN.active" \
    "--$CHAIN.$NETWORK" \
    "--$CHAIN.node"="btcd" \
    "--$BACKEND.rpccert"="/rpc/rpc.cert" \
    "--$BACKEND.rpchost"="blockchain" \
    "--$BACKEND.rpcuser"="$RPCUSER" \
    "--$BACKEND.rpcpass"="$RPCPASS" \
    --debuglevel="$DEBUG" \
    "$@"

BTCD

FROM golang:1.12-alpine as builder

MAINTAINER bruwbird

# Install build dependencies such as git and glide.
RUN apk add --no-cache git gcc musl-dev

WORKDIR $GOPATH/src/github.com/btcsuite/btcd

# Grab and install the latest version of of btcd and all related dependencies.
RUN git clone https://github.com/btcsuite/btcd.git . \  
    &&  GO111MODULE=on go install -v . ./cmd/...

# Start a new image
FROM alpine as final

# Expose mainnet ports (server, rpc)
EXPOSE 8333 8334

# Expose testnet ports (server, rpc)
EXPOSE 18333 18334

# Expose simnet ports (server, rpc)
EXPOSE 18555 18556

# Expose segnet ports (server, rpc)
EXPOSE 28901 28902

# Copy the compiled binaries from the builder image.
COPY --from=builder /go/bin/addblock /bin/  
COPY --from=builder /go/bin/btcctl /bin/  
COPY --from=builder /go/bin/btcd /bin/  
COPY --from=builder /go/bin/findcheckpoint /bin/  
COPY --from=builder /go/bin/gencerts /bin/

COPY "start-btcctl.sh" .  
COPY "start-btcd.sh" .

RUN apk add --no-cache \  
    bash \
    ca-certificates \
    &&  mkdir "/rpc" "/root/.btcd" "/root/.btcctl" \
    &&  touch "/root/.btcd/btcd.conf" \
    &&  chmod +x start-btcctl.sh \
    &&  chmod +x start-btcd.sh \
    # Manually generate certificate and add all domains, it is needed to connect
    # "btcctl" and "lnd" to "btcd" over docker links.
    && "/bin/gencerts" --host="*" --directory="/rpc" --force

# Create a volume to house pregenerated RPC credentials. This will be
# shared with any lnd, btcctl containers so they can securely query btcd's RPC
# server.
# You should NOT do this before certificate generation!
# Otherwise manually generated certificate will be overridden with shared
# mounted volume! For more info read dockerfile "VOLUME" documentation.
VOLUME ["/rpc"]

start-btcd.sh

#!/usr/bin/env bash

# exit from script if error was raised.
set -e

# error function is used within a bash function in order to send the error
# message directly to the stderr output and exit.
error() {  
    echo "$1" > /dev/stderr
    exit 0
}

# return is used within bash function in order to return the value.
return() {  
    echo "$1"
}

# set_default function gives the ability to move the setting of default
# env variable from docker file to the script thereby giving the ability to the
# user override it durin container start.
set_default() {  
    # docker initialized env variables with blank string and we can't just
    # use -z flag as usually.
    BLANK_STRING='""'

    VARIABLE="$1"
    DEFAULT="$2"

    if [[ -z "$VARIABLE" || "$VARIABLE" == "$BLANK_STRING" ]]; then

        if [ -z "$DEFAULT" ]; then
            error "You should specify default variable"
        else
            VARIABLE="$DEFAULT"
        fi
    fi

   return "$VARIABLE"
}

# Set default variables if needed.
RPCUSER=$(set_default "$RPCUSER" "devuser")  
RPCPASS=$(set_default "$RPCPASS" "devpass")  
DEBUG=$(set_default "$DEBUG" "info")  
NETWORK=$(set_default "$NETWORK" "simnet")

PARAMS=""  
if [ "$NETWORK" != "mainnet" ]; then  
   PARAMS=$(echo --$NETWORK)
fi

PARAMS=$(echo $PARAMS \  
    "--debuglevel=$DEBUG" \
    "--rpcuser=$RPCUSER" \
    "--rpcpass=$RPCPASS" \
    "--datadir=/data" \
    "--logdir=/data" \
    "--rpccert=/rpc/rpc.cert" \
    "--rpckey=/rpc/rpc.key" \
    "--rpclisten=0.0.0.0" \
    "--txindex"
)

# Set the mining flag only if address is non empty.
if [[ -n "$MINING_ADDRESS" ]]; then  
    PARAMS="$PARAMS --miningaddr=$MINING_ADDRESS"
fi

# Add user parameters to command.
PARAMS="$PARAMS $@"

# Print command and start bitcoin node.
echo "Command: btcd $PARAMS"  
exec btcd $PARAMS

上記のイメージ構築し、事前にGCR上に保存する必要がある。まず、Container Registry を認証するには、gcloud を Docker 認証ヘルパーとして使用する。

gcloud auth configure-docker

これができたら、imageをbuildし、gcrにpushする。ホスト名と Google Cloud Platform Console のプロジェクト ID とイメージ名を組み合わせたものを指定する必要がある。

[HOSTNAME]/[PROJECT-ID]/[IMAGE]

docker build . -t  gcr.io/<PROJECT_NAME>/lnd:latest  
docker build . -t  gcr.io/<PROJECT_NAME>/btcd:latest

下記を実行すると、下記の用にイメージが表示されるはずだ。

$ docker images
REPOSITORY                                                  TAG                 IMAGE ID            CREATED             SIZE  
gcr.io/kouzoh-p-bruwbird/btcd                               latest              aaaaaaaaaaaa        25 hours ago        59.7MB  
gcr.io/kouzoh-p-bruwbird/lnd                                latest              aaaaaaaaaaaa        25 hours ago        72.7MB

続いて、このイメージをGCRにプッシュする。

docker push gcr.io/<PROJECT_NAME>/btcd:latest  
docker push gcr.io/<PROJECT_NAME>/lnd:latest

GCPコンソールでGCRを確認すると、イメージが存在する。

コマンドでも可。

gcloud container images list

deploy

statefulsetを利用する。フルノードのデータやキーデータ等は、再起動に関わらずstatefulに保持したいためだ。定義ファイルは下記の通り。今後ここにmonitor等を追加していく。

apiVersion: apps/v1  
kind: StatefulSet  
metadata:  
  name: lnd-btcd
spec:  
  selector:
    matchLabels:
      app: lnd-btcd # Label selector that determines which Pods belong to the StatefulSet
                 # Must match spec: template: metadata: labels
  serviceName: "lnd-btcd"
  replicas: 2
  template:
    metadata:
      labels:
        app: lnd-btcd # Pod template's label selector
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: btcd
        image: gcr.io/{yourproject}/btcd:latest
        command:
        - ./start-btcd.sh
        env:
        - name: DEBUG
        - name: MINING_ADDRESS
        - name: NETWORK
        - name: RPCPASS
        - name: RPCUSER
        volumeMounts:
        - mountPath: /rpc
          name: lndcontainer-shared
        - mountPath: /data
          name: lndcontainer-bitcoin
      - name: lnd
        image: gcr.io/{yourproject}/lnd:latest
        command:
        - ./start-lnd.sh
        env:
        - name: CHAIN
        - name: DEBUG
        - name: NETWORK
        - name: RPCPASS
        - name: RPCUSER
        volumeMounts:
        - mountPath: /rpc
          name: lndcontainer-shared
  volumeClaimTemplates:
  - metadata:
      name: lndcontainer-bitcoin
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
  - metadata:
      name: lndcontainer-shared
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

上記の設定ファイルを対象にして、applyする。

$ kubectl apply -f LND.yaml --record
statefulset.apps/lnd-btcd created

一定時間が経過すると、podsが作成されていることが確認できる。

$ kubectl get pods
NAME         READY     STATUS    RESTARTS   AGE  
lnd-btcd-0   2/2       Running   1          99m  
lnd-btcd-1   2/2       Running   0          99m

volumeは下記の通り。

kubectl get pvc -o wide  
NAME                              STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE  
lndcontainer-bitcoin-lnd-btcd-0   Bound     pvc-b6c7ecca-e4ac-11e9-9cc8-42010a92001a   1Gi        RWO            standard       3m36s  
lndcontainer-bitcoin-lnd-btcd-1   Bound     pvc-c91f2495-e4ac-11e9-9cc8-42010a92001a   1Gi        RWO            standard       3m5s  
lndcontainer-shared-lnd-btcd-0    Bound     pvc-b6c721c1-e4ac-11e9-9cc8-42010a92001a   1Gi        RWO            standard       3m36s  
lndcontainer-shared-lnd-btcd-1    Bound     pvc-c922fec9-e4ac-11e9-9cc8-42010a92001a   1Gi        RWO            standard       3m5s

動作

実際にコンテナの中に入り、動作確認をする。

kubectl exec -it lnd-btcd-0 -c lnd bash  
lncli create  
Input wallet password:  
Confirm password:


Do you have an existing cipher seed mnemonic you want to use? (Enter y/n): n

Your cipher seed can optionally be encrypted.  
Input your passphrase if you wish to encrypt it (or press enter to proceed without a cipher seed passphrase): 

Generating fresh cipher seed...

!!!YOU MUST WRITE DOWN THIS SEED TO BE ABLE TO RESTORE THE WALLET!!!

---------------BEGIN LND CIPHER SEED---------------
 1. abstract   2. high       3. biology   4. slight 
 5. weekend    6. tonight    7. mystery   8. submit 
 9. easily    10. royal     11. wood     12. figure 
13. benefit   14. ordinary  15. ceiling  16. item  
17. lottery   18. next      19. opera    20. clump  
21. faith     22. copper    23. song     24. tuition  
---------------END LND CIPHER SEED-----------------

!!!YOU MUST WRITE DOWN THIS SEED TO BE ABLE TO RESTORE THE WALLET!!!

lnd successfully initialized!

無事動作しそうだ。作業が終わったら、一旦クラスタごと削除する。

gcloud container clusters delete test-cluster

続いて、pvcも削除する。クラスタを削除すると、コンテキストももちろん削除されるので、コンソールからやったが、これを先にやったほうが良かったかもしれない。

Menu

LND-BTCDフルノード環境をGKEに構築する

前提

k8s

docker

LND

BTCD

deploy

動作