storm中基于k8s的安装示例
这篇文章主要介绍了storm中基于k8s的安装示例，具有一定借鉴价值，感兴趣的朋友可以参考下，希望大家阅读完这篇文章之后大有收获，下面让小编带着大家一起了解一下。
目前创新互联已为上1000家的企业提供了网站建设、域名、网络空间、网站托管、服务器租用、企业网站设计、桓台网站维护等服务，公司将坚持客户导向、应用为本的策略，正道将秉承"和谐、参与、激情"的文化，与客户和合作伙伴齐心协力一起成长，共同发展。
首先我们先来编写一下Dockerfile
FROM ip:80/bigdata/centos7-jdk:1.8
#config storm env
ENV STORM_USER=storm \
STORM_INSTALL_BASE=/home/frank/platform \
STORM_HOME=/home/frank/platform/storm \
STORM_CONF_DIR=/home/frank/conf/storm \
STORM_DIR=/home/frank/hard_disk/storm \
STORM_BLOBS=/home/frank/hard_disk/storm/blobs \
STORM_NIMBUS=/home/frank/hard_disk/storm/nimbus \
STORM_LOG_DIR=/home/frank/logs/storm \
STORM_ROOT=/home/frank \
STORM_LOG=/home/frank/logs
ENV PATH $PATH:$STORM_HOME/bin
#add FLINK user and make dirs
RUN set -x \
&& adduser "$STORM_USER" \
&& mkdir -p "$STORM_INSTALL_BASE" "$STORM_CONF_DIR" "$STORM_LOG" "$STORM_LOG_DIR" "$STORM_BLOBS" "$STORM_NIMBUS"
#install storm libs
ADD apache-storm-1.2.2.tar.gz $STORM_INSTALL_BASE
RUN cd $STORM_INSTALL_BASE && ln -s -f apache-storm-1.2.2 storm && mkdir -p /home/storm
RUN chown -R "$STORM_USER:$STORM_USER" "$STORM_ROOT"
COPY docker-entrypoint.sh /home/storm
RUN chmod 755 /home/storm/docker-entrypoint.sh
USER storm
ENTRYPOINT ["/home/storm/docker-entrypoint.sh"]
其中的脚本如下：
#!/bin/sh
# If unspecified, the hostname of the container is taken as the JobManager address
ACTION_CMD="$1"
source /etc/profile
export STORM_CONF_DIR=/home/frank/conf/storm
export STORM_CONF_FILE=/home/frank/conf/storm/storm.yaml
sed -i -e "s/storm.zookeeper.servers: \[localhost\]/storm.zookeeper.servers: \[${STORM_ZK_QUORUM}\]/" /home/frank/conf/storm/storm.yaml
sed -i -e "s/storm.zookeeper.root: \"\/storm\"/storm.zookeeper.root: \/${STORM_ZK_ROOT}/" /home/frank/conf/storm/storm.yaml
# if use cluster model, pod ${JOB_CLUSTER_NAME}-0,${JOB_CLUSTER_NAME}-1 as jobmanager
if [ ${ACTION_CMD} == "cluster" ]; then
nimbuses=(${NIMBUS_HOSTS//,/ })
ACTION_CMD="supervisor"
for i in ${!nimbuses[@]}
do
if [ "$(hostname -s)" == "${nimbuses[i]}" ]; then
NIMBUS_ADDRESS=${NIMBUS_ADDRESS:-$(hostname -f)}
ACTION_CMD="nimbus"
sed -i -e "s/nimbus.seeds: \[localhost\]/nimbus.seeds: \["${NIMBUS_ADDRESS}"\]/" /home/frank/conf/storm/storm.yaml
echo "pod hostname match nimbus config host, change action to nimbus."
fi
done
fi
# if ha model, replace ha configuration
if [ ${ACTION_CMD} == "help" ]; then
echo "Usage: $(basename "$0") (cluster|nimbus|supervisor|ui|logviewer|help)"
exit 0
elif [ ${ACTION_CMD} == "nimbus" ]; then
echo "Starting nimbus ui logviewer supervisor"
nohup /home/frank/platform/storm/bin/storm nimbus >> /tmp/nimbus.log 2>&1 & \
nohup /home/frank/platform/storm/bin/storm ui >> /tmp/ui.log 2>&1 & \
/home/frank/platform/storm/bin/storm logviewer
elif [ ${ACTION_CMD} == "supervisor" ]; then
echo "Starting supervisor"
/home/frank/platform/storm/bin/storm supervisor
fi
镜像构建命令如下：
docker build --network=host -f Dockerfile -t ip:80/bigdata/storm:1.2.2 .
下面我们提供一下service的创建文件
apiVersion: v1
kind: Service
metadata:
name: storm-cluster
labels:
app: storm-cluster
namespace: storm
spec:
clusterIP: None
ports:
- port: 8080
name: ui
selector:
app: storm-cluster
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: storm-cluster
namespace: storm
spec:
selector:
matchLabels:
app: storm-cluster
serviceName: storm-cluster
replicas: 4
podManagementPolicy: Parallel
template:
metadata:
labels:
app: storm-cluster
spec:
terminationGracePeriodSeconds: 2
containers:
- name: storm-cluster
imagePullPolicy: Always
image: ip:80/bigdata/storm:1.2.2
command: ["/bin/sleep"]
args: ["30d"]
volumeMounts:
- name: storm-conf
mountPath: /home/conf/storm
- name: storm-log
mountPath: /home/logs/storm
resources:
requests:
memory: "4096Mi"
cpu: 4
limits:
memory: "4096Mi"
cpu: 4
env:
- name: NIMBUS_HOSTS
value: "storm-cluster-2,storm-cluster-3"
- name: STORM_ZK_QUORUM
value: "ip"
- name: STORM_ZK_ROOT
value: "stormNew"
ports:
- containerPort: 6627
name: nimbus
- containerPort: 8000
name: logviewer
- containerPort: 8080
name: storm-ui
- containerPort: 6700
name: sv0
- containerPort: 6701
name: sv1
- containerPort: 6702
name: sv2
- containerPort: 6703
name: sv3
- containerPort: 22
name: ssh
volumes:
- name: storm-conf
configMap:
name: storm-conf
- name: storm-log
hostPath:
path: /stormtmp
type: Directory
service的创建脚本如下：
kubectl create -f storm-svc.yaml
statefulSet的创建脚本如下：
kubectl create -f storm-statefulSet.yaml
这个创建脚本要注意如下问题：
command:/bin/sleep
程序需要手动启动
/home/vipshop/hard_disk/storm
drwxr-xr-x 2 storm storm 4096 Jul 12 12:20 blobs
drwxr-xr-x 2 storm storm 4096 Jul 12 12:20 nimbus
drwxrwxr-x 5 storm storm 4096 Jul 29 00:00 supervisor
这个hard_disk目录需要有storm权限
注：nimbus.thrift.max_buffer_size: 10485760
这个参数需要是2的倍数
感谢你能够认真阅读完这篇文章，希望小编分享的“storm中基于k8s的安装示例”这篇文章对大家有帮助，同时也希望大家多多支持创新互联，关注创新互联行业资讯频道，更多相关知识等着你来学习!
网站题目：storm中基于k8s的安装示例
文章地址：http://scyanting.com/article/pdcoig.html
