ganglia install

ganglia master install

1
yum -y install ganglia ganglia-web ganglia-gmond ganglia-gmetad ganglia-devel ganglia-gmond-python pcp-import-ganglia2pcp nordugrid-arc-gangliarc libnodeupdown-backend-ganglia

ganglia node install

1
yum -y install ganglia-gmond

ganglia 简介

1
与Cacti、Nagios、Zabbix等工具相比,Ganglia更关注整个集群的性能和可用性。可以用于集群的性能监控、分析和优化。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
1.Gmetad程序:
Gmetad程序可以部署在集群内的任一个节点或者通过网络连接到集群的独立主机,它通过单播路由的方式与gmond通信负责周期性的到各个集群的数据源(datasource)节点去pool各个集群的数据,并更新到rrd数据库中.
一个数据源节点就是是一个gmond节点;一个gmetad节点可以设置多个数据源节点,每个数据源节点可以有多个备份,一个数据源节点失败了还可以从其他节点取数据.可以把Gmetad理解为服务器.
Gmetad只有TCP通道,一方面它向datasource发送请求,获取数据源节点的XML文件,另一方面会使用一个8651的默认TCP端口发布自身收集的XML文件,因此,Gmetad即可以从Gmond也可以从其他Gmetad节点获取XML数据

2.Gmond程序:
Gmond收集本机的监控数据,然后发送到集群内的其他机器上,并获取其他机器上的监控数据,Gmond之间通过udp通信,传递文件格式为XDL
Gmond收集到的数据供Gmetad读取,Gmond通过默认端口8649监听到Gmetad请求后将XML格式数据发送给Gmetad.可以将Gmond理解为客户端.
Gmond收集数据有muticast和unicast两种.Gmond本身具有UDP的发送(send)和接受(recv)通道以及TCP的接收(recv)通道.
其中UDP通道用于向其他Gmond节点发送或接收数据,TCP通道主要接受来自Gmetad的请求,向Gmetad发送XML文件.
在muticast模式下,Gmond节点之间通过UDP向多播目标相互传递数据.

Gmond节点模块结构主要有三个模块组成:
1.collect and publish模块
该模块周期性的调用一些内部命令获得metric data,然后将这些数据通过UDP通道发布给其他Gmond节点
2.Listen Threads模块
监听其他Gmond节点发送的UDP数据,并将这些数据存放在内存中
3.XML Export Threads模块
负责将数据以XML格式发布出去,比如交给Gmetad

在unicast模式下,多个Gmond节点通过UDP向单播的目标主机host的Gmond发送数据,Gmetad然后向目标主机的Gmond请求XML文件
在Unicast模式下,Gmond、Gmetad、rrd数据库以及web前端通常位于集群内的同一个节点上,该节点负责收集、存储、显示被监控的各节点的状态信息.
Web前端通常和Gmetad安装在同一个节点上,它从Gmetad中取数据,并且读取rrd数据库,生成图片显示出来.
1
2
3
4
5
6
7
8
9
10
11
在动手部署Ganglia之前,首先要对监控体系进行初步的规划.主要考虑两方面的问题:
1.单集群 or 多集群
如果节点较少,使用单集群配置起来更容易;如果节点很多,使用多集群可以避免广播风暴
但是需要为每个集群配置不同的组播通道(通过端口区分),同时要配置gmetad同时监听这多个通道

2.组播模式 or 单播模式
组播模式是ganglia的默认模式,同一集群的多个gmond之间互相交换数据,gmetad中可以指定集群中的任意一个或多个节点作为"data_source";
组播模式可能会带来网络的"抖动(Jitter)"据说设置节点的时钟同步可以避免抖动的问题;但如果网络环境不支持组播()比如Amazon’s AWS EC2)就需要使用单播模式
单播模式时,将大部分节点的gmond.conf中,global的deaf设置改为"yes",则这些节点只发生数据,不接收其他节点的数据,同样也不能作为gmetad中的"data_source"。
单播模式中还需要设置"send_metadata_interval",比如30秒,以强制发送元数据.
ganglia将一个gmetad覆盖的所有集群/节点称为一个grid.可以在/etc/ganglia/gmetad.conf中通过gridname指定其名称.多个grid的数据也可以聚合到一个上级gmetad中.

  1. ganglia 安装前选择

    1
    2
    3
    选择多集群,单播模式:
    server端: ganglia-web gmetad gmond
    client端: gmond
  2. 安装基础软件包

    1
    2
    3
    [root@10 ~]# yum makecache
    [root@10 ~]# yum -y install curl wget vim lrzsz net-tools dstat
    [root@10 ~]# yum -y install libart_lgpl-devel gcc gcc-c++ python-devel libconfuse-devel expat-devel apr-devel pcre-devel rpm-build libconfuse libconfuse-devel rrdtool-perl rrdtool-devel
  3. ganglia master install

    1
    2
    3
    4
    5
    6
    7
    [root@10 srv]# yum -y install ganglia ganglia-web ganglia-gmond-python ganglia-gmond ganglia-gmetad ganglia-devel

    [root@10 ~]# mkdir -p /mnt/data
    [root@10 ~]# cp -r /var/lib/ganglia /mnt/data/ganglia
    [root@10 ~]# chown -R apache.apache /mnt/data/ganglia/conf
    [root@10 ~]# chown -R apache.apache /mnt/data/ganglia/dwoo
    [root@10 ~]# chown -R ganglia.ganglia /mnt/data/ganglia/rrds
  4. 修改配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    [root@10 ~]# vim /etc/ganglia/gmetad.conf
    data_source "ssp" 10.0.2.15
    gridname "ssp"
    setuid_username ganglia
    xml_port 8651
    interactive_port 8652
    rrd_rootdir "/mnt/data/ganglia/rrds"
    case_sensitive_hostnames 0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    [root@10 ~]# vim /etc/ganglia/gmond.conf
    globals {
    daemonize = yes
    setuid = yes
    user = ganglia
    debug_level = 0
    max_udp_msg_len = 1472
    mute = no
    deaf = yes
    allow_extra_data = yes
    host_dmax = 86400 /*secs. Expires (removes from web interface) hosts in 1 day */
    host_tmax = 20 /*secs */
    cleanup_threshold = 300 /*secs */
    gexec = no
    # By default gmond will use reverse DNS resolution when displaying your hostname
    # Uncommeting following value will override that value.
    # override_hostname = "mywebserver.domain.com"
    # If you are not using multicast this value should be set to something other than 0.
    # Otherwise if you restart aggregator gmond you will get empty graphs. 60 seconds is reasonable
    send_metadata_interval = 30 /*secs */
    }
    cluster {
    name = "ssp"
    owner = "ssp"
    latlong = "unspecified"
    url = "unspecified"
    }
    host {
    location = "unspecified"
    }
    udp_send_channel {
    #bind_hostname = yes # Highly recommended, soon to be default.
    # This option tells gmond to use a source address
    # that resolves to the machine's hostname. Without
    # this, the metrics may appear to come from any
    # interface and the DNS names associated with
    # those IPs will be used to create the RRDs.
    #mcast_join = 239.2.11.71
    port = 8649
    ttl = 1
    host = 10.0.2.15
    }
    udp_recv_channel {
    mcast_join = 239.2.11.71
    port = 8649
    bind = 239.2.11.71
    retry_bind = true
    # Size of the UDP buffer. If you are handling lots of metrics you really
    # should bump it up to e.g. 10MB or even higher.
    # buffer = 10485760
    }
    tcp_accept_channel {
    port = 8649
    # If you want to gzip XML output
    gzip_output = yes
    }
    1
    2
    3
    4
    5
    [root@10 ~]# vim /etc/ganglia/conf.php
    <?php
    $conf['gweb_confdir'] = "/mnt/data/ganglia";
    $conf['gmetad_root'] = "/mnt/data/ganglia";
    ?>
  5. 启动服务

    1
    2
    3
    [root@10 ~]# systemctl start gmetad
    [root@10 ~]# systemctl start gmond
    [root@10 ~]# systemctl start httpd

jenkins install

jenkins 安装

  1. 基础软件安装

    1
    2
    3
    [root@jenkins ~]# yum -y install epel-release
    [root@jenkins ~]# yum -y install gcc gcc-c++ make cmake bison libtool autoconf automake zip unzip bzip2 zlib zlib-devel openssl openssl-devel pcre pcre-devel bison-devel ncurses-devel tcl tcl-devel perl-Digest-SHA1 GeoIP GeoIP-devel gperftools gperftools-devel libatomic_ops-devel gtest gtest-devel glibc-devel unixODBC-devel fop libperl libpython readline readline-devel python2-pip readline readline-devel readline-static openssl openssl-devel openssl-static sqlite-devel bzip2-devel bzip2-libs openldap-devel curl-devel expat-devel gettext-devel perl-ExtUtils-MakeMaker
    [root@jenkins ~]# yum -y install git lftp ntpdate vim wget telnet dstat tree lrzsz net-tools nmap-ncat nmap sysstat
  2. 创建一个普通用户

    1
    [root@jenkins ~]# useradd -s /sbin/nologin jenkins
  3. java 安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@jenkins app]# tar xzf jdk-8u131-linux-x64.tar.gz
    [root@jenkins app]# mv jdk1.8.0_131 /mnt/app/java
    [root@jenkins app]# chown -R jenkins.jenkins /mnt/app/java

    [root@jenkins app]# echo 'export JAVA_HOME=/mnt/app/java'|tee /etc/profile.d/java.sh
    [root@jenkins app]# echo 'export JRE_HOME=${JAVA_HOME}/jre'|tee -a /etc/profile.d/java.sh
    [root@jenkins app]# echo 'export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib'|tee -a /etc/profile.d/java.sh
    [root@jenkins app]# echo 'export PATH=${JAVA_HOME}/bin:${PATH}'|tee -a /etc/profile.d/java.sh
    [root@jenkins app]# source /etc/profile

    [root@jenkins ~]# java -version
    java version "1.8.0_131"
    Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
    Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
  4. maven安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@jenkins app]# tar xzf apache-maven-3.5.0-bin.tar.gz
    [root@jenkins app]# mv apache-maven-3.5.0 /mnt/app/maven
    [root@jenkins app]# chown -R jenkins.jenkins /mnt/app/maven
    [root@jenkins app]# echo 'MAVEN_HOME=/mnt/app/maven' | tee -a /etc/profile.d/maven.sh
    [root@jenkins app]# echo 'export PATH=${PATH}:${MAVEN_HOME}/bin' | tee -a /etc/profile.d/maven.sh
    [root@jenkins app]# source /etc/profile

    [root@jenkins app]# mvn --version
    Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T03:39:06+08:00)
    Maven home: /mnt/app/maven
    Java version: 1.8.0_131, vendor: Oracle Corporation
    Java home: /mnt/app/java/jre
    Default locale: en_US, platform encoding: UTF-8
    OS name: "linux", version: "3.10.0-123.el7.x86_64", arch: "amd64", family: "unix"
  5. Tomcat 安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    [root@jenkins app]# tar xzf apache-tomcat-8.5.16.tar.gz
    [root@jenkins app]# mv apache-tomcat-8.5.16 /mnt/app/jenkins
    [root@jenkins app]# chown -R jenkins.jenkins /mnt/app/jenkins

    [root@jenkins ~]# mkdir /mnt/data/jenkins
    [root@jenkins ~]# chown -R jenkins.jenkins /mnt/data/jenkins
    [root@jenkins ~]# mkdir -p /mnt/log/jenkins
    [root@jenkins ~]# chown -R jenkins.jenkins /mnt/log/jenkins

    [root@jenkins ~]# vim /mnt/app/jenkins/conf/server.xml
    <Connector port="8080" protocol="HTTP/1.1"
    connectionTimeout="20000"
    redirectPort="8443" />

    <Host name="localhost" appBase="/mnt/data/jenkins"
    unpackWARs="true" autoDeploy="true">

    <Valve className="org.apache.catalina.valves.AccessLogValve" directory="/mnt/log/jenkins"
    prefix="jenkins_access_log" suffix=".txt"
    pattern="%h %l %u %t &quot;%r&quot; %s %b" />
    </Host>

    [root@jenkins ~]# vim /mnt/app/jenkins/conf/logging.properties
    1catalina.org.apache.juli.AsyncFileHandler.directory = /mnt/log/jenkins
    2localhost.org.apache.juli.AsyncFileHandler.directory = /mnt/log/jenkins
    3manager.org.apache.juli.AsyncFileHandler.directory = /mnt/log/jenkins
    4host-manager.org.apache.juli.AsyncFileHandler.directory = /mnt/log/jenkins

    [root@jenkins ~]# vim /mnt/app/jenkins/bin/catalina.sh
    JAVA_OPTS="-server -Xmx14g -Xms8g -Xmn1024m -Xss512k -XX:SurvivorRatio=6 -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"
  6. jenkins 安装

    1
    2
    [root@jenkins app]# cp jenkins.war /mnt/data/jenkins/
    [root@jenkins app]# chown -R jenkins.jenkins /mnt/data/jenkins
  7. jenkins启动脚本

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    [root@jenkins ~]# cat > /etc/init.d/jenkins <<EOF
    #!/bin/bash

    if [ $# != 1 ];then
    echo "Please input:
    /etc/init.d/$0 start
    /etc/init.d/$0 stop
    /etc/init.d/$0 status
    "
    exit 0
    fi

    STATUS=$1
    CATALINA="/mnt/app/jenkins/bin/catalina.sh "
    ROLE="jenkins"

    if [ $STATUS == "start" ];then
    $CATALINA start
    fi

    if [ $STATUS == "stop" ];then
    NUM=$(ps -ef|grep -w ${ROLE}|grep -w '/mnt/app/jenkins'|grep -v grep|awk '{print $2}'|wc -l)
    if [ $NUM != 1 ];then
    echo "Please check $ROLE, There are $NUM processes."
    fi

    PID=$(ps -ef|grep -w ${ROLE}|grep -w '/mnt/app/jenkins'|grep -v grep|awk '{print $2}')
    kill -9 ${PID}
    if [ $? == 0 ];then
    echo "$ROLE stoped success."
    else
    echo "$ROLE stoped fail."
    fi
    fi

    if [ $STATUS == "status" ];then
    NUM=$(ps -ef|grep -w ${ROLE}|grep -w '/mnt/app/jenkins'|grep -v grep|awk '{print $2}'|wc -l)

    if [ $NUM == 0 ];then
    echo "$ROLE is stopped."
    exit 0
    fi

    if [ $NUM != 1 ];then
    echo "Please check $ROLE, There are $NUM processes."
    fi

    PID=$(ps -ef|grep -w ${ROLE}|grep -w '/mnt/app/jenkins'|grep -v grep|awk '{print $2}')
    echo "$ROLE is running. PID: $PID"
    fi
    EOF
    [root@jenkins ~]# chmod +x /etc/init.d/jenkins
  8. jenkins systemd脚本

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    [root@jenkins ~]# cat > /usr/lib/systemd/system/jenkins.service <<EOF
    [Unit]
    Description=Apache Tomcat/8.5.16
    After=syslog.target network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/jenkins/bin/catalina.sh start
    ExecReload=kill -s HUP $MAINPID
    ExecStop=kill -s QUIT $MAINPID
    PrivateTmp=true
    User=jenkins
    Group=jenkins

    [Install]
    WantedBy=multi-user.target
    EOF
    [root@jenkins ~]# systemctl daemon-reload
    [root@jenkins ~]# systemctl start jenkins
    [root@jenkins ~]# systemctl enable jenkins
  9. jenkins 启动

    1
    2
    3
    4
    5
    6
    7
    [root@jenkins ~]# su - jenkins
    [jenkins@jenkins ~]$ /mnt/app/jenkins/bin/catalina.sh configtest
    [jenkins@jenkins ~]$ /mnt/app/jenkins/bin/catalina.sh start
    or:
    [jenkins@jenkins ~]$ /etc/init.d/jenkins start
    [jenkins@jenkins ~]$ /etc/init.d/jenkins stop
    [jenkins@jenkins ~]$ /etc/init.d/jenkins status
  10. jenkins nginx反向代理

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    [root@192 vhost]# cat jenkins.conf
    # jenkins
    upstream switchjenkins{
    server 192.168.13.214:8080 max_fails=3 fail_timeout=30s;
    }

    server {
    listen 80;
    server_name jenkins.domain.cn;

    location / {
    rewrite ^/ http://jenkins.domain.cn/jenkins;
    }

    location /jenkins {
    proxy_pass http://switchjenkins;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    access_log /mnt/log/nginx/jenkins.domain.cn.log main;
    }
  11. 访问jenkins,并初始化

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    //1.访问web界面
    http://jenkins.domain.cn
    //2.输入密码:
    [jenkins@jenkins ~]$ cat ~/.jenkins/secrets/initialAdminPassword
    357b1ccc008d4406ba3eca72a741aa7e
    //3.安装插件(等待安装)
    我们先选择推荐插件进行安装"Install suggested plugins"
    //4.创建管理员账号
    admin/admin123
    //5.启动jenkins

    //日常操作
    http://jenkins.domain.cn/jenkins/stop 关闭jenkins
    http://jenkins.domain.cn/jenkins/start 开启jenkins
    http://jenkins.domain.cn/jenkins/restart 重启jenkins
    http://jenkins.domain.cn/jenkins/reload 重新加载jenkins
  12. 安装常用插件

    1
    2
    * Deploy to Container Plugin   支持将代码部署到tomcat容器
    * GIT plugin

supersord install

  1. 安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@plat1 srv]# wget https://bootstrap.pypa.io/ez_setup.py -O - | python
    [root@plat1 srv]# easy_install supervisor

    or:
    [root@plat1 srv]# pip install supervisor
    [root@plat1 srv]# echo_supervisord_conf |tee /etc/supervisord.conf
    [root@plat1 srv]# cat >> /etc/supervisord.conf <<EOF
    [program:hello]
    command=python /mnt/web/tornado/hello.py --port=8888
    directory=/mnt/web/tornado/
    autorestart=true
    redirect_stderr=true
    EOF
    [root@dev tornado]# supervisord
  2. 使用

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    [root@plat1 srv]# cat /etc/supervisord.conf
    [program:openvpn]
    directory=/etc/openvpn
    command=/usr/local/openvpn/sbin/openvpn --config /etc/openvpn/server.conf
    process_name=%(program_name)s
    numprocs=1
    user=root
    autorstart=true
    autorestart=true
    stdout_logfile=/tmp/supervisor_openvpn.log
  3. supersord启动

    1
    2
    [root@plat1 srv]# /usr/bin/supervisord -c /etc/supervisord.conf
    [root@plat1 openvpn]# supervisorctl reload

openvpn install

  1. 基础服务安装

    1
    [root@plat1 srv]# yum -y install gcc gcc-c++
  2. 安装openssl

    1
    2
    3
    4
    5
    [root@plat1 srv]# ls -l openssl-1.0.1f.tar.gz
    -rw-r--r--. 1 root root 4509212 10月 13 17:48 openssl-1.0.1f.tar.gz
    [root@plat1 srv]# tar xzf openssl-1.0.1f.tar.gz
    [root@plat1 srv]# cd openssl-1.0.1f
    [root@plat1 openssl-1.0.1f]# ./config --prefix=/usr/local --openssldir=/usr/local/ssl && make && make install
  3. 安装lzo

    1
    2
    3
    4
    5
    6
    7
    [root@plat1 srv]# wget http://www.oberhumer.com/opensource/lzo/download/lzo-2.03.tar.gz
    [root@plat1 srv]# tar xzf lzo-2.03.tar.gz
    [root@plat1 srv]# cd lzo-2.03
    [root@plat1 lzo-2.03]# ./configure --prefix=/usr/local/lzo && make && make install

    [root@plat1 ~]# ln -s /usr/local/lzo/lib/liblzo2.a /usr/local/lzo/lib/liblzo.a
    [root@plat1 ~]# ln -s /usr/local/lzo/lib/liblzo2.la /usr/local/lzo/lib/liblzo.la
  4. 安装openvpn

    1
    2
    3
    4
    5
    6
    [root@plat1 srv]# ls -l openvpn-2.0.9.tar.gz
    -rw-r--r--. 1 root root 669076 10月 13 17:24 openvpn-2.0.9.tar.gz
    [root@plat1 srv]# tar xzf openvpn-2.0.9.tar.gz
    [root@plat1 srv]# cd openvpn-2.0.9
    [root@plat1 openvpn-2.0.9]# ./configure --prefix=/usr/local/openvpn --with-ssl-headers=/usr/local/ssl/include/openssl --with-ssl-lib=/usr/local/ssl/lib --with-lzo-headers=/usr/local/lzo/include/lzo --with-lzo-lib=/usr/local/lzo/lib
    [root@plat1 openvpn-2.0.9]# make && make install
  5. 准备openvpn-server端认证环境

    1
    2
    3
    4
    [root@plat1 openvpn-2.0.9]# mkdir /etc/openvpn
    [root@plat1 openvpn-2.0.9]# cp -r easy-rsa /etc/openvpn/
    [root@plat1 openvpn-2.0.9]# cp sample-config-files/server.conf /etc/openvpn/
    [root@plat1 openvpn-2.0.9]# cd /etc/openvpn/easy-rsa/2.0/
  6. 修改 认证 环境变量

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    [root@plat1 2.0]# vim vars
    export EASY_RSA="`pwd`"
    export OPENSSL="openssl"
    export PKCS11TOOL="pkcs11-tool"
    export GREP="grep"
    export KEY_CONFIG=`$EASY_RSA/whichopensslcnf $EASY_RSA`
    export KEY_DIR="$EASY_RSA/keys"
    echo NOTE: If you run ./clean-all, I will be doing a rm -rf on $KEY_DIR
    export KEY_SIZE=1024
    export CA_EXPIRE=3650
    export KEY_EXPIRE=3650
    export KEY_COUNTRY="CN"
    export KEY_PROVINCE="BJ"
    export KEY_CITY="BeiJing"
    export KEY_ORG="OpenVPN ORG"
    export KEY_EMAIL="smallasa@sina.com"
    [root@plat1 2.0]# . ./vars
  7. 清理key目录

    1
    [root@plat1 2.0]# ./clean-all
  8. 生成CA证书

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    [root@plat1 2.0]# ./build-ca
    Generating a 1024 bit RSA private key
    ............................++++++
    ......................................++++++
    writing new private key to 'ca.key'
    -----
    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [CN]:
    State or Province Name (full name) [BJ]:
    Locality Name (eg, city) [BeiJing]:
    Organization Name (eg, company) [OpenVPN ORG]:
    Organizational Unit Name (eg, section) []:
    Common Name (eg, your name or your server's hostname) [OpenVPN ORG CA]:
    Email Address [smallasa@sina.com]:

    [root@plat1 2.0]# ls keys/
    ca.crt ca.key index.txt serial

    [root@plat1 2.0]# ./build-dh
    Generating DH parameters, 1024 bit long safe prime, generator 2
    This is going to take a long time
    ....+.......+..........+.................................................+...+..................................................................................................................+..................................................................................................................................+........................................+......................................................................................................................+..................................+..............+.................................+..............................................................................................+..................................+.......................+............................++*++*++*

    [root@plat1 2.0]# ls keys/
    ca.crt ca.key dh1024.pem index.txt serial
  9. 生成openvpn server端key

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    [root@plat1 2.0]# ./build-key-server test
    Generating a 1024 bit RSA private key
    .....++++++
    ....................................++++++
    writing new private key to 'test.key'
    -----
    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [CN]:
    State or Province Name (full name) [BJ]:
    Locality Name (eg, city) [BeiJing]:
    Organization Name (eg, company) [OpenVPN ORG]:
    Organizational Unit Name (eg, section) []:
    Common Name (eg, your name or your server's hostname) [test]:
    Email Address [smallasa@sina.com]:
    Please enter the following 'extra' attributes
    to be sent with your certificate request
    A challenge password []:
    An optional company name []:
    Using configuration from /etc/openvpn/easy-rsa/2.0/openssl.cnf
    Check that the request matches the signature
    Signature ok
    The Subject's Distinguished Name is as follows
    countryName :PRINTABLE:'CN'
    stateOrProvinceName :PRINTABLE:'BJ'
    localityName :PRINTABLE:'BeiJing'
    organizationName :PRINTABLE:'OpenVPN ORG'
    commonName :PRINTABLE:'test'
    emailAddress :IA5STRING:'smallasa@sina.com'
    Certificate is to be certified until Oct 11 10:30:42 2026 GMT (3650 days)
    Sign the certificate? [y/n]:y
    1 out of 1 certificate requests certified, commit? [y/n]y
    Write out database with 1 new entries
    Data Base Updated
  10. 将 openvpn server端生成的key拷贝到/etc/openvpn/目录

    1
    2
    [root@plat1 2.0]# cd keys/
    [root@plat1 keys]# cp ca.crt ca.key test.crt test.key dh1024.pem /etc/openvpn/
  11. 生成 openvpn client key(根据用户不同可以按此方法生成相应的用户key)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    [root@plat1 2.0]# . ./vars
    [root@plat1 2.0]# ./build-key client
    Generating a 1024 bit RSA private key
    ......................++++++
    ...........................................................................++++++
    writing new private key to 'client.key'
    -----
    You are about to be asked to enter information that will be incorporated
    into your certificate request.
    What you are about to enter is what is called a Distinguished Name or a DN.
    There are quite a few fields but you can leave some blank
    For some fields there will be a default value,
    If you enter '.', the field will be left blank.
    -----
    Country Name (2 letter code) [CN]:
    State or Province Name (full name) [BJ]:
    Locality Name (eg, city) [BeiJing]:
    Organization Name (eg, company) [OpenVPN ORG]:
    Organizational Unit Name (eg, section) []:test
    Common Name (eg, your name or your server's hostname) [client]:client
    Email Address [smallasa@sina.com]:
    Please enter the following 'extra' attributes
    to be sent with your certificate request
    A challenge password []:
    An optional company name []:smallasa
    Using configuration from /etc/openvpn/easy-rsa/2.0/openssl.cnf
    Check that the request matches the signature
    Signature ok
    The Subject's Distinguished Name is as follows
    countryName :PRINTABLE:'CN'
    stateOrProvinceName :PRINTABLE:'BJ'
    localityName :PRINTABLE:'BeiJing'
    organizationName :PRINTABLE:'OpenVPN ORG'
    organizationalUnitName:PRINTABLE:'test'
    commonName :PRINTABLE:'client'
    emailAddress :IA5STRING:'smallasa@sina.com'
    Certificate is to be certified until Oct 12 06:10:37 2026 GMT (3650 days)
    Sign the certificate? [y/n]:y
    1 out of 1 certificate requests certified, commit? [y/n]y
    Write out database with 1 new entries
    Data Base Updated
  12. 将用户key 和 ca证书打包,拷贝给用户

    1
    2
    3
    [root@plat1 2.0]# cd keys
    [root@plat1 keys]# tar czf test.tar.gz test.* ca.*
    [root@plat1 keys]# sz test.tar.gz
  13. 配置openvpn server端服务配置

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    [root@plat1 keys]# cd /etc/openvpn/
    [root@plat1 openvpn]# mkdir /var/log/openvpn
    [root@plat1 openvpn]# vim server.conf
    local 10.10.16.3
    port 1194
    proto tcp
    dev tun
    ca ca.crt
    cert test.crt
    key test.key
    dh dh1024.pem
    server 10.8.0.0 255.255.255.0
    ifconfig-pool-persist ipp.txt
    push "route 10.10.16.0 255.255.255.0"
    push "route 192.168.14.0 255.255.255.0"
    client-to-client
    keepalive 10 120
    comp-lzo
    max-clients 30
    user nobody
    group nobody
    persist-key
    persist-tun
    status /var/log/openvpn/openvpn-status.log
    log /var/log/openvpn/openvpn.log
    verb 3
  14. 配置 openvpn server端防火墙和访问策略

    1
    2
    3
    [root@plat1 openvpn]# sed -i '/net.ipv4.ip_forward/s/0/1/g' /etc/sysctl.conf
    [root@plat1 openvpn]# sysctl -w net.ipv4.ip_forward=1
    [root@plat1 openvpn]# iptables -t nat -I POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE
  15. 启动openvpn server服务

    1
    [root@plat1 openvpn]# /usr/local/openvpn/sbin/openvpn --config /etc/openvpn/server.conf &
  16. linux系统下openvpn客户端安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    [ec2-user@ip-192-168-110-244 ~]$ sudo su -
    [root@ip-192-168-110-244 ~]# yum -y install openvpn
    [root@ip-192-168-110-244 ~]# cd /etc/openvpn/
    [root@ip-192-168-110-244 openvpn]# tar xzf test.tar.gz
    [root@ip-192-168-110-244 openvpn]# ls -l
    total 20
    -rw-r--r-- 1 root root 1245 Oct 13 18:27 ca.crt
    -rw------- 1 root root 916 Oct 13 18:27 ca.key
    -rw-r--r-- 1 root root 3815 Oct 14 14:10 client.crt
    -rw-r--r-- 1 root root 733 Oct 14 14:10 client.csr
    -rw------- 1 root root 916 Oct 14 14:10 client.key
  17. 修改openvpn client 配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@ip-192-168-110-244 openvpn]# cat client.conf
    client
    dev tun
    proto tcp
    remote 124.127.242.73 1194
    resolv-retry infinite
    nobind
    persist-key
    persist-tun
    ca ca.crt
    cert client.crt
    key client.key
    comp-lzo
    verb 3
  18. 启动openvpn Client服务

    1
    2
    [root@ip-192-168-110-244 ~]# cd /etc/openvpn/
    [root@ip-192-168-110-244 openvpn]# /usr/sbin/openvpn --config /etc/openvpn/client.conf
  19. 进行测试

  20. supervisor安装

    1
    2
    3
    [root@plat1 srv]# wget https://bootstrap.pypa.io/ez_setup.py -O - | python
    [root@plat1 srv]# easy_install supervisor
    [root@plat1 srv]# echo_supervisord_conf > /etc/supervisord.conf
  21. 使用supervisord支持openvpn

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    [root@plat1 srv]# cat /etc/supervisord.conf
    [program:openvpn]
    directory=/etc/openvpn
    command=/usr/local/openvpn/sbin/openvpn --config /etc/openvpn/server.conf
    process_name=%(program_name)s
    numprocs=1
    user=root
    autorstart=true
    autorestart=true
    stdout_logfile=/tmp/supervisor_openvpn.log
  22. supersord启动

    1
    2
    [root@plat1 srv]# /usr/bin/supervisord -c /etc/supervisord.conf
    [root@plat1 openvpn]# supervisorctl reload

git gogs source install

1.go env

1
2
3
4
5
6
[root@gitserver ~]# mkdir -p /mnt/app
[root@gitserver ~]# tar xzf go1.7.4.linux-amd64.tar.gz -C /mnt/app/
[root@gitserver ~]# echo 'export GOROOT=/mnt/app/go' |tee /etc/profile.d/go.sh
[root@gitserver ~]# echo 'export GOPATH=$HOME/work' |tee -a /etc/profile.d/go.sh
[root@gitserver ~]# echo 'export PATH=$GOROOT/bin:$PATH' |tee -a /etc/profile.d/go.sh
[root@gitserver ~]# source /etc/profile

2.gogs download

1
2
3
4
5
6
7
[root@localhost ~]# echo $GOPATH
/root/work

[root@gitserver ~]# go get -v -u github.com/gogits/gogs
[root@localhost ~]# ls ~/work/src/github.com/gogits/gogs/
appveyor.yml conf Dockerfile Dockerfile.rpi LICENSE models packager README.md routers templates
cmd docker Dockerfile.aarch64 gogs.go Makefile modules public README_ZH.md scripts vendor

3.gogs 编译安装

1
2
[root@localhost ~]# cd $GOPATH/src/github.com/gogits/gogs
[root@localhost gogs]# go build

4.gogs 启动

1
2
3
4
5
6
7
8
9
10
[root@localhost gogs]# ./gogs web
2017/03/17 17:49:39 [ WARN] Custom config '/root/work/src/github.com/gogits/gogs/custom/conf/app.ini' not found, ignore this if you're running first time
2017/03/17 17:49:39 [TRACE] Custom path: /root/work/src/github.com/gogits/gogs/custom
2017/03/17 17:49:39 [TRACE] Log path: /root/work/src/github.com/gogits/gogs/log
2017/03/17 17:49:39 [TRACE] Log Mode: Console (Trace)
2017/03/17 17:49:39 [ INFO] Gogs 0.10.18.0313
2017/03/17 17:49:39 [ INFO] Cache Service Enabled
2017/03/17 17:49:39 [ INFO] Session Service Enabled
2017/03/17 17:49:39 [ INFO] Run Mode: Development
2017/03/17 17:49:40 [ INFO] Listen: http://0.0.0.0:3000

Go默认不支持一些功能,如果需要支持,需要使用build tag来支持

1
2
3
[root@gitserver ~]# go get -u -tags "sqlite pam cert" github.com/gogits/gogs
[root@gitserver ~]# cd $GOPATH/src/github.com/gogits/gogs
[root@localhost gogs]# go build

kafka offset monitor

kafka offset monitor

  1. 下载

    1
    Download: https://github.com/quantifind/KafkaOffsetMonitor
  2. 安装

    1
    2
    3
    4
    5
    [root@10 ~]# mkdir /mnt/app/kafkaoffsetmonitor
    [root@10 ~]# ls -l /mnt/app/kafkaoffsetmonitor/
    total 51700
    -rw-r--r-- 1 root root 52937155 Mar 17 17:09 KafkaOffsetMonitor-assembly-0.2.1.jar
    [root@10 ~]# chown -R wisdom.wisdom /mnt/app/kafkaoffsetmonitor
  3. 启动

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    [root@10 ~]# su - wisdom
    [wisdom@10 ~]$ vim /mnt/app/kafkaoffsetmonitor/kafkaoffsetmonitor.sh
    java -Xms512M -Xmx512M -Xss1024K -XX:PermSize=256m -XX:MaxPermSize=512m \
    -cp KafkaOffsetMonitor-assembly-0.2.1.jar \
    com.quantifind.kafka.offsetapp.OffsetGetterWeb \
    --zk 10.0.3.65:2181 \
    --port 8888 \
    --refresh 10.seconds \
    --retain 2.days &
    [wisdom@10 ~]$ chmod +x /mnt/app/kafkaoffsetmonitor/kafkaoffsetmonitor.sh

    [wisdom@10 ~]$ cd /mnt/app/kafkaoffsetmonitor/
    [wisdom@10 kafkaoffsetmonitor]$ ./kafkaoffsetmonitor.sh
  4. 访问

    1
    http://{IP}:8888

kafka manage web

kafka manage web

  1. kafka manage install

    1
    2
    3
    4
    5
    [root@10 app]# git clone https://github.com/yahoo/kafka-manager.git
    [root@10 app]# cd kafka-manager/
    [root@10 kafka-manager]# ./sbt /mnt/app/java clean dist
    or:
    [root@10 kafka-manager]# JAVA_HOME=/mnt/app/java PATH=${JAVA_HOME}/bin:$PATH ./sbt -java-home /mnt/app/java clean dist
  2. 解压安装

    1
    2
    3
    4
    [root@10 kafka-manager]# cp target/universal/kafka-manager-1.3.3.4.zip .
    [root@10 kafka-manager]# unzip kafka-manager-1.3.3.4.zip
    [root@10 kafka-manager]# mv kafka-manager-1.3.3.4 /mnt/app/kafka-manager
    [root@10 kafka-manager]# chown -R wisdom.wisdom /mnt/app/kafka-manager
  3. 配置

    1
    2
    3
    4
    5
    6
    7
    8
    [root@10 app]# su - wisdom
    [wisdom@10 ~]$ vim /mnt/app/kafka-manager/conf/application.conf
    kafka-manager.zkhosts="10.0.3.65:2181"
    ... ...
    basicAuthentication.enabled=true
    basicAuthentication.username="admin"
    basicAuthentication.password="admin"
    basicAuthentication.realm="Kafka-Manager"
  4. 启动

    1
    [wisdom@10 ~]$ /mnt/app/kafka-manager/bin/kafka-manager &
  5. 访问

    1
    http://{IP}:9000

zabbix windows agent

zabbix windows agent 安装和配置

  1. 安装

    1
    在官网下载windows zabbix agent客户端,解压到C盘(C:\zabbix),并在此目录下创建两个目录log(C:\zabbix\log)和conf.d(C:\zabbix\conf.d)
  2. 修改配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    C:\zabbix\conf\zabbix_agentd.win.conf

    LogFile=C:\zabbix\log\zabbix_agentd.log
    LogFileSize=20
    DebugLevel=3
    SourceIP={server_agent_ip}
    EnableRemoteCommands=1
    LogRemoteCommands=1
    Server={server_ip}
    ListenPort=10050
    StartAgents=3
    ServerActive={server_ip}:10051
    Hostname={server_agent_ip}
    HostMetadata=windows IIS SQLServer
    HostMetadataItem=system.uname
    RefreshActiveChecks=120
    BufferSend=5
    BufferSize=200
    MaxLinesPerSecond=100
    Timeout=10
    Include=C:\zabbix\conf.d/*.conf
    UnsafeUserParameters=1
  3. 启动agent

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    Ctrl+C 输入 cmd,进入dos终端:

    //进入zabbix 执行目录
    cd C:\zabbix\bin\win64\

    //将服务安装到windows服务里面
    zabbix_agentd.exe -c C:\zabbix\conf\zabbix_agentd.win.conf -i
    通过services.msc命令,找到zabbix,进行启动

    //启动服务
    zabbix_agentd.exe -c C:\zabbix\conf\zabbix_agentd.win.conf -s

    //停止服务
    zabbix_agentd.exe -c C:\zabbix\conf\zabbix_agentd.win.conf -x

    //卸载客户端
    zabbix_agentd.exe -c C:\zabbix\conf\zabbix_agentd.win.conf -d

nginx location

user nginx;
worker_processes auto;
worker_rlimit_nofile 65535;

events {
use epoll;
worker_connections 10240;
}

virtual host

http {
log_format main ‘$remote_addr $remote_user [$time_local] “$request” $http_host ‘
‘[$status] [$upstream_status] [$body_bytes_sent] “$http_referer” ‘
‘“$http_user_agent” $ssl_protocol $ssl_cipher $upstream_addr ‘
‘[$request_time] [$upstream_response_time]’;
access_log /dev/stdout main;

upstream haha{
server 192.168.99.225:80 max_fails=2 fail_timeout=30s;
}

upstream hehe{
server 192.168.99.232:80 max_fails=2 fail_timeout=30s;
}

server {
listen 80;
server_name a.com;

location / {
  root   html;
        index  index.html index.htm;
}

location /nginx_status {
  stub_status on;
  access_log off;
}

location  ^/(haha) {
  proxy_pass http://haha/haha.html;
  proxy_set_header   Host    $host;  
  proxy_set_header   X-Real-IP   $remote_addr;   
  proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;  
}

location ^~ /hehe {
  proxy_pass http://hehe/hehe.html;
  proxy_set_header   Host    $host;
  proxy_set_header   X-Real-IP   $remote_addr;
  proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
}


access_log  /mnt/log/nginx/main_access.log  main;

}
}

mysql command

  1. mysql创建数据库(utf8)

    1
    mysql> create database {dbname} default character set utf8 collate utf8_general_ci;
  2. 数据库授权

    1
    2
    mysql> grant all privileges on *.* to '{user}@'%' identified by '{password}' with grant option;
    mysql> flush privileges;
  3. 实时查看参数

    1
    [root@10 ~]# mysqladmin -uroot -S /mnt/data/mysql/3306/mysql.sock extended-status -r -i 1 |egrep 'Com_(select|insert|update|delete)'
  4. 数据库查看超时时间

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    mysql> show global variables like "%timeout%";
    +-----------------------------+----------+
    | Variable_name | Value |
    +-----------------------------+----------+
    | connect_timeout | 10 |
    | delayed_insert_timeout | 300 |
    | innodb_flush_log_at_timeout | 1 |
    | innodb_lock_wait_timeout | 50 |
    | innodb_rollback_on_timeout | OFF |
    | interactive_timeout | 300 |
    | lock_wait_timeout | 31536000 |
    | net_read_timeout | 30 |
    | net_write_timeout | 60 |
    | rpl_stop_slave_timeout | 31536000 |
    | slave_net_timeout | 3600 |
    | wait_timeout | 120 |
    +-----------------------------+----------+