automation►salt►salt manage module

salt manage module

salt 针对管理对象操作

  1. salt module组件

    1
    Module是我们日常使用SaltStack接触最多的一个组件,它是SaltStack通过Push的方式进行管理的入口.比如:我们日常简单的执行命令,查看包安装情况,查看服务运行情况等工作都是通过SaltStack Module来实现的

  2. 查看modules

    1
    2
    3
    [root@salt-master base]# salt 'salt-minion-187' sys.list_modules
    [root@salt-master base]# salt 'salt-minion-187' sys.list_functions sys
    [root@salt-master base]# salt 'salt-minion-187' sys.doc sys

  3. 执行modules

    1
    2
    [root@salt-master base]# salt 'salt-minion-187' test.echo
    [root@salt-master base]# salt 'salt-minion-187' test.echo,cmd.run

  4. 查看file_roots位置

    1
    2
    3
    4
    [root@salt-master ~]# vim /etc/salt/master
    file_roots:
      base:
        - /mnt/data/salt.repo/salt/base

  5. 创建_modules目录

    1
    2
    3
    4
    5
    6
    7
    8
    [root@salt-master ~]# mkdir -p /mnt/data/salt.repo/salt/base/_modules
    def world():
        """
        This is my first function.
        CLI Example:
          salt '*' hello.world
        """
        return 'Hello, world!'

  6. 将模块推送到minion端

    1
    2
    3
    [root@salt-master ~]# salt 'salt-minion-187' saltutil.sync_modules
    salt-minion-187:
        - modules.hello

  7. 查看minion端的模块

    1
    2
    [root@salt-master ~]# salt 'salt-minion-187' sys.list_modules|grep hello
        - hello

  8. 执行minion端的模块

    1
    2
    3
    [root@salt-master ~]# salt 'salt-minion-187' hello.world
    salt-minion-187:
        Hello, world!

  9. 特别说明

    1
    我们可以通过"__salt__","__grains__"和"__pillar__"函数调用所有其他执行模块,就像使用salt命令一样简单
automation►salt►salt manage pillar

salt manage pillar

salt 数据管理中心

  1. salt pillar组件

    1
    Pillar是SaltStack组件中非常重要的组件之一,它在SaltStack中主要的作用就是存储和定义配置管理中需要的一些数据,比如:软件版本号,用户名密码等信息.

  2. 查看pillar位置

    1
    2
    3
    4
    5
    6
    7
    8
    [root@salt-master]# cat /etc/salt/master
    pillar_roots:
      base:
        - /mnt/data/salt.repo/pillar/base
      dev:
        - /mnt/data/salt.repo/pillar/dev
      prod:
        - /mnt/data/salt.repo/pillar/prod

  3. 在pillar工作目录创建top.sls文件并引用两个sls文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    [root@salt-master]# vim /mnt/data/salt.repo/pillar/base/top.sls
    base:
      '*':
        - packages
        - services

    [root@salt-master]# vim /mnt/data/salt.repo/pillar/base/packages.sls
    zabbix:
      package-name: zabbix
      version: 2.2.4

    [root@salt-master]# vim /mnt/data/salt.repo/pillar/base/services.sls
    zabbix:
      port: 10050
      user: zabbix

  4. 使用refresh_pillar命令进行刷新,只能放在master端

    1
    [root@salt-master]# salt 'salt-minion-187' saltutil.refresh_pillar

  5. 通过pillar.items获取相关信息

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    [root@salt-master]# salt 'salt-minion-187' pillar.item zabbix
    salt-minion-187:
        ----------
        zabbix:
            ----------
            package-name:
                zabbix
            port:
                10050
            user:
                zabbix
            version:
                2.2.4

    [root@salt-master]# salt 'salt-minion-187' pillar.item zabbix:package-name
    salt-minion-187:
        ----------
        zabbix:package-name:
            zabbix

  6. 查看相关grains命令和帮助

    1
    2
    salt minion sys.list_functions pillar
    salt minion sys.doc pillar

  7. 根据操作系统版本进行包安装

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    [root@salt-master]# vim /mnt/data/salt.repo/pillar/base/top.sls
    base:
      '*':
        - packages

    [root@salt-master]# vim /mnt/data/salt.repo/pillar/base/packages.sls
    pkgs:
      {% if grains['os'] == 'CentOS' %}
      apache: httpd
      git: git
      {% elif grains['os'] == 'SUSE' %}
      apache: apache2
      git: git-core
      {% endif %}

    [root@salt-master]# salt 'salt-minion-187' saltutil.refresh_pillar

    [root@salt-master]# salt 'salt-minion-187' pillar.item pkgs
    salt-minion-187:
        ----------
        pkgs:
            ----------
            apache:
                httpd
            git:
                git

  8. state组件里调用pillar

    1
    2
    3
    4
    5
    6
    7
    8
    9
    apache:
      pkg.installed:
        - name: {{ pillar['pkgs']['apache'] }}

    或者:

    apache:
      pkg.installed:
        - name: {{ salt['pillar.get']('pkgs:apache', 'httpd') }}

  9. 敏感数据加密

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    */pillar/base/database.sls:
    dbname: project
    dbuser: username
    dbpass: password
    dbhost: localhost

    # cat website.conf
    // MySQL settings
    define('DB_NAME', '{{ pillar['dbname'] }}');
    // MySQL database username
    define('DB_USER', '{{ pillar['dbuser'] }}');
    // MySQL database password
    define('DB_PASSWORD', '{{ pillar['dbpass'] }}');
    // MySQL hostname
    define('DB_HOST', '{{ pillar['dbhost'] }}');
automation►salt►salt manage grains

salt manage grains

salt 管理对象属性

  1. salt grains组件

    1
    2
    3
    Grains是SaltStack组件中非常重要的组件之一,它是SaltStack记录Minion的一些静态信息的组件,例如:CPU,内存,磁盘,网络信息等.
    Minions的Grains信息是Minions启动的时候采集汇报给Master的,在实际应用环境中我们需要根据自己的业务需求去自定义一些Grains.
    Grains存储的是静态信息,静态信息,静态信息!!!

  2. 查看minion所有grains信息

    1
    salt minion grains.items

  3. 自定义grains方法

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    方法一: 通过Minion配置文件定义
    1.修改配置文件/etc/salt/minion.d/grains:
    grains:
      roles:
        - webserver
        - memcache
      deployment: datacenter4
      cabinet: 13
      cab_u: 14-15
    2.重启salt-minion
    3.salt-master验证
      salt 'minion' grains.item roles

    方法二: 通过Grains相关模块定义
    1.定义grains
      salt 'minion' grains.append hosttype 'online'
      salt 'minion' grains.item hosttype
      或:
      salt 'minion' grains.setvals "{'idc':'Z1','city':'BJ'}"
      salt 'minion' grains.item idc

    方法三: 通过Python脚本定义
    1.查看file_roots
    [root@salt-master ~]# cat /etc/salt/master
    file_roots:
      base:
        - /mnt/data/salt.repo/salt/base
    2.创建_grains目录
    [root@salt-master ~]# mkdir /mnt/data/salt.repo/salt/base/_grains
    3.创建python脚本
    [root@salt-master ~]# cat /mnt/data/salt.repo/salt/base/_grains/get_time.py
    #!/usr/bin/env python
    # -*- coding:utf-8 -*-
    from datetime import datetime
    def get_server_time():
        grains = {}
        grains['server_time'] = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
        return grains
    4.使用sync_grains命令同步脚本到minion主机
    [root@salt-master ~]# salt 'salt-minion-187' saltutil.sync_grains
    salt-minion-187:
        - grains.get_time
    5.通过grains.item获取相关信息
    [root@salt-master ~]# salt 'salt-minion-187' grains.item server_time
    salt-minion-187:
        ----------
        server_time:
            2017-11-14 17:05:33

  4. grains优先级

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    grains配置优先级:
    minion端自定义grains配置(/etc/salt/minion.d/grains.conf或/etc/salt/minion) >
    master端自定义grains脚本(minion端/var/cache/salt/minion/extmods/grains目录下) >
    master端Grains模块定义(minion端/etc/salt/grains文件)

    注意: 当同步自定义grains脚本时,最好如下操作
    [root@salt-master ~]# mv /mnt/data/salt.repo/salt/base/{_grains,_grains.bak}
    [root@salt-master ~]# salt 'salt-minion-187' saltutil.sync_grains
    [root@salt-master ~]# mv /mnt/data/salt.repo/salt/base/{_grains.bak,_grains}
    [root@salt-master base]# salt 'salt-minion-187' saltutil.sync_grains

  5. 查看相关grains命令和帮助

    1
    2
    salt minion sys.list_functions grains
    salt minion sys.doc grains
automation►salt►salt manage target

salt manage target

salt 管理对象

  1. 管理对象方法
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    1.正则匹配
      salt -E 'min*' test.ping

    2.列表匹配
      salt -L minion1,minion2 test.ping

    3.Grians匹配(Minion的Grains的键值对)
      salt -G 'os:centos' test.ping

    4.组匹配(groups是在master配置文件中定义的组名称)
      salt -N groups test.ping

    5.复合匹配
      salt -C 'G@os:centos or L@Minion1'

    6.Pillar值匹配(key:value是Pillar系统中定义的一组键值,类似Grians键值对)
      salt -I 'key:value' test.ping

    7.CIDR匹配
      salt -S '192.168.1.0/24' test.ping

    8.Target参数
      L => L@minion1,minion2
      G => G@os:centos
      E => E@minion[1-3]
      P => P@os:(centos|redhat|ubuntu)
      I => I@key:value
      S => S@10.10.1.0/24 or S@10.10.2.10
      R => R@%foo.bar
      C => G@os:centos or L@minion1,minon3
database►Mongo►mongo replication auth install

mongo replication auth install

mongo replication auth install

  1. 创建用户

    1
    [root@mongodb app]# useradd -s /sbin/nologin mongo

  2. 解压和安装

    1
    2
    3
    [root@mongodb app]# tar xzf mongodb-linux-x86_64-rhel70-3.4.10.tgz
    [root@mongodb app]# mv mongodb-linux-x86_64-rhel70-3.4.10 /mnt/app/mongo
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo

  3. 设置环境变量

    1
    2
    3
    4
    5
    [root@mongodb app]# cat >/etc/profile.d/mongo.sh <<EOF
    export MONGO_HOME=/mnt/app/mongo
    export PATH=\${MONGO_HOME}/bin:\$PATH
    EOF
    [root@mongodb app]# source /etc/profile

  4. 创建mongo配置文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/app/mongo/conf
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/conf

  5. 创建mongo存放keyfile目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/app/mongo/key
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/key

  6. 创建mongo数据文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/data/mongo/{27017,27018,27019}
    [root@mongodb app]# chown -R mongo.mongo /mnt/data/mongo

  7. 创建mongo日志目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/log/mongo/{27017,27018,27019}
    [root@mongodb app]# chown -R mongo.mongo /mnt/log/mongo

  8. 生成mongo.key

    1
    2
    3
    4
    5
    生成mongo.key加密文件(注意:mongo.key里面密码的长度不能超过1024),并将mongo.key拷贝到副本集中其它几台机器上

    [root@mongodb ~]# openssl rand -base64 741 > /mnt/app/mongo/key/mongo.key                         
    [root@mongodb ~]# chmod 600 /mnt/app/mongo/key/mongo.key
    [root@mongodb ~]# chown -R mongo.mongo /mnt/app/mongo/key

  9. 设置mongo配置文件(primary)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27017.conf <<EOF
    dbpath=/mnt/data/mongo/27017
    logpath=/mnt/log/mongo/27017/mongo.log  
    logappend=true  
    port=27017  
    fork=true  
    nohttpinterface=true
    objcheck=true
    rest=false
    #auth=true  

    journal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset

    keyFile=/mnt/app/mongo/key/mongo.key
    EOF

  10. 设置mongo systemd文件(primary)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27017.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27017.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27017.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# systemctl daemon-reload
    [root@mongodb app]# systemctl start mongo-27017
    [root@mongodb app]# systemctl enable mongo-27017

  11. 副本集初始化(primary)

    1
    2
    3
    4
    [root@mongodb ~]# mongo --port 27017
    > config={_id: "dbset",version: 1,members: [{ _id: 0, host : "192.168.13.218:27017" }]}
    > rs.initiate(config)
    dbset:PRIMARY>

  12. 设置用户权限(primary,(auth=false))

    1
    2
    3
    4
    5
    6
    dbset:PRIMARY> use admin
    dbset:PRIMARY> db.createUser({user: "root",pwd:"root",roles:[{role:"root",db:"admin" }]})
    dbset:PRIMARY> db.auth('root','root')
    dbset:PRIMARY> db.createUser({user: "admin",pwd:"admin",roles:[{role:"userAdminAnyDatabase",db:"admin" }]})
    dbset:PRIMARY> exit
    bye

  13. 重启mongo(primary,(auth=true))

    1
    2
    3
    4
    5
    [root@mongodb ~]# vim /mnt/app/mongo/conf/mongo.27017.conf
    #auth=true
    改为:
    auth=true
    [root@mongodb ~]# systemctl status mongo-27017

  14. 验证(primary)

    1
    2
    3
    dbset:PRIMARY> use admin
    dbset:PRIMARY> db.auth('root','root')
    dbset:PRIMARY> rs.status()

  15. 配置文件设置

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    //secondary配置文件
    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27018.conf <<EOF
    dbpath=/mnt/data/mongo/27018
    logpath=/mnt/log/mongo/27018/mongo.log  
    logappend=true  
    port=27018  
    fork=true  
    nohttpinterface=true
    objcheck=true
    rest=false
    auth=true  

    journal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset

    keyFile=/mnt/app/mongo/key/mongo.key
    EOF

    //arbiter配置文件
    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27019.conf <<EOF
    dbpath=/mnt/data/mongo/27019
    logpath=/mnt/log/mongo/27019/mongo.log  
    logappend=true  
    port=27019  
    fork=true  
    nohttpinterface=true
    objcheck=true
    rest=false
    auth=true  

    nojournal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset

    keyFile=/mnt/app/mongo/key/mongo.key
    EOF

    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/conf

  16. 设置mongo systemd文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27018.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27018.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27018.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27019.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27019.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27019.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# systemctl daemon-reload
    [root@mongodb app]# systemctl start mongo-27018
    [root@mongodb app]# systemctl start mongo-27019
    [root@mongodb app]# systemctl enable monog-27018
    [root@mongodb app]# systemctl enable monog-27019

  17. 副本集添加节点

    1
    2
    3
    4
    5
    6
    7
    8
    [root@mongodb ~]# mongo --port 27017
    dbset:PRIMARY> use admin
    dbset:PRIMARY> db.auth('root','root')
    dbset:PRIMARY> rs.status()

    dbset:PRIMARY> rs.add("192.168.13.218:27018")
    dbset:PRIMARY> rs.addArb("192.168.13.218:27019")
    dbset:PRIMARY> rs.status()

  18. 副本集状态

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    dbset:PRIMARY> rs.status()
    {
            "set" : "dbset",
            "date" : ISODate("2017-11-13T09:12:44.740Z"),
            "myState" : 1,
            "term" : NumberLong(1),
            "heartbeatIntervalMillis" : NumberLong(2000),
            "optimes" : {
                    "lastCommittedOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    },
                    "appliedOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    },
                    "durableOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    }
            },
            "members" : [
                    {
                            "_id" : 0,
                            "name" : "192.168.13.218:27017",
                            "health" : 1,
                            "state" : 1,
                            "stateStr" : "PRIMARY",
                            "uptime" : 641,
                            "optime" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2017-11-13T09:12:37Z"),
                            "electionTime" : Timestamp(1510564115, 2),
                            "electionDate" : ISODate("2017-11-13T09:08:35Z"),
                            "configVersion" : 3,
                            "self" : true
                    },
                    {
                            "_id" : 1,
                            "name" : "192.168.13.218:27018",
                            "health" : 1,
                            "state" : 2,
                            "stateStr" : "SECONDARY",
                            "uptime" : 122,
                            "optime" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDurable" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2017-11-13T09:12:37Z"),
                            "optimeDurableDate" : ISODate("2017-11-13T09:12:37Z"),
                            "lastHeartbeat" : ISODate("2017-11-13T09:12:43.815Z"),
                            "lastHeartbeatRecv" : ISODate("2017-11-13T09:12:43.814Z"),
                            "pingMs" : NumberLong(0),
                            "syncingTo" : "192.168.13.218:27017",
                            "configVersion" : 3
                    },
                    {
                            "_id" : 2,
                            "name" : "192.168.13.218:27019",
                            "health" : 1,
                            "state" : 7,
                            "stateStr" : "ARBITER",
                            "uptime" : 58,
                            "lastHeartbeat" : ISODate("2017-11-13T09:12:43.814Z"),
                            "lastHeartbeatRecv" : ISODate("2017-11-13T09:12:40.893Z"),
                            "pingMs" : NumberLong(0),
                            "configVersion" : 3
                    }
            ],
            "ok" : 1
    }

  19. 副本集常用命令

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    //primary secondary切换
    dbset:PRIMARY> rs.stepDown()

    //副本集中移除
    dbset:PRIMARY> rs.remove("192.168.13.218:27019")

    //副本集配置
    dbset:PRIMARY> rs.conf()

    //查看副本同步状态
    dbset:PRIMARY> db.printSlaveReplicationInfo();

  20. mongo内核优化

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    [root@mongodb ~]# systemctl stop mongo && systemctl start mongo

    [root@mongodb ~]# cat >>/etc/rc.local <<EOF

    ### mongodb close hugepage
    if test -f /sys/kernel/mm/transparent_hugepage/enabled;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    fi
    if test -f /sys/kernel/mm/transparent_hugepage/defrag;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    fi
    EOF
database►Mongo►mongo error

mongo error

mongo error

  1. mongo 推荐使用XFS文件系统
    1
    2
    3
    4
    在Linux环境下,MongoDB推荐使用 XFS 文件系统,如果MongoDB数据存放的文件系统不是XFS,会出现如下提示:
    2017-04-17T13:37:40.045+0800 I STORAGE [initandlisten]
    2017-04-17T13:37:40.045+0800 I STORAGE [initandlisten] ** WARNING: Using the XF S filesystem is strongly recommended with the WiredTiger storage engine
    2017-04-17T13:37:40.045+0800 I STORAGE [initandlisten] ** See http://d ochub.mongodb.org/core/prodnotes-filesystem
database►Mongo►mongo auth

mongo auth

mongo auth

  1. mongo 权限说明

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    MongoDB数据库在默认是没有用户名及密码,不用安全验证的,只要连接上服务就可以进行CRUD操作.

    概念理解:
    1.用户概念
      Mongodb的用户是由"用户名+所属库名"组成
      注意:帐号是跟着库走的,所以在指定库里授权,必须也在指定库里验证(auth)
      例如:
      登录mongo库test1,创建用户user,创建的用户为:user@test1
      登录mongo库test2,创建用户user,创建的用户为:user@test2

    2.角色概念
      Mongodb的授权采用了角色授权的方法,每个角色包括一组权限
      Mongodb已经定义好了的角色叫内建角色,我们也可以自定义角色

      内建角色:
      * 数据库用户角色: read readWrite
      * 数据库管理角色: dbAdmin dbOwner userAdmin
      * 集群管理角色: clusterAdmin clusterManager clusterMonitor hostManager
      * 备份恢复角色: backup restore
      * 所有数据库角色: readAnyDatabase readWriteAnyDatabase userAdminAnyDatabase dbAdminAnyDatabase
      * 超级用户角色: root  
      * 内部角色: __system

      角色权限说明:
      * Read 允许用户读取指定数据库
      * readWrite 允许用户读写指定数据库
      * dbAdmin 允许用户在指定数据库中执行管理函数,如索引创建,删除,查看统计或访问system.profile
      * userAdmin 允许用户向system.users集合写入,可以找指定数据库里创建,删除和管理用户
      * clusterAdmin 只在admin数据库中可用,赋予用户所有分片和复制集相关函数的管理权限
      * readAnyDatabase 只在admin数据库中可用,赋予用户所有数据库的读权限
      * readWriteAnyDatabase 只在admin数据库中可用,赋予用户所有数据库的读写权限
      * userAdminAnyDatabase 只在admin数据库中可用,赋予用户所有数据库的userAdmin权限
      * dbAdminAnyDatabase 只在admin数据库中可用,赋予用户所有数据库的dbAdmin权限
      * root 只在admin数据库中可用.超级账号,超级权限

  2. 创建超级权限用户(auth=false)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    [root@mongodb ~]# mongo
    > use admin
    > db.createUser(
       {
         user: "root",
         pwd: "root",
         roles: [ { role: "root", db: "admin" } ]
       }
     )
    > db.auth('root','root')
    > db.createUser(
       {
         user: "admin",
         pwd: "admin",
         roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
       }
     )

  3. 重启mongo(auth=true)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    [root@mongodb ~]# mongo
    > use admin
    > db.auth("root", "rootroot" )
    1
    > show dbs
    admin  0.000GB
    local  0.000GB
    > exit
    bye

    或者:
    [root@mongodb ~]# mongo --port 27017 -u "root" -p "rootroot" --authenticationDatabase "admin"
    > show dbs
    admin  0.000GB
    local  0.000GB
    > exit
    bye

  4. 用户授权过程

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    //创建用户并授权
    创建用户条件: 首先选择在哪个数据库下进行创建;其次创建用户有三项需要提供:用户名,密码,角色列表

    1.先切换到超级权限用户
    > use admin
    > db.auth('root','rootroot')

    2.创建用户
    > use penn
    > db.createUser({user:"test1",pwd:"test2",roles:[{role:"readWrite",db:"penn"},{role:"userAdmin",db:"penn"}]})

    3.查看用户
    > use penn
    > show users;

    4.修改密码
    > use penn
    > db.changeUserPassword('test1','test1');

    5.权限验证
    > use penn
    > db.auth('test1','test1')
    > show tables;

    6.添加角色权限
    > use penn
    > db.grantRolesToUser("test1",[{role:"read",db:"admin"}])

    7.回收角色权限
    > use penn
    > db.revokeRolesFromUser("test1",[{role:"read",db:"admin"}])

    8.删除用户
    > use penn
    > db.dropUser("test1")

    9.查看所有账号
    > use admin
    > db.auth('root','rootroot')
    > db.system.users.find().pretty()
    > db.system.users.find().count()
database►Mongo►mongo standard install

mongo standard install

mongo standard install

  1. 创建用户

    1
    [root@mongodb app]# useradd -s /sbin/nologin mongo

  2. 解压和安装

    1
    2
    3
    [root@mongodb app]# tar xzf mongodb-linux-x86_64-rhel70-3.4.10.tgz
    [root@mongodb app]# mv mongodb-linux-x86_64-rhel70-3.4.10 /mnt/app/mongo
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo

  3. 设置环境变量

    1
    2
    3
    4
    5
    [root@mongodb app]# cat >/etc/profile.d/mongo.sh <<EOF
    export MONGO_HOME=/mnt/app/mongo
    export PATH=\${MONGO_HOME}/bin:\$PATH
    EOF
    [root@mongodb app]# source /etc/profile

  4. 创建mongo配置文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/app/mongo/conf
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/conf

  5. 创建mongo数据文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/data/mongo
    [root@mongodb app]# chown -R mongo.mongo /mnt/data/mongo

  6. 创建mongo日志目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/log/mongo
    [root@mongodb app]# chown -R mongo.mongo /mnt/log/mongo

  7. 设置mongo配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.conf <<EOF
    dbpath=/mnt/data/mongo
    logpath=/mnt/log/mongo/mongo.log  
    logappend=true  
    port=27017  
    fork=true  
    nohttpinterface=true  
    #auth=true  
    EOF

  8. 设置mongo systemd文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF
    [root@mongodb app]# systemctl daemon-reload

  9. 启动mongo

    1
    2
    [root@mongodb app]# systemctl start mongo
    [root@mongodb app]# systemctl enable mongo

  10. 验证

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    [root@mongodb ~]# mongo
    MongoDB shell version v3.4.10
    connecting to: mongodb://127.0.0.1:27017
    MongoDB server version: 3.4.10
    Server has startup warnings:
    2017-11-13T15:13:35.914+0800 I CONTROL  [initandlisten]
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten]
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten]
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten]
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
    2017-11-13T15:13:35.915+0800 I CONTROL  [initandlisten]
    > show dbs
    admin  0.000GB
    local  0.000GB
    > exit
    bye

  11. mongo内核优化

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    [root@mongodb ~]# systemctl stop mongo && systemctl start mongo

    [root@mongodb ~]# cat >>/etc/rc.local <<EOF

    ### mongodb close hugepage
    if test -f /sys/kernel/mm/transparent_hugepage/enabled;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    fi
    if test -f /sys/kernel/mm/transparent_hugepage/defrag;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    fi
    EOF
database►Mongo►mongo replication standard install

mongo replication standard install

mongo replication standard install

  1. 创建用户

    1
    [root@mongodb app]# useradd -s /sbin/nologin mongo

  2. 解压和安装

    1
    2
    3
    [root@mongodb app]# tar xzf mongodb-linux-x86_64-rhel70-3.4.10.tgz
    [root@mongodb app]# mv mongodb-linux-x86_64-rhel70-3.4.10 /mnt/app/mongo
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo

  3. 设置环境变量

    1
    2
    3
    4
    5
    [root@mongodb app]# cat >/etc/profile.d/mongo.sh <<EOF
    export MONGO_HOME=/mnt/app/mongo
    export PATH=\${MONGO_HOME}/bin:\$PATH
    EOF
    [root@mongodb app]# source /etc/profile

  4. 创建mongo配置文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/app/mongo/conf
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/conf

  5. 创建mongo数据文件目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/data/mongo/{27017,27018,27019}
    [root@mongodb app]# chown -R mongo.mongo /mnt/data/mongo

  6. 创建mongo日志目录

    1
    2
    [root@mongodb app]# mkdir -p /mnt/log/mongo/{27017,27018,27019}
    [root@mongodb app]# chown -R mongo.mongo /mnt/log/mongo

  7. 设置mongo配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27017.conf <<EOF
    dbpath=/mnt/data/mongo/27017
    logpath=/mnt/log/mongo/27017/mongo.log  
    logappend=true  
    port=27017  
    fork=true  
    nohttpinterface=true  
    #auth=true  

    journal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset
    EOF

    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27018.conf <<EOF
    dbpath=/mnt/data/mongo/27018
    logpath=/mnt/log/mongo/27018/mongo.log  
    logappend=true  
    port=27018  
    fork=true  
    nohttpinterface=true  
    #auth=true  

    journal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset
    EOF

    [root@mongodb app]# cat > /mnt/app/mongo/conf/mongo.27019.conf <<EOF
    dbpath=/mnt/data/mongo/27019
    logpath=/mnt/log/mongo/27019/mongo.log  
    logappend=true  
    port=27019  
    fork=true  
    nohttpinterface=true  
    #auth=true  

    nojournal=true
    oplogSize=2048
    smallfiles=true
    replSet=dbset
    EOF
    [root@mongodb app]# chown -R mongo.mongo /mnt/app/mongo/conf

  8. 设置mongo systemd文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27017.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27017.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27017.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27018.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27018.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27018.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# cat > /usr/lib/systemd/system/mongo-27019.service <<EOF
    [Unit]
    Description=mongodb
    After=network.target remote-fs.target nss-lookup.target

    [Service]
    Type=forking
    ExecStart=/mnt/app/mongo/bin/mongod --config /mnt/app/mongo/conf/mongo.27019.conf
    ExecReload=/bin/kill -s HUP \$MAINPID
    ExecStop=/mnt/app/mongo/bin/mongod --shutdown --config /mnt/app/mongo/conf/mongo.27019.conf
    PrivateTmp=true
    User=mongo
    Group=mongo

    [Install]
    WantedBy=multi-user.target
    EOF

    [root@mongodb app]# systemctl daemon-reload

  9. 启动mongo

    1
    2
    3
    4
    5
    6
    [root@mongodb app]# systemctl start mongo-27017
    [root@mongodb app]# systemctl start mongo-27018
    [root@mongodb app]# systemctl start mongo-27019
    [root@mongodb app]# systemctl enable monog-27017
    [root@mongodb app]# systemctl enable monog-27018
    [root@mongodb app]# systemctl enable monog-27019

  10. 副本集初始化

    1
    2
    3
    4
    5
    6
    [root@mongodb ~]# mongo --port 27017
    > rs.initiate({_id: "dbset",version: 1,members: [{ _id: 0, host : "192.168.13.218:27017" }]})
    dbset:SECONDARY>
    dbset:PRIMARY> rs.add("192.168.13.218:27018")
    dbset:PRIMARY> rs.addArb("192.168.13.218:27019")
    dbset:PRIMARY> rs.status()

  11. 副本集状态

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    dbset:PRIMARY> rs.status()
    {
            "set" : "dbset",
            "date" : ISODate("2017-11-13T09:12:44.740Z"),
            "myState" : 1,
            "term" : NumberLong(1),
            "heartbeatIntervalMillis" : NumberLong(2000),
            "optimes" : {
                    "lastCommittedOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    },
                    "appliedOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    },
                    "durableOpTime" : {
                            "ts" : Timestamp(1510564357, 1),
                            "t" : NumberLong(1)
                    }
            },
            "members" : [
                    {
                            "_id" : 0,
                            "name" : "192.168.13.218:27017",
                            "health" : 1,
                            "state" : 1,
                            "stateStr" : "PRIMARY",
                            "uptime" : 641,
                            "optime" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2017-11-13T09:12:37Z"),
                            "electionTime" : Timestamp(1510564115, 2),
                            "electionDate" : ISODate("2017-11-13T09:08:35Z"),
                            "configVersion" : 3,
                            "self" : true
                    },
                    {
                            "_id" : 1,
                            "name" : "192.168.13.218:27018",
                            "health" : 1,
                            "state" : 2,
                            "stateStr" : "SECONDARY",
                            "uptime" : 122,
                            "optime" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDurable" : {
                                    "ts" : Timestamp(1510564357, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2017-11-13T09:12:37Z"),
                            "optimeDurableDate" : ISODate("2017-11-13T09:12:37Z"),
                            "lastHeartbeat" : ISODate("2017-11-13T09:12:43.815Z"),
                            "lastHeartbeatRecv" : ISODate("2017-11-13T09:12:43.814Z"),
                            "pingMs" : NumberLong(0),
                            "syncingTo" : "192.168.13.218:27017",
                            "configVersion" : 3
                    },
                    {
                            "_id" : 2,
                            "name" : "192.168.13.218:27019",
                            "health" : 1,
                            "state" : 7,
                            "stateStr" : "ARBITER",
                            "uptime" : 58,
                            "lastHeartbeat" : ISODate("2017-11-13T09:12:43.814Z"),
                            "lastHeartbeatRecv" : ISODate("2017-11-13T09:12:40.893Z"),
                            "pingMs" : NumberLong(0),
                            "configVersion" : 3
                    }
            ],
            "ok" : 1
    }

  12. 副本集常用命令

    1
    2
    3
    4
    5
    6
    7
    8
    //primary secondary切换
    dbset:PRIMARY> rs.stepDown()

    //副本集中移除
    dbset:PRIMARY> rs.remove("192.168.13.218:27019")

    //副本集配置
    dbset:PRIMARY> rs.conf()

  13. mongo内核优化

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    [root@mongodb ~]# echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    [root@mongodb ~]# systemctl stop mongo && systemctl start mongo

    [root@mongodb ~]# cat >>/etc/rc.local <<EOF

    ### mongodb close hugepage
    if test -f /sys/kernel/mm/transparent_hugepage/enabled;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/enabled
    fi
    if test -f /sys/kernel/mm/transparent_hugepage/defrag;then
       echo never |tee /sys/kernel/mm/transparent_hugepage/defrag
    fi
    EOF
Monitor►zabbix►monitor►zabbix monitor mem

zabbix monitor mem

zabbix monitor mem

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@smallasa]# free
             total       used       free     shared    buffers     cached
Mem:      65922752    1178064   64744688        840     266968     330884
-/+ buffers/cache:     580212   65342540
Swap:      8191996          0    8191996

第1行:
total   内存总数(65922752)
used    已经使用的内存数(1178064)
free    空闲的内存数(64744688)
shared  多个进程共享的内存总额(840)
buffers 系统分配但未被使用的buffers数量(266968)
cached  系统分配但未被使用的cache数量(330884)

关系: total = used + free

第2行:
-/+ buffers/cache:
-buffers/cache 的内存数(580212) = (used - buffers - cached)
+buffers/cache 的内存数(65342540) = (free + buffers + cached)

"-buffers/cache"反映的是被程序实实在在吃掉的内存
"+buffers/cache"反映的是可以挪用的内存总数

第3行:
total   Swap内存总数(8191996)
used    Swap已经使用的内存数(0)
free    Swap空闲内存数(8191996)