hadoop►logstash►logstash install

logstash install

logstash install

  1. logstash安装

    1
    2
    3
    4
    5
    6
    7
    8
    [root@10 app]# tar xzf logstash-all-plugins-2.4.0.tar.gz
    [root@10 app]# mv logstash-2.4.0 /mnt/app/logstash
    [root@10 app]# chown -R wisdom.wisdom /mnt/app/elasticsearch

    [root@10 app]#  mkdir /mnt/app/logstash/conf
    [root@10 app]#  mkdir /mnt/log/logstash
    [root@10 app]# chown -R wisdom.wisdom /mnt/app/logstash/conf
    [root@10 app]# chown -R wisdom.wisdom /mnt/log/logstash

  2. logstash 配置

    1
    2
    3
    4
    5
    [root@10 app]# su - wisdom
    [wisdom@10 ~]$ cat >/mnt/app/logstash/conf/test.conf <<EOF
    > input { stdin { } }
    > output { stdout {} }
    > EOF

  3. logstash 配置检查

    1
    2
    [wisdom@10 ~]$ /mnt/app/logstash/bin/logstash -f /mnt/app/logstash/conf/test.conf --configtest
    Configuration OK

  4. logstash 测试

    1
    2
    3
    4
    5
    [wisdom@10 ~]$ /mnt/app/logstash/bin/logstash -f /mnt/app/logstash/conf/test.conf
    Settings: Default pipeline workers: 1
    Pipeline main started
    hello world  =>输入字符串
    2016-10-12T09:16:18.058Z ubuntu hello world

  5. logstash 设置启动需要的内存大小

    1
    2
    [wisdom@10 ~]$ vim /mnt/app/logstash/bin/logstash
    LS_HEAP_SIZE="8G"

  6. logstash 后台执行

    1
    [wisdom@10 ~]$ /mnt/app/logstash/bin/logstash -f /mnt/app/logstash/conf/test.conf -l /mnt/log/logstash/test.log -w 8 -b 125 -u 5 --auto-reload --reload-interval 3 &

    ```bash

  • The –pipeline-workers or -w parameter determines how many threads to run for filter and output processing.If you find that events are backing up, or that the CPU is not saturated, consider increasing the value of this parameter to make better use of available processing power.
  • The –pipeline-batch-size or -b parameter defines the maximum number of events an individual worker thread collects before attempting to execute filters and outputs. Larger batch sizes are generally more efficient, but increase memory overhead.
  • The –pipeline-batch-delay option rarely needs to be tuned. Pipeline batch delay is the maximum amount of time in milliseconds that Logstash waits for new messages after receiving an event in the current pipeline worker thread. After this time elapses, Logstash begins to execute filters and outputs.
    ``