hadoop►kafka►kafka add topic replicats

kafka add topic replicats

kafka修改topic的replicats数量(添加)

  1. 修改前检查

    1
    2
    3
    4
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-topics.sh --zookeeper 10.0.2.15:2181/kafka --describe --topic test-1
    Topic:test-1    PartitionCount:2        ReplicationFactor:2     Configs:
            Topic: test-1   Partition: 0    Leader: 4       Replicas: 4,3   Isr: 3,4
            Topic: test-1   Partition: 1    Leader: 3       Replicas: 3,4   Isr: 3,4

  2. 编写json文件,将partition的副本由2个扩大到4个

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    penn@ubuntu:~$ cat increate_replica_partition.json
    {
      "partitions": [
        {
          "topic": "test-1",
          "partition": 0,
          "replicas": [
            1,
            2,
            3,
            4
          ]
        },
        {
          "topic": "test-1",
          "partition": 1,
          "replicas": [
            1,
            2,
            3,
            4
          ]
        }
      ],
      "version": 1
    }

  3. 执行副本集扩容

    1
    2
    3
    4
    5
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-reassign-partitions.sh --zookeeper 10.0.2.15:2181/kafka --reassignment-json-file ./increate_replica_partition.json --execute
    Current partition replica assignment
    {"version":1,"partitions":[{"topic":"test-1","partition":1,"replicas":[3,4]},{"topic":"test-1","partition":0,"replicas":[4,3]}]}
    Save this to use as the --reassignment-json-file option during rollback
    Successfully started reassignment of partitions.

  4. 修改后检查

    1
    2
    3
    4
    5
    6
    7
    8
    9
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-reassign-partitions.sh --zookeeper 10.0.2.15:2181/kafka --reassignment-json-file ./increate_replica_partition.json --verify
    Status of partition reassignment:
    Reassignment of partition [test-1,0] completed successfully
    Reassignment of partition [test-1,1] completed successfully

    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-topics.sh --zookeeper 10.0.2.15:2181/kafka --describe --topic test-1
    Topic:test-1    PartitionCount:2        ReplicationFactor:4     Configs:
            Topic: test-1   Partition: 0    Leader: 4       Replicas: 1,2,3,4       Isr: 3,4,2,1
            Topic: test-1   Partition: 1    Leader: 3       Replicas: 1,2,3,4       Isr: 3,4,2,1

kafka修改topic的replicats数量(缩减)

  1. 缩容前检查

    1
    2
    3
    4
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-topics.sh --zookeeper 10.0.2.15:2181/kafka --describe --topic test-1
    Topic:test-1    PartitionCount:2        ReplicationFactor:4     Configs:
            Topic: test-1   Partition: 0    Leader: 1       Replicas: 1,2,3,4       Isr: 3,4,2,1
            Topic: test-1   Partition: 1    Leader: 1       Replicas: 1,2,3,4       Isr: 3,4,2,1

  2. 缩容json

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    penn@ubuntu:~$ cat increate_replica_partition.json
    {
      "partitions": [
        {
          "topic": "test-1",
          "partition": 0,
          "replicas": [
            1,
            2
          ]
        },
        {
          "topic": "test-1",
          "partition": 1,
          "replicas": [
            1,
            2
          ]
        }
      ],
      "version": 1
    }

  3. 执行json

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-reassign-partitions.sh --zookeeper 10.0.2.15:2181/kafka --reassignment-json-file ./increate_replica_partition.json --execute
    Current partition replica assignment
    {"version":1,"partitions":[{"topic":"test-1","partition":1,"replicas":[1,2,3,4]},{"topic":"test-1","partition":0,"replicas":[1,2,3,4]}]}
    Save this to use as the --reassignment-json-file option during rollback
    Successfully started reassignment of partitions.

    4. 缩减后检查
    ```bash
    penn@ubuntu:~$ /mnt/app/kafka.1/bin/kafka-topics.sh --zookeeper 10.0.2.15:2181/kafka --describe --topic test-1                                                      Topic:test-1    PartitionCount:2        ReplicationFactor:2     Configs:
            Topic: test-1   Partition: 0    Leader: 1       Replicas: 1,2   Isr: 2,1
            Topic: test-1   Partition: 1    Leader: 1       Replicas: 1,2   Isr: 2,1