Thrift Consul load balancing is achieved by Consul-template-Nginx

Share a load balancing scenario with Consul today, and many small partners know that Nginx can achieve load balancing, but may not have combined Consul so share it today.

The overall architecture

Let's first look at what the architecture of the entire framework looks like, where we have three service providers and three service callers that are load balanced through Consul and Nginx and Consul-template

Description This example is load balancing for RPC, which is a tcp protocol, so Nginx configures the tcp module to support tcp load balancing.

1. Consul clusters are used for service registration, registration of multiple service instances, and external provision of RPC services.
2. Consul-template is used to monitor the status of Consul in Consul in real time, working with one of its own template files to generate A profile for Nginx
3. Nginx uses its own profile and the second-step generated profile for load balancing.

Nginx installation

1. Install the latest version of Nginx to ensure that Nginx version is above 1.9.0
2. Version 1.9.0 or more TCP forwarding, it is said that the module is not installed by default, the installation is complete can be queried, if there is a --with-stream parameter, indicating that TCP is already supported. If not, recompile the increase parameter installation.
3. 
4. My Nginx is installed in /etc/nginx directory
5. When the installation is complete, use nginx -t to monitor for success.

Consul-template

The purpose of this article is load balancing, and Consul clustering is not introduced.

1. Download the corresponding system version file https://releases.hashicorp.com/consul-template/

2. Unzip and copy to the PATH path

[silence@centos145 ~]$ tar xzvf consul-template_0.19.4_linux_amd64.tgz
[silence@centos145 ~]$ mv ./consul-template /usr/sbin/consul-template

3. Find a place to create a new folder and create three files

4. config.hcl is primarily used to configure consul the startup parameter items of consul-template including the address of the server, the location of the template file, the location of the generated profile, and so on. I n addition to consul and template blocks, other parameters are optional. Reference https://github.com/hashicorp/consul-template

5. Consul Consul block configuration server address and port

consul {
  auth {
    enabled  = false
    username = "test"
    password = "test"
  }


  address = "172.20.132.196:8500"
  retry {
    enabled = true
    attempts = 12
    backoff = "250ms"
    max_backoff = "1m"
  }


}

6. template block configures the path to the template and where the file was generated, as well as the commands that need to be executed after the file is generated. Here we need nginx to reload the profile, so set the command to nginx -s reload

template {
  source = "/etc/nginx/consul-template/template.ctmpl"
  destination = "/etc/nginx/consul-template/nginx.conf"
  create_dest_dirs = true
  command = "/usr/sbin/nginx -s reload"
  command_timeout = "30s"
  error_on_missing_key = false
  perms = 0600
  backup = true
  left_delimiter  = "{{"
  right_delimiter = "}}"
  wait {
    min = "2s"
    max = "10s"
  }
}

7. template.ctmpl is written, because only the server address and port number are required here, so the template file is as follows:

[root@centos145 consul-template]# cat template.ctmpl
stream {


    log_format main '$remote_addr - [$time_local] '
      '$status';


    access_log /var/log/nginx/tcp_access.log main;


    upstream cloudsocket {
 \{\{range service "ad-rpc-device-server"}}server \{\{.Address}}:\{\{.Port}};{{end}}
    }


    server {
 listen 8888;
 proxy_pass cloudsocket;
    }
}

consul-template consul-template -config=./config.hcl

The config.hcl profile is used to simplify the command consul-template-consul-addr=172.20.132.196:8500 -template./template.ctmpl:./nginx.conf

9. The initial nignx.conf file is empty and the contents are after startup

[root@centos145 consul-template]# cat nginx.conf
stream {


    log_format main '$remote_addr - [$time_local] '
      '$status';


    access_log /var/log/nginx/tcp_access.log main;


    upstream cloudsocket {
 server 172.20.139.77:8183;
    }


    server {
 listen 8888;
 proxy_pass cloudsocket;
    }
}

Make sure that the service has successfully registered with Consul so that you can see that the server address and port are configured.

10. Introduce the profile include nginx in nginx.conf of the consul-template include /etc/nginx/consul-template/nginx.conf;

Note that the resulting profile cannot drink the contents of the nginx itself in the profile repeated!!!

11. Launching a service instance and looking at the generated nginx.conf file will reveal that the list of services is dynamically increased in upstream cloudsocket{} and changes dynamically as the service joins and leaves.

[root@centos145 consul-template]# cat nginx.conf
stream {


    log_format main '$remote_addr - [$time_local] '
      '$status';


    access_log /var/log/nginx/tcp_access.log main;


    upstream cloudsocket {
 server 172.20.139.77:8183;
    }


    server {
 listen 8888;
 proxy_pass cloudsocket;
    }
}

Start one more and the list of services becomes two

[root@centos145 consul-template]# cat nginx.conf
stream {


    log_format main '$remote_addr - [$time_local] '
      '$status';


    access_log /var/log/nginx/tcp_access.log main;


    upstream cloudsocket {
 server 172.20.139.77:8183;server 172.20.139.77:8184;
    }


    server {
 listen 8888;
 proxy_pass cloudsocket;
    }
}

12. thrift client only needs to configure the address and port of Nginx when calling, does not need to configure the address and port of the service, Nginx will automatically do forwarding.

(Recommended tutorial: Nginx Getting Started Guide)

summary

Today, we introduce a new load balancing implementation scheme, this scheme for some small-scale clusters is still very good, of course, if it is a large cluster, or the use of Alibaba Cloud or Tencent Cloud to provide the best solution. It's fun to try to build a small partner environment that you want to implement yourself.

Source: Public Number - Java Geek Technology

Author: Duck Blood Fans

Here's a look at the information about Consul-template-Nginx's Thrift Consul load balancing, which I hope will help you.