Recently, Spring Cloud Tencent was officially open sourced on June 14, as Tencent’s open source one-stop microservice framework, Spring Cloud Tencent implements Spring Cloud standard microservice SPI, developers can quickly develop Spring Cloud microservice architecture applications based on Spring Cloud Tencent. As Tencent’s open source one-stop microservice framework, Spring Cloud Tencent implements the Spring Cloud standard microservice SPI, which allows developers to quickly develop Spring Cloud microservice architecture applications based on Spring Cloud Tencent. The core of Spring Cloud Tencent relies on Tencent’s open source one-stop service discovery and governance platform PolarisMesh to implement various distributed microservice scenarios.
1. Why Spring Cloud Tencent
Spring Boot + Spring Cloud is still the most dominant framework in the Java ecosystem
Spring Boot released version 1.0 in April 2014 and after 8 years of development, Spring Boot has become the de facto standard for Java development frameworks. In the area of distributed microservices, Spring Cloud was released in January 2016 as Angel.SR5. Spring Cloud inherits the core idea of Spring Boot’s core component-based, low-configuration, rapid integration and defines a standard microservices SPI on which Spring Cloud Netfix and other excellent microservice solution implementation suites. In terms of remote service invocation frameworks, Feign and RestTemplate are based on the pervasive HTTP protocol and have natural advantages in terms of ease of use, observability, and cross-language. That’s why Spring Cloud has flourished since its first release in 2016.
From an industry perspective, Spring Boot + Spring Cloud is one of the most widely used development frameworks for Java today.
ServiceMesh, represented by Istio, started to incubate in the community in 2018, and by 2022 there will be a very large number of ServiceMesh products.The core idea of ServiceMesh is to sink the underlying capabilities related to service invocation to a separate Sidecar process that governs traffic by means of a traffic proxy. No solution is a panacea, and the Sidecar model has some problems. For example, Sidecar is highly dependent on the underlying Paas capabilities to govern Sidecar (injection, version management, upgrade, etc.), Sidecar requires extra resources, increases network latency, and makes troubleshooting more difficult. Therefore, there are not many companies that can really implement ServiceMesh.
To sum up, we think Spring Boot + Spring Cloud will remain the mainstream microservice solution for Java for a long time, although it does not look as advanced as Istio and Dapr. In the premise of meeting the business development requirements of enterprises, the best solution is a low-cost, efficient and stable architecture.
Tencent’s PolarisMesh open source in 2021 provides a one-stop microservice solution
PolarisMesh is a one-stop microservice solution that integrates registration center, configuration center, and service governance center. It has covered 90% of Tencent’s business internally, and the number of registered instance nodes has reached 5 million. After 21 years of open source, it has been produced and landed in the community by external companies.
In-house architects often make technology selections, such as Zookeeper, Consul, Nacos for registry, Apollo, Nacos for configuration center, Sentinel for flow restriction and Circuit Breaker. Multiple components also mean that multiple services need to be maintained, taking up more resources and making it difficult to achieve consistency in user experience.
So a one-stop microservices solution can greatly simplify technology selection, operations and maintenance, and resource costs. Of course, you can also use Polaris as part of the solution, for example, only use Polaris’ service registration discovery, and still choose Apollo as the configuration center. After all, there is no one-size-fits-all solution, and the best solution is the one that suits your business requirements.
In addition, Polaris has certain advantages in certain capabilities in horizontal comparison. For example, the completely stateless registry is easier to operate and maintain, and the powerful service routing capability supports complex business scenarios. Specific product features will be described in more detail in Part II.
Summary
For these two core reasons, it is logical to use Polaris as an out-of-the-box implementation kit for Spring Cloud. It can satisfy both Spring Cloud users and Polaris Java users. Of course, Spring Cloud Tencent currently only docks the capabilities of Polaris, and will subsequently support more of Tencent’s excellent open source products.
2. Spring Cloud Tencent Module Details
Currently, Spring Cloud Tencent mainly provides the common service registration and discovery, configuration center, service routing, flow restriction and Circuit Breaker, and metadata link pass-through capabilities in microservices. Each part of the capabilities will be described in detail in the next section.
2.1 Service Registration and Discovery (Spring Cloud Tencent Polaris Discovery)
Service registration and discovery is one of the core features of Spring Cloud Tencent, providing microservice applications with fast access to the Polaris service registry by implementing Spring Cloud’s standard interface for service registration and discovery. Developers can use Polaris’ service registration and discovery capabilities by simply introducing Spring Cloud Tencent service registration and discovery dependencies. After accessing service registration and discovery, you can also use the powerful service governance capabilities provided by Polaris on demand, such as scenario-based service routing capabilities, service circuit breaker capabilities, and more. It is easy for developers to make personalized service governance configurations for actual production scenarios of microservices.
Polaris’ service model includes namespaces, services and service instances.
Namespaces
Namespaces provide a mechanism for logical isolation of resources under the same registry. Services named under the same namespace must be unique, but services with the same name are allowed to exist across namespaces. Namespaces are often used to distinguish between different environments or to isolate services between multiple services.
Service
Services are also logical concepts that provide service capabilities for a specific business domain. Examples include order services, user services, transfer services, etc.
Service Instance
A service instance is a specific physical node under a service.
Spring Cloud Tencent provides some extended capabilities on top of the basic service registration discovery. First, Spring Cloud Tencent integrates some routing plugins from Polaris to enable dynamic up and down of service instances by changing the isolation status or weight value of the service instance in the Polaris console page, as shown in the figure above.
Spring Cloud Tencent also provides advanced features for multi-service registration and discovery. For example, if multiple departments or organizations within a company are using different service registries, when making decisions to unify the technology stack or migrate to the Polaris registry, you need to use a smooth way to transform the business instead of directly replacing the original SDK to connect to the Polaris registry. At this point, you can use Spring Cloud Tencent’s multi-service registration and discovery capabilities to help developers transition their microservice applications through the awkward period of technology stack conversion.
Spring Cloud Tencent provides such a series of peripheral functions for service registration and discovery, improving the governance and control of microservice architecture.
2.2 Configuration Center (Spring Cloud Tencent Polaris Config)
In the first half of this year Polaris started to support Configuration Center capabilities. The core configuration triad model for Polaris Configuration Center is
Namespace
A capability for logically isolating clusters, such as commonly used for isolated environments.
FileGroup
A configuration file group, a collection of configuration files. In Spring Cloud Tencent, our recommended best practice is to have one application as a FileGroup, and for framework class configurations, the framework name is used as a FileGroup, for example dubbo.
File
Configuration file, e.g. properties, yml formatted configuration file. A configuration file is the smallest management unit, not a configuration item in a configuration file.
[Namespace, FileGroup, File] uniquely locates a configuration file. When we designed the model, we referred to the industry’s mainstream configuration center products, and we believe that the concept of configuration file and configuration file group is the configuration domain model that is widely known and least understood by developers, such as the concept of local disk folders and files.
The core capabilities of the configuration center are configuration management capabilities and dynamic real-time push capabilities. In terms of configuration management, an application often has a very large number of configuration files, and how to clearly manage configuration files is a very important capability. When we designed the UI in the control panel, we pioneered to display the file names in the form of a tree with / as the separator. As shown in the figure below, you can divide the directory by application module, and through the directory method, you can clearly manage the cluttered configuration files under one application.
In addition, for Spring Cloud integration, Spring Boot is known to automatically load the application.yml, application.properties, and the higher priority application-activeProfile.yml files in the application resources directory.
For the SpringCloudTencentPolarisConfig integration, we have followed this native configuration loading mechanism exactly. That is, SpringCloudTencentPolarisConfig will automatically load the application.yml and application-{activeProfile}.yml files under the application file group into the Spring container at startup. When users do the migration, they just need to upload all the configuration files in the resources directory to Polaris as is.
2.3 Service Routing (Spring Cloud Tencent Polaris Router)
In the microservices space, services have become very light and flexible due to the fine-grained splitting of services for deployment. It becomes more powerful when combined with k8s cloud-native extreme elasticity capabilities. However, the underlying Paas capability only provides the basic elasticity capability, and the real ability to leverage the capability needs to rely on the upper layer of traffic provisioning capabilities.
Looking at the Spring Cloud ecosystem, there are not many component suites that can deeply integrate with Spring Cloud to provide scenario-based service routing capabilities. Here is an explanation of scenario-based, service invocation framework based on certain rules to achieve the ability to achieve service routing we call the underlying atomic capabilities. Atomic capabilities are very general capabilities, but many times they are not directly used in specific business scenarios. For example, the common test environment grouping, proximity routing, blue-green release, etc. are called scenarios. Service routing can only really serve the business out of the box after doing scenarios.
Spring Cloud Tencent Polaris Router currently implements two kinds of scenario-based routing capabilities and a very flexible rule-based routing capability.
Metadata Routing
Service instances are attached to a set of metadata, such as environmental information, room information, etc. Metadata routing simply means that the metadata information is used as the basis for routing in. This is still a bit abstract, so let’s use a test environment example to explain.
The above diagram is a very classic solution for resolving test environment conflicts. In one iteration, SvcA needs to be co-located with SvcD. When the team size is small, you can directly deploy the stable environment as development branch code and then co-locate it. However, when multiple development tasks are in parallel, there is contention for the environment. One solution is to deploy a full-linked environment for each development task independently, which is time-consuming and ineffective. The industry’s most mainstream approach is shown above, where each development task subenvironment only needs to deploy a co-located application, and services on the link that are not in the subenvironment are routed back to the stable environment.
To achieve the above purpose, you only need to add the following two environment variables for each service deployment.
- SCT_METADATA_CONTENT_ENV=dev1
- SCT_METADATA_CONTENT_TRANSITIVE=ENV
The Spring Cloud Tencent Polaris Router component automatically reads the above environment variables and prioritizes each service call to the target instance with the same ENV value as the current instance.
Metadata routing is used in a wide range of scenarios, so check out the Github Wiki for more details.
Rule Routing
Metadata routing is essentially filtering based on the metadata of a service instance and is a built-in service routing capability to support specific and concrete scenarios. There is no need to issue any routing rules and it is very simple to use.
The actual business scenarios are very complex, such as the following typical business scenarios.
- Internal employees are routed to a set of production grayscale environments, while external normal users are routed to the production formal environment.
- VIP customers are routed to a set of high security environment, and ordinary customers are routed to the normal environment.
The above two business scenarios cannot be achieved through metadata routing. This is because business request parameters are involved, not system dimensional environment variables. Rule routing is a set of rule-based service routing implementations used to meet complex business scenarios.
A typical rule is shown in the following figure.
The meaning of the above diagram is: HTTP Query Param with uid parameter value of 100 invokes the instance grouping to ENV=gray. Most complex business scenarios can be described by routing rules.
For ease of use, Spring Cloud Tencent has a set of built-in expression tag rules that automatically parse tag values from HTTP requests. The currently supported expression rules are
- ${http.query.xxx}
- ${http.header.xxx}
- ${http.cookie.xxx}
- ${http.method}
- ${http.uri}
Rule routing is relatively complex, so check out the Github Wiki for more details.
Proximity Routing
Production environment services often require multiple cabinets, multiple rooms, and multiple geographic deployments for high availability, disaster recovery, and other capabilities.
As shown in the figure above, the range is from small to large: Campus < Zone < Region < All where Campus, Zone and Region are uniformly defined as metadata in the service registration and discovery domain model, which is a special kind of location metadata.
As the name implies, service invocation selects the target service instance according to the priority of same Campus, same Zone, and same Region in descending order. The core is to reduce the network time consumed by service invocation due to physical distance. Essentially, proximity routing is a type of metadata routing based on a specific set of location metadata.
To implement proximity routing with Spring Cloud Tencent, simply add the following environment variables to the service instance.
- SCT_METADATA_CAMPUS
- SCT_METADATA_ZONE
- SCT_METADATA_REGION
2.4 Service Flow Limiting (Spring Cloud Tencent Polaris Ratelimit)
With growing business development, network requests are increasing, leading to bursts of traffic on the service side of business applications in some scenarios, and therefore some protection needs to be given to the service provider. Through the service flow limiting function, developers can control the QPS to avoid being overwhelmed by transient traffic spikes and thus guarantee the high availability of the system. There are two main application scenarios for service flow limiting, overload protection and frequent requests. Spring Cloud Tencent has built-in flow limiting filters for Spring Web and Spring WebFlux scenarios to help businesses quickly access Polaris’ flow limiting capabilities.
Spring Cloud Tencent supports two types of service flow limiting capabilities provided by Polaris, i.e., standalone flow limiting and distributed flow limiting.
Single machine flow restriction
Single-machine flow limiting is a level of flow limiting for a single deployed instance, and the traffic limit only takes effect for the current deployed instance and is not shared, as shown in the figure below. Single machine flow restriction is generally applied to protect the service itself from being broken, and the quota is calculated according to the capacity of each single machine of the service cluster.
The effect of single-machine flow limiting is divided into fast failure and even queuing. Fast failure means that when the QPS reaches the quota specified by the flow limiting rule, a flow limiting type error response is immediately returned to all requests that exceed the threshold. Uniform queuing is a peak and valley limiting method based on the leaky bucket algorithm, helping the server to ensure a uniform processing state when the traffic flood arrives, so that some requests can be processed after a period of queuing and waiting time, instead of failing directly. See the official Github Wiki for more details on even queuing.
Distributed Flow Limiting
Distributed flow limiting is for flow limiting at the level of all instances under a service, where multiple service instances share the same global traffic limit, as shown in the figure below. Distributed flow limiting is generally applied to protect third-party services or public services (e.g., protecting databases); or flow limiting at the gateway level for back-end services that are accessed through the gateway.
Spring Cloud Tencent provides the ability to customize flow restriction rules, so developers can customize the corresponding flow restriction rules according to their business scenarios.
2.5 Service Circuitbreaker (Spring Cloud Tencent Polaris Circuitbreaker)
In microservices architecture operation and maintenance scenarios, sometimes there is a single point of service instance failure, and if it cannot be removed in time, then there will still be requests forwarded to the failed service instance.Spring Cloud Tencent provides the service Circuitbreaker capability to determine whether the called party’s service is failing by reporting the results of each inter-service call, and then to detect it. It then blocks it and starts a timed task to probe the Circuitbreaker instance to live. After the recovery condition is reached, it is half-opened for recovery. After the half-open recovery, a small number of requests are released for real business request probing. And based on the result of real business probing, we determine whether the recovery is complete. This feature can effectively weed out abnormal service instances and provides an important help for service governance.
Summary
The above is just a brief introduction to some of the Spring Cloud Tencent capabilities, to learn more about them please visit our official Github page.
3. Planning and Vision
At the beginning of the article, we mentioned why we want to do Spring Cloud Tencent, we strongly believe that Spring Cloud will remain the mainstream solution for microservices in Java domain for a long time. We hope to combine with Polaris’ one-stop microservice capability to lower the threshold of microservice architecture and provide a full set of microservice solutions for enterprises out of the box. This will enable enterprises to focus more on their own business development and improve productivity.
A product that works well needs to withstand rich scenarios to polish stability, ease of use, and continuous improvement of its own product power. The following are some of the points we currently think of that need support and improvement. Of course, as the product develops and more and more users use it, there will be more requests and we will continue to iterate on it.
Github: https://github.com/Tencent/spring-cloud-tencent
Reference https://www.oschina.net/news/200252/spring-cloud-tencent-open-source