1. What You Will Learn
-
How to use Ribbon as a client side load balancer
-
How to use a Ribbon enabled
RestTemplate
2. Start the foundation services
-
Start the
config-serverin a terminal window. You may have terminal windows still open from previous labs. They may be reused for this lab.$ cd config-server $ mvn spring-boot:run -
Start the
service-registry$ cd service-registry $ mvn spring-boot:run -
Start two instances of the
fortune-service, as follows:-
Start your first instance as usual:
$ cd fortune-service $ mvn spring-boot:runThis will give us a running fortune-service on http://localhost:8787/
-
Start a second instance by picking a different port for the second instance to listen on:
Mac, Linux:
SERVER_PORT=8788 mvn spring-boot:runWindows:
$ set SERVER_PORT=8788 $ mvn spring-boot:run -
Now check the eureka dashboard at http://localhost:8761/ and verify that you have two local instances of the fortune service running
-
3. Set up greeting-ribbon
No additions to the pom.xml
In this case, we don’t need to explicitly include Ribbon support in the pom.xml. Ribbon support is pulled in through transitive dependencies (dependencies of the dependencies we have already defined).
-
Review the class
greeting-ribbon/src/main/java/io/pivotal/greeting/GreetingController.java. Notice theloadBalancerClient. It is a client-side load balancer (Ribbon). Review thefetchFortuneServiceUrl()method. Ribbon is integrated with Eureka so that it can discover services as well. Notice how theloadBalancerClientchooses a service instance by name.package io.pivotal.greeting; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.loadbalancer.LoadBalancerClient; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.client.RestTemplate; @Controller public class GreetingController { private final Logger logger = LoggerFactory.getLogger(GreetingController.class); private final LoadBalancerClient loadBalancerClient; public GreetingController(LoadBalancerClient loadBalancerClient) { this.loadBalancerClient = loadBalancerClient; } @RequestMapping("/") String getGreeting(Model model) { logger.debug("Adding greeting"); model.addAttribute("msg", "Greetings!!!"); RestTemplate restTemplate = new RestTemplate(); String fortune = restTemplate.getForObject(fetchFortuneServiceUrl(), String.class); logger.debug("Adding fortune: {}", fortune); model.addAttribute("fortune", fortune); return "greeting"; // resolves to the greeting.ftl template } private String fetchFortuneServiceUrl() { ServiceInstance instance = loadBalancerClient.choose("fortune-service"); logger.debug("uri: {}", instance.getUri().toString()); logger.debug("serviceId: {}", instance.getServiceId()); return instance.getUri().toString(); } } -
Open a new terminal window. Start the
greeting-ribbonapp.$ cd greeting-ribbon $ mvn spring-boot:run -
After the a few moments, check the
service-registrydashboard http://localhost:8761. Confirm thegreeting-ribbonapp is registered. -
Browse to the
greeting-ribbonapplication at http://localhost:8080/. Confirm you are seeing fortunes. As you refresh the greeting-ribbon app’s main page, observe the log output forgreeting-ribbonin the terminal. Inspect the loggeduriandserviceIdvalues. You should see these go back and forth between the twofortune-serviceinstances, in round-robin fashion. Ribbon is doing client-side load balancing! -
Stop the
greeting-ribbonapplication.
4. Refactor to use a load-balanced RestTemplate
LoadBalancerClient provides a simple way to ask Ribbon to choose() an application instance from among multiple running instances. Spring Cloud Netflix provides an alternative, more elegant mechanism to make load-balanced REST API calls, which transparently invokes the Ribbon load balancer, via a cross-cutting annotation named @LoadBalanced.
Let’s see how this works by refactoring our existing implementation:
-
In our application class,
GreetingRibbonApplication, explicitly inject aRestTemplatebean, and decorate it with the@LoadBalancedannotation, as shown in this diff:package io.pivotal; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; +import org.springframework.cloud.client.loadbalancer.LoadBalanced; +import org.springframework.context.annotation.Bean; +import org.springframework.web.client.RestTemplate; @SpringBootApplication @EnableDiscoveryClient public class GreetingRibbonApplication { public static void main(String[] args) { SpringApplication.run(GreetingRibbonRestApplication.class, args); } + @LoadBalanced + @Bean + RestTemplate restTemplate() { + return new RestTemplate(); + } }Simply adding the annotation will ensure that calls made using this
RestTemplateinstance will be load-balanced by Ribbon. -
To use this
RestTemplateinstance, let’s inject it into ourGreetingControllerusing Spring constructor injection, as shown by this diff:- private final LoadBalancerClient loadBalancerClient; + private final RestTemplate restTemplate; - public GreetingController(LoadBalancerClient loadBalancerClient) { + public GreetingController(RestTemplate restTemplate) { - this.loadBalancerClient = loadBalancerClient; + this.restTemplate = restTemplate; } -
Next, inside our
getGreeting()method, we can remove the line that explicitly instantiates aRestTemplateand use the Autowired instance in its place. Furthermore, instead of callingfetchFortuneServiceUrl(), we use a clever String parameter that looks like a url, but that is in fact a template or placeholder for the actual url that Ribbon will derive from it:@RequestMapping("/") String getGreeting(Model model) { logger.debug("Adding greeting"); model.addAttribute("msg", "Greetings!!!"); - RestTemplate restTemplate = new RestTemplate(); - String fortune = restTemplate.getForObject(fetchFortuneServiceUrl(), String.class); + String fortune = restTemplate.getForObject("http://fortune-service", String.class); logger.debug("Adding fortune: {}", fortune); model.addAttribute("fortune", fortune); return "greeting"; // resolves to the greeting.ftl template }Basically
http://fortune-servicereferences the name of the application we wish to invoke. Spring Cloud Netflix automatically parses the string, looks up the application name against a locally cached copy of the Eureka registry, and asks Ribbon to choose an instance, whose url is then inserted in the place of the application name. -
We can now delete the unused method
fetchFortuneServiceUrl()
A reference implementation of greeting-ribbon refactored as described above is available under the project named greeting-ribbon-rest.
|
Let’s take our refactored implementation for a spin:
$ cd greeting-ribbon
$ mvn clean spring-boot:run
-
After the a few moments, check the
service-registrydashboard at http://localhost:8761. Confirm thegreeting-ribbonapp is registered once more. -
Browse to http://localhost:8080/ to the
greeting-ribbonapplication. Confirm you are seeing fortunes. Refresh as desired. Review the terminal output for thegreeting-ribbonapp. -
More interestingly, review the logs for the two
fortune-serviceapplications. With each refresh ofgreeting-ribbon, one of these two will serve the response, in alternating fashion.
5. Retrying Failed Requests
Conduct the following experiment:
-
Make sure you have two instances of
fortune-servicerunning -
Visit the greeting application, and again, verify that requests round-robin between the two instances
-
Stop or kill one of the two
fortune-serviceinstances -
Continue making requests to the greeting application
One of two requests simply fails.
If we wait long enough for Eureka to remove the insance from the list (since it won’t be receiving heartbeats any longer), and for the greeting application to fetch an updated registry, this matter will resolve itself.
Given that a second instance is available to handle these requests, it’s preferable if Ribbon were to just retry the request against the surviving instance.
Let’s rectify this problem.
Netflix Ribbon supports retrying failed requests. The Ribbon GitHub Wiki cites configuration properties such as MaxAutoRetries and MaxAutoRetriesNextServer that govern this behavior. The Spring Cloud team supports this capability via the Spring Retry project. Ryan Baxter, a member of the Spring Cloud team, discusses this integration work in this blog (so does the Spring Cloud Netflix reference documentation, for that matter).
➜ Add the Spring Retry dependency to pom file for greeting-ribbon:
@@ -41,6 +41,10 @@
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-starter-service-registry</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.springframework.retry</groupId>
+ <artifactId>spring-retry</artifactId>
+ </dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId>
➜ Configure MaxAutoRetriesNextServer for the ribbon client:
---
fortune-service:
ribbon:
MaxAutoRetriesNextServer: 1
This setting can be interpreted as if a request to one of the servers from the list fails, try another server.
➜ Also, configure the logging level for the package org.springframework.retry to DEBUG:
logging:
level:
org.springframework.retry: DEBUG
Finally, repeat the test:
-
Ensure two instances of
fortune-serviceare running and registered with eureka. -
Start
greeting-ribbon, and ensure that requests round-robin between the two instances. -
Take down one of the
fortune-serviceinstances. -
Access the greeting application a few more times: you should see no failures.
Also observe that the console output for greeting-ribbon contains a log message from the Spring Retry library stating that the retry count was 1 (i.e. that it actually retried a request):
... o.s.retry.support.RetryTemplate : Retry: count=1 io.pivotal.greeting.GreetingController : Adding fortune: You can always find happiness at work on Friday io.pivotal.greeting.GreetingController : Adding greeting ...
6. Customize the Load Balancing Rule
The Ribbon API was designed to be flexible, with multiple interfaces whose implementations define its behavior:
-
IRule: defines the rules that Ribbon uses for load balancing. The default isRoundRobinRule -
IPing: the mechanism Ribbon uses to ping services (to check if they are alive) -
ILoadBalancer: a more general interface that includes how Ribbon obtains the list of servers to load-balance across
Out of the box, Ribbon provides multiple alternative implementations of IRule. Our task here is to configure Ribbon to use the WeightedResponseTimeRule instead of the default RoundRobinRule.
We can simulate two instances of fortune-service with different response times by using an artificial delay in the response for one of our two fortune-service instances. The fortune service has been configured with the property (or environment variable) DELAY_MS to allow us to do just that.
-
Start two instances of
fortune-serviceonce more, but this time, as follows:mvn spring-boot:run..and:
Mac, Linux:
SERVER_PORT=8788 DELAY_MS=1000 mvn spring-boot:runWindows:
$ set SERVER_PORT=8788 $ set DELAY_MS=1000 $ mvn spring-boot:runAt this point you can verify once more that you have two instances of
fortune-serviceregistered with Eureka, and that one of them takes longer to respond than the other. -
Refer to the following Spring Cloud Ribbon documentation which discusses how different aspects of Ribbon can be configured using properties.
-
In
greeting-ribbonsrc/main/resourcesfolder, create theapplication.ymlfile and edit it as follows:fortune-service: ribbon: NFLoadBalancerRuleClassName: com.netflix.loadbalancer.WeightedResponseTimeRuleNote above how a spring boot application can define and configure multiple independent Ribbon clients. Each client has a name, in this case corresponding to the service being called.
-
Start the
greeting-ribbonapplication once more, and refresh the page to see Ribbon again load-balance requests across the twofortune-serviceinstances.-
You should begin to see log messages indicating that a background thread is running that computes updated response times for each instance.
-
Within a short time,
greeting-ribbonwill adapt to invoking the fasterfortune-serviceinstance in proportion to its response time; i.e. it will be invoked more often than the slower instance.
-
Congratulations! You’ve now configured Ribbon with a custom load balancing rule. You can now stop the config-server, service-registry, fortune-service and greeting-ribbon applications.
7. Deploy the greeting-ribbon to PCF
7.1. PCF and the Eureka Registration Method
In the previous lab, we used the route registration method. This had the effect of giving client applications the same url for all application instances. This setting defeats Ribbon, in that no matter which application instance we select, we end up with the same application url, which by definition will be routed through the PCF GoRouter.
In order to bypass the GoRouter and leverage Ribbon, the registration method has to be set to direct.
When using the direct registration method, all application instances register their internal IP address and port number with Eureka. This can be a problem in PCF however, for a number of reasons:
-
Cloud Foundry may not be configured for direct container-to-container networking,
-
Even with container-to-container networking enabled, platform security policy may not allow applications to call one another directly
A new feature of PCF version 1.10 known as "container-to-container" networking, provides a mechanism for an operator for applications to call each other directly by IP address, effectively bypassing the GoRouter.
A recent blog post by Chris Sterling gives a great overview of how this feature works together with Spring Cloud Services.
Assuming you’re working with an instance of Cloud Foundry that has direct networking enabled, let’s begin by modifying our application configuration:
-
In your
app-configgit repository, locateapplication.yml -
Modify the value of the property
spring.cloud.services.registrationMethodfromroutetodirect -
Commit and push your changes back to your github repository
-
Finally, restart your running
fortune-serviceinstances, to cause them to re-register with eureka using their IP addresses:cf restart fortune-service
7.2. Deploy 'greeting-ribbon'
-
Package and push the
greeting-ribbonapplication.$ mvn clean package $ cf push greeting-ribbon -p target/greeting-ribbon-0.0.1-SNAPSHOT.jar -m 768M --random-route --no-start -
Bind services for the
greeting-ribbonapplication.cf bind-service greeting-ribbon config-server..and:
cf bind-service greeting-ribbon service-registryYou can safely ignore the TIP: Use 'cf restage' to ensure your env variable changes take effect message from the CLI. We don’t need to restage at this time.
-
Start the
greeting-ribbonapp.cf start greeting-ribbon -
After the a few moments, check the
service-registry. Confirm thegreeting-ribbonapp is registered.
|
Instead of deploying
Manifest files simplify the task of deploying an application, and a single |
Refresh the greeting-ribbon root endpoint.
This should fail.
In a nutshell, you’ll need to invoke the following command to allow direct communication from the greeting-ribbon application to the fortune-service application:
cf add-network-policy greeting-ribbon --destination-app fortune-service --protocol tcp --port 8080
Now retry refreshing the greeting-ribbon root endpoint. If fortune-service is still scaled at three instances, you should see Ribbon load balance requests across these instances.