1. What You Will Learn
-
How to use Ribbon as a client side load balancer
-
How to use a Ribbon enabled
RestTemplate
2. Start the foundation services
-
Start the
config-server
in a terminal window. You may have terminal windows still open from previous labs. They may be reused for this lab.$ cd config-server $ mvn spring-boot:run
-
Start the
service-registry
$ cd service-registry $ mvn spring-boot:run
-
Start two instances of the
fortune-service
, as follows:-
Start your first instance as usual:
$ cd fortune-service $ mvn spring-boot:run
This will give us a running fortune-service on http://localhost:8787/
-
Start a second instance by picking a different port for the second instance to listen on:
Mac, Linux:
SERVER_PORT=8788 mvn spring-boot:run
Windows:
$ set SERVER_PORT=8788 $ mvn spring-boot:run
-
Now check the eureka dashboard at http://localhost:8761/ and verify that you have two local instances of the fortune service running
-
3. Set up greeting-ribbon
No additions to the pom.xml
In this case, we don’t need to explicitly include Ribbon support in the pom.xml
. Ribbon support is pulled in through transitive dependencies (dependencies of the dependencies we have already defined).
-
Review the class
greeting-ribbon/src/main/java/io/pivotal/greeting/GreetingController.java
. Notice theloadBalancerClient
. It is a client-side load balancer (Ribbon). Review thefetchFortuneServiceUrl()
method. Ribbon is integrated with Eureka so that it can discover services as well. Notice how theloadBalancerClient
chooses a service instance by name.package io.pivotal.greeting; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.loadbalancer.LoadBalancerClient; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.client.RestTemplate; @Controller public class GreetingController { private final Logger logger = LoggerFactory.getLogger(GreetingController.class); private final LoadBalancerClient loadBalancerClient; public GreetingController(LoadBalancerClient loadBalancerClient) { this.loadBalancerClient = loadBalancerClient; } @RequestMapping("/") String getGreeting(Model model) { logger.debug("Adding greeting"); model.addAttribute("msg", "Greetings!!!"); RestTemplate restTemplate = new RestTemplate(); String fortune = restTemplate.getForObject(fetchFortuneServiceUrl(), String.class); logger.debug("Adding fortune: {}", fortune); model.addAttribute("fortune", fortune); return "greeting"; // resolves to the greeting.ftl template } private String fetchFortuneServiceUrl() { ServiceInstance instance = loadBalancerClient.choose("fortune-service"); logger.debug("uri: {}", instance.getUri().toString()); logger.debug("serviceId: {}", instance.getServiceId()); return instance.getUri().toString(); } }
-
Open a new terminal window. Start the
greeting-ribbon
app.$ cd greeting-ribbon $ mvn spring-boot:run
-
After the a few moments, check the
service-registry
dashboard http://localhost:8761. Confirm thegreeting-ribbon
app is registered. -
Browse to the
greeting-ribbon
application at http://localhost:8080/. Confirm you are seeing fortunes. As you refresh the greeting-ribbon app’s main page, observe the log output forgreeting-ribbon
in the terminal. Inspect the loggeduri
andserviceId
values. You should see these go back and forth between the twofortune-service
instances, in round-robin fashion. Ribbon is doing client-side load balancing! -
Stop the
greeting-ribbon
application.
4. Refactor to use a load-balanced RestTemplate
LoadBalancerClient
provides a simple way to ask Ribbon to choose()
an application instance from among multiple running instances. Spring Cloud Netflix provides an alternative, more elegant mechanism to make load-balanced REST API calls, which transparently invokes the Ribbon load balancer, via a cross-cutting annotation named @LoadBalanced.
Let’s see how this works by refactoring our existing implementation:
-
In our application class,
GreetingRibbonApplication
, explicitly inject aRestTemplate
bean, and decorate it with the@LoadBalanced
annotation, as shown in this diff:package io.pivotal; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; +import org.springframework.cloud.client.loadbalancer.LoadBalanced; +import org.springframework.context.annotation.Bean; +import org.springframework.web.client.RestTemplate; @SpringBootApplication @EnableDiscoveryClient public class GreetingRibbonApplication { public static void main(String[] args) { SpringApplication.run(GreetingRibbonRestApplication.class, args); } + @LoadBalanced + @Bean + RestTemplate restTemplate() { + return new RestTemplate(); + } }
Simply adding the annotation will ensure that calls made using this
RestTemplate
instance will be load-balanced by Ribbon. -
To use this
RestTemplate
instance, let’s inject it into ourGreetingController
using Spring constructor injection, as shown by this diff:- private final LoadBalancerClient loadBalancerClient; + private final RestTemplate restTemplate; - public GreetingController(LoadBalancerClient loadBalancerClient) { + public GreetingController(RestTemplate restTemplate) { - this.loadBalancerClient = loadBalancerClient; + this.restTemplate = restTemplate; }
-
Next, inside our
getGreeting()
method, we can remove the line that explicitly instantiates aRestTemplate
and use the Autowired instance in its place. Furthermore, instead of callingfetchFortuneServiceUrl()
, we use a clever String parameter that looks like a url, but that is in fact a template or placeholder for the actual url that Ribbon will derive from it:@RequestMapping("/") String getGreeting(Model model) { logger.debug("Adding greeting"); model.addAttribute("msg", "Greetings!!!"); - RestTemplate restTemplate = new RestTemplate(); - String fortune = restTemplate.getForObject(fetchFortuneServiceUrl(), String.class); + String fortune = restTemplate.getForObject("http://fortune-service", String.class); logger.debug("Adding fortune: {}", fortune); model.addAttribute("fortune", fortune); return "greeting"; // resolves to the greeting.ftl template }
Basically
http://fortune-service
references the name of the application we wish to invoke. Spring Cloud Netflix automatically parses the string, looks up the application name against a locally cached copy of the Eureka registry, and asks Ribbon to choose an instance, whose url is then inserted in the place of the application name. -
We can now delete the unused method
fetchFortuneServiceUrl()
A reference implementation of greeting-ribbon refactored as described above is available under the project named greeting-ribbon-rest .
|
Let’s take our refactored implementation for a spin:
$ cd greeting-ribbon
$ mvn clean spring-boot:run
-
After the a few moments, check the
service-registry
dashboard at http://localhost:8761. Confirm thegreeting-ribbon
app is registered once more. -
Browse to http://localhost:8080/ to the
greeting-ribbon
application. Confirm you are seeing fortunes. Refresh as desired. Review the terminal output for thegreeting-ribbon
app. -
More interestingly, review the logs for the two
fortune-service
applications. With each refresh ofgreeting-ribbon
, one of these two will serve the response, in alternating fashion.
5. Retrying Failed Requests
Conduct the following experiment:
-
Make sure you have two instances of
fortune-service
running -
Visit the greeting application, and again, verify that requests round-robin between the two instances
-
Stop or kill one of the two
fortune-service
instances -
Continue making requests to the greeting application
One of two requests simply fails.
If we wait long enough for Eureka to remove the insance from the list (since it won’t be receiving heartbeats any longer), and for the greeting application to fetch an updated registry, this matter will resolve itself.
Given that a second instance is available to handle these requests, it’s preferable if Ribbon were to just retry the request against the surviving instance.
Let’s rectify this problem.
Netflix Ribbon supports retrying failed requests. The Ribbon GitHub Wiki cites configuration properties such as MaxAutoRetries
and MaxAutoRetriesNextServer
that govern this behavior. The Spring Cloud team supports this capability via the Spring Retry project. Ryan Baxter, a member of the Spring Cloud team, discusses this integration work in this blog (so does the Spring Cloud Netflix reference documentation, for that matter).
➜ Add the Spring Retry dependency to pom file for greeting-ribbon
:
@@ -41,6 +41,10 @@
<groupId>io.pivotal.spring.cloud</groupId>
<artifactId>spring-cloud-services-starter-service-registry</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.springframework.retry</groupId>
+ <artifactId>spring-retry</artifactId>
+ </dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId>
➜ Configure MaxAutoRetriesNextServer
for the ribbon client:
---
fortune-service:
ribbon:
MaxAutoRetriesNextServer: 1
This setting can be interpreted as if a request to one of the servers from the list fails, try another server.
➜ Also, configure the logging level for the package org.springframework.retry
to DEBUG
:
logging:
level:
org.springframework.retry: DEBUG
Finally, repeat the test:
-
Ensure two instances of
fortune-service
are running and registered with eureka. -
Start
greeting-ribbon
, and ensure that requests round-robin between the two instances. -
Take down one of the
fortune-service
instances. -
Access the greeting application a few more times: you should see no failures.
Also observe that the console output for greeting-ribbon
contains a log message from the Spring Retry library stating that the retry count was 1 (i.e. that it actually retried a request):
... o.s.retry.support.RetryTemplate : Retry: count=1 io.pivotal.greeting.GreetingController : Adding fortune: You can always find happiness at work on Friday io.pivotal.greeting.GreetingController : Adding greeting ...
6. Customize the Load Balancing Rule
The Ribbon API was designed to be flexible, with multiple interfaces whose implementations define its behavior:
-
IRule
: defines the rules that Ribbon uses for load balancing. The default isRoundRobinRule
-
IPing
: the mechanism Ribbon uses to ping services (to check if they are alive) -
ILoadBalancer
: a more general interface that includes how Ribbon obtains the list of servers to load-balance across
Out of the box, Ribbon provides multiple alternative implementations of IRule
. Our task here is to configure Ribbon to use the WeightedResponseTimeRule
instead of the default RoundRobinRule
.
We can simulate two instances of fortune-service
with different response times by using an artificial delay in the response for one of our two fortune-service
instances. The fortune service has been configured with the property (or environment variable) DELAY_MS
to allow us to do just that.
-
Start two instances of
fortune-service
once more, but this time, as follows:mvn spring-boot:run
..and:
Mac, Linux:
SERVER_PORT=8788 DELAY_MS=1000 mvn spring-boot:run
Windows:
$ set SERVER_PORT=8788 $ set DELAY_MS=1000 $ mvn spring-boot:run
At this point you can verify once more that you have two instances of
fortune-service
registered with Eureka, and that one of them takes longer to respond than the other. -
Refer to the following Spring Cloud Ribbon documentation which discusses how different aspects of Ribbon can be configured using properties.
-
In
greeting-ribbon
src/main/resources
folder, create theapplication.yml
file and edit it as follows:fortune-service: ribbon: NFLoadBalancerRuleClassName: com.netflix.loadbalancer.WeightedResponseTimeRule
Note above how a spring boot application can define and configure multiple independent Ribbon clients. Each client has a name, in this case corresponding to the service being called.
-
Start the
greeting-ribbon
application once more, and refresh the page to see Ribbon again load-balance requests across the twofortune-service
instances.-
You should begin to see log messages indicating that a background thread is running that computes updated response times for each instance.
-
Within a short time,
greeting-ribbon
will adapt to invoking the fasterfortune-service
instance in proportion to its response time; i.e. it will be invoked more often than the slower instance.
-
Congratulations! You’ve now configured Ribbon with a custom load balancing rule. You can now stop the config-server
, service-registry
, fortune-service
and greeting-ribbon
applications.
7. Deploy the greeting-ribbon
to PCF
7.1. PCF and the Eureka Registration Method
In the previous lab, we used the route
registration method. This had the effect of giving client applications the same url for all application instances. This setting defeats Ribbon, in that no matter which application instance we select, we end up with the same application url, which by definition will be routed through the PCF GoRouter.
In order to bypass the GoRouter and leverage Ribbon, the registration method has to be set to direct
.
When using the direct
registration method, all application instances register their internal IP address and port number with Eureka. This can be a problem in PCF however, for a number of reasons:
-
Cloud Foundry may not be configured for direct container-to-container networking,
-
Even with container-to-container networking enabled, platform security policy may not allow applications to call one another directly
A new feature of PCF version 1.10 known as "container-to-container" networking, provides a mechanism for an operator for applications to call each other directly by IP address, effectively bypassing the GoRouter.
A recent blog post by Chris Sterling gives a great overview of how this feature works together with Spring Cloud Services.
Assuming you’re working with an instance of Cloud Foundry that has direct networking enabled, let’s begin by modifying our application configuration:
-
In your
app-config
git repository, locateapplication.yml
-
Modify the value of the property
spring.cloud.services.registrationMethod
fromroute
todirect
-
Commit and push your changes back to your github repository
-
Finally, restart your running
fortune-service
instances, to cause them to re-register with eureka using their IP addresses:cf restart fortune-service
7.2. Deploy 'greeting-ribbon'
-
Package and push the
greeting-ribbon
application.$ mvn clean package $ cf push greeting-ribbon -p target/greeting-ribbon-0.0.1-SNAPSHOT.jar -m 768M --random-route --no-start
-
Bind services for the
greeting-ribbon
application.cf bind-service greeting-ribbon config-server
..and:
cf bind-service greeting-ribbon service-registry
You can safely ignore the TIP: Use 'cf restage' to ensure your env variable changes take effect message from the CLI. We don’t need to restage at this time.
-
Start the
greeting-ribbon
app.cf start greeting-ribbon
-
After the a few moments, check the
service-registry
. Confirm thegreeting-ribbon
app is registered.
Instead of deploying
Manifest files simplify the task of deploying an application, and a single |
Refresh the greeting-ribbon
root endpoint.
This should fail.
In a nutshell, you’ll need to invoke the following command to allow direct communication from the greeting-ribbon
application to the fortune-service
application:
cf add-network-policy greeting-ribbon --destination-app fortune-service --protocol tcp --port 8080
Now retry refreshing the greeting-ribbon
root endpoint. If fortune-service
is still scaled at three instances, you should see Ribbon load balance requests across these instances.