§The Play WS API
Sometimes we would like to call other HTTP services from within a Play application. Play supports this via its WS library, which provides a way to make asynchronous HTTP calls.
There are two important parts to using the WS API: making a request, and processing the response. We’ll discuss how to make both GET and POST HTTP requests first, and then show how to process the response from the WS. Finally, we’ll discuss some common use cases.
§Making a Request
To use WS, first add javaWs
to your build.sbt
file:
libraryDependencies ++= Seq(
javaWs
)
Now any controller or component that wants to use WS will have to add the following imports and then declare a dependency on the WSClient
:
import javax.inject.Inject;
import play.mvc.*;
import play.libs.ws.*;
import java.util.concurrent.CompletionStage;
public class Application extends Controller {
@Inject WSClient ws;
// ...
}
To build an HTTP request, you start with ws.url()
to specify the URL.
WSRequest request = ws.url("http://example.com");
This returns a WSRequest
that you can use to specify various HTTP options, such as setting headers. You can chain calls together to construct complex requests.
WSRequest complexRequest = request.setHeader("headerKey", "headerValue")
.setRequestTimeout(1000)
.setQueryParameter("paramKey", "paramValue");
You end by calling a method corresponding to the HTTP method you want to use. This ends the chain, and uses all the options defined on the built request in the WSRequest
.
CompletionStage<WSResponse> responsePromise = complexRequest.get();
This returns a CompletionStage<WSResponse>
where the WSResponse
contains the data returned from the server.
§Request with authentication
If you need to use HTTP authentication, you can specify it in the builder, using a username, password, and an WSAuthScheme
. Options for the WSAuthScheme
are BASIC
, DIGEST
, KERBEROS
, NONE
, NTLM
, and SPNEGO
.
ws.url(url).setAuth("user", "password", WSAuthScheme.BASIC).get();
§Request with follow redirects
If an HTTP call results in a 302 or a 301 redirect, you can automatically follow the redirect without having to make another call.
ws.url(url).setFollowRedirects(true).get();
§Request with query parameters
You can specify query parameters for a request.
ws.url(url).setQueryParameter("paramKey", "paramValue");
§Request with additional headers
ws.url(url).setHeader("headerKey", "headerValue").get();
For example, if you are sending plain text in a particular format, you may want to define the content type explicitly.
ws.url(url).setHeader("Content-Type", "application/json").post(jsonString);
// OR
ws.url(url).setContentType("application/json").post(jsonString);
§Request with timeout
If you wish to specify a request timeout, you can use setRequestTimeout
to set a value in milliseconds. A value of -1
can be used to set an infinite timeout.
ws.url(url).setRequestTimeout(1000).get();
§Submitting form data
To post url-form-encoded data you can set the proper header and formatted data.
ws.url(url).setContentType("application/x-www-form-urlencoded")
.post("key1=value1&key2=value2");
§Submitting JSON data
The easiest way to post JSON data is to use the JSON library.
import com.fasterxml.jackson.databind.JsonNode;
import play.libs.Json;
JsonNode json = Json.newObject()
.put("key1", "value1")
.put("key2", "value2");
ws.url(url).post(json);
§Streaming data
It’s also possible to stream data.
Here is an example showing how you could stream a large image to a different endpoint for further processing:
CompletionStage<WSResponse> wsResponse = ws.url(url).setBody(largeImage).execute("PUT");
The largeImage
in the code snippet above is an Akka Streams Source<ByteString, ?>
.
§Processing the Response
Working with the WSResponse
is done by applying transformations such as thenApply
and thenCompose
to the CompletionStage
.
§Processing a response as JSON
You can process the response as a JsonNode
by calling response.asJson()
.
CompletionStage<JsonNode> jsonPromise = ws.url(url).get()
.thenApply(WSResponse::asJson);
§Processing a response as XML
Similarly, you can process the response as XML by calling response.asXml()
.
CompletionStage<Document> documentPromise = ws.url(url).get()
.thenApply(WSResponse::asXml);
§Processing large responses
Calling get()
, post()
or execute()
will cause the body of the response to be loaded into memory before the response is made available. When you are downloading a large, multi-gigabyte file, this may result in unwelcomed garbage collection or even out of memory errors.
WS
lets you consume the response’s body incrementally by using an Akka Streams Sink
. The stream()
method on WSRequest
returns a CompletionStage<StreamedResponse>
. A StreamedResponse
is a simple container holding together the response’s headers and body.
Any controller or component that wants to levearge the WS streaming functionality will have to add the following imports and dependencies:
import javax.inject.Inject;
import akka.stream.Materializer;
import akka.stream.javadsl.*;
import akka.util.ByteString;
import play.mvc.*;
import play.libs.ws.*;
import play.libs.F.Promise;
import scala.compat.java8.FutureConverters;
public class MyController extends Controller {
@Inject WSClient ws;
@Inject Materializer materializer;
// ...
}
Here is a trivial example that uses a folding Sink
to count the number of bytes returned by the response:
// Make the request
CompletionStage<StreamedResponse> futureResponse =
ws.url(url).setMethod("GET").stream();
CompletionStage<Long> bytesReturned = futureResponse.thenCompose(res -> {
Source<ByteString, ?> responseBody = res.getBody();
// Count the number of bytes returned
Sink<ByteString, scala.concurrent.Future<Long>> bytesSum =
Sink.fold(0L, (total, bytes) -> total + bytes.length());
// Converts the materialized Scala Future into a Java8 `CompletionStage`
Sink<ByteString, CompletionStage<Long>> convertedBytesSum =
bytesSum.mapMaterializedValue(FutureConverters::toJava);
return responseBody.runWith(convertedBytesSum, materializer);
});
Alternatively, you could also stream the body out to another location. For example, a file:
File file = File.createTempFile("stream-to-file-", ".txt");
FileOutputStream outputStream = new FileOutputStream(file);
// Make the request
CompletionStage<StreamedResponse> futureResponse =
ws.url(url).setMethod("GET").stream();
CompletionStage<File> downloadedFile = futureResponse.thenCompose(res -> {
Source<ByteString, ?> responseBody = res.getBody();
// The sink that writes to the output stream
Sink<ByteString,scala.concurrent.Future<scala.runtime.BoxedUnit>> outputWriter =
Sink.<ByteString>foreach(bytes -> outputStream.write(bytes.toArray()));
// Converts the materialized Scala Future into a Java8 `CompletionStage`
Sink<ByteString, CompletionStage<?>> convertedOutputWriter =
outputWriter.mapMaterializedValue(FutureConverters::toJava);
// materialize and run the stream
CompletionStage<File> result = responseBody.runWith(convertedOutputWriter, materializer)
.whenComplete((value, error) -> {
// Close the output stream whether there was an error or not
try { outputStream.close(); }
catch(IOException e) {}
})
.thenApply(v -> file);
return result;
});
Another common destination for response bodies is to stream them back from a controller’s Action
:
// Make the request
CompletionStage<StreamedResponse> futureResponse = ws.url(url).setMethod("GET").stream();
CompletionStage<Result> result = futureResponse.thenApply(response -> {
WSResponseHeaders responseHeaders = response.getHeaders();
Source<ByteString, ?> body = response.getBody();
// Check that the response was successful
if (responseHeaders.getStatus() == 200) {
// Get the content type
String contentType =
Optional.ofNullable(responseHeaders.getHeaders().get("Content-Type"))
.map(contentTypes -> contentTypes.get(0)).
orElse("application/octet-stream");
// If there's a content length, send that, otherwise return the body chunked
Optional<String> contentLength = Optional.ofNullable(responseHeaders.getHeaders()
.get("Content-Length"))
.map(contentLengths -> contentLengths.get(0));
if (contentLength.isPresent()) {
return ok().sendEntity(new HttpEntity.Streamed(
body,
Optional.of(Long.parseLong(contentLength.get())),
Optional.of(contentType)
));
} else {
return ok().chunked(body).as(contentType);
}
} else {
return new Result(Status.BAD_GATEWAY);
}
});
As you may have noticed, before calling stream()
we need to set the HTTP method to use by calling setMethod
on the request. Here follows another example that uses PUT
instead of GET
:
CompletionStage<StreamedResponse> futureResponse =
ws.url(url).setMethod("PUT").setBody("some body").stream();
Of course, you can use any other valid HTTP verb.
§Common Patterns and Use Cases
§Chaining WS calls
You can chain WS calls by using flatMap
.
final CompletionStage<WSResponse> responseThreePromise = ws.url(urlOne).get()
.thenCompose(responseOne -> ws.url(responseOne.getBody()).get())
.thenCompose(responseTwo -> ws.url(responseTwo.getBody()).get());
§Exception recovery
If you want to recover from an exception in the call, you can use recover
or recoverWith
to substitute a response.
CompletionStage<WSResponse> responsePromise = ws.url("http://example.com").get();
CompletionStage<WSResponse> recoverPromise = responsePromise.handle((result, error) -> {
if (error != null) {
return ws.url("http://backup.example.com").get();
} else {
return CompletableFuture.completedFuture(result);
}
}).thenCompose(Function.identity());
§Using in a controller
You can map a CompletionStage<WSResponse>
to a CompletionStage<Result>
that can be handled directly by the Play server, using the asynchronous action pattern defined in Handling Asynchronous Results.
public CompletionStage<Result> index() {
return ws.url(feedUrl).get().thenApply(response ->
ok("Feed title: " + response.asJson().findPath("title").asText())
);
}
§Using WSClient
WSClient is a wrapper around the underlying AsyncHttpClient. It is useful for defining multiple clients with different profiles, or using a mock.
The default client can be called from the WSClient class:
WSClient client = WS.client();
You can instantiate a WSClient directly from code and use this for making requests. Note that you must follow a particular series of steps to use HTTPS correctly if you are defining a client directly:
import org.asynchttpclient.*;
import play.api.libs.ws.WSClientConfig;
import play.api.libs.ws.ning.NingWSClientConfig;
import play.api.libs.ws.ning.NingWSClientConfigFactory;
import play.api.libs.ws.ssl.SSLConfigFactory;
import play.api.libs.ws.ning.NingAsyncHttpClientConfigBuilder;
import scala.concurrent.duration.Duration;
import akka.stream.Materializer;
import akka.stream.javadsl.*;
import akka.util.ByteString;
// Set up the client config (you can also use a parser here):
scala.Option<String> noneString = scala.None$.empty();
WSClientConfig wsClientConfig = new WSClientConfig(
Duration.apply(120, TimeUnit.SECONDS), // connectionTimeout
Duration.apply(120, TimeUnit.SECONDS), // idleTimeout
Duration.apply(120, TimeUnit.SECONDS), // requestTimeout
true, // followRedirects
true, // useProxyProperties
noneString, // userAgent
true, // compressionEnabled / enforced
SSLConfigFactory.defaultConfig());
NingWSClientConfig clientConfig = NingWSClientConfigFactory.forClientConfig(wsClientConfig);
// Build a secure config out of the client config:
NingAsyncHttpClientConfigBuilder secureBuilder = new NingAsyncHttpClientConfigBuilder(clientConfig);
AsyncHttpClientConfig secureDefaults = secureBuilder.build();
// You can directly use the builder for specific options once you have secure TLS defaults...
AsyncHttpClientConfig customConfig = new AsyncHttpClientConfig.Builder(secureDefaults)
.setProxyServer(new org.asynchttpclient.proxy.ProxyServer("127.0.0.1", 38080))
.setCompressionEnforced(true)
.build();
WSClient customClient = new play.libs.ws.ning.NingWSClient(customConfig, materializer);
CompletionStage<WSResponse> responsePromise = customClient.url("http://example.com/feed").get();
NOTE: if you instantiate a NingWSClient object, it does not use the WS plugin system, and so will not be automatically closed in
Application.onStop
. Instead, the client must be manually shutdown usingclient.close()
when processing has completed. This will release the underlying ThreadPoolExecutor used by AsyncHttpClient. Failure to close the client may result in out of memory exceptions (especially if you are reloading an application frequently in development mode).
You can also get access to the underlying AsyncHttpClient
.
org.asynchttpclient.AsyncHttpClient underlyingClient =
(org.asynchttpclient.AsyncHttpClient) ws.getUnderlying();
This is important in a couple of cases. WS has a couple of limitations that require access to the client:
WS
does not support multi part form upload directly. You can use the underlying client with RequestBuilder.addBodyPart.WS
does not support streaming body upload. In this case, you should use theFeedableBodyGenerator
provided by AsyncHttpClient.
§Configuring WS
Use the following properties in application.conf
to configure the WS client:
play.ws.followRedirects
: Configures the client to follow 301 and 302 redirects (default is true).play.ws.useProxyProperties
: To use the system http proxy settings(http.proxyHost, http.proxyPort) (default is true).play.ws.useragent
: To configure the User-Agent header field.play.ws.compressionEnabled
: Set it to true to use gzip/deflater encoding (default is false).
§Timeouts
There are 3 different timeouts in WS. Reaching a timeout causes the WS request to interrupt.
play.ws.timeout.connection
: The maximum time to wait when connecting to the remote host (default is 120 seconds).play.ws.timeout.idle
: The maximum time the request can stay idle (connection is established but waiting for more data) (default is 120 seconds).play.ws.timeout.request
: The total time you accept a request to take (it will be interrupted even if the remote host is still sending data) (default is 120 seconds).
The request timeout can be overridden for a specific connection with setTimeout()
(see “Making a Request” section).
§Configuring WS with SSL
To configure WS for use with HTTP over SSL/TLS (HTTPS), please see Configuring WS SSL.
§Configuring AsyncClientConfig
The following advanced settings can be configured on the underlying AsyncHttpClientConfig.
Please refer to the AsyncHttpClientConfig Documentation for more information.
play.ws.ning.allowPoolingConnection
play.ws.ning.allowSslConnectionPool
play.ws.ning.ioThreadMultiplier
play.ws.ning.maxConnectionsPerHost
play.ws.ning.maxConnectionsTotal
play.ws.ning.maxConnectionLifeTime
play.ws.ning.idleConnectionInPoolTimeout
ws.ning.webSocketIdleTimeout
play.ws.ning.maxNumberOfRedirects
play.ws.ning.maxRequestRetry
play.ws.ning.removeQueryParamsOnRedirect
play.ws.ning.useRawUrl