I have a requirement to read and write compressed (GZIP) streams without intermediate storage. Currently, I’m using Spring
RestTemplate to do the writing, and Apache HTTP client to do the reading (see my answer here for an explanation of why
RestTemplate can’t be used for reading large streams). The implementation is fairly straightforward, where I slap a
GZIPInputStream on the response
InputStream and move on.
Now, I’d like to switch to using Spring 5 WebClient (just because I’m not a fan of status quo). However,
WebClient is reactive in nature and deals with
Flux<Stuff>; I believe it’s possible to get a
Flux<DataBuffer>, where DataBuffer is an abstraction over
ByteBuffer. Question is, how do I decompress it on the fly without having to store the full stream in memory (
OutOfMemoryError, I’m looking at you), or writing to local disk? It’s worth mentioning that
WebClient uses Netty under the hood.
I’ll admit to not knowing much about (de)compression, however, I did do my research, but none of the material available online seemed particularly helpful.
Also related to Spring integration issue 2300