I work on a microservices based REST application. On the overview, I am facing an issue where in a service calls another service over HTTP and the code proceeds in the calling service before the response is received.
- Let there be service X in java, Y in node.js and Z in java.
- The flow in the application is that there is a button on UI which calls an API in X. X does some work and then calls API in Y. Y in turn calls syncronously an API in Z. The responses travel back from Z->Y->X.
- There is no multithreaded scenario or asynchronous calls anywhere in the path.
- The HTTP client is in-house and is common for all the services but this issue is only happening for a particular api.
The normal flow where the button is pressed once works fine. The problem arises when the button on the UI is presses twice very quickly such that the X’s API gets called 2 times almost simultaneously.
In the above case, X’s API does its work for both the calls, then calls Y’s API but in the first call from X to Y, X does not wait for Y’s API response, though Y does send the response (as verified from the logs). The second call from X to Y gets perfect response as expected. This happens very randomly but the frequency of this happening is 5/100 calls in production.
What are the possibles causes for the scenario and how should I fix it?