Dealing with Elasticsearch concurrency

Using Elasticsearch has a lot of advantages and as everything in life also disadvantages when compared to a more traditional SQL relational database. One of the disadvantage is that the client has to deal with the control of concurrency when the same resource (or document if we use Elasticsearch terminology) is being modified at the same time by different processes.

If you are using Elasticsearch as your main data repository, sooner or later you will face the concurrency problem of different processes writing (changing) the same document at the same time. In this post I am going to explain the approach we took to solve the problem.

Actually this is the typical problem of concurrent programming, known as “Race Condition”. Lets try to explain the problem as easy as possible:

  • We have document A stored in Elasticsearch
  • Process P1 reads the document A (gets a copy of document A) at time T1
  • Process P2 reads the document A (gets a copy of document A) at time T2
  • Process P1 applies some transformation to the copy of document A, lets call it A1
  • Process P2 applies some transformation to the copy of document A, lets call it A2
  • Now P2 decides to send back document A2 to Elasticsearch at time T5
  • Notice that Elasticsearch has now document A2 (the new version created by P2)
  • Now P1 decides to send back document A1 to Elasticsearch at time T6
  • Now Elasticsearch has document A1, with the changes produced by P1 but changes created by P2 are lost, that is the problem !!!

There are different solutions to this problem, but all of them will always impact the overall performance in terms of writing. The solution we took was good enough for our needs, and was basically the following:

  • Process P1 reads the document A  (gets a copy of document A) and stores its version at time T1
  • Process P2 reads the document A (gets a copy of document A) and stores its version at time T2
  • Process P1 applies some transformation to the copy of document A, lets call it A1
  • Process P2 applies some transformation to the copy of document A, lets call it A2
  • Now P2 decides to send back document A2 to Elasticsearch, passing the document version stored at time T2
  • Elasticsearch accepts the new document since it still has the same version. Notice that Elasticsearch has now document A2 (the new version created by P2), and the version has been increased
  • Now P1 decides to send back document A1 to Elasticsearch passing the document version stored at time T1
  • Now Elasticsearch rejects the passed document, since the passed version is older than the current stored by Elasticsearch.
  • P1 has to read the document A again from Elasticsearch (with P2 modification), apply the transformation again and send it back again. This time, nobody has modified the document in the meantime so the document version has not been increased and Elasticsearch accepts the new document.

This approach is basically an optimistic lock, in which the document is never locked, but the clients have to give the document version when sending the modified document. If the document hasn’t change in the meanwhile the update succeed, otherwise Elasticsearch denies the update and the client has to read the document again, re-apply the logic and send it back again.

I wrote a simple Java application in order to simulate the problem and the solution. You can download to play with it from here.

 

Leave a reply:

Your email address will not be published.

Site Footer