Top 3 Projects for Java Concurrency
The threading model is important for achieving high performance and responsiveness for your application, but getting it right is a challenge. In this article we’ll explore my top 3 picks for toolkits and frameworks for concurrency and multithreading.
Multithreading helps us write software that can interact efficiently with the user and the IO devices such as the network, the disk, keyboard, mouse and others. Most computers today have multiple cores. Multithreading allows us to utilize the full performance capabilities of our computer by running tasks completely in parallel on multiple cores in the same time.
A correct multithreading model for your particular application can improve performance by an order of magnitude, and make it much easier to scale your code and keep it maintainable.
Building the multithreading boiler plate code, for every single application is tedious and error prone, and keeping it reusable for future projects is a full time job. That is why some people made it their full time job, and created some amazing, reusable projects so YOU can use your multithreading knowledge to pick the right technology and decide how to use it, rather than writing your own solution every single time.
Let’s look at the top 3 innovative projects that made software concurrency easy for all of us.
1. Vert.x
Based on Netty, Vert.x is one of the most versatile toolkits for writing high performance multithreaded code. It is both easy to use and completely non-opinionated.
It doesn’t put your code into a frame (therefore it is not a framework), but rather provides you with many easy, non intrusive ways to write your multithreaded code.
Vert.x comes with separate modules for anything you need, and is based and relies on the core module for everything.
A few examples are
Vert.x Web and Vert.x Web Client modules for HTTP server and client application development.
MongoDB client and Redis Client for data access.Reactive Streams for back pressure and working with streams of data.
Vert.x Service Discovery and Vert.x Circuit Breaker for managing communication between microservices
And many other
The main concept of Vert.x is the Verticle which is essentially an event loop running on a single thread. An event loop is an infinite loop that
Checks for IO events coming from the OS (in most operating systems it’s done in constant time)
Checks for our own scheduled events’ completion (such as timers),
Runs all the callback functions registered for incoming events.
And goes to the next iteration
For those familiar with NodeJS, this is exactly the same concept. This is called the Reactor Model.
The second main concept Vert.x is advocating for is completely asynchronous, non blocking programming.
Essentially every time you catch yourself writing blocking code like Thread.sleep(1000) or networkDataStream.readLine() you are doing it wrong. (With respect to Vert.x)
In general the problem with writing blocking code is the thread becomes blocked and unusable until that method unblocks. If you still have data coming in from somewhere, you will end up creating more threads to handle that data, and then block those threads again. The more threads you create, the more management work your OS is doing, and less CPU left for us. (Thrashing)
Vert.x provides us with all the tools to write non blocking code and avoid such a situation. So you will need to create only a handful of threads and no more.
For example instead of Thread.sleep(1000) , in Vert.x you write:
long delay = 1000; long timerId = vertx.setTimer(delay, timerId -> { System.out.println(“This code is triggered after 1000 ms”); });
This way you simply register a callback to be executed after the delay has passed, but the thread is never blocked.
If you write all your code in a non blocking, asynchronous way, all you need is a small number of threads, each thread with a single Verticle (Event loop). That pattern is called a Multi Reactor.
Creating Verticle threads with Vert.x is easy.
All you need to do is
Extend the abstractVerticle class.
Put your code in the start() method
Deploy your verticle.
public class CustomVerticle extends AbstractVerticle { // This is called when the verticle is deployed public void start() { //Your code goes here } } ... Vertx vertx = Vertx.vertx(); CustomVerticle customVerticle = new CustomVerticle(); vertx.deployVerticle(customVerticle);
Creating an HTTP server that runs on 8 threads is just as easy:
public class WebServerVertcile extends AbstractVerticle { public void start() { HttpServer server = getVertx().createHttpServer(); server.requestHandler(request -> { // Code that handles the requests }); server.listen(8080); } } ... Vertx vertx = Vertx.vertx(); DeploymentOptions options = new DeploymentOptions().setInstances(8); vertx.deployVerticle("WebServerVerticle", options);
That’s it. That is all the code you need!!
Vert.x also supports RX Java, Fibers, programming using the Actor Model and many other ways for you to write your multithreaded code. If you still need to write blocking code, Vert.x provides with ways to do that too.
Supported languages: Java, JavaScript, Groovy, Ruby, Ceylon, Scala and Kotlin.
2. LMAX Disruptor
LMAX Disruptor comes from LMAX Exchange which is a foreign exchange trading facility, based in London. To achieve low level and high throughput, so important for financial trading, LMAX focused on the area of optimizing the queues which pass data in between stages of the system.
Although created initially for financial trading, LMAX Disruptor aims to be a general purpose framework for solving the concurrency problem.
LMAX Disruptor is extremely cache efficient and lock free, which comes with very high performance benefits.
Here are some impressive performance numbers.
In the heart of LMAX Disruptor is an extremely hardware efficient Ring Buffer, which serves as the vehicle for moving events from producers to consumers.
To pass an event from a producer to consumers you need to claim a sequence number in the Ring Buffer. After claiming the sequence number you produce the desired event and publish it to the consumers:
Lets start with creating the producer and consumer classes:
Producer:
public class CustomEventProducer { private final RingBuffer<CustomEvent> ringBuffer; public CustomEventProducer(RingBuffer<CustomEvent> ringBuffer) { this.ringBuffer = ringBuffer; } public void publishDataToConsumer(String data) { long sequence = ringBuffer.next(); try { CustomEvent event = ringBuffer.get(sequence); event.set(data); } finally { ringBuffer.publish(sequence); } } }
Consumer:
public class CustomEventProducer { private final RingBuffer<CustomEvent> ringBuffer; public CustomEventProducer(RingBuffer<CustomEvent> ringBuffer) { this.ringBuffer = ringBuffer; } public void publishDataToConsumer(String data) { long sequence = ringBuffer.next(); try { CustomEvent event = ringBuffer.get(sequence); event.set(data); } finally { ringBuffer.publish(sequence); } } }
Unlike Vert.x that uses the Netty event loop and threading model, LMAX Disruptor is using the general Java Executors for creating the threads.
You could create a simple executor like so
Executor executor = Executors.newCachedThreadPool();
And pass that executor into the LMAX Disruptor
Disruptor<CustomEvent> disruptor = new Disruptor<>(CustomEvent::new, BUFFER_SIZE, executor);
Now all we need to do is create a producer and a consumer and pass the consumer into the disruptor.
RingBuffer<CustomEvent> ringBuffer = disruptor.getRingBuffer(); CustomEventProducer producer = new CustomEventProducer(ringBuffer); CustomEventConsumer consumer = new CustomEventConsumer(); disruptor.handleEventsWith(consumer);
Start the disruptor and produce data
disruptor.start(); producer.produce("Some piece of Data");
That’s it!
If your application is designed to be performed in stages, such as a transformation pipeline, LMAX Disruptor will do it the most efficiently for you, and it leaves the threading boiler plate to the Java standard Executors which comes in many flavors and configurations.
Supported languages: Java, .Net (Port of LMAX Disruptor)
3. Akka
Akka aims to make it easy for developers to write concurrent, message driven applications and is based on the Actor Model.
In that model you decouple your code into actors that maintain state and act on your data. The actors communicate with each other through messages that they send to each other’s “mailbox”.
The actors are arranged in a hierarchical structure where each actor has exactly one supervisor. And the overall master actor takes the original work and splits it among its children/worker actors. This way each actor is responsible for only one task. And if the actor fails to perform that task, he can asks his supervisor for help.
Similarly to Vert.x, Akka’s actors should not block the thread.
In addition, actors should not pass any mutable objects between each other. This way no synchronization between threads is needed, and each actor thread can operate on its data without contention with other threads.
Unlike Vert.x and LMAX Disruptor that aim to provide the developers with control over threading to achieve the desired performance, Akka is aiming to abstract the multithreading from the developer almost entirely, making it easy to focus on the logic and stay worry free about concurrency as long as you follow Akka’s best practices.
To get the ball rolling with Akka, more boiler plate code is needed than with LMAX Disruptor or Vert.x, but the idea is pretty straightforward.
Every actor has to extend the UntypedActor class and implement the public void onReceive(Object message) method which is called when an actor receives a message.
An example for a typical actor:
public class WorkerActor extends UntypedActor { @Override public void onReceive(Object message) { //handle message and send a message to another actor } }
Besides creating the actor classes, we also need to define the message class for every actor to actor communication.
So for example if we have a master actor and a worker actor, and the master actor wants to pass a task to the worker, and get a result back, the general structure would be like this:
public class MasterToWorkerMessage { private final String message; public MasterToWorkerMessage(String message) { this.message = message; } public String getMessage() { return message; } } public class WorkerToMasterMessage { private final String result; public MasterToWorkerMessage(String result) { this.result = result; } public String getResult() { return result; } }
And the worker class would like like this
public class WorkerActor extends UntypedActor { ... @Override public void onReceive(Object message) { if(message instanceof MasterToWorkerMessage) { MasterToWorkerMessage work = (MasterToWorkerMessage)message; String result = calculateResult(work.getMessage()); getSender().tell(new WorkerToMasterMessage(result), getSelf()); } else { unhandled(message); } } private String calculateResult(String message) { ... } }
The master class has a little more boiler plate code to set up the router and configure how many children workers to create and how many message to send to each child at a time, but the idea is very simple.
For more information about how to create actors in Java you can follow the official example.
Just like Vert.x, Akka also provides separate modules such as
Akka HTTP for everything relating to building HTTP servers.
Akka Streams for back pressure and working with streams of data
And many other modules for persistence, cluster management, and others
Supported languages : Scala, Java
It's important to note that using either of those projects is not a substitute to knowing core Java Multithreading and Concurrency. And without that knowledge you may easily introduce data races, performance bottlenecks and many other issues so hard to debug and troubleshoot. To master those topics I recommend taking the Java Multithreading, Concurrency & Performance Optimization course which covers all those topics in depth with strong emphasis on high performance and best practices.
Summary
There is no magic bullet and no one framework or toolkit to solve all problems. As software engineers and architects we need to learn the requirements of our system and use our judgment to decide which technology to use and when.
Vert.x is an excellent choice when your application is IO bound and you need to have full control over your threading model.
Lmax Disruptor is most efficient in message passing between stages of your application.
Akka provides great abstraction based on the actor model and immutability, to free developers from dealing with concurrency issues in the first place.
