A MicroService WebSocket Chat

This article describes how to create a high performing chat application that functions as a MicroService. Within this MicroService, I will be using Caucho Technology’s Baratine ( v 1.0) framework to demonstrate the following:

  1. Building .jars not .wars
  2. Asynchronous nonblocking methods
  3. Pub/Sub functionality
  4. Performance & Scalability
  5. API/interface based programming
  6. Single-threaded frameworks as a solution for multi-threaded concurrency issues

For reference, the most recent dzone article that detailed a WebSocket chatroom can be found here https://dzone.com/articles/creating-websocket-chat. While the article was a great starting point for people to get familiar with WebSockets, this chat application will demonstrate how open source framework, in particular Baratine, can help developers implement the best current known practices in the industry.

Running the example

To run the example, do the following:

  1. Ensure that you have installed the gradle build tool
  2. Clone the repository from: https://github.com/swily/WebSocketChat.git
  3. Run gradle build
  4. Run gradle run
  5. Open your browser and navigate to localhost:8080

Overview of Files

While the application includes a few POJO classes, majority of the work is done in the following 3 files:

index.js / index.html – Displays chat messages and chatroom

ChatWebSocket.java – Listens to the websocket wire, connects the PubSub message pipes

ChatService.java – Support for long-polling, starts the Baratine server

ChatWebSocket.java

ChatWebSocket relies on Baratine’s implementation of the WebSocket protocol. In order to have access to the WebSocket wire, we implement ServiceWebSocket with our parameterized type Message. Message is our own defined class of what we expect to be coming across the WebSocket wire.

Beyond listening to the WebSocket wire, we also set up our PubSub messaging inside of Baratine. PubSub in Baratine has three components

  1. A PipeBroker
  2. A Pipe Subscriber (PipeSub)
  3. A Pipe Publisher (PipePub)

You’ll notice that we also make use of a _messagePipeHandler to ensure that we can properly implement “close” functionality when a user leaves the chatroom.

Setting up of the pipe is done in lines 88-93 in the following order:

  1. Create a PipeSub endpoint to tell when messages should flow out of the pipe, in our case I want these messages passed to my OnPipeReceive method. I utilize a lambda to achieve this.
  2. I subscribe the instance of my PipeBroker to the PipeSub endpoint.
  3. I create a handler to close the pipe when the connection is closed.

Here is the setup of the pipe:

Screen Shot 2016-08-05 at 3.56.47 PM

ChatService.java

ChatService is responsible for creating ChatMessages and passing them into the pipe. The methods are fairly straightforward, however scanning the class three important pieces might jump out:

  1. Annotations such as @Query on the send method provide long-polling fallback for http. Methods can be called directly over WebSocket by clients, or accessed over HTTP.
  2. Our main method uses the Baratine specific (Web.include(foo.class)) in order to provide the Baratine server with access to these classes.
  3. Baratine’s Result is used as an asynchronous callback so that methods do not block. Instead, a method finishing processing when given the .ok() call.

We start our Baratine web server with a simple Web.start() method as such:

Screen Shot 2016-08-05 at 3.57.09 PM

Performance

This MicroService was benchmarked on a MacBook Air Core i7 3667U with the following results

20 million messages/second/instance with no acknowledge messages

9 million messages/second/instance with acknowledge messages

6 million messages/second/instance with full ping-pong acknowledgement on every message

Baratine is able to achieve these results due to its automatic batching and underlying single-threaded implementation. What’s more is that the @Service annotation on a class provides a lock free and ordered inbox for processing messages on a service. Because of this, Baratine does not need to have synchronized code blocks for concurrency; data is only ever accessed by a single service owning thread, thus making dreaded concurrent access a thing of the past.

In fact, because of this improved underlying programming model, Baratine can persist data to internal data stores without the need for a data schema or an outside data source. Baratine does have MySQL drivers for the database you have already licensed, but many applications and services can rely on operating with their data in-memory and persisting to Baratine’s internal data store when needed.

Conclusion

Baratine 1.0 is officially released under the GPL license and offers a significant improvement over many of the issues that plague and prevent agile development in today’s programming world. With Baratine, developers can simply code an API and know that the implementation will be ready to be consumed by today’s IoT clients from any language (JSON permitting). This application can be embedded into an existing Java EE project, or used as a standalone service. This is the promise of MicroServices as features to an application can be added and removed without an overall impact to the current architecture.

While you may or may not need a specific chat implementation added to your current web app, I am hoping that people will take away the structure of the underlying architecture in this example application and apply it for their specific use case. Feel free to fork the github and continue the development!

Implementing a BookStore as a MicroService in 40 Lines of Code

MicroServices are the new trend in web service development. Much has been written about their architecture and benefits, but when it comes to actual implementation, many developers are using the same underlying technologies that they have used before.

In this post, we will look at an implementation of the common “Bookstore” application as a single MicroService. We’ll discuss the architectural benefits & performance the MicroService coded in Baratine presents compared to a JAX-RS implementation.

In doing so, we put forth the idea that if developers are still using similar multithreaded components to build with, they are in fact building monolithic solutions with the same inherently flawed monolithic models, just on a smaller scale.

Part 1: The service

Our Baratine Bookstore is only 40 lines of code in its entirety, so let’s take a look at the entire application:

Bookstore Baratine

  1. @Service creates a single-threaded service. Because it is single-threaded, we do not need synchronization and can use a simpleHashMap.
  2. @Get(“/books”) maps the method to a GET request at the URL “/books”.
  3. Then result.ok() completes the asynchronous request.

That’s all there is to it. The service is fully self contained, and because we used Baratine’s Result, all of our methods are nonblocking (do not tie up threads in the background) and can be called concurrently. It’s also worthwhile to point out this API can be consumed by a client of any language that can understand JSON.

Now let’s take a look at JAX-RS:

JAX_RS2

JAX_RS

web.xml:

JAX_RS3

  1. JAX-RS requires three files: an application class, a path class, and an empty web.xml.
  2. It must use a ConcurrentHashMap because the application is multi-threaded.
  3. It must be deployed in a container like Tomcat or JBoss.

In terms of complexity, JAX-RS is more complicated than Baratine (albeit more configurable). It requires at least 3 files, a servlet container, and the developer needs to mind concurrency issues.

Performance:

Even though Baratine is single-threaded, it easily outperforms JAX-RS (RESTEasy) by over 2x. You can scale out Baratine to take advantage of all the CPU cores on your machine and Baratine will scale up linearly – something that you cannot do with JAX-RS.

Persistence:

Our bookstore example stores the data in memory. What about saving it to a database? Persistence is out of the scope of JAX-RS and it forces you to use another library like JPA. With Baratine, it comes with a document-style persistence layer. No additional libraries and only a trivial change to our bookstore:

BookstorePersist

There are only two changes to our bookstore:

  1. @Data added to the class, to tell Baratine to save this object into its internal reactive database.
  2. @Modify on addBook() to tell Baratine that this object has been modified and that Baratine should add it to the save queue.

Even though the new bookstore is persistent, we still get the same high performance as before because the persistence is asynchronous and bookstore operates mostly in-memory. That is the beauty of Baratine and only possible because Baratine is reactive. There are no comparable platforms out there.

If you would like to expand this application, take note of the following Baratine maven dependency that was used:

mvn

Baratine vs Akka

Akka vs Baratine (Benchmark Included!)

Philosophy:

To understand the differences in programming model between the two, you first need to understand the purpose of why Caucho has spent so much time creating Baratine in the first place.

Concurrency, parallelism, or something more?

The principles of SOA have always been sound. They have revolved around the ideas of:

• Explicit boundaries
• Shared contract and Schema, not class
• Policy-driven
• Loosely-coupled
• Vendor independent
• and more

While these principles should have led to very agile programming, the reality is that the implementation has never able to meet these requirements. The complexity of data modeling and business requirements forced data to be abstracted away from business logic operations and placed into separate entities: databases. The resulting databases struggled to maintain high throughout and were often the cause of many bottlenecks within distributed applications. While distributed caches were able to provide faster content, they also introduced complex cache invalidation logic into an application. Besides the numerous components that make up a software stack, underlying frameworks have also been a point of contention with SOA. Multithreading throughout different portions of the application have been the cause of many thread deadlocks, starvation, & race conditions.

We believe that many of these misfortunes were cause by a fundamental flaw within the SOA model. By failing to address data locality and thread execution as principles that must be addressed within a SOA service, many implementations functioned as freeways without lanes. That is, the ability to block or break an application was available by any portion of the application as threads were free to operate outside of a given “lane” of execution.

An Evolution of SOA?

Baratine is a reworked solution, from the thread and data level up. It is designed as a framework that does not have any inherit bottlenecks or limitations when accessing data. Baratine is able to do this because it defines an abstraction level that merges thread execution with the service data. This allows services to call out to other services through serialized message passing, allowing for greater thread affinity within an application. The result is that these services can then be scaled independently while serving millions of requests per second per instance.

What’s more is that Baratine abstracts away the low level details including sharding, batching, message construction, and synchronization providing developers with an API and Object Oriented framework they are familiar with without having ever programmed in it directly. When programming in Baratine, developers are tasked with writing method calls to coordinate business logic throughout an application. This is precisely the object-oriented programming they have been doing for years, just not on a framework that allows groups of methods to function at the millions of operations/sec.

Baratine is not Akka… Akka is not Baratine

Akka handles distributed concurrency through its unique thread abstraction utilizing actors as passing messaging queues. It turns everything asynchronous, allowing for actors to route requests, parallel the execution of the requests. Great for execution, and can answer using futures and observables to decide what action should be taken.

Actors do not merge the persistence/locality of data and the execution of that data into a single entity. Untyped actors mean message types are not being enforced around the system, which results in hard to trace callbacks of failed services. Most developers are using actors to move data through an execution pipeline very fast while integrating an outside entity like Spark for managing the analytics/processing of the data. The execution of threads is handled by Akka at cost of pushing data into a separate entity.

Baratine manages both. This allows for coding that stays at the method level abstraction, has a cleaner handle on threads, and introduces no integration points of contention or failure.

Both models are looking to answer: how can we get more out of distributed concurrent programming? We know that greater concurrency will allow us to have more portions of our application functioning at once, so how can we handle this? Currently, you can try managing your own thread pools, futures, lambdas, etc, but still be stuck with an underlying framework that will use blocking techniques. Baratine and Akka have the ability to free you from a blocking framework… but at what cost? The cost is complexity.

Because Baratine addresses the data locality (including persistence & transactions) as well as the thread execution (batching & synchronization) the amount of code in a full blown application in Baratine is significantly less than Akka. The level of understanding is also much higher: a junior developer can comb through the code of a Baratine program and begin contributing services immediately.

The costs are where the fundamental differences within Baratine and Akka appear. Although both models are unfamiliar to developers, if you have built a object oriented program in Java or any given language before, you already know 90% of the Baratine framework.

Performance Matters!

Because the two frameworks are attempting to provide solutions for the same space, we thought it important to ensure high performance at the base level implementation. We benchmarked Akka and Baratine in a batching message passing service within a single JVM & within a local machine. The results we achieved are as follows:

within jvm               Akka               baratine
batch-size-1           1.0m               2.9m
batch-size-4           2.2m              3.7m
batch-size-16         2.6m              4.8m
batch-size-64        2.7m              5.2m

within local machine
batch-size-1           4.5k                 12.5k
batch-size-4           9.0k                43k
batch-size-16          11k-13k         120k
batch-size-64         13k-15k         230k

Our results showed an impressive 2-3x performance advantage held by Baratine over Akka! While message passing is a fundamental focus of Akka, Baratine handles message passing for you ( as batched method calls ), making it not just easier to build a simple messaging service, but an entire asynchronous applications such as the Single Page Auction Application.

Conclusion

If you have built a fully functioning message based app only, then the actor model of Akka will make sense to you. However, handling callbacks, parsing untyped messages from actors, implementing your own batching and most importantly, touching actual data is difficult. Akka is not actually a data store, it is closer a control flow for many messages going through it. Baratine is for all types of services: cloud based, mobile, REST, HTTP, & more.

Head over to Baratine’s Homepage to explore more about this powerful framework!

Technologies Supported By Baratine

Baratine is Caucho’s latest project. As more developers have began to use Baratine, we have noticed a common set of questions and have written this response to get users to understand the programming model Baratine presents.

Overview

A large percentage of web applications today are composed as a mix of JavaEE / Spring and similar REST frameworks. These architectures are considered “3-tiered” and can be pictured as below:

Screen Shot 2015-12-16 at 3.43.10 PM

  Throughout this architecture, a developer can expect to find different modules, protocols, and components that use different frameworks or JSRs to accomplish their specific task. Some, for example a cache, may be used to speed up the performance of an application at the added cost of complexity to the overall framework. This architecture also has a certain amount of familiarity as it has been used since the early days of web applications. While this architecture is familiar, it is not without its documented shortcomings (http://microservices.io/patterns/microservices.html).

  Between the load balancer and cache sits a logical separation from what is considered application code versus hardware or software responsible for proxying client requests into the application. Starting at the red line and containing everything within it, is what is best captured by the statement SOA-tier. This code is often complex and critical to an applications uptime & responsiveness. Over the course of the years, developers have looked to standardize, at least conceptually, how this portion of an application should be written (http://www.infoq.com/articles/tilkov-10-soa-principles).

  While these best practices have been coveted by developers, they are seldom implemented. This is in part due to (1) the business demand to prototype or have a partial solution up and running and (2) the learning curve and integration required to use frameworks that offer only a partial benefit and add to the overall complexity.

Enter Baratine

  Baratine is not “yet another framework to learn”. Baratine presents a programming model that most developers already know: Object-Oriented.

Screen Shot 2015-12-17 at 3.32.38 PM

As illustrated by the image above, Baratine has the capabilities to replace or integrate with the SOA-tier of an application’s architecture. While the internals of how Baratine does this is a full discussion worthy of a blog post in itself, each layer above can be described in Baratine as follows:

Cache – There is no cache in Baratine. The data that would normally sit in a cache, instead sits within individual Baratine services. These services can pull from a database, keep everything in-memory, or use a combination of the two. Services use high-performance techniques such as batching & high-throughput queues to maintain performance.

Web Server – Baratine contains a web server capable of communicating through Websockets, HTTP, or Long-Polling. However, clients do not call into a Baratine server directly. Instead, they call into a Baratine service that is deployed on a server. This allows clients to call methods directly on the service.

App Server – Baratine does not need an application server. Baratine processes application business logic in the form of POJOs developers code. Because of this, developers code to standard Java 8 conventions and Baratine handles the wiring, service discovery, and lifecycle of services.

Database – Baratine services can persist data and store data in multiple ways:

  1. They can use an outside entity such as a NoSQL or MySQL database.
  2. They can use a <k,v>  or database store local to the service.
  3. They can store to Baratine’s File System

What does this mean for developers?

What this means to developers, is that they can achieve greater performance, scalability, and resilience with straightforward Java 8 code rather than looking to integrate multiple components from different vendors. Code is more standard and coding is more agile. Baratine does not support any of the JSR technologies directly.

The core features of Baratine allow for services that are polyglot (any client capable of understanding JSON can communicate), individually scalable, and can be embedded within current environments or function as standalone servers.

Baratine is designed for the SOA-tier of applications. Having been in the industry since 1998 and responsible for an application server implementation that predates Tomcat, Caucho has seen both the best practices as well as fallacies of application architecture. Current frameworks, although popular, leave developers exposed to mitigating synchronization errors, thread starvation, database bottlenecks, and a laundry list of others.

Baratine is the SOA layer redesigned from the thread & data locality level up. This model prevents possible integration point failures that exist in current frameworks such as between the web server and cache. Baratine applications implement a set of principles best described by the Reactive Manifesto(http://www.reactivemanifesto.org/).  In short, each service is capable of millions of operations per second per instance, can be individually scaled and sharded, and can have its functionality isolated from bottlenecking or bringing down an entire application. These are principles described in Michael T. Nygard’s book Release It!: Design and Deploy Production-Ready Software

The Baratine API

In Baratine, your POJO is your service and your data. A POJO contains business logic + whatever data it needs to maintain (For example a Counter POJO might contain an instance variable such as: long counter). Once the @Service Annotation is placed on that POJO, it is made into a Baratine Service. A Baratine Service can be pictured as follows with the definition below:

service-inbox-outbox

A Baratine Service is an annotated POJO with:

  • Its own internal storage
  • An inbox that automatically batches method requests
  • An outbox for returning response calls 

Instead of persisting this data to an outside entity (namely a database), the Baratine framework provides access to storage on the service itself. A service will store each instance created at a unique URL. This allows the owning service and data to reside on the same node. For the developer, this provides an easy way to access data for a given service instance:  it takes in a url and uses this id as a unique identifier to store the data for the POJO object at. This is how Baratine is able to maintain state on different object types and continually run methods on these objects in a nonblocking manner. Each server instance is managed on a single thread with batched methods coming through the inbox, because data is only ever accessed on a single thread, the service cannot suffer from possible synchronization issues, thread starvation, or bottlenecked resource consumption.

How services communicate

Baratine allows developers to continue to code object-oriented Java applications. This means that services communicate through normal method calls. Clients in Baratine can connect using Websockets, Http, or fallback to long-polling as a third option. Once a client has established a connection to a service, communication is done through normal Java method calls. The wire protocol responsible for transmitting the message is JAMP. JAMP communication allows a service to speak JSON to any client calling into it, because of this Baratine is a polyglot language.

This style of communication contrasts with the common practice of wiring REST endpoints to Java objects and exposing their methods. Normally, a developer would be concerned about the URLs placed on servlets or other objects answering requests. This is not needed in Baratine. While it is possible to test a hello service(http://doc.baratine.io/v0.8/patterns/hello-tour/#examples-hello-tour) method using curl as such:

curl http://localhost:8080/webapp/hello?m=hello

and receive the following response:

{“status”:”ok”,”value”:”hello”}

This style is only to be used for testing! When implementing clients that actually communicate with services, a client will lookup or build a service reference, and call methods on that returned Service instance as follows:

{

  ServiceManager manager = ServiceManager.current()

  Hello myHello = manager.lookup(“pod://pod/hello”)

                         .as(Hello.class);

  out.println(“Hello: ” + myHello.hello(“world”));

  hello.helloSend(“data”);

}

For a full description and walkthrough of this service see http://doc.baratine.io/v0.8/manual/http-websocket-clients/.

Summary

We hope this overview of the Baratine framework and what technologies it supports contributes to your understanding of the framework. If you have further questions about Baratine, please post them on our Baratine-io Google Group ( https://groups.google.com/forum/#!forum/baratine-io ) or email us directly at sales@caucho.com.