Building Performant Data Platforms with Cyrex
19 November 2020
Platforms with large amounts of data can run slow. Having to wait for your own data to respond can be frustrating. Special tools and engines like Elasticsearch are key in keeping your applications operating at their best. Utilising the proper tools can ensure your system remains effective and efficient. At Cyrex, we keep the process smooth and fast with a few key tools and concepts.
Web socket, real-time communication
With regular web communication, you typically have to rely on the client-side sending its request to the server. This is usually their web browser, making a request and the server responds. This results in the client-side having to ping every few seconds, checking in if there’s any new information. The system will have to consistently check for any updates so the server can respond. This increases the load to keep up, to keep pinging and check if anything new has come up.
Currently, most applications operate this way. There’s no option for the server to initiate the communication and provide information on an update, instead it must wait for the client-side to make its own request.
With a web socket, it acts like a circuit. It remains open. The browser connects to it, identifies it, and the security kicks in to ensure safety for the parties involved. The server then has the option to send data back to the client without waiting for the client-side to initiate.
This keeps the consistent pings for changes low, keeping the server load low and far more manageable. A web socket system means there is no delay in communication, everything happens in real-time on your web platform.
Front-End Frameworks, strength in the user
PCs and the end user have much stronger systems in their hands, so the front end can take a portion of the load and keep the system moving quickly and efficiently. With the average user working with far stronger CPUs, more RAM, and a significant amount of processing power, we can push a lot of the usage to the front-end rather than fully relying on the back-end. This reduces the workload on the server, which helps with hosting costs for your application.
We work with strong front-end frameworks for situations just like this and use the latest version of Angular to do so.
Multitenancy, under one roof
Another concept we often work with is multitenancy architecture. This means keeping separate entities all located and stored on a single system. They each have their own database, separated and isolated from one another. Therefore, security for each company involved is not compromised.
This set up keeps all the data on a single, secure platform.
With all of the data on a single platform but with each entity on its own private database, they do not collide and they don’t mix. However, on the backend for upkeep and maintenance, we have the entire linked database indexed and easily navigable.
Elasticsearch, bounce back searches faster than before
Elasticsearch is not a typical search engine. Traditional engines use relational databases, meaning searching for a keyword will return any results that literally contain your search. Elasticsearch instead searches using a score which is determined by the data’s relevance to your search.
In addition, Elasticsearch is kept in memory as opposed to an actual file directory. These two details result in a faster and more accurate search engine when you are looking for non-singular records or you want to gather data under particular criteria, like register date or role.
There are many advantages to Elasticsearch. It provides analytics and insights into your data almost instantly. It is a fast-performing engine, using an index system to ensure almost immediate results from very large sets of data. It can search through large datasheets at High-speed and take into account human error such as spelling mistakes.
When developing and building high performing data platforms, we look to Elasticsearch to help do it. The speed is incredible and because performance is important to us, our developers have learnt its unique query language.