Why we use MongoDB as our main database

I originally chose MongoDB in 2018 based on my previous experience with the database. MongoDB has proven reliable, and developing on it is fast.

Early on in the company history we had only a limited idea of what we would need in the future. Since MongoDB has no strictly defined schema, we had more opportunities to develop our systems to the direction we needed to as the business and the product grew. It’s a clear benefit of the NoSQL paradigm: we could store any sort of data before we knew what we would need it for, or even if we would need it at all.

I like to call Mongo a “Swiss army multitool”. Even if it’s not the best at any one single thing, it can handle pretty much any use case. Even if you didn’t know what you should use as the database, Mongo is always at least a decent option that you can’t go wrong with.

Why we choose Go for our main backend development language

The main benefit I see for Go is its performance and capability for parallel computing. At our heart, we are a data company and the data volumes we need to handle are huge at all times. The volumes alone make multithreading a necessity for us. Go is definitely great in that area, as it has a very elegant and powerful way of handling concurrency. The developer experience of Go is sublime, as it’s both very efficient while being surprisingly simple. 

Another major benefit is that Go is a strictly typed language. Many potential bugs don’t get through in the first place, and the ones that do are found in the code faster and more easily.

It doesn’t hurt either that there is a large community around Go. Whatever challenges might come up, there’s probably some sort of answer to be found in the documentation created for the language. Go developers tend to be very keen on solving difficult problems and sharing information. Most of the time the documentation is of high quality as well. 

Docker for container packaging

With as large a software entity as our Cloud, containerization makes life a lot easier. Moving the developed software from the developer’s computer to the live environment is more efficient, and the software also works in the live environment as well. Easier testing is a clear benefit.

Docker is pretty much an industry standard, to the point where dockerizing is a common phrase. Since packaging software into containers is a basic need, it might be a better question to ask “Why not Docker?”

Kubernetes for orchestrating the containers

Our cloud service consists of a huge amount of software ranging from the UI to APIs and the management of data traffic. Kubernetes helps us make sure that the Cloud stays up all the time. As the most versatile software that I know, I can’t list all the benefits here. Just for starters, the main benefits include:

• Restarting software automatically if something crashes
• Connects all the parts of the Cloud from services & networking to config files to be one singular entity
• Allows us to manage all the Cloud software in one place.
  

We chose Kubernetes based on two main reasons. First, it is backed up by Google, so it is never out of date thanks to constant updates. Second, the community around K8s is wide, and therefore the documentation is good and plentiful. Getting started with Kubernetes was a bit challenging back when we first implemented it, but putting the time to learn it has really paid off.

Our Cloud runs on Amazon servers on multiple availability zones. The way Kubernetes is set up, the availability zones ensure high availability and fault tolerance, both of which are central for the reliability of our service.

Why Redis & Redis Sentinel

We chose Redis for its versatility. Redis plays three roles in the Logmore Cloud:

1. It acts as cache. The most used data is stored in Redis for quick fetching. Since Redis operates in RAM, using the Cloud is faster and the payload is lighter.
2. User sessions are stored in Redis to keep them logged in.
3. Queue system. Our data architecture is largely event-based. By processing events with Redis, we can organize our data to an easily readable and indexed format..
   

In addition to the versatility, Redis is really stable. The resources put into Redis development are huge, and it is kept up to date all the time. It is simply something you can rely on and trust it works. As with Kubernetes, the fault tolerance and high availability are important factors.

Why Linux

At least 95% of all servers in the world run on Linux. Open source and the amount of people working on it ensures that Linux-based systems get developed all the time, and there’s a lot of pairs of eyes tracking the development. In our case, most of our developers work on Macs, which makes the UNIX-based OS handy. The almost identical command lines make development work much easier.

The theoretical options would be to use Linux or Microsoft, both of which are industry standards. For open source, Linux takes the victory, and as with the previous notes, the amount of developers working on Linux is huge. We appreciate open source in our tools in general thanks to the versatility it offers.

Why Amazon Web Services

I briefly mentioned Amazon previously. Of the three major players in the industry, all use millions of dollars on development annually, so the reliability and security of all of them is top-notch. You can’t really go wrong with any of the three, but we have been satisfied with AWS.