Synergy SKY offers multiple software solutions, however the one i am going to talk about here is an analytic solution for video related CDRs (Call Detail Records).
The solution (Synergy SKY Analytic Server, shortened SAS) is harvesting CDR data from various sources, then trying to correlate, group and match data together. This gives us the fundament for various analytics and reporting services on our platform.The platform is utilizing XMPP as a carrier bus, together with web-frontend in angularjs. This all requires a lot of various components to be installed (xmpp server, database, web server, misc runtimes and various homebrewed bins), configured and managed on each installation. Also since we are using XMPP, the general architecture is distributed by nature (a final deployment consists of two servers). At an early point in the development we figured out we needed an easy way of deploying our software, this lead us to some choices.
One is to avoid the whole customer premise install and make a pure cloud solution, however this kind of architecture has too many caveat in terms of data storage, compute-requirements and regulation (also given the new GDRP in EU). Second option would be a on-premise installation of the software, which is what we probably needed to do… But how, when the stack is complex?We landed on that we wanted to push out an appliance hardware, eventually a virtual appliance using VMWare or similar.
This gives us the ability to create a controlled environment with the full stack included, and in expected working state.
In a very very very simplified way, the diagram looks like this:
Now we needed to decide on what would be the OS of the hardware or virtual hardware (The one called "SAS Appliance at Customer location" in the diagram)? Linux of course.. we had experience with CentOS from previous product and building virtual appliance. However the more we talked about it, the more we realized that we should be looking around for options.
One major issue we experienced was that Systemd managed to crash the whole dbus-systemd connection if our software took too much memory, leaving the system in unstable state, with reboot as only option (I don’t think we can blame CentOS directly for that, however pulling in such premature technology into a proclaimed stable platform, makes no sense at all!!). We had some other minor issues as well which is probably weeded out in the latest versions of CentOS (and not worth mentioning).
We had already a lot of experience with CentOS Linux, which is a very good thing when deciding on what to do. And it is usually better to know the problems and work around them then starting on scratch with a new distro/OS!
What we looked into in regards of building the platform was:
- Container technology (docker was tested extensively, LXC was tested some)
- Package and repo manager (yum/dnf)
- Automated install (never tested with CentOS, but we knew it was possible)
- Service management system (Systemd)
- How to deal with basic configuration of system (network, service start, etc.)
But still with the knowledge in CentOS we wanted to broaden our view.
Next up where FreeBSD - which a colleague had been tinkering with, for pure fun - installing FreeBSD 9, compiling anything from ground up on his home computer.He was very positive about the concept of FreeBSD being a full OS, with a proper base system, well designed package system for building your own packages and the demarcation between OS and Userland package system, leaving the system easy to maintenance. Also the jail technology and was interesting since we were investigating using docker as container on the server side, and looked at this as an option to that.
However, the thought of building anything from scratch was not appealing. However i agreed on testing out getting our early version of the software with the required stack up and running quote: "to get some impression, before moving on" - with quests for another/the perfect Linux distro.
And this is where it turned.
First impression of FreeBSD 10.3 was way beyond expectations. The installation was done within minutes, the online handbook where exceptionally well written. To my surprise, there were precompiled packages, and everything just worked neatly out of the box. I was amazed with the fact that after a few hours with FreeBSD, i had the whole stack running.
With this experience and the appealing case of FreeBSD Jail and ZFS (which is matured and tested solutions we both needed), we stopped looking around for alternatives instantly, and instead focused on testing and learning more about FreeBSDThe table from above became looking like this.
- Container technology (docker was tested extensively) ⇒ FreeBSD Jails
- Package and repo manager (yum/dnf) ⇒ Pkgng / pkg / ports
- Automated install ⇒ bsdinstall (we even build a pxe installer image that would get you up and running with a dev. System within minutes)
- Service management system (Systemd) ⇒FreeBSD rc, and later on we also adapted runit for some services.
- How to deal with basic configuration of system (network, service start, etc.) ⇒ FreeBSD rc and rc.conf
We now started building our application around the FreeBSD appliance utilizing Pkgng (pkg) as our binary package format for distributing. We have of course had some minor challenges here and there but never with stability and performance. The clear separation between OS and userland software (port/pkg repo) is logical and perfectly executed. This also automatically makes the whole system cleaner in sense of "where do i find the configuration", and also - where should i place my data.
FreeBSD has given us the "it just works" experience, without sacrificing the full control. It feels and behaves mature and stable with great performance!Using FreeBSD has been a great, i have also changed all my workstations to FreeBSD to learn the platform better, and the few times i misses Linux (Windows or OS X) is when i want Spotify and/or Netflix who both require compiled binaries which does not exists for FreeBSD.
I am very glad we took our head out of the sand and looked around, because FreeBSD turned out to be a real gem!
Its now been 2 years into our journy with FreeBSD and we haven´t looked back.
Other honorable mentions which makes FreeBSD great:
- Freebsd-update, keeps your base OS updated and is separated from the pkg system.
- Pkg repository is up-to-date, which is not always the case for ubuntu/debian and/or CentOS.
- Small footprint out of the box.
- Very easy to modify, rebuild and distribute custom modifications to packages using the ports system.
- Tightly integrated with ZFS (not just a userland driver).
- Easy license to deal with commercially.
- Network stack (there are in total 3 different firewalls that's included into FreeBSD, we landed on pf)