Image this: You as a FIWARE IoT platform owner have your Orion Context Broker running. You have different databases created in Mongo by different Tenants. Multi-tenancy complete! Then someone comes and tells you that “Yeah, multi-tenancy is nice but I want HYPER_TENANCY!” and you are like “What?! Who is this guy and what is he talking about? “
Here is what this Hyper-Tenancy guys is all about: Say you have a setup where the is a one*1 of everything (like in Paderborn K8S based solution): One Keyrock, one Umbrella, one Orion and so on. This is all good for simple use case, but the hyper-guy wants to have one platform for multiple cities/stakeholders who have their own data lake via logical or physical separation. Cities want to ensure that data is separated and access is controlled.
There are a few ways to go about this. The most obvious is that everyone has their own setup and no-one else is allowed. It’s like in the neighborhood everyone has their own lawn mover and no-one is sharing. This is cool if you have tons of money and you know, who cares. But if you want to be responsible, ecological, cool and use resources effectively, sharing is caring.
Which architecture allows you to pool your resources best? Usually, a good answer begins with the words “It depends.” Nevertheless I give you two cool alternatives and we’ll discuss those separate so you get some idea what the possibilities are: 1) Isolation via Kubernettes clusters 2) Isolation via Application control and selective component scaling.
Isolation via Kubernettes clusters.
Isolation via Kubernettes clusters. This is fairly simple. For each of the stakeholders, you have one cluster. This will still allow stakeholders to share some of the resources, but have independence. Each have their separate CI/CD pipes, they can run different components and in different stages of CI/CD pipe. Data sharing can be done via NGSI API and sharing Tenant accesses.
It’s worth remembering that many times something needs many times maintenance and operations.
Isolation via Application control and selective component duplication.
Isolation via Application isolation and selective component duplication. In this, stakeholders need to work more in tandem; they need to be ok that there is single version on a given component running. On top of this they need to yield control to a impartial authority over User Management (Keyrock). Why? Because Keyrock Admin can see all the organisations (if the tenant-manager system is in use) and organisations translate to Tenants. If yielding control is not something you like, this can be circumvented by adding features to the.. Tenant-Manager!
In this scenario it is good to have a Crate DB license. Crate DB free version has a limitation; single user. This has the nasty side effect that if we connect with this user to Grafana, this user can see all the data AND modify all the data. With Crate license in place, it is possible to have different user, who can connect to different Crate DB databases. It is possible that you do not need a license, BUT then it’s the same jam as with Keyrock. You need to give control to impartial authority, which can control Grafana and grant access to new users and organisations.
In the previous scenario, the Grafana Admin user needs to be given to partial authority. Or duplicate Crate DBs. This is one example of the “selective component duplication”. Bit more is discussed in the “What about the Ops” section, because.. it depends.
There is one (yeah, sure you can have all sorts of permutations) more I want to mention: Isolation via namespaces. In this one, each stakeholder has their own namespaces but, say, a single Gitlab CI/CD instance. This gets then bit more trickier in the CI/CD pipe, as breaking the rules allows you to deploy to a wrong name space, as deployment rules are based on RegEx. There are few other variables which can be used to tune the accesses, like “tenant-servicepath” Header. These tunings then depend on the use case.
What about Ops?
Running a High Available can be both challenging and scary. However, if you plan your setup correctly, many of these worries will go away. There are several good approaches to this and I will not discuss those here. Instead I will make few remarks how those two main scenarios affect Production setup and Ops. With Ops I mean that “something went bang and them IT folks need to login to the server and investigate.” You should not grant all accesses even if someone tells that it makes life easier. Not true.
If you have separate Kubernettes clusters, then this is fairly easy. Control via kubeconfig. Each of the designated clusters should have their own kubeconfig. It’s like a key or a Bearer token. One who possesses the kubeconfig, can access the system*2 on server level.
If you have Isolation via Application control scenario, things get bit more interesting. In principle there is only one system and once the ops folks have access to one place, they have access to everything*2. This is not a bad thing by default, but just something that you need to be aware of. There are ways to change this and then the “selective component duplication” kicks in.
If you want to limit the access, then you can run several namespaces, and run multiple instances of selected components. Kubeconfigs can be set up per namespace granularity and this way you can limit access to resources. This is the hybrid approach.
Final words.
I wanted to write this bit to help on the selection when going ahead with the project/deployment planning. The core take away here is that there are few ways and by understanding the requirements you can formulate the correct approach.
You may have noticed that this post is pretty specific to certain tech. There are other possibilities for IoT platform setups. If those make more sense in your use case, go for it!
If you need more info or want to share some thoughts, please do so by commenting below or hit me up on ilari.mikkonen@profirator.fi.
*1 There are many QuantumLeaps and Crate DBs but just for horizontal scaling.
*2 Oversimplification, you need some other info/access control too.
P.S. Hyper-tenancy is a term coined (AFAIK) by me. If you don’t like it, fight me.