Architecting Multi-Tenant Infra - AWS CDK

Overview: smallcase Gateway is a platform to help users transact stocks, mutual funds, ETFs, and smallcases in your app or website with their own brokerage account.

In this blog, we are going to talk about how we enabled smallcase distribution for gateway partners by providing them with a customizable frontend offering and absorbing all the engineering complexity for infrastructure.

Situation: To enable smooth partner onboarding, we had to make distributing smallcases easier. Architecting it to a multi-tenant model was a much-needed engineering call. From an infrastructure perspective, onboarding them manually became a huge task.

Task: To make the integration easier and standardized, we wanted to have an offering where the frontend + infra is also maintained by us and can be customized according to partner requirements. These are some of the critical infra requirements:

• This is a server-rendered app, where the server would potentially need to serve 100s of users across different tenants. So it should be possible to scale the server.
• Even though it is a server-rendered app, there is a lot of static content, which does not change frequently, so we need a caching layer in front, so that not all calls go to the server.
• Since we need to support multiple different partners within a single setup, allowing customizations through the config, it was decided to build it in a multi-tenant way, where the tenant would be identified through the URL.
• We need to make sure that the URL is relayed correctly to the app server, through all of the infra pieces that it passes
• Since this is expected to scale to 100s of tenants, setting up the infra manually every time we onboard a partner does not make sense. We need to make sure that setting up the entire infrastructure for a new partner is hassle-free and quick

In a nutshell, we’ve to handle

• Networking (Route53, load-balancers)
• Caching (CloudFront)
• Autoscaling/Target groups,

Action: To make it smooth, it had to be through infrastructure as code. With our complete workloads hosted on AWS, the obvious choice was Cloudformation, with CDK it gets even more interesting. The comfort of writing logic in typescript is way more powerful.

CDK with typescript lets you define your boilerplate your way through constructs. Constructs are the basic building blocks of AWS CDK apps. A construct represents a “cloud component” — Like Route53, Cloudfront, ALBs, etc.

When working and reading about CDK you often see the terms Constructs, Stacks, and Apps, a brief about how they’re stitched together.

The solution is majorly divided into 3 parts:

DNS entry for onboarding partner: All the entries were to be created under the same R53 hosted zone, but with a different record entry.

Time for our first construct that stitches Route 53 to CloudFront, route53.ts

Cache handling: The obvious choice for this had to be Cloudfront, since we did not want maintenance overhead for multiple CloudFront distributions, we choose to minimize the distributions by maintaining origin as ALB and multiple Alternate domain names (cnames) for the single distribution. This has a limit of 100 cnames, so we’re keeping a check here.

A special note on cache policy:

• We wanted a minimum TTL of 180s.
• We wanted host header to be carried to ALB for the backend to respond accordingly.
• The communication between CF and ALB had to be on port 443.

Time for our 2nd construct, cloudfront.ts

Load Balancing: We’re using ALB here since our use case is limited to path-based routing. Time for the third construct, alb.ts. This chunk absorbs whatever is passed by CloudFront.

Target group handling: A key point to notice here is partners were sharing the same backend, hence the same target group for all the partners, this also means keeping it in powerful instances under ASG.

A final construct that stitches all this together.

Result: Partners can now be easily onboarded by raising PR with just this piece of detail. :)

PS: we’re taking care of deployment through argo workflow. Stay tuned for the details in the next blog. :)