Why Joot chose Serverless
Serverless was the all-in-one solution for Joot. Joot’s team could install, set-up a project and from day one get something up and running. Serverless development provides an unprecedented amount of choice and flexibility to meet a vast array of scenarios - from web API’s to machine learning. It also allows you to do it super fast. The speed at which you can get things done is remarkable.
A scalable system that could scale up and down based on cluster demand and their machine-learning compute needs
Unlike dealing with a cluster of machines where there is no indication as to how many requests or how much actual traffic the cluster can handle, serverless services have a very specific limit attached to the service. This is great because your application won’t collapse under the weight of heavy traffic if you hit these limits, instead they return very clear responses via the SDK you are using to interact with them. Meaning that you can, in turn, much more cleanly handle service limits for your end users without resorting to the browsers default 500 error page.
It also means, however, that when you have times of no load or traffic, you aren’t still paying for a machine sitting around doing nothing. Serverless services only bill for when they are actually used, otherwise they scale down to zero!
Furthermore, if you find you are going to be using more of that service than the limits allow, in the vast majority of cases, you can just send AWS support a message to request an increase.
Complete (and ever-growing) suite of web-services to fit nearly any requirement
AWS services used:
If you take a look at AWS and the list of services they provide,
there is an enormous variety to choose from. And even limiting the selection to serverless services allows for unlimited customization and creativity.
The image below shows the architecture Joot uses for image processing. All of Joot’s users are managed using Cognito. Amplify was used as the library for the web app. Lambda was used to coordinate the user authentication. When a user comes into the system, they upload an image which is saved directly into the S3. Triggers set on the image upload bucket go in and then pull that image out, resize it, do some processing and pull meta-data out and store it in DynamoDB.
Joot early on wanted to rely heavily on event-driven architecture using EventBridge.
They also knew that early on there were a lot of properties they wanted to pull out of an image, quality, color, classification, etc. That event would then place that image information to be processed into several SQS queues each attached with Lambdas that would pull the image out and do the type of processing that they needed to. Once processing was completed, a new event would be triggered that would be picked up from another web service that would check to see if the model had been built for that market or that particular customer and then send it to a scoring queue. That would then take the image and build a training record to pass on to a model endpoint that had been built for that customer.
one of the options is to host a multi-model or single-model endpoint. Joot has single-model endpoints for each model that was built for a customer, so when a customer built a model using Sagemaker through Joot’s system they had their own special endpoint to use to score for that market.