MVP on a Shoestring: Our Serverless Experiment

Introduction

One of our clients had a specific requirement: build an MVP with minimal infrastructure costs. This led us to revisit the concept of serverless architecture. The idea of not having to manage servers, automatic scaling, and paying only for what you use made serverless a compelling choice for this project.

The Experiment

After a brainstorming session, my team and I decided to use the AWS serverless ecosystem due to our familiarity with it.

  • We chose AWS Lambda to deploy our Node.js code as serverless functions.
  • Used MongoDB Atlas for our serverless storage needs.
  • AWS API Gateway to trigger the Lambda functions.
  • For handling asynchronous tasks, we employed AWS SQS (Simple Queue Service).
  • VueJS UI was deployed using AWS CloudFront.

All set and ready to go. However, a few weeks into development, we encountered several significant challenges that made us question the viability of a fully serverless architecture:

Scheduling Tasks:  In a traditional server-based setup, scheduling tasks with cron jobs is straightforward. In a serverless environment, scheduling was initially a headache. AWS CloudWatch Events offered some help, but it wasn’t always straightforward.

Complex Business Domains:  The Lambda functions could not handle business domains as single deployable units. Real-world business logic is complex, and the Lambda functions started getting bloated.

Long-Running Processes:  Traditional servers manage long-running processes efficiently. In contrast, long-running APIs in a serverless environment suffered from severe latency issues and cold start delays, making it difficult to develop performant APIs.

Database Connections:  Optimizing database connections and implementing effective caching strategies is more challenging in a serverless environment. The lack of connection pooling affected performance.

Typical, right? There’s a tech that promises a revolution only to deceive in the end. But does it? I want to take a step back and reexamine some of these challenges.

Query Optimization:   DB query optimization is always an afterthought for the development team. Most teams do not care about DB query performance on day one. During the development stages, your application doesn’t have much data, so all your queries appear to work well. Once your software is live for a considerable period, this changes. I have seen teams redesign features/user experience when they realize that the product cannot handle the data demands. At this stage, it becomes super expensive to make the change. With limited computing power and no connection pooling, a developer is forced to think about query optimization from day zero. Some might call it premature optimization, but I call it adherence to coding best practices.

Doesn’t sound that bad after all, right? Especially when we were able to reduce our fixed infrastructure costs by 600%. This approach is ideal for products that are still finding their market fit, where minimizing fixed costs is crucial.

Conclusion

Reflecting on our journey, serverless computing has been both challenging and rewarding. It pushed us to optimize and adhere to best practices constantly. Despite the significant challenges, the benefits of serverless architecture—such as cost efficiency and scalability—make it a compelling choice for many applications. Our experience underscores the importance of disciplined development practices when embracing serverless architecture.

-Prashant Srinivasan,
Director of Engineering.
Scroll to Top