When you’re building a startup, you move fast. You want to test ideas quickly, keep costs low, and be ready to change directions anytime. That’s why the tech stack you choose early on matters a lot.
What works for big companies often doesn’t fit early-stage startups. If your product is still new and you’re exploring the market, don’t build a complicated architecture too soon. You don’t need microservices for everything. Instead, start with one backend and one frontend that handles everything. This keeps your infrastructure easy to manage and lets you focus more on your product.
This article presents an example of a simple core and modern startup stack on Azure, and discusses its components. This helps you:
- Spend more time building the product, not managing tools.
- Keep your costs low.
- Move faster as you learn from users.
Example use case: A startup building a RAG app
Let’s take an example of a startup building a Contoso Retrieval-Augmented Generation (RAG) application.
This app allows users to ask questions, and it finds answers by searching their own uploaded documents using OpenAI models. The startup also wants to support multiple customers, each with their own private document storage and AI experience.
They’ve built their backend using Python (FastAPI) and frontend in JavaScript(React). Now they want to deploy on Azure in a way that is:
- Easy to manage
- Cost-efficient
- Ready to scale
- Security in mind
Architecture overview
Here’s the architecture components this startup uses on Azure:
Azure Entra ID
Azure Entra ID (formerly Azure AD) is used for authenticating and managing users securely inside a single customer tenant, perfectly fits SaaS scenarios as the startup grows.
Azure DNS + Application Gateway
Handles incoming traffic from the browser and routes it securely through an internal Application Gateway and DNS. The gateway helps load-balance and protect the app using Web Application Firewall (WAF).
Azure Virtual Network
Azure Virtual Network protects your internal resources by isolating them from the internet. Your app communicates safely with the Neon database, storage, KeyVault and OpenAI through secure networking.
Static Web App + App Service Plan
The frontend is deployed as static content on Azure Static Web Apps, while the backend API (FastAPI) is hosted inside Azure App Service containers. Azure App Service is ideal as it reduces the need for server maintenance and supports quick scaling as your startup grows. Three deployment slots are used:
ProductionStagingLast good(rollback)
Azure OpenAI
Gives secure access to OpenAI models like GPT-4 or vector embedding text-embedding-ada-002. It works great for RAG use cases, especially when you need enterprise security, compliance, and region control.
Neon Serverless PostgreSQL
Once your app runs, it needs data storage. Neon Serverless PostgreSQL provides a modern, serverless relational database that scales effortlessly, requiring no manual database management. Neon also stores customer metadata, vector embeddings, and structured app data. Azure also has managed database services to consider such as SQL, MySQL, MariaDB, MongoDB, Apache Cassandra, Gremlin, and Redis. Neon is purpose-built for modern applications — it automatically scales to zero when idle and only charges for active usage, which helps startups save costs from day one. As we use multi-staging environments with dev, staging, and prod in our example architecture, Neon’s instant branching makes it easier to test changes in these environments and deploy data and schema to different environments via CI/CD.
Azure Blob Storage
Azure Blob Storage handles storage and delivery of static files, freeing up your app servers from extra load. Also, it stores raw customer files (PDFs, docs) that will be embedded and searched.
Azure Key Vault
Manages secrets and credentials securely. Only accessible via a private endpoint.
GitHub Actions + Container Registry
Frequent, automatic deployments are essential. GitHub Actions helps you automate deployment processes, avoid manual errors, and quickly deliver your latest updates.
- Developers push code to GitHub.
- GitHub Actions builds containers and deploys infrastructure.
- Container images are pushed to Azure Container Registry and pulled by App Service.
Azure Monitor + Log Analytics
Used to gather logs, metrics, and diagnostic information across the app for better observability.
Core stack
What your startup gets with this stack:
- Fast to launch
- Low cost with serverless tools
- AI-powered using OpenAI
- Easy to scale for multiple customers
- Secure and enterprise-ready
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | React (Static Web App) | User interface served via App Service |
| Backend | Python (FastAPI) in Azure App Service | Handles API requests, document parsing, vector search, OpenAI calls |
| AI Layer | Azure OpenAI Service | GPT model for answer generation |
| Database | Neon Serverless PostgreSQL | Stores documents, embeddings, metadata |
| Storage | Azure Blob Storage | Raw customer file storage (private endpoint) |
| CI/CD | GitHub Actions + Container Registry | Automates build, test, deploy |
| Security & Identity | Azure Key Vault + Entra ID | Secrets management and user authentication |
| Monitoring | Azure Monitor + Log Analytics | Logging, diagnostics, and alerts |
Dataflow
- A user opens the app in their browser.
- DNS routes the request through App Gateway to the App Service backend.
- The user’s question is received by the FastAPI backend.
- The backend queries Neon to retrieve vector embeddings and relevant document snippets.
- The context and question are sent to Azure OpenAI over a private endpoint.
- The generated answer is returned to the frontend and displayed to the user.
- Logs and performance data are sent to Azure Monitor and Log Analytics.
- Developers push changes via GitHub. CI/CD pipelines automatically build and deploy new app versions.
Built to scale
This architecture is simple enough to get your first version live — but it’s also powerful enough to grow with you as you add more customers, more data, and more usage.
From Day 1, you’re using managed services that handle complexity for you. As your app becomes more popular and your customer base grows, each layer in this design can scale without requiring a full rewrite or major re-architecture.
Here’s how scaling works in this stack:
Scale compute with App Service Plans
Start with a basic App Service Plan for minimal cost, and upgrade as your app traffic grows. Azure App Service supports vertical scaling (more CPU/RAM) and horizontal scaling (more instances). You can even set auto-scaling rules to handle spikes in traffic.
Scale data using Neon’s branching or per-tenant isolation
Neon makes it easy to handle multiple customers:
- Branching: Create separate database branches for testing, staging, or customer-specific workspaces — instantly and cost-efficiently.
- Isolation per customer: Use separate databases or schemas for each customer to ensure data security and regulatory compliance.
- Usage-based pricing: Neon only charges for compute when it’s used, which keeps costs low when tenants are idle.
This allows you to grow from 1 tenant to 1000+ with confidence.
Control OpenAI usage at scale
As users interact with the AI, you can:
- Track usage per customer and apply quotas to control costs.
- Use caching to avoid repeated requests for the same queries.
- Switch to Azure OpenAI’s multi-region deployments to ensure low-latency responses for customers around the globe.
- Use deployment slots and routing rules to test new prompts or agents before going live.
Global reach with Azure Front Door
Azure Front Door acts as your global edge — routing traffic intelligently to the closest region, caching static content, and protecting your app with built-in Web Application Firewall (WAF).
As your customer base spreads across regions:
- Add regional app deployments with automatic failover
- Use custom domains and SSL for each tenant or brand
- Easily integrate with Azure CDN and Traffic Manager
Multi-tenant security from day one
Even as you scale to support more companies:
- Azure Entra ID (formerly Azure AD) lets you support enterprise SSO and role-based access.
- Azure Key Vault with private endpoints ensures secrets never leave your secure network.
- Virtual Network isolation keeps critical components like Neon, OpenAI, and Storage hidden from the public internet.
You’re not just scaling in size — you’re scaling in security and control, too.
Continuous deployment that scales with teams
- GitHub Actions makes it easy to scale your release workflow as your team grows.
- Use branch-based pipelines for dev, staging, and production environments.
- Set up feature flags or canary deployments to test features with specific customers before full rollout.
Summary
Using this modern startup stack on Azure with Neon helps your startup move fast, stay lean, and focus more on building a great product. This startup is building a Retrieval-Augmented Generation (RAG) application to support multiple customers, each with their own private data and AI assistant experience. The entire solution is designed with scalability, isolation, and security in mind.
Resources
- Try Neon on Azure Marketplace
- Try the example project and experiment with Neon-powered AI search.
- Core startup stack on Azure
