Intermediate: Decentralized Solution
Last updated
Last updated
This solution is better suitable for large scale platforms with multiple geographically distributed warehouses and also for the high traffic environments. In this, each warehouse has its own independent inventory node.
High Level Components:
Inventory nodes: Each warehouse or group of small warehouses with its own inventory service.
Global inventory aggregator: To combine global stock information from all inventory nodes and update availability.
Order processing system: Routes order to the appropriate warehouse based on the availability and proximity.
Notification and reorder service: For alerts and reorders.
Worker service: Syncs Inventory changes across warehouses and incoming shipments.
Database: Use , (AWS), (Azure), (Google).
Distributed Caching: To reduce the database load for inventory systems and speed up the availability checks at the warehouses.
Message Queue: This is to handle stock changes during purchase, on return, on cancellation. (Kafka)
Stream Processing: For faster real time processing of the inventory items with large amount of data ( , )
Prometheus & Grafana: To monitor inventory trends and track stock outs / over selling incidents. You can also use for reports.
Monitoring & logging: When dealing with multiple nodes, tracing the logs is very important. Use Open Telemetry to track warehouse specific performance metrics.
Scalability Considerations:
Horizontally scalable both inventory updates and global aggregators.
Sharded databases per warehouse.
Use gRPC instead of REST for faster communication between the microservices.
In Kafka, use topic partitioning for scalability. By partitioning topics, each service can process messages in parallel and independently, allowing better throughput and load distribution.
Considering the distributed nature of the system, data flow should be designed in such a way to have eventual consistency.
For real time processing of the inventory items, use streaming engines
How does the above system work for the above functional requirements ?
It should track real-time inventory updates for products across multiple warehouses and locations.
Every warehouse has its own inventory node thanks to the decentralized system. Each warehouse may track real-time modifications locally, and the global inventory aggregator offers global inventory aggregation, which enables ongoing product tracking across several locations.
On purchase: Deduct stock at the warehouse node, then propagate changes
On return: Add stock back at the relevant warehouse node
On cancellation: Release reserved stock and propagate
It should support product variants like size and color.
System can handle product variants at each warehouse's inventory node, allowing each node to track variations independently. This ensures that the stock is accurately tracked and the order is processed with the correct variant.
It should handle high traffic and concurrent transactions.
Horizontal scaling is made possible by the decentralized structure, in which every warehouse functions independently. Because each warehouse node is capable of managing its own transactions and traffic, bottlenecks are avoided and the burden is distributed. Furthermore, high concurrency and transaction volumes can be handled by utilizing scalable technologies like Apache Flink for stream processing and Kafka for message queuing and using gRPC for faster communication.
It should provide low-stock notifications and automatic reorder triggers.
Notification and reorder service helps to trigger the notifications when inventory reaches a predefined low threshold for each warehouse separately and also it allows reorder actions based on local stock levels.
It should generate accurate inventory reports and forecasts.
Global inventory aggregators can collect data from all warehouse nodes to provide accurate and aggregated reports. The use of real-time stream processing, distributed caching, and scalable databases enables fast and accurate reporting. The worker service can also sync inventory changes, ensuring that reports are generated with up-to-date data.
Pros:
Improved fault tolerance with independent nodes
Faster local inventory lookups and order processing
Scales better for global scaling
Cons:
More complex to maintain and also implementation
Eventual consistency requires conflict resolution when dealing the items at global inventory system
Periodic reconciliation may introduce slight delays. This delay could cause over selling or incorrect product availability.
Caching:
For product listing and search scenarios, implementing the cache-aside pattern helps maintain a balance between freshness and performance. When a request is made, the system first checks the cache. If the data is not found, it fetches it from the database, caches the result, and returns it to the user. This prevents stale data issues while improving read latency.
Questions you should ask yourself while designing a database:
Does the system require structured or unstructured data storage?
How often do you need complex queries with joins and aggregations ?
Do you need real time updates or eventual consistency ?
Do we need secondary indexes for lookups ?
Does the inventory system need horizontal scaling?
How frequently does data change? High read-heavy or write-heavy workloads?
Do you need low-latency reads/writes across multiple regions?
Do you need multi-region replication for disaster recovery?
Can the system tolerate occasional downtimes?
Is the inventory model evolving rapidly?
Do you frequently add new attributes?
*** These are the questions from AI