Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

1. Overview

  • Objective: REAL-510 - Getting issue details... STATUS

  • Scope: setGetApi, API Gateway , AWS API Gateway , Amazon Simple Queue Service

2. System Components

  • Web UI: Client interface for uploading data.

  • Kairos Processing Modules: Details on kairos-file-process, kairos-product-manager, and kairos-products that prepare data for storage.

  • API Gateway: Manages client requests and routes them to SQS for processing.

  • Amazon SQS: Queues data to ensure order and manage asynchronous processing.

  • setGetApi: Retrieves and processes data from SQS and performs validation.

  • Redis: Serves as both a temporary and long-term data storage layer.

  • Scheduler: Nightly task that backs up Redis data to MongoDB.

  • MongoDB: Stores a secondary copy of data for archival purposes.

3. Data Flow

  • Describe the sequential data journey:

    • From Web UI upload to Kairos modules for processing.

    • API Gateway routes processed data to SQS.

    • setGetApi retrieves data from SQS, processes it, and stores it in Redis.

    • The Scheduler copies Redis data to MongoDB each night.

4. Data Persistence and Storage

  • Redis: Long-term storage and temporary data retrieval cache.

  • MongoDB: Archival storage updated nightly.

5. Architecture

for Set Api

Screenshot from 2024-11-15 14-57-48.png

for Get Api

Request from KP(webhook Key)→ setGetApi (retrieves data from Redis) → Response to KP.

6. Advantages

  • Storing Multiple Webhook Keys:
    The ability to store an arbitrary number of webhook keys enhances flexibility, allowing for different systems or endpoints to receive the same data. This improves the extensibility and scalability of your integration.

  • Reduced Load on Kairos-Products and ActiveMQ:
    By handling webhook keys and managing the distribution of data within this module, the system offloads processing from the Kairos-products module and internal ActiveMQ. This reduces the risk of overloading these critical components, improving overall performance and reliability.

  • Configurable TTL (Time-To-Live):
    The TTL configuration on both the account and flow level allows for greater control over how long data is retained before it expires or is removed. This flexibility is especially useful for handling transient data or controlling cache behavior on an account-level or flow-level basis, ensuring efficient memory and resource usage.

  • No labels

0 Comments

You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.
 

This document has been developed by Phonon.io for the sole and exclusive use of the customer / prospective customer with whom this document is being shared. Further, this document has been provided by Phonon.io to the recipient in good faith and based on request from the recipient for the same. This document is a confidential document and contains confidential product technology, workflow and commercial details that are for the sole usage of the intended recipients of this document. Recipients are advised not to share this document with any third party that is not the intended recipient of this document and neither to bring this document in full or parts into the public domain. Any unauthorized access may be brought to Phonon.io’s notice immediately. Phonon.io is free to take any legal action it deems necessary against any person or entity that violates this confidentiality agreement. Phonon.io is bound and governed by the rules of the state of Gujarat in India. In case you are not in agreement with the terms set in this clause or are not an intended recipient of this document, please destroy the document and intimate us of the same at info@phonon.io.