Campaign Manager Phase 2(Scrubbing) - Technical Documentation
- 1 Requirement
- 1.1 Scrubbing Manager
- 1.1.1 Technical Approach
- 1.1.2 Story Details
- 1.1.3 New Table Required
- 1.2 Exclusion
- 1.1 Scrubbing Manager
Requirement
Scrubbing Manager
Our customers follow specific rules daily before uploading data to our central platform. This repetitive process consumes significant time and leads to inefficiencies.
As a user, I want the ability to create and manage fully customizable data scrubbing filters, so I can define specific validation rules and control system functionality to meet my unique data validation requirements. To address this, a new section, "Scrubbing Management," should be introduced, allowing users to configure and manage scrubbing filters that meet their unique data validation requirements.
Ex . The customer will create a filter based on mobile numbers and upload a file containing 100,000 mobile numbers for filtering. After the filter is created, the customer will upload a file with 500,000 records and apply the filter to the uploaded file. During the processing of the uploaded campaign file, our system will mark 100,000 records as 'SCRUBBED' and will not process them, while the remaining 400,000 records will be processed.
Technical Approach
The file processing on file data will process for KFP and store in storage(Redis).
The data filtering process will be applied on KFP.
All filter TLL must be set in storage, after expiry will be removed from storage.
We need to add limitations on active filters.
We need to add a limit on the apply filter at the time of uploading the file.
At the time of adding condition users can choose Regular Expression OR Operator.
Need to set limits for adding conditions.
Users can upload scrubbing data files multiple times if active.
Need to remove all filter data when the user deletes the filter.
Show all filter lists on Scrubbing Manager if TTL expired ??
Story Details
Sr No | Story Name | URL |
|---|---|---|
1 | Scrubbing Management Tab | Scrubbing Manager | |
2 | Scrubbing Management tab | UI Changes | |
3 | Scrubbing Manager | Upload manager changes Exclusion | Slot calculation | Set timezone | |
4 | Scrubbing Manager | Exclusion file read and store in DB | |
5 | Scrubbing Manager | Filter approval from Support UI | |
6 | Scrubbing Manager | Process Filtered Data File and store in DB |
Module effected :
Web UI
Kairos-Service.
Support-UI
KFP
KPM
Kairos-report
New Table Required
ScrubManagerFilter
FilterRule
ScrubFileUpload
FilterVariable
FilterVariableValue
Modify Table
UploadedFileLog
Detail Description :
ScrubManagerFilter
Column Name | Data Type | Null/Not Null |
filterId | int | Primary key |
filterName | varchar(100) | Not Null |
validity | dateTime | Not Null |
status | int | Not Null |
accountId | Int | Not Null |
createTime | dateTime | Not Null |
FilterRule
Column Name | Data Type | Null/Not Null |
ruleId | int | Primary key |
filterId | int | Foreign Key |
keyVariableName | varchar(100) | Null |
ConditionType | enum | OR/AND |
regularExpression | Text | Null |
operator | varchar(50) | Null |
ScrubFileUpload
Column Name | Data Type | Null/Not Null |
fileId | int | Primary key |
actualFileName | varchar(255) | Not Null |
fileName | varchar(255) | Null |
size | int | Null |
header | Text | Not Null |
uploadedRecords | int | Null |
fileCountRecords | int | Null |
statusId | int | Not Null |
processStartTime | dateTime | Null |
processEndTime | dateTime | Null |
canceledTime | dateTime | Null |
ipAddress | varchar(50) | Not Null |
uploadedBy | varchar(255) | Not Null |
FilterVariable
Column Name | Data Type | Null/Not Null |
variableId | int | Primary key |
filterId | int | Foreign Key |
keyVariableName | varchar(255) | NOT NULL |
endTime | dateTime | NOT NULL |
FilterVariableValue
Column Name | Data Type | Null/Not Null |
Id | int | Primary key |
variableId | int | Foreign Key |
variablValue | varchar(255) | NOT NULL |
Exclusion
Exclusion functionality is on for campaign then after uploading one day slot campaign will be pause state will not execute next slot auto , user first upload exclusion number then un-pause it , what ever number uploaded for exclusion will not process and marks as “EXCLUSION_NUMBER“.
Exclusion will be on at the time of upload campaign.
Upload Exclusion number and un-pause campaign.
KFP File Processing
KPM Process
This document has been developed by Phonon.io for the sole and exclusive use of the customer / prospective customer with whom this document is being shared. Further, this document has been provided by Phonon.io to the recipient in good faith and based on request from the recipient for the same. This document is a confidential document and contains confidential product technology, workflow and commercial details that are for the sole usage of the intended recipients of this document. Recipients are advised not to share this document with any third party that is not the intended recipient of this document and neither to bring this document in full or parts into the public domain. Any unauthorized access may be brought to Phonon.io’s notice immediately. Phonon.io is free to take any legal action it deems necessary against any person or entity that violates this confidentiality agreement. Phonon.io is bound and governed by the rules of the state of Gujarat in India. In case you are not in agreement with the terms set in this clause or are not an intended recipient of this document, please destroy the document and intimate us of the same at info@phonon.io.