Understanding Moving parts of Enterprise AI environments – Turn Complexity into Clarity

 

Generative AI environments and ML systems gets really complex with a lot of moving parts.
What makes AI security complicated? The Answer is moving parts.

Best way to secure AI is to start right now…and..see where you are.

AI Environments are Complex.
AI Environments are Multi-pronged.

To secure AI, First need to understand, what AI system means, actually.
Not from what it means to others.
From what is means to you. Your organization, actually.

“TURN COMPLEXITY INTO CLARITY”

 

Understanding the moving pieces in Generative AI and ML systems

 

 

 

 

The moving parts in the Generative AI and ML Systems very distributed, across clusters, regions, zones even multi-cloud. They are resilent, fault-tolerant., highly cohesive but loosely coupled when performing its functions.

 

 

 

 

 

Data Source-> Model Training-> Inference

 

Data Source- Batch or Streaming

The streaming dataset example Kafka, the batch data source example cloud storages offered by Amazon AWS,Microsoft Azure, GCP.The cloud storages have an entry point to input the data.The data is in raw format and needs to be formatted .The data source acts as the interface point for the feature store to preprocess the data.

Data Extraction – (ETL processing) The required data is Extracted from the data source  by sending in queries.For this purpose , there are tools available like pandas which are python libraries or luigi,to retrieve the needed data.

Data Load -The load module is to forward the retrieved data by using  python libraries on to the warehouse also called data lakes for storage of data.The data may be loaded in batches.The data can be loaded to the warehouse first and retrieved to perform transformation or vice versa.The former approach would be faster and efficient.

Data Warehouses-The data warehouse provides the storage layer. It provides an interfacing point to transform the data.This data warehouse is the centralized repository for the information needed by data scientists.

Data transform -The transformation process is to transform the data to the required format by performing the table operations.The process may  involve aggregation of the data from the table,joining the data or filtering the tables.

Data Science-The formatted data is the resource to use for data science.This formatted data is used to train the model.The data is retrieved from data warehouses and or data lakes and       used for training purposes.

 

 

Security addressed in Data Pipeline 

 

As the data pipelines involves a number of process to get to the formatted training data, it draws many security concerns.Data ingestion,User authentication,account taken over,credentials theft,

crafting irrelevant queries,hindering tables,third party malware plugins when using libraries,data poisoning,network intrusions,ai firewall detections,Inputting biased data.

 

Alert AI addresses the security issues.

 

Model Pipeline

 

Model train-> model deploy-> model evaluate

 

Model pipeline encircles the model training ,model evaluation and validation and its deployment This relates to Model governance and preparing the model for production ready.

 

Model train-The model is trained with the training data.The business intelligence is incorporated and data is used to compute the results. This compiles the model to visualize its performance,strengths and weaknesses.The model is stabilized and run through many iterations to reach the needed accuracy.They are python modules and depends on the context of the input data and applicable to different industries.While there are classifiers for image classifications ,there are LLMs for generative text,text classification and question answering.There are models available for audio and visual training.

 

Model Evaluate and Validation-The models are evaluated for its performance and reliability.The aim is to establish robust models,to give accurate prediction.In case of LLMs we are looking at the correctness of the relevant information,its factuality,and its faithfulness. In the case of image classification we want to get the image identified correctly.The success of the model is based on the score and probability decisions.

 

Model Deploy-Once the model has been verified for its accuracy it becomes Production Ready. and deployable.When actual data is given as input,an AI/ML system is able to output the inference to the user given data.

 

Evaluations Pipeline

 

Analytics->Metrics-> Predictions

This is the last stage of model prediction.The model undergoes a rigorous evaluation to produce the best outcomes.There are quite a lot of evaluation metrics and the AI/ML systems need to choose the right metric for checking the correctness of their model.Depending on the type of classifier and model used, different analytics is performed.Metrics are used to the test the models behavior.During this process if there are any anomalies the model has to be reiterated with the all its steps to fix the anomalies.Some of the evaluation metrics are outlined below:

 

Embedding models relate to the correctness of dialogue generation used in LLMs.

 

Truthful QA to measure the truthfulness of the generated output and to check if the questions are correctly answered used in LLMs.

 

HuggingFace models are used in the evaluation pipeline and hold a repository of evaluation metrics serving models of different kinds.

 

Factual to test the factuality and the relevance of the generated text in RAGs.

 

Perplexity is used in NLP to test how well a probability model predicts the sample. Given the input text sequence, perplexity measures how well the model generates the input text sequence.

 

BLEU(Bilingual Evaluation Understudy) is an algorithm to evaluate the quality of the text when generated from one language to another used in chat AIs.

 

Recall metrics tests if the positive samples were correctly classified as true positives when considered along with false negatives samples used in classifications(TP/(TP+FN)), where TP is true positive,fn is false negative.

 

Hella Swag metric to test the grounded common sense inference used in LLMs.

 

Security in Model Pipeline

 

Crafting proxy models can occur when adversaries gain access to the system.Reverse engineering the model to produce an undesired result occurs if the attackers gain entry.Model injection,poison attack by providing wrong configurations can occur in the model pipeline.Resource consumption by prompting the model with multiple instructions can occur if the intent is to harm the model are some of the security concerns in the model pipeline. Governing the model and safeguarding it becomes essential in the growing world of cyber threats.

 

 

 

 

 

 

 

 

 

 

Alert AI , security platform provides several services that help address these concerns and risks.

Discovery, Visibility, Tracking Analytics

Pipeline analytics

Threat detection and Alerting in AI Footprint and Incidents.

Model behavior Analytics
–Training, Eval, Inference alerts
–Log anomaly detection
–Metric anomaly detection

Alert AI Security platform 360 view and dectections in Gen AI applications and AI Stack:

• Data alerts
• Model alerts
• Pipeline alerts
• Evaluation alerts
• Training alerts
• Inference alerts
• Model Vulnerabilities
• Llm vulnerability
• Privacy
• Threats
• Resources
• Environments
• Governance and compliance

About ALERT AI

What is at stake AI & Gen AI in Business? We are addressing exactly that. Generative AI security solution for Healthcare, Insurance, Retail, Banking, Finance, Life Sciences, Manufacturing.

Alert AI is end-to-end, Interoperable Generative AI security platform to help enhance security of Generative AI applications and workflows against potential adversaries, model vulnerabilities, privacy, copyright and legal exposures, sensitive information leaks, Intelligence and data exfiltration, infiltration at training and inference, integrity attacks in AI applications, anomalies detection and enhanced visibility in AI pipelines. forensics, audit,AI  governance in AI footprint.

Despite the Security challenges, the promise GenAI & ML systems is enormous.
We are committed to enhance the security of Generative AI applications and workflows in industries and enterprises to reap the benefits .