What is AI Trust, Risk, and Security Management (AI TRiSM)?
According to Gartner, AI Trust, Risk, and Security Management (AI TRiSM) is a framework that delivers "AI model governance, trustworthiness, fairness, reliability, robustness, efficacy and data protection." AI TRiSM enables the preemptive identification and mitigation of:
- AI-powered risks
- Security and capability gaps of AI systems.
With AI TRiSM, organizations can contextually respond to the rapidly expanding AI-enabled attack surface by leveraging a consolidated stack of tools. Besides, AI TRiSM can uphlold several ethical principles pertaining to AI innovation and usage.
Also read: How are countries dealing with AI regulation - An overview
Pillars of the AI TRiSM framework
AI TRiSM is the combined deployment of five components:
- Explainability
- ModelOps
- Data anomaly detection
- Adversarial attack resistance
- Data protection
AI TRiSM framework
Explainability:
This refers to the processes that make AI systems, their inputs, outcomes, and mechanisms, unambiguous and understandable to human users. By discarding the black box-like nature of traditional AI systems, Explainable AI's (XAI's) traceability helps stakeholders identify capability gaps and improve its performance. Additionally, XAI reassures users of its reliability and receptiveness to course correction.
ModelOps:
Similar to DevOps, ModelOps refers to the tools and processes that constitute the software development lifecycle of an AI-powered solution. According to Gartner, ModelOps' primary focus lies "on the governance and life cycle management of a wide range of operationalized artificial intelligence (AI) and decision models, including machine learning, knowledge graphs, rules, optimization, linguistic and agent-based models." The components of ModelOps include:
- Design, development, and deployment
- Testing
- Versioning
- Model store
- Model rollback
- Continuous integration and continuous deployment
With ModelOps, AI development becomes scalable by design, and ensures that AI solutions perpetually scale up and undergo holistic betterment.
Adversarial Attack Resistance:
Adversarial attacks manipulate AI and other deep learning to deliver rogue outcomes as bad actors embed erroneous data (known as adversarial samples) into input data sets. These malicious input are indistinguishable from existing data sets, and makes AI systems susceptible to cyberattacks. AI TRiSM hardens AI from adversarial attacks by introducing multiple process, some of which include:
- Adversarial training: By deliberately factoring adversarial samples into the training schema, the AI program gains awareness about potential discrepancies. Adversarial training enables AI to differentiate adversarial data from clean data and eliminate classification errors.
- Defensive distillation: As opposed to training a single model, defensive distillation includes two AL/ML models: the teacher and the student. The teacher is trained on a data set, and arrives at a probability of the model's accuracy in outcome. This probability factor derived from the teacher model is then treated as a baseline for training the student model.
- Model ensembling: Refers to the process of training a consolidated group of multiple models at the same time. Ensembling makes it hard for attackers to target the process due to the difficulty of finding vulnerabilities across multiple AI models.
- Feature squeezing: Squeezing (compressing) the size of an input, such as reducing the color-bit depth of an input image, or spatial smoothing, results in a reduced search space for adversaries.
Data Anomaly Detection
Data is the backbone of an AI program's developmental journey, as it draws its generative and other critical capabilities by analyzing an exhaustive collection of data sets. Compromise of AI data can result in anomalous, inaccurate, and potentially risky outcomes, such as biased results. Data anomaly detection keeps the integrity of AI systems intact by:
- Mitigating errors and related to the training data
- Monitoring and correcting instances of model drift
Data Protection
Much like how data accuracy is crucial to AI, data privacy weighs equal importance, considering its direct impact on users. Fortifying data by applying application security controls,and accounting users' consent in the data processing journey form this integral component of AI TRiSM.
Use cases of AI TRiSM
AI TRiSM has extensive applications for organizations that deploy and/or develop enterprise AI. Environments implementing AI TRiSM can avoid false positives and inaccuracies generated by their AI deployments, while making them reliable and compliant to data privacy regulations.
Building trustworthy AI systems
Several proposed AI regulatory frameworks, such as the European Union's AI Act and the NIST's AI Risk Management Framework, emphasize the need for trustworthiness in AI design. With data and transparency of processes being the core functionalities that inform trustworthiness, AI TRiSM's data protection and explainability capabilities ensure that the AI initiatives of organizations are reliable.
To demonstrate a real-life example on how AI TRiSM capabilities helps build trustworthy AI, let's take the case of the Danish Business Authority (DBA). This organization created a protocol for developing and deploying ethical AI systems which involved two key processes:
- Creating a model monitoring framework
- Periodically monitoring model predictions against fairness tests
By leveraging the framework, the DBA was able to develop, deploy, and manage 16 AI tools that oversee high-volume financial transactions worth billions of Euros.
AI TRiSM implementation: Key requirements to consider
For organizations that are considering implementing AI TRiSM, or any other watchdog framework that governs AI assets, it is imperative to lay down the necessary groundwork to facilitate seamless implementation and functioning.
Skill training
Educating employees on the core technologies of AI TRiSM and bridging the gap of skills plays a crucial role in enhancing human participation, which determines the functioning and optimization of the framework. Effective collaboration between disparate teams holds the key to the seamless transfer of knowledge between employees vis-à-vis expediting employee education. Post training, organizations should form a task force to manage their AI operations.
Having clear documentation
Any expansive architecture starts its journey from a unified standard that defines:
- Risk assessment methodologies
- Scope of the framework
- Use cases
- Best practices
- Continuity and reproducibility plans
- Definition of key processes and terminologies
- Systems overview
- Quality testing methods
Having comprehensible documentation in place helps organizational leaders educate key stakeholders and employees on the AI TRiSM framework and its applications. Besides, documentation standardizes essential processes. To make documentation and governance of AI TRiSM more dynamic, it is important for organizations to collaborate with subject matter experts and policymakers belonging to diverse fields. For instance, to build an AI TRiSM that is efficient in detecting social bias, the organization should have legal and social justice experts on board.
Prioritize AI transparency
Business leaders and C-level executives overseeing the AI journeys within their respective organizations should mandate the necessary toolkits and infrastructure that favors XAI. Some of the important XAI processes include:
- Local interpretable model-agnostic explanations (LIME)
- SHAPley Additive exPlanations (SHAP)
- Algorithmic fairness
- Human-in-the-loop feedback models
- Partial dependence plots
Implement optimal security practices
Organizations looking to deploy AI should minimize their attack surface by deploying foolproof security practices and frameworks within their network. To avoid the over-exposure of AI systems and data sets, organizations should implement the Zero Trust architecture, Secure Access Service Edge, and other security programs that emphasize microsegmentation of critical resources, continuous evaluation and monitoring of assets, and contextual authentication and authorization of users.
Future of AI TRiSM
Gartner's AI TRiSM Market Guide predicts that:
- By 2026, organizations that implement AI transparency, trust and security will see their AI models achieve a 50% improvement with respect to adoption, business goals and user acceptance.
- By 2027, at least one global company will see its AI deployment banned by a regulating body for non-compliance with data protection or AI governance laws.
- By 2027, at least two vendors that deliver AI risk management functionality will be acquired by enterprise risk management vendors that deliver broader functionality.
The guide also charted a five-point market roadmap of the AI TRiSM architecture.
Phase 1 — Model Life Cycle Scope Expansion (2020-2024):
Organizational leaders involved in building the AI TRiSM framework will collaborate with ''data scientists, AI developers, security leaders, and business stakeholders'' to ensure they understand the significance of AI TRiSM tools and can be contextually applied to the security design of the framework.
Phase 2 — Feature Collision (2023-2025):
This refers to the overlapping capabilities present within AI TRiSM. Features including ModelOps, explainability, model monitoring, continuous model testing, privacy functions (including privacy-enhancing technologies) and AI application security overlap with each other.
Phase 3 — Model Management and Feature Convergence (2024-2026):
With the proliferation of AI TRiSM tools, ModelOps vendors are projected to widen their capabilities to accommodate the entire AI model lifecycle.
Stage 4 — Market Consolidation (2025-2028):
As organizations start adopting AI TRiSM tools, Gartner expects the AI TRiSM market to consolidate around two of its key capabilities: ModelOps and privacy functions.
Stage 5 — AI-Augmented TRiSM (2029 onward):
Gartner predicts the introduction of AI-Augmented TRiSM, which orchestrates AI regulation with human oversight.