Levi, Ray & Shoup, Inc.

Are you ready for AI model risk management?

9/22/2022 by Steve Cavolick

A flurry of recent news stories has focused a spotlight on the ethical use of AI and data and necessitates the need for organizations to have risk management processes around AI.

Cases such as one with WW International (the company formerly known as Weight Watchers) will become more common. The FTC ruled in March of this year that the company must destroy AI algorithms that were built using information collected from children as young as 8 without parental permission. WW International was also fined $1.5 million and ordered to delete the illegally collected data.

In another example, New York City passed a law that goes into effect in 2023 which will prohibit companies from using AI algorithms for hiring or personnel promotions unless it has passed a third-party bias audit within the past 12 months.

Spurred by the pandemic, organizations accelerated digitalization of their business and now data and AI are at the heart of most business models. Knowing that AI and algorithm regulation is coming, it is important for your organization to understand the implications of the laws and prepare for them.

LRS believes there are four things you can do now to ready yourself for the coming AI regulation.

The first step is to establish an AI risk management program that is integrated with business operations. This should include establishing an AI governance structure that is tied in with data risk management. Organizations will need to inventory all AI systems, assign each one a risk value, and have risk mitigation plans in place. This will also include independent audits of algorithms.

Secondly, companies should automate the process of creating AI models. Products like IBM’s AutoAI automates the data preparation, model building and deployment process. The best part is that AutoAI requires no coding. Just select the field or fields in your data you want to predict and AutoAI does the rest. It creates multiple model pipelines and you can select the best one based on metrics that the platform calculates for you. Workers on audit or risk teams can also use AutoAI to create challenger models to validate that the correct model is chosen for deployment.

Once a model is deployed, products such as IBM’s OpenScale automate the testing of algorithms that are in production. It can detect bias and drift and generates a comprehensive model explanation. This eliminates the need to write and execute test scripts so you can focus on strategic challenges and provides the foundation for AI model audits.

Finally, it is important to avoid putting AI risk in its own silo. Many companies are using Governance, Risk, and Compliance (GRC) platforms, such as IBM OpenPages, to manage all regulatory and risk challenges. OpenPages has a Model Risk Governance component that maps your AI models to regulatory requirements across jurisdictions and geographies. With this approach, you can track model issues, assign roles for model ownership, and report on model inventories from a central place as part of an enterprise GRC program.

AI regulation has arrived, but more is coming. To avoid fines, reputational harm, and the mandated destruction of data and models, you will need capabilities at your fingertips to ensure AI models are free from bias, can easily be explained to business users, and are auditable.

If you are interested in learning more about how LRS can help you put governance around your data, build AI models and measure them for bias over time, and help you reduce risk from AI to your organization, please contact us to request a meeting.  If you are not using AI yet and don’t know how to get started, our strategic roadmapping services will help identify the gaps in your analytics approach that prevent you from achieving your business goals and recommend the capabilities you need to attain them.

About the author

Steve Cavolick is a Senior Solution Architect with LRS IT Solutions. With over 20 years of experience in enterprise business analytics and information management, Steve is 100% focused on helping customers find value in their data to drive better business outcomes. Using technologies from best-of-breed vendors, he has created solutions for the retail, telco, manufacturing, distribution, financial services, gaming, and insurance industries.