New York Department of Financial Services Proposes Circular to Address Insurance Companies’ Use of Artificial Intelligence

On Jan. 17, the New York Department of Financial Services (“NYDFS” or “Department”) issued for public comment a proposed circular letter (“Circular”) for addressing the use of artificial intelligence systems (“AIS”) and external consumer data and information sources (“ECDIS”) by insurance companies licensed by the State of New York. The NYDFS acknowledges that AIS/ECDIS can expedite and improve the underwriting and pricing of insurance products. The Department is desirous, however, that AIS/ECDIS not be used in an unfairly discriminatory manner. The Circular, not surprisingly, is long on compliance requirements but short on specifics regarding the uses of AIS/ECDIS and means of measuring bias.

The Circular addresses AIS/ECDIS used to supplement traditional medical, property, and casualty underwriting and pricing. It does not address uses of AIS/ECDIS in other aspects of the life cycle of insurance products, including marketing and claims where it has been most widely used.

Unlawful Discrimination

An insurance company contemplating the use of AIS/ECDIS will need to first establish a “governance and risk management framework” that ensures the underwriting or pricing guidelines deployed (as modified by AIS/ECDIS) do not unfairly discriminate against similarly situated individuals or any protected class. The insurer’s initial assessment would include:

• Whether the use of AIS/ECDIS produces disproportionate adverse effects (e.g., denials or increased pricing) in underwriting or pricing of similarly situated insureds, applicants, including insureds/applicants of a protected class;
• If there is a prima facie showing of a disproportionate adverse effect, the insurer would need to establish a legitimate, lawful, and fair explanation for the differential effect on similarly situated insureds/applicants;
• If a legitimate, lawful, and fair explanation can account for the differential effect, the insurer must still search for a less discriminatory alternative variable or methodology that would still meet the insurer’s business needs.

The first two prongs of the initial assessment currently apply to all underwriting and pricing of insurance products in New York and elsewhere. The third prong leaves the Department with the discretion to challenge even lawful discrimination, based on sound actuarial underwriting and pricing.

If the initial assessment determines that the insurance company’s use of AIS/ECDIS is not unfairly discriminatory it has ongoing obligations to verify and document its processes as well as periodically test its data sets:

• Documenting the processes and reasoning behind its methodologies and analyses used for determining unlawful discrimination;
• Testing for unlawful discrimination on a regular cadence or when changes are made in the use of AIS/ECDIS;
• Evaluate data and model outputs using multiple statistical metrics that may include:
  • Adverse impact ratio that analyzes rates of favorable outcomes between protected classes and control groups
  • Denial odds ratios that compute the odds of adverse decisions for protected classes compared to control groups
  • Marginal effects assessing the effect of marginal change in a predictive variable on the likelihood of unfavorable outcomes, particularly on protected classes
  • Standardized mean differences that measure the difference in average outcomes between protected classes and control groups
  • Z-test and T-tests that ascertain whether differences in outcomes between protected classes and control groups are statistically significant
  • Drivers of disparity identifying variables in AIS that cause differences in outcomes for protected classes relative to control groups

None of these statistical metrics are mandated, perhaps to provide an insurance company flexibility in measuring bias, depending on the insurance product and data used. The comment period will undoubtedly highlight each metric’s usefulness and accuracy in identifying unlawful discrimination.

An insurance company should be able to readily demonstrate that its use of AIS/ECDIS is supported by generally accepted actuarial standards and based on actual or “reasonably anticipated experience,” including “statistical studies, predictive modeling, and risk assessments.” Moreover, an insurer must demonstrate that the ECDIS used for underwriting and pricing does not serve as a proxy for any protected class.

Governing Framework

The Circular requires a governance framework commensurate with the “nature, scale, and complexity of the insurer.” This is good news for middle market insurers not content to wait for an AXA or a State Farm to set the standard for AIS/ECDIS compliance. Regardless of the size of the insurer or the complexity of its use of AIS/ECDIS, the framework requires (i) significant participation of senior management and the board of directors (most will undoubtedly create a board subcommittee); (ii) written policies and procedures for its use of AIS/ECDIS; (iii) clearly defined management roles and responsibilities; (iv) periodic employee training; and (v) a comprehensive account of its uses and implementation of AIS/ECDIS. The Circular’s description of a comprehensive account suggests the equivalent of a diary or journal, although those terms are not used. The diary would provide a real-time account of the company’s uses, methods, implementation, outcomes, findings, and changes in connection with AIS/ECDIS, especially its testing for unlawful discrimination. The diary is to be updated at regular intervals and to capture any substantive changes. Given the speed at which AI is advancing and the law’s attempt to keep pace, the diary would seemingly be a continual work in progress for the foreseeable future.

Transparency

Transparency has long been a guiding principle of insurance regulation. An applicant subject to denial, non-renewal, rate differential, or rate increase, has the right to an explanation for such an adverse decision. This ensures that the applicant is aware of the data being used to underwrite one’s life, house, auto, etc., and can correct any misinformation used (e.g., driving record, criminal history, medical or credit report).  The Circular relies on these existing regulations to address transparency of insurers’ use of AIS/ECDIS. The new issue is how readily an insurer (or its third-party vendor) can provide the details (in plain English) of the information used in any declination, limitation, rate differential, or other adverse underwriting decision. This may be difficult in cases where it is not a single factor, but a combination effect of two or more factors leading to the adverse decision. Moreover, the Circular states that an insurer may not rely upon the proprietary nature of its (or a vendor’s) algorithmic processes to evade disclosure or avoid liability by relying upon the vendor’s attestation of non-biased data or processes in underwriting and pricing.

Aside from the technical difficulties, the most successful insurers are typically also the best underwriters and keep their advantages confidential. The insurance industry will have to fashion a process whereby an insurer can readily explain its reasoning for adverse decisions to both the NYDFS and applicants without disclosing its AIS/ECDIS competitive underwriting advantages.

Conclusion

The comment period closes on March 17, 2024. Insurers and their AI vendors are already moving quickly to address some of the issues addressed in this Alert. Clark Hill and Howden Tiger advise clients on pending circulars, insurance regulations, and creating governing frameworks for the use of AIS/ECDIS.

This publication is intended for general informational purposes only and does not constitute legal advice or a solicitation to provide legal services. The information in this publication is not intended to create, and receipt of it does not constitute, a lawyer-client relationship. Readers should not act upon this information without seeking professional legal counsel. The views and opinions expressed herein represent those of the individual author only and are not necessarily the views of Clark Hill PLC. Although we attempt to ensure that postings on our website are complete, accurate, and up to date, we assume no responsibility for their completeness, accuracy, or timeliness.