Clifford Chance

The New York State Department of Financial Services (DFS) recently issued a proposed circular letter regarding the "Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing" (PCL).  The PCL incorporates obligations contained in the DFS's 2019 Insurance Circular Letter No. 1 and tracks key themes of the recent National Association of Insurance Commissioners model bulletin on the "Use of Artificial Intelligence Systems by Insurers" and the Colorado regulations on the use of algorithms and predictive models by life insurers.

Other relevant recent developments include draft regulations issued on November 27, 2023 by the California Privacy Protection Agency, which would apply to businesses, including insurance, using automated decision making technology, a bulletin issued by the California Insurance Commissioner on June 30, 2022, expressing concerns that use of AI by insurers could result in unfair discrimination, and a similar bulletin issued by the Connecticut Insurance Department on April 20, 2022.   

The PCL's scope is broader (in that it applies to life and non-life insurers) and more detailed than these other developments, but it is limited to insurance underwriting and pricing and does not include claim settlement practices.  While the PCL aims to address the risks of artificial intelligence (AI) and third-party data use in insurance, it may impact AI regulation more broadly.  The PCL, once finalized, would not have the force of law, but would reflect the DFS' interpretation of existing insurance laws and regulations and its expectations when examining insurers. 

Scope

The PCL applies to licensed fraternal benefit societies, the New York State Insurance Fund, and all "insurers authorized to write insurance in New York State" that use ECDIS and AIS in their underwriting or pricing processes. 

• ECDIS is defined as data used to supplement or as a proxy for traditional medical, property or casualty underwriting or pricing, or to establish "lifestyle indicators" that may contribute to an underwriting or pricing assessment of an insurance applicant.  
• AIS is defined as "any machine-based system designed to perform functions normally associated with human intelligence, such as reasoning, learning, and self-improvement" used to supplement or as a proxy for traditional medical, property or casualty underwriting or pricing, or to establish "lifestyle indicators" that may contribute to an underwriting or pricing assessment of an insurance applicant. 

Fairness

Fairness and antidiscrimination remain important and key focus areas for the insurance industry.  The PCL outlines core fairness principles to which insurers should adhere when using ECDIS or AIS.  An insurer cannot use ECDIS or AIS for underwriting or pricing if such use would be unlawful, unfairly discriminatory, or an unfair trade practice under existing insurance laws and regulations.  Further, an insurer cannot use ECDIS or AIS unless it can establish that the data source or model, as applicable, does not use and is not based on any protected class.  Use of ECDIS and AIS must be supported by generally acceptable actuarial standards of practice and based on actual or reasonably anticipated experience, such as predictive modelling and risk assessments.  ECDIS cannot be a proxy for any protected class. 

An insurer should initially assess:

• whether the use of ECDIS or AIS produces disproportionate adverse effects in underwriting or pricing of similarly situated insureds or applicants
• if there is a prima facie showing of a disproportionate adverse effect, the insurer would need to establish a legitimate, lawful and fair explanation for it
• if a legitimate, lawful, and fair explanation can account for the differentiated effect, the insurer must still search for a less discriminatory alternative or methodology that would meet the insurer's business needs.  

Following the initial assessment, insurers have an ongoing obligation to verify and document processes and periodically test their data sets.

An insurer's documentation of processes and its reasoning in support of testing methodologies and analyses should be commensurate with its use of ECDIS and AIS and the complexity and materiality of its ECDIS and AIS.  The DFS may request to see such documentation.  Testing and analyses should be performed on a regular cadence and insurers are encouraged to use qualitative as well as quantitative assessments, tracking such metrics as adverse impact ratio, Z-tests and T-tests and marginal effects.

Governance and Risk Management

An insurer is required to have a corporate governance and risk management framework that is appropriate for the insurer's nature, scale, and complexity.  A board of directors is obligated to oversee the insurer's use of ECDIS and AIS and ensure the framework's effectiveness.  The board is responsible for the day-to-day implementation of ECDIS and AIS systems, including assigning competent staff and overseeing risk model management.  Senior management should ensure that appropriate lines of reporting are established to provide regular, quality information to the board, and engage all of an insurer's relevant operation areas.  The PCL requires that an insurer have in place appropriate policies, procedures, and documentation necessary for governance and risk management, including training tailored to staff responsibilities and comprehensive documentation detailing the use of ECDIS and AIS.

An insurer is also required to have an internal audit function to provide general and specific tests, reviews, and audits, including assessment of potential biases in the ECDIS or other data, verification of records regarding AIS use, and assessment of supporting operational systems.  If a vendor is engaged in any aspects of underwriting and pricing, the insurer is required to understand such vendor's tools, ECDIS and AIS and is responsible for the vendor's compliance with all applicable laws and requirements.

Transparency

Transparency is an existing guiding principle of insurance regulation and an applicant who is subject to an adverse decision, such as denial, non-renewal or rate increase or differential, has the right to an explanation. In case of an adverse underwriting or pricing decision using ECDIS or AIS, an insurer must provide notice to the insured or potential insured that discloses:

• whether the insurer uses AIS in its underwriting or pricing process
• whether the insurer uses data about the person obtained from external vendors
• that such person has the right to request information, including contact information for such a request.  

Failure to adequately disclose to the insured or potential insured specific reasons for an adverse action may be deemed an unfair or deceptive act or practice and a violation of insurance laws.  The PCL clarifies that an insurer may not rely on the proprietary nature of its or its vendors' algorithmic processes to avoid liability or disclosure.

An insurer should also be prepared to respond to consumer complaints and inquiries about its use of ECDIS and AIS and to maintain records of such complaints, which must be available to the DFS upon request. 

Audits and Examinations by DFS

The DFS may audit and examine an insurer's use of ECDIS and AIS from time to time, including within the scope of regular or targeted examinations, and as part of requests for special reports under applicable insurance laws.

What Insurers Can Do Now

• Review existing processes with respect to ECDIS, AIS, governance, and risk management, including testing procedures and vendor management, to understand potential gaps.  Map potential necessary technical, operational, procedural, financial and people changes.
• Continue horizon scanning with a focus on ECDIS and AIS from other states as we expect that there will be increased legislation and focus on this area in the coming year.
• Consider submitting comments on the PCL to the DFS at innovation@dfs.ny.gov.  The deadline for submitting feedback is March 17, 2024.