NYC Issues Final Rules for AI-Based Workplace Decision-Making Tools and (Again) Delays Enforcement

After a few iterations of proposed rules, the New York City Department of Consumer and Worker Protection (“DCWP”) has finally adopted final rules regarding the implementation of NYC’s new law regulating Automated Employment Decision Tools (“AEDTs”).   In announcing the final rules, the DCWP further delayed enforcement of the new law (“Local Law 144”) until July 5, 2023.  NYC employers and employment agencies that utilize AEDTs in their hiring or promotion processes should pay close attention to these final rules and work with vendors to ensure that all covered programs and practices are compliant before the July 5th enforcement date.

Background

As we discussed more fully here, Local Law 144 prohibits New York City employers and employment agencies from using AEDTs in employment decision-making processes unless they take a number of specific and affirmative steps, including: (i) subjecting the AEDT to a bias audit performed by an “independent auditor” within one year of the AEDTs use; (ii) making the results of such bias audit(s) publicly available; and (iii) providing notice to employees and job candidates that an AEDT will be used in connection with a given assessment or decision, and providing instructions for how these individuals can request an alternative evaluation process or a reasonable accommodation.  The law itself is scant on definition or detail, leaving the DCWP to spell out many of the particulars, including what constitutes a covered AEDT, who can conduct an independent bias audit, how an independent bias audit must be conducted, and chiefly how employers can fully comply with all obligations under the law.

Final Rules

The final rules adopted by the DCWP alter some of the earlier September and December 2022 proposed rules, while keeping the majority of the previous proposals intact.  Among the changes from earlier versions are: (i) the DCWP’s expansion of the scope of what constitutes “machine learning, statistical modeling, data analytics, or artificial intelligence” (and thus what might qualify as an AEDT); (ii) modifications to the bias audit standards; (iii) clarification of the type of audit information that employers and employment agencies must disclose; and (iv) clarification regarding when an employer or employment agency may rely on bias audits that use historical data or test data from other employers or employment agencies.

Definition of Covered AEDTs

Local Law 144 defines the term “AEDT” as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation” and that is used to “substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.” 

The final rules further expand upon what constitutes “machine learning, statistical modeling, data analytics, or artificial intelligence” to mean “a group of mathematical, computer-based techniques” that: (i) “generate a prediction, meaning an expected outcome” or “that generate a classification, meaning an assignment of an observation to a group (such as categorizations based on skill sets or aptitude) ,” and (ii) “for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.” Deleted from an earlier version of the proposed rule was the requirement that these techniques have inputs and parameters that “are refined through cross-validation or by using training and testing data” to qualify as an AEDT.

The last prong in the definition of AEDT – “substantially assist or replace” – remains unchanged from the December 2022 proposed rules, meaning that to fall within the ambit of Local Law 144, an AEDT must: (i) rely solely on simplified output (e.g., scores, tags, classifications, or rankings) without consideration of other factors; (ii) use simplified outputs as “one of a set of criteria” that is weighed more heavily than others; or (iii) use simplified outputs that overrule conclusions from other factors including human decision-making.   

Composition and Independence of Bias Audits

When AEDTs are used to select candidates for hiring, advancement, or promotion, the employer or employment agency must ensure that the AEDT has been subjected to a compliant “bias audit.”  The bias audit is meant to evaluate an AEDT’s potential disparate impact on a group of job applicants or employees based on particular demographic categories that mirror EEO-1 reportable data: sex, race, and ethnicity. The final rules clarify that bias audits are sufficient under the law when they:

1. Calculate the selection rate for each demographic category;
2. Calculate the impact ratio for each demographic category;
3. Calculate separately the impact ratio for (i) sex categories; (ii) race/ethnicity categories; and (iii) intersectional categories of sex, ethnicity, and race (e.g., the impact ratio for selection of Hispanic or Latino male candidates vs. Not Hispanic or Latino Black or African American female candidates);
4. Ensure that the above-referenced calculations are performed for each group, if an AEDT classifies candidates for employment or employees being considered for promotion into specified groups (e.g., leadership styles); and
5. Indicate the number of individuals the AEDT assessed that are not included in the required calculations because they fall within an unknown category (this fifth criterion is a newly implemented requirement in the final rules).

The final rules clarify two other aspects of the independent bias audit: (i) that an independent auditor may exclude a demographic category that represents less than 2% of the data being used for the bias audit from the required calculations for impact ratio; and (ii) that where such a category is excluded, the summary of results must include the independent auditor’s justification for the exclusion, as well as the number of applicants and scoring rate or selection rate for the excluded category.

The final rules also provide further clarity regarding the definition of an “independent auditor,” opting to adopt the December 2022 proposed rule on that topic.  Specifically, an “independent auditor” is a “person or group that is capable of exercising objective and impartial judgment on all issues within the scope of a bias audit of an AEDT.”  The rules identify three criteria that would disqualify a proposed auditor under the rules, namely if the proposed person or group: (i) is or was involving in using, developing, or distributing the AEDT; (ii) at any point during the bias audit, has an employment relationship with an employer or employment agency that seeks to use or continue to use the AEDT or with a vendor that developed or distributes the AEDT; or (iii) at any point during the bias audit, has a direct financial interest or a material indirect financial interest in an employer or employment agency that seeks to use or continue to use the AEDT or in a vendor that developed or distributed the AEDT.”

Data Source Requirements

The final rules largely adopt the December 2022 proposed rules with respect to allowing the use of historical data and test data in bias audits.  With respect to historical data, DCWP clarifies that multiple employers using the same AEDT may rely on the same bias audit conducted using historical data of other employers or employment agencies that use the AEDT where the employer or employment agency provides the independent auditor with historical data from its own use of the AEDT, or where the employer or employment agency has never used the AEDT before.  With respect to test data, the employer’s or employment agency’s independent bias audit can rely upon test data in instances where (i) “insufficient historical data is available to conduct a statistically significant bias audit”; and (ii) the summary of results for the bias audit explains why historical data was not used and how test data was generated and obtained.  

Bias Audit Results & Disclosure Requirements

Employers and employment agencies using AEDTs subject to the law’s requirements have notice and publication obligations:

1. Before an employer or employment agency in NYC may utilize an AEDT, it must publish on the employment section of its website (or via a hyperlink to same) the date of the most recent bias audit of the AEDT and a summary of the bias audit results which includes the following information and must remain active on the website until at least 6 months after its latest use of the AEDT’s use for employment decisions:

• The source and explanation of the data used to conduct the bias audit;
• The number of individuals the AEDT assessed that fall within an unknown category;
• The number of applicants or candidates, the selection or scoring rates, as applicable, and the determined bias audit impact ratios for all demographic categories; and
• The distribution date of the AEDT.

2. Employers and employment agencies must provide notice to job applicants and employees who reside in NYC (including those being considered for a promotion), no less than 10 business days prior to its use of an AEDT that an AEDT will be utilized in connection with employment decisions.  The notice must describe the job qualifications and characteristics the AEDT will evaluate in such an assessment and provide instructions on how an individual can request a reasonable accommodation or an alternate selection process.  If not otherwise disclosed on the employer or employment agency’s website, this notice must also provide the type and source of data collected for the AEDT.  Additionally, the employer’s or employment agency’s data retention policy must be made available upon written request.  The final rules also set forth three ways employers and employment agencies can distribute this notice: (i) via its website in a clear and conspicuous way; (ii) via a job posting; or (iii) via an email or regular mailing.  Notice to employees being considered for promotion can also be provided via a written policy or procedure.   

Next Steps

NYC employers and employment agencies now have additional certainty regarding the DCWP’s finalized rules and compliance expectations.  Given the current landscape and the approaching July enforcement date, NYC employers should:

• Review Local Law 144 and the final rules to understand new compliance obligations.
• Assess what categories of automated tools and technologies the employer uses in its workplace decision-making schemes, and determine with counsel whether these are within the ambit of AEDTs and whether this law impacts the employer’s ability to use those tools.  If so, employers should decide whether to continue using such tools (and, if so, how to ensure compliance) or otherwise whether the compliance framework may be too burdensome to justify continued use.
• Review practices and procedures regarding data retention and develop policies and procedures for data retention required by Local Law 144.
• Work with third-party vendors to ensure their compliance with Local Law 144 as necessary, and determine whether any updates are needed to existing service agreements to account for requirements under the law.
• Train Human Resource professionals and other managers involved in hiring and employment decision-making processes to ensure they are familiar with the new legal requirements surrounding AEDTs and are able to address any issues that could arise with the implementation of and/or use of the tools.
• Ensure candidates or employees are provided adequate notice under Local Law 144 when an AEDT is used in connection with a given assessment or evaluation.

In addition, employers should remain mindful that other government entities have taken an increased interest in ensuring fairness in the AI/employment space, including the federal Equal Employment Opportunity Commission (“EEOC”).  Of note, the EEOC recently developed an agency webpage devoted to its nascent “Artificial Intelligence and Algorithmic Fairness Initiative” which focuses on promotion equal employment opportunity in the AI-utilized workplace, and the EEOC has noted this topic as a strategic enforcement priority over the next several years.

For additional guidance on how employers can deploy a holistic approach to AEDTs in the workplace in light of NYC’s law and various other federal, state, and local guidance, see our previous post here.

Subscribe To Viewpoints