Scheduled to go into effect in January 2023, the New York City Council recently passed legislation that seeks to prevent unjust hiring and promotion practices arising from automated decision tools widely used by employers, such as artificial intelligence.

The legislation also includes notice elements that require employers to inform employees or candidates if an automated decision tool was used to make decisions related to employment situations.

Background

Although federal legislation is currently under consideration, there are no laws at that level yet, and City Council Member Laurie Cumbo, a sponsor of the bill, stated at a November 10 hearing of the Council’s Committee on Technology, that it was needed in NYC due to the fact that the city is “home to some of the world’s largest corporations.”

As to the prevalence of the use of automated decision-making tools by such corporations, NYU Center for Responsible AI founding director Julia Stoyanovich recently stated that there is “reason to believe that every major company uses some algorithmic hiring.”

Bias Audits as Primary Method of Holding Automated Employment Decision Tools Accountable

The legislation defines  an automated employment decision tool as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”

As to such automated employment decision making tools, the bill requires that a bias audit be required prior to the use of such tools, with a bias audit being defined as “an impartial evaluation by an independent auditor,” which “shall include but not [be] limited to the testing of an automated employment decision [taken] to assess the tool’s disparate impact on persons.” To be used, the automated tools must have undergone a bias audit no more than one year prior to the proposed use.

Notice and Transparency Requirements

Specific language as to notification requirements holds that the bill will “require that candidates or employees that reside in the city be notified about the use of such tools in the assessment or evaluation for hire or promotion, as well as, be notified about the job qualifications and characteristics that will be used in the automated employment decision tool.” Ms. Stoyanovich, cited above, has highlighted this notice requirement as potentially being more significant than the bias audit.

In terms of further increasing transparency, a summary of the most recent bias audit of an employer’s automated tool as well as the distribution date of the tool to which such audit applies must be made publicly available on the website of the employer or employment agency prior to use.

Compliance and Enforcement

Failure to comply with the law will result in what might be considered relatively low civil monetary penalties, with concern having been voiced by some organizations that such penalties may not provide sufficient deterrent.

The corporation counsel for the city shall be responsible for direct or indirect enforcement and may initiate court action against employers in violation.

Categories: AI, Big Tech in NYC, Comparative Advantage, Digital Economy, Government v Tech, Jobs & Industries, Regulation, Uncategorized