PITTSBURGH — The Justice Department has been scrutinizing a controversial artificial intelligence tool used by a Pittsburgh-area child protective services agency following concerns that it could result in discrimination against families with disabilities, The Associated Press has learned.revealed potential bias and transparency issues about the opaque algorithm that is designed to assess a family’s risk level when they are reported for child welfare concerns in Allegheny County.
A third person told AP that the same group of federal civil rights attorneys also spoke with them in November as part of a broad conversation about how algorithmic tools could potentially exacerbate disparities, including for people with disabilities. That conversation explored the design and construction of Allegheny’s influential algorithm, though the full scope of the Justice Department’s interest is unknown.
FILE - This image provided by Allegheny County Department of Human Services in Pennsylvania shows the interface for the Allegheny Family Screening Tool, which the county says social workers use to predict the likelihood that a child will be placed in foster care in the two years after they are investigated. Child welfare officials in the county say the cutting-edge algorithmic tool uses data to support agency workers as they seek to protect children from neglect.
Mark Bertolet, spokesman for the Allegheny County Department of Human Services, said by email that the agency had not heard from the Justice Department and declined interview requests. The Allegheny Family Screening Tool was specifically designed to predict the risk that a child will be placed in foster care in the two years after the family is investigated. It has used a trove of detailed personal data collected from child welfare history, as well as birth, Medicaid, substance abuse, mental health, jail and probation records, among other government data sets. When the algorithm calculates a risk score of 1 to 20, the higher the number, the greater the risk.
Allegheny’s algorithm, in use since 2016, has at times drawn from data related to Supplemental Security Income, a Social Security Administration program that provides monthly payments to adults and children with a disability; as well as diagnoses for mental, behavioral and neurodevelopmental disorders, including schizophrenia or mood disorders, AP found.