Technology & AI1w ago · 105 comments
Should AI be banned from making decisions in courts and hiring?
Faster and free of human bias, or a black box deciding your job and your freedom with no soul and no appeal? Where do we draw the line?
Join the debate to comment
Reading is free. Members pay to post — that's why it stays clean.
Add your comment- Taylor1w ago
I lost a job to an AI screening tool. I know this because the rejection came literally 4 minutes after I submitted my application on a Sunday night. No human read that. I had 11 years of directly relevant experience. I emailed asking for feedback and got a form response. I'm not a person to them. I'm a data point that failed a filter I'm not allowed to see.
- Taylor _x6d ago
This happened to my dad. Laid off at 54, applying everywhere, rejection after rejection within minutes. He thought he'd lost his mind, kept asking what he was doing wrong. It took me months to explain to him that a machine that had never heard of him was deciding he was too old — not in those words but effectively. Watching someone who worked hard their whole life get processed like that broke something in me.
- Priya1d ago
I'm 58. I've been in software engineering for over 30 years. I was laid off last year and the automated screeners won't touch me — I've A/B tested my own resume, younger-sounding language gets more callbacks. The system is actively discriminating against experienced workers and everyone's treating it like an unfortunate side effect rather than a scandal. Age discrimination is illegal. No one cares.
- Yuki1w ago
my cousin spent 14 months unemployed because resume screening software kept flagging her name as 'low priority.' she's a software engineer with a masters. the system literally filtered out the person most qualified to build the system. let that sink in.
- Casey M.5d ago
Public defender. I've seen clients sentenced with the help of risk scores that neither they nor I could examine in full because they were proprietary. The defendant has a constitutional right to understand the evidence against them. A risk score is evidence. 'Trade secret' cannot override due process. Cannot. This is already a constitutional crisis and the courts are moving too slowly to recognize it.
- Marco S.5d ago
lol at 'constitutional crisis.' everything is a constitutional crisis these days. courts have allowed plenty of forms of evidence the defense can't fully interrogate.
- Jamie5d ago
Such as? I'm genuinely asking because I'd like a specific example of a different type of evidence the defense isn't allowed to examine that's actually analogous to a proprietary algorithm scoring someone's likelihood to reoffend. Because I can't think of one.
- Nina5d ago
This is genuinely the most important point in the thread. The sixth amendment implications alone should have shut these tools down years ago. The fact that they haven't tells you something about whose interests the legal system prioritizes.
- Theo4d ago
I work in HR. Not for a tech company, just a regular mid-size firm. We adopted an AI screening tool two years ago. In the first three months it filtered out two candidates who went on to get hired manually after internal referrals — both became top performers. When I raised this with leadership I was told the tool 'performs well at scale.' What that means is: we're optimizing for throughput, not for people. I hate what this industry is becoming.
- Drew1w ago
Former judge here, retired. I'll tell you what nobody else will: judicial discretion isn't always a feature. Sometimes it's a bug. I watched colleagues — good people, smart people — make decisions based on mood, fatigue, personal history with certain defense attorneys. If AI can enforce some baseline consistency I'm not ready to dismiss it wholesale. That said, I'd never let it near the sentencing phase. There's too much that can't be quantified.
- Leo 923d ago
I'm a public defender. I see risk assessment tools used in bail hearings almost every day. Judges treat the scores like gospel. I have never — not once — seen a judge ask to examine the methodology. The algorithm has effectively already replaced the judge, we just haven't acknowledged it publicly.
- Priya1w ago
The COMPAS risk assessment tool. Look it up if you haven't. It was predicting recidivism in criminal courts and was about as accurate as a coin flip — but only for white defendants. For Black defendants the error pattern was systematically different. Journalists had to do an investigative piece to even surface this. The company called their algorithm proprietary. Think about that. A coin flip with trade secret protection.
- Casey1w ago
The coin flip line is a bit misleading. It wasn't random. Random errors would actually be less harmful — the systematic directional bias is precisely what made it so bad. Please don't accidentally soften the actual critique when citing it.
- Iris B.1w ago
I keep seeing COMPAS cited like it proves the case against AI full stop. It proves the case against deploying unaudited, proprietary systems with no transparency. Those are regulatory failures, not proof that the entire category is irredeemable.
- Hana T.2d ago
I study algorithmic fairness for a living. The honest truth my field doesn't like to advertise: there is no mathematically consistent definition of fairness that satisfies all the criteria we'd want simultaneously. It was proven formally in 2016. Any AI system you deploy will be 'fair' by some metrics and discriminatory by others. This is not an engineering problem waiting for a clever solution. It's a fundamental limitation.
- Feli 921w ago
An AI trained on decades of biased decisions doesn't remove the bias, it launders it into 'objective math' nobody's allowed to question. That's worse, not better.
- Hana B.3d ago
My grandfather was a judge for 30 years. I grew up hearing about the weight of decisions, the sleepless nights, the second-guessing. He once overturned his own ruling mid-sentence when a defendant started crying and something didn't add up. An algorithm cannot have that moment. I don't care how accurate your F1 score is.
- Theo S.1w ago
Every time I read this debate I think the same thing: the people most harmed by these systems are the least likely to have a platform to talk about it. The folks building and defending these tools are comfortable. The people getting auto-rejected or risk-scored into longer sentences aren't in this comments section. Keep that in mind when you weigh the opinions here.
- Taylor S.6d ago
I teach computer science. The thing my students consistently underestimate is how much the choice of what to optimize for IS a moral decision. If you train a hiring model to select people who stayed longest at previous jobs, you've baked in a preference against people who left abusive workplaces, against caregivers, against anyone who had a rough patch. That's not neutral. It never was.
- Elena6d ago
okay but human hiring also optimizes for job hopping, length of tenure, prestige of past employers, etc. so the bias was already there. the AI is just faster at applying it.
- Priya S.6d ago
That's kind of the point though? 'The bias was already there so it doesn't matter that we industrialized it' is not the defense you think it is.
- Hana6d ago
This is the point that needs to be in every article about this. The 'objective algorithm' myth is the most dangerous assumption in this whole space. Every model reflects the values — and the blindspots — of whoever designed it.
- Nina1w ago
Worked as a public defender for nine years. The thing nobody talks about is that judges and juries are ALREADY using risk scores to inform bail, sentencing, parole. We just don't call it AI because it makes people uncomfortable. The dishonesty about what's already happening is staggering.
- Marco4d ago
Explainability laws sound great until you realize the companies will just hire lawyers to write explanations that technically comply while revealing absolutely nothing. 'Your application did not meet our scoring criteria.' There. Explained.
- Priya B.4d ago
The mandatory explainability argument is intellectually appealing but practically hollow. What does it even mean to 'explain' a gradient boosted model with 400 features to a layperson? You'd be explaining an explanation, not the decision. It's turtles all the way down.
- Yuki5d ago
I genuinely don't understand the 'free of human bias' claim. An AI cannot be free of human bias because it was made by humans, trained on human-generated data, evaluated by humans, and deployed to serve human-defined goals. 'Free of bias' is not a thing that exists. Anyone selling you that is selling something.
- Ravi5d ago
The correct claim is 'free of certain specific in-context biases like mood, tiredness, physical appearance in a blind resume screen.' That's a narrower and actually defensible claim. 'Bias-free' is marketing.
- Drew2d ago
What bothers me most is the asymmetry. When a human judge makes a wrong call, we can appeal, we can identify the judge, we can call them to account publicly. The wrongness has an address. When the algorithm errs, there's no address, no face, no consequence. The error just... evaporates into policy.
- Diego1w ago
The 'no bias' argument falls apart immediately when you ask who built the training data. Spoiler: humans did. Biased humans. With biased historical records. You don't launder prejudice by converting it to floating point numbers.
- Alex1w ago
The framing of 'ban vs. allow' is itself the problem. We ban specific uses of specific tools all the time without banning the tool entirely. We don't ban cars, we ban drunk driving. Why is AI any different? The nuance is completely missing from this conversation.
- Elena1w ago
ok but who decides which uses are ok? the companies selling the AI? the government? because both of those options terrify me equally
- Marco3d ago
Consistent? A study out of MIT showed the same resume with a traditionally Black-sounding name got lower scores from AI screeners trained on historical hiring data. So your 'consistent mediocrity' is consistently discriminatory. That's not better.
- Priya1w ago
What I want to know is why we're debating this in the abstract when these systems are ALREADY deployed. Amazon used AI hiring tools that systematically penalized women and scrapped it quietly. Courts in multiple US states use risk scoring. The question isn't 'should we allow this.' It's 'what do we do about what's already happening.' Catch up, everyone.
- Morgan R.4d ago
The people most enthusiastically defending AI in hiring have never been unemployed for more than six weeks. I would bet my left hand on it.
- Casey B.4d ago
I've been unemployed for eight months. I have applied to over 200 positions. I have received exactly three responses that appeared to come from a human. I am a person with two degrees and over a decade of experience. I have started to feel like I don't exist. Make of that what you will.
- Priya S.1w ago
Transparency is the crux. If a hiring algorithm rejects you, you should be legally entitled to a plain-language explanation of exactly why. If a risk assessment informs a bail decision, the defendant should be able to see and challenge the inputs. Without that it's just a black box with a veneer of legitimacy.
- Nina1w ago
Some of these models are literally trade secrets. The company that built them has argued in court that the defendant doesn't have the right to examine the algorithm because it's proprietary IP. Intellectual property law and the right to a fair trial are currently in direct collision and courts have largely sided with the IP. Read that again.
- Noah1w ago
It is right. State v. Loomis, Wisconsin Supreme Court, 2016. Upheld use of COMPAS score in sentencing even though the defendant couldn't examine the algorithm. This is real, documented, and terrifying.
- Maya 216d ago
Hot take incoming: we should be MORE worried about AI in hiring than courts. At least in courts there are lawyers, appeals, constitutional protections, public scrutiny. In private hiring there's almost none of that. The sector with fewer rights is the one where the black box runs unchecked.
- Jamie R.2d ago
Falsifiable only if the company releases the model. Which they won't because 'trade secret.' The auditability argument assumes a transparency that structurally cannot exist under current IP law. Fix that first.
- Reese1w ago
Courts specifically are a different animal than hiring. One is a commercial decision, the other is liberty. I could maybe — MAYBE — accept AI as a screening tool for job applications. I will never accept a model having any weight in whether a human being goes to prison. That's a moral red line.
- Elena K.1w ago
COMPAS has been doing exactly that in US courts for years already. This isn't a hypothetical debate. People are sitting in cells right now because an algorithm said their 'recidivism risk' was high. The ship has sailed and nobody checked the tickets.
- Hana M.1w ago
The recidivism scoring tools also have an absolutely documented pattern of racial disparity in their outputs. ProPublica did a whole investigation. This is not a theoretical concern it is a measured, documented harm happening to real people.
- Elena5d ago
Look I've built these systems. From the inside, the optimism about AI neutrality is real and genuine and also completely wrong. Engineers are not trying to launder bias. They mostly just don't see the bias they carry into their feature choices. That's almost scarier than malice. Malice you can prosecute.
- Feli5d ago
Thank you for saying this. The 'it's just engineers being naive' vs 'it's deliberate discrimination' debate misses that systemic harm doesn't require intent. Outcomes are what matter. Good intentions don't undo them.
- Iris2d ago
Because money. That's the whole answer. There are entire consultancies selling bias audits that are methodologically worthless. The certification industry around 'ethical AI' is mostly theater and everyone in the field knows it.
- Omar L.1w ago
They aren't mutually exclusive. You can demand human final authority AND better decision support AND transparency. The framing of this entire debate as AI vs humans is a false binary pushed by vendors who want adoption and activists who want clicks.
- Diego2d ago
Hybrid systems just give humans the illusion of oversight while the AI does all the real filtering. If 90% of applicants are eliminated before a human sees anything, the 'human review' at the end is theater. I've studied this. Automation bias is brutal and well-documented.
- Marco R.1w ago
I'm a hiring manager. The dirty secret is we get 400+ applications for mid-level roles. Humans simply cannot review all of them meaningfully. The real choice isn't 'AI screening vs. careful human review.' It's 'AI screening vs. the first 30 applications a tired recruiter opens on a Monday morning.' Neither is great.
- Casey B.1w ago
That's a real structural problem and the answer is probably 'post fewer jobs with better requirements so the volume is manageable' not 'automate a consequential decision because the volume we've created is unmanageable.' You've described a symptom and called it a solution.
- Hana R.5d ago
My concern isn't AI replacing judges. It's AI replacing the public's confidence in courts. Even if a system were statistically perfect — which it isn't — justice has always required a performance of reason, of deliberation, of human wrestling with hard cases. That performance IS part of the legitimacy. A printout is not.
- Reese K.5d ago
ok philosopher king over here lol. the 'performance of reason' in actual courts often looks like one guy reading a risk assessment score out loud and sentencing accordingly. the theatrical legitimacy you're describing is already gone in a lot of courtrooms.
- Leo4d ago
Both things can be true: current courts are imperfect AND AI in courts makes things worse. 'Existing system has problems' doesn't automatically justify any proposed alternative. This is a logic error that keeps coming up here.
- Drew R.4d ago
This is honestly a point I hadn't considered and I'm still thinking about it. There's something to the idea that visible, explainable deliberation is itself constitutive of legitimacy, not just instrumental to it.
- Riley1w ago
Everyone keeps talking about courts and hiring like they're the same problem. They're not. Hiring is a private company choosing between candidates — that's awful but survivable. Courts are the STATE depriving you of LIBERTY. These need completely different conversations and different legal frameworks. Mixing them muddies both.
- Riley _x6d ago
Disagree actually. Both are gatekeepers to fundamental life outcomes. Unemployment, homelessness, despair — these aren't minor inconveniences either. Access to employment is as structurally important as many rights we protect legally. The 'it's just private' framing is a cop-out.
- Liam R.6d ago
Hard agree. The constitutional stakes in criminal sentencing are categorically different from resume screening. One is inconvenient. The other is years of your life. Treating them as one debate is a category error.
- Avery1w ago
I work in HR and honestly? The bias problem cuts both ways. Our human interviewers consistently favor people who went to the same universities they did, share hobbies, even have similar accents. At least an algorithm's biases can theoretically be audited and corrected. Your gut feeling cannot.
- Diego T.1w ago
The company that sold it? The company that bought it? Definitely not the person it rejected. Nobody is auditing these things in any meaningful way and you know it.
- Hana3d ago
The 'reasonable middle' in questions of liberty and livelihoods isn't a compromise, it's a capitulation. Some things should be decided by humans for humans, period. The efficiency argument has been used to justify every erosion of individual rights in the last century.
- Riley L.1w ago
The deeper question nobody's asking: even if AI decisions are MORE accurate, do we have a right to be judged by something that cannot understand us? Like a philosophical right to human judgment regardless of outcome quality? I think we might. And I think that matters.
- Drew L.1w ago
this is a great question and i have genuinely no idea what the answer is which is rare for me on the internet
- Liam 921w ago
Respectfully, 'the right to be judged by a human' sounds philosophical until you remember that human judges have sentenced innocent people to death. At some point outcomes have to matter more than the process feeling emotionally meaningful.
- Ravi4d ago
The real answer nobody wants to say: we need mandatory explainability laws. Not bans. Any automated decision affecting employment or liberty must be explainable to the person affected, in plain language, with a genuine right to challenge it before a human. That's it. That solves 80% of the legitimate concerns raised here without throwing out tools that do have real value. Why is this not already law.
- Morgan2d ago
Imagine defending the status quo by saying 'bans are blunt instruments' when the status quo is that a black-box model can already end your career or extend your prison sentence and you have no recourse whatsoever. Sometimes blunt is appropriate.
- Diego R.3d ago
That story is beautiful and also completely irrelevant to whether the average judge, not your exceptional grandfather, should be trusted over a well-designed model. We don't build systems for the best-case human, we build them for the median one.
- Noah _x1w ago
The scary part isn't AI deciding, it's that there's no one to blame when it's wrong. Accountability is the whole point of a system, and the machine has none.
- Avery2d ago
Counterpoint to the 'bias laundering' argument: human decision-makers also cannot explain their decisions in any rigorous sense. 'Gut feeling' has decided more court cases and job offers than anyone wants to admit. At least with an algorithm you have something falsifiable to audit.
- Zara2d ago
okay but appeals courts overturn like 20% of cases which means 80% of wrong decisions by human judges just... stick. so the accountability you're describing is extremely theoretical for most people who can't afford to appeal
- Diego1w ago
Hard ban. Full stop. Some decisions are too consequential to outsource to a system that cannot understand context, remorse, circumstances, or the simple human fact that people change.
- Jordan4d ago
you people talk about this like it's hypothetical. in china social credit systems are already doing exactly what you're debating here. you have maybe a decade before this is normalized everywhere and then there's no debate to have
- Elena1d ago
Courts and hiring are not the same thing and treating them identically in this debate muddies the water. Hiring is bad and getting worse, agreed. But courts are about liberty, bodily autonomy, years of someone's life. The stakes are categorically different and they need different standards. Can we please separate these two conversations.
- Leo M.5d ago
The EU AI Act actually bans certain high-risk AI applications in justice and employment with no carve-outs. So there are jurisdictions doing exactly this. Whether it works is a different question but let's not pretend a ban is inconceivable or unprecedented.
- Elena1w ago
Is anyone going to acknowledge that AI tools also catch human bias? I work in HR and before we implemented structured scoring, our department had a documented tendency to interview candidates with certain university names at twice the rate of equally qualified candidates without them. The AI didn't create that problem. It was already there, invisible, shaping every hire.
- Riley1w ago
Catching bias in humans vs. not introducing new bias yourself are two completely different bars and you're conflating them. A biased tool that is 'less biased than your specific bad process' is not a vindication of the tool.
- Casey6d ago
Also worth noting: when a human HR manager shows bias, you can potentially challenge it, get a lawyer, file a complaint. When an algorithm does it, the company says 'that's just our process' and you have essentially no recourse. The legal framework to fight algorithmic discrimination barely exists.
- Zara1w ago
Human judges are tired, hungry, and provably harsher before lunch. 'Soulless' might be exactly what justice needs. The bar isn't perfection, it's better than us.
- Iris1w ago
Ban them from both. I said what I said. These are the two highest stakes individual decisions most people will face — keeping their income and keeping their freedom. If we can't protect human judgment there, we might as well admit the experiment failed.
- Zara1w ago
Define 'ban.' Because if you mean 'no algorithm can touch any part of the process' then you're also banning spell-checkers on application forms and search tools on court databases. The word does a lot of work and people use it sloppily.
- Theo M.1w ago
You know exactly what people mean and you're being deliberately obtuse. Nobody is conflating ChatGPT with a spell checker and this bad faith hair-splitting is a technique to exhaust the opposition, not actually engage with it.
- Drew3d ago
Hot take: AI in hiring is actually MORE democratic than the current system. Right now getting past a screener depends enormously on whether you went to the right school, know the right people, or have a name that sounds familiar. At least an algorithm is consistent. Consistent mediocrity beats wildly inconsistent nepotism.
- Feli T.2d ago
The question 'should AI be BANNED' is doing a lot of work here. Bans are blunt instruments. The real policy question is what specific uses require what specific safeguards and who enforces them. Collapsing it all into 'ban vs allow' is how we get bad legislation written by people who don't understand the technology.
- Ravi1w ago
I think people are conflating 'AI helps decide' with 'AI decides alone.' Nobody serious is proposing fully autonomous sentencing. The actual proposal is AI as decision support — flagging things a human might miss. That's categorically different and the hysteria isn't helping the real debate.
- Zara1w ago
The problem with 'AI as support tool' is decades of research on automation bias showing humans overwhelmingly defer to whatever the algorithm suggests, especially under cognitive load. In practice 'the AI recommends' becomes the decision. The human becomes the rubber stamp.
- Leo1w ago
^^^ THIS. There's a reason they call it automation bias. Judges, doctors, pilots — once a system gives a confident recommendation, overriding it requires active effort that most humans under pressure won't make. The 'human in the loop' often isn't really in the loop at all.
- Iris1w ago
So the solution is better training and process design, not a blanket ban. Doctors use diagnostic AI that dramatically improves outcomes. The answer is governance, not prohibition.
- Maya 211w ago
Governance. Sure. From the same regulators who still don't fully understand what an algorithm does. From the same legislatures where the median age is 63. Peak copium.
- Theo2d ago
if this is actually true and not just academic posturing then why are companies still selling 'fair AI' products. genuinely asking
- Nina3d ago
why does everyone frame this as AI vs humans like those are the only two options. what about AI that flags decisions for mandatory human review when confidence is below a threshold. hybrid systems exist and nobody in this thread has mentioned them
- Hana5d ago
If we banned everything that could be abused or misconfigured we'd ban cars, prescription drugs, and kitchen knives. The solution is regulation, transparency requirements, mandatory audits, and right-of-appeal for affected individuals. A ban is intellectually lazy.
- Riley K.5d ago
The kitchen knife comparison is doing a LOT of heavy lifting here. A knife used badly injures one person. An AI system deployed at scale makes the same error millions of times before anyone catches it. Scale is a morally relevant variable and your analogy erases it.
- Theo5d ago
also 'just regulate it' has been the answer to every tech concern for 30 years and the regulation almost never comes or comes too late to matter. not exactly a track record that inspires confidence
- Theo _x1w ago
I got a job through an AI screening system and I think it was actually fairer than previous processes I'd been through? The questions were standardized, there was no small talk that could go weird, no one could decide they didn't like my handshake. I don't know. My experience was positive and I feel weird about this debate.
- Casey T.1w ago
Individual positive experiences don't tell us much about systemic effects. Glad it worked out for you genuinely. But 'it worked for me' is the same reasoning people use to oppose disability accommodations, paid leave, all kinds of things. What worked for you may exclude others in ways neither of you can see.
- Reese R.3d ago
this is one of those debates where both extremes are obviously wrong and the reasonable middle is what everyone's pretending they can't find because outrage is more fun
- Casey2d ago
I got hired through an AI system and it's been the best job I've ever had. So not every story here is a horror story. Just saying.
- Avery 921w ago
okay but statistically speaking if an AI correctly predicts outcomes 80% of the time and humans only do it 65% of the time, isn't that just... better? like im not saying its perfect but better is better right
- Feli1w ago
also where did those accuracy numbers come from. who measured them. against what baseline. im begging people to ask these questions before citing percentages like they fell from heaven
- Leo1w ago
That 20% you're so breezy about is hundreds of thousands of wrongful rejections, extended sentences, missed paroles. 'Better on average' is meaningless comfort to the individual it destroyed. Statistics aren't justice.
- Alex1w ago
Got auto-rejected by a hiring bot for a job I was perfect for, no human ever saw it. Try appealing to an algorithm. There's no one home.
More debates people can't stop arguing about
Is it wrong to choose not to have children just because you don't want to?
110 comments
Netflix's 'Adolescence' shows a teenage boy becoming a killer — did the show actually get modern boyhood right, or did it miss the point entirely?
109 comments
Would you still eat meat if you had to kill the animal yourself?
109 comments
Is judging someone by their height as shallow as judging by their weight?
108 comments