What AI Is Doing to Analytical Judgment

↓

​Learn more about the People Analytics course →​

Hi Friends,,

You have seen this before.

Your CHRO pings you:

"Why did regrettable turnover among mid-terure software engineers jump 4% last quarter?"

Before, or maybe even today, the answer would take about 5 steps and maybe a few days.

1. Pulling raw HRIS data
2. Verifying tenure buckets and exit survey timestamps
3. Running cross-tabulations against performance ratings
4. Cleaning tenure data
5. Piecing together a story

Today?

You can open an AI assistant, upload your turnover data, and type: "Analyze our turnover data and tell me why engineering attrition spiked."

Thirty seconds later (+/-), you have a well formatted, bulleted response:

• Compensation competitiveness fell behind market rates
• Manager feedback frequency dropped in Q1
• Return-to-office mandates created friction
• Career growth sentiment dipped among 2–4-year tenure bands

Looks about right.

The points make sense.

And, you can easily just copy and paste the bullets into a deck.

But...

Did you just perform an analysis, or did you just accept a story?

The "Good Enough" Problem:

In a recent column, writer John Nosta explored a concept that should haunt anyone working in data: the line between Iterative Intelligence and Cognitive Surrender.

h/t to Montarious Usher for pointing me to this piece.

Nosta wasn't really writing about analytics; instead, he talked about human behavior with AI tools:

When AI gives us an answer that is obviously wrong, we challenge it.

We fix it. We engage.

When the output is plausible, however...

When an answer looks reasonable,

clean,

and analytical,

human brains default to cognitive offloading.

We don't engage.

Instead, we accept the machine's certainty because working through the problem ourselves requires effort.

Yes, the good old effort.

In People Analytics, this is the Plausibility Problem.

When you read that 30-second turnover summary, did you verify whether:

• The sample size of leaving engineers was reliable?
• Engagement survey drops occurred before employees submitted resignations, or after they mentally checked out?
• RTO friction was confounded by the fact that remote engineers were concentrated in legacy business units facing budget cuts?
• The data was even capable of answering "why" in the first place?

The AI generated a plausible narrative.

It didn't generate a real contextualized explanation.

And because the narrative arrived instantly, it eliminated the very friction that used to force analysts to understand their data better than anyone else in their business.

Technical Friction vs. Intellectual Friction

Historically, working with workforce data required immense technical friction. Believe me, I know.

• Data Cleaning
• Reconciling
• Debugging code
• Formatting chart axes

Automating technical friction is fantastic.

No analyst should spend four hours manually cleaning survey timestamps if a model can handle it in four seconds.

The problem arises when we confuse technical friction with intellectual friction.

Intellectual friction is the work of reasoning:

• Questioning whether a correlation is meaningful
• Testing competing hypotheses and counterfactuals
• Spotting selection bias in exit surveys
• Knowing when the available data cannot support a causal claim

When analysts use AI to bypass technical friction, they accelerate their work.

When they use AI to bypass intellectual friction, they trade real understanding for fake certainty.

Knowing People Analytics means moving beyond simply accepting plausible-sounding narratives to actually testing hypotheses and validating data rigor.

Cognitive Surrender vs. Iterative Analytics

The goal isn't to stop using AI in People Analytics.

That debate was over before it started.

The shift is moving from Cognitive Surrender (asking AI for the final answer) to Iterative Analytics (using AI to sharpen your own thinking).

Here is what that difference looks like in practice:

Scenario 1: Attrition Drivers

• Cognitive Surrender:
  • "Look at this turnover dataset and tell me what's driving attrition."
• Iterative Analytics:
  • "Engineering turnover rose 4%. My hypothesis is that the increase is isolated to engineers with 2–4 years of tenure who were impacted by RTO policy changes. What evidence in this dataset would support or contradict that hypothesis? What alternative explanations should I test?"

Scenario 2: Engagement Surveys

• Cognitive Surrender:
  • "Summarize the key takeaways from these 500 employee survey comments."
• Iterative Analytics:
  • "I'm seeing a drop in career-development sentiment among sales reps. Summarize the open-ended comments for that specific group, but highlight any instances where comment sentiment contradicts their quantitative scores."

Scenario 3: Predictive Modeling

• Cognitive Surrender:
  • "Build me a model predicting flight risk for our sales team."
• Iterative Analytics:
  • "Before we build a retention risk model, help me audit what decisions this model will actually inform, what data would realistically be available at prediction time, and where target leakage might corrupt our predictions."

In the surrender model, AI generates your conclusion.

In the iterative model, AI acts like a skeptical senior colleague sitting across the table, trying to prove your conclusion wrong.

AI as a Sparring Partner

If you run a People Analytics team or lead an HR function, the most valuable way to deploy AI today is as an adversarial reviewer, devil's advocate, a person who can challenge you.

Instead of prompting AI to write your narrative, try prompts like these:

• "Here is my conclusion on why sales attrition increased. Give me 5 alternative explanations that would fit the exact same data patterns."
• "What implicit assumptions am I making in this compensation analysis that could be wrong?"
• "Pretend you are an unpersuaded CFO. Attack the methodology of this workforce recommendation."
• "What confounding variables could explain this relationship that I haven't accounted for?"

This brings human judgment back into the center of the loop via iteration.

The AI doesn't replace the analyst's sensemaking.

Instead, it accelerates cognitive development of your team.

The People Analytics AI Playbook If you want your team to build AI literacy without sacrificing analytical judgment, institute these five operational rules:

1. Hypothesize First, Prompt Second: Never ask AI to analyze a dataset before you have written down at least two plausible human hypotheses.
2. Require Alternative Explanations: Every major analytical finding presented to leadership must include at least two competing explanations that were tested and disproven.
3. Audit the Data Lineage: If an AI assistant highlights a pattern, the analyst must manually trace that pattern back to raw, un-aggregated data before it enters a report.
4. Use AI to Challenge, Not Confirm: Default to using LLMs as skeptical peer reviewers rather than report generators.
5. Acknowledge Data Boundaries: Explicitly state what the dataset cannot answer. AI loves to invent certainty where sample design only supports speculation.

If you're trying to figure out what this should look like inside your organization—moving from automated reports to genuine analytical judgment—that's a conversation we can have.

We work with organizations on People Analytics capabilities, workforce planning, and using workforce data to make better decisions. ​

And yes, we do help teams think when they are using AI.

​→ Learn more about our consulting work​.​

K

Whenever you’re ready, there are 2 ways I can help you:

#1

If you’re still looking to get started in People Analytics, I recommend starting with my affordable course:

​Practical People Analytics: Build data-driven HR programs to 10x your professional effectiveness, business impact, and career. This comprehensive course will teach you everything from building an HR dashboard for business results to driving growth through more advanced analytics (i.e., regression). Join your peers today! ​
​

#2

If you are looking for support in your human capital programs, such as engagement, retention, and compensation & benefits, and want to take a more data-driven approach, contact me at Tskhay & Associates for consulting services. Or simply reply to this email!