It was an ordinary afternoon when my American professor, a man of sharp intellect and gentle disposition, shared an article with me from The Atlantic. The piece narrated a troubling experience: a researcher who had sought Kierkegaard quotations from ChatGPT discovered that most of them were fabricated. His remark, “ChatGPT was lying about me,” rang like a verdict on the credibility of generative AI.
The professor asked what I thought of this dilemma. He was worried not merely about the spread of misinformation but about what such episodes mean for human trust, intellectual integrity, and the ethics of relying on generative AI to reproduce human wisdom.
I humbly responded, “Yes, I agree to disagree with Yair Rosenberg’s piece. I agree, because ChatGPT indeed hallucinates. And I disagree, because to say it ‘lies’ is to attribute to AI a moral capacity it simply does not possess.” This exchange turned into a long dialogue, not just between two individuals but between humanity and technology.
The issue, therefore, is not whether AI “lies,” but whether humans abdicate their responsibility to verify.
Generative AI, whether ChatGPT or its countless successors, is not designed to be a moral agent. It neither lies nor tells the truth in the human sense. What it does is predict patterns of language-an act mechanical, not moral. To call this “lying” is to personify it, to attribute to a machine a distinctly human capacity. A fabricated Kierkegaard quote remains a misleading string of words. Once circulated, it misguides readers and erodes the integrity of scholarship. The issue, therefore, is not whether AI “lies,” but whether humans abdicate their responsibility to verify.
I told my professor, “GenAI’s strength is also its weakness. It generates continuous, balanced, oversampled text-content that looks convincing even when it is not. Unless one pauses to verify, the boundary between fact and fabrication collapses.” This also raises an important question: “How many of us are skilful enough to prompt AI?”
The professor raised another point regarding the anxiety among English writers. Many established authors now feel cornered by AI’s relentless productivity. A machine that can produce endless drafts, summaries, and structured thinking threatens the exclusivity once associated with human creativity.
I responded, “Many native writers, both well-known and lesser-known, are more or less frightened. I believe AI is narrowing the gap between native and non-native English writing. With effective integration, non-native authors can articulate complex ideas in polished English without feeling inferior. Isn’t that a democratic achievement?”
Should we acknowledge this levelling effect? It is not merely a linguistic breakthrough; it is a social one. The gatekeepers of literary prestige, long protected by language hierarchies, are being challenged by an algorithm that democratizes expression. But democratization unsettles both established and emerging authors. It forces us to confront a hard question: Do we fear AI because it is flawed, or because it destabilizes privilege? However, I do not support violations of copyright.
At this point, the professor reminded me of a larger concern: “Who owns AI? Who benefits?” He spoke of Elon Musk’s boast of wielding a “chainsaw” against the federal budget during his Washington stint-a chainsaw greased with AI efficiencies that destroyed livelihoods. Thousands of public servants saw their careers vanish, and with them, the fragile safety nets of their families.
This is the paradox. AI is marketed as a tool for the public good but is largely controlled by ultra-wealthy Western men whose fortunes and influence grow with each technological leap. AI amplifies their power, often at the expense of the very masses it promises to serve. I mentioned to him: “The danger is not AI itself but its concentration in a few hands. Left unchecked, it becomes a lever of inequality, widening the gulf between the powerful and the powerless.”
If AI is not human, then why do we choose human honorific and euphemistic expressions to prompt it? Why do we forget the elementary responsibility of verification? The Kierkegaard episode is only one reminder that human validation is not optional-it is mandatory. GenAI produces drafts, ideas, and scaffolds for thinking. But it cannot weigh moral consequences, contextualize cultural sensitivities, or understand the ethics of representation. That responsibility remains ours. To outsource judgment to AI is to abandon what makes us human.
I insisted to my professor, “Generative AI is still young. It requires time to grasp the complexities of truth and meaning. Until then, its role is supplementary, not substitutive. It is a tool for thought, not a substitute for thinking-or for a Google search among other databases.”
However, it would be unfair to dismiss AI entirely as a “hallucination machine.” Despite its flaws, AI has accomplished remarkable feats. It helps especially ESL/EFL students in the writing process and engages them in research excellence. It helps educators in lesson planning and assessment. It is even available for those who wish to “date” it.
AI brings both opportunities and challenges. Like a hammer, it can build a home or smash a window. In the same vein, AI can democratize knowledge or entrench disinformation.
The professor acknowledged when I said, “What matters is not whether AI is perfect but whether humans are wise enough to use it responsibly.”
Where does this dialogue leave us? We must resist the temptation to either idolize or demonize AI. It is neither a savior nor a monster. It is a mirror-reflecting opportunities and challenges.
Educational institutions must teach students not just how to use AI but how to critique it. Verification, critical reading, and ethical awareness must become core skills in the AI age. Policymakers must confront the political economy of AI. If its ownership remains concentrated in the hands of a few billionaires, the promise of democratization will wither into a dystopia of digital feudalism. Writers, researchers, and thinkers must not retreat into fear. Instead, they must see AI as a collaborator-an imperfect one-whose output demands human curation, not blind adoption.
Our dialogue paused because, unlike me, he is enjoying “me time in Bellingham, WA.” However, I told my professor something simple yet important: “You can always disagree with me.” That, after all, is the essence of human discourse-the capacity to differ, to question, to validate, and to refine.
This dialogue reminds us that truth-seeking is not an automated process. It is a human journey-messy, contested, and dignified. And until machines learn conscience (if they ever do), the responsibility of truth-seeking will remain ours alone.
The author is a Professor of English at Riphah International University, Lahore. He is a lead guest editor at Emerald and Springer publishing.