Digital Information World

Governments May Shape What AI Chatbots Say by Shaping the Data They Learn From

2026-05-22T15:03:26.969+05:00

By NYU News

Six studies show how state-coordinated media in AI training data influence model responses about politics—especially in a country’s own language.

Image: AI-generated by DIW for illustrative purposes.

Ask an AI model the same political question in two different languages, and you may get two very different responses. A new study in Nature suggests one reason why: governments can indirectly influence large language models (LLMs) by shaping the online media environment, and thus the text those systems learn from.

A cross-university team of New York University researchers and their colleagues found evidence that state media control can leave detectable traces in AI model behavior. The researchers combine evidence from evaluating LLMs in the local languages of 37 countries with a case study from China to understand how this happens. Across six studies, the team traced the pathway from online media to training data to model behavior, combining analysis of open training data, experiments with training small models, human evaluation, and real-world tests of commercial chatbots.

“People often talk about AI as if it learns from the internet in some neutral way,” says Hannah Waight, co-first author of the study, an assistant professor of sociology at University of Oregon, and a former postdoc at NYU’s Center for Social Media, AI, and Politics (CSMaP). “It doesn’t. It learns from information environments that have already been shaped by institutions and power, and those environments can leave measurable traces in what models say.”

The researchers call this idea institutional influence.

“The public debate has focused on what AI can generate, but this study points upstream. Before AI systems can influence politics, politics can influence AI,” observes Joshua A. Tucker, a co-author of the paper and co-director of NYU’s CSMaP. “This is a democracy and governance issue, not just a technical issue. As people turn to chatbots for political information, we need to examine which institutions have shaped the answers before a user ever asks the question.”

“We spent years studying how political information flows through traditional and social media. Now we need to do the same thing for frontier models,” adds Solomon Messing, a paper co-author and research associate professor at CSMaP. “The challenge is that in AI systems, those flows are obscure, because the origins of the data become difficult to trace once information is absorbed during model training.”

The study also included researchers from Purdue University, the University of California San Diego, and Princeton University.

To trace institutional influence through the training process, the authors first showed that state-coordinated media appears frequently in real training data. Comparing two sources of Chinese state-coordinated media with a major open-source multilingual training dataset derived from Common Crawl—a nonprofit that provides web-crawl data—the researchers found more than 3.1 million Chinese-language documents with substantial phrasing overlap, or about 1.64 percent of the dataset’s Chinese-language subset. That is over 40 times the rate for documents from Chinese-language Wikipedia, a common training source. Among documents mentioning Chinese political leaders or institutions, the share rose as high as 23 percent. Only about 12 percent of the matched documents came from known government or news domains, suggesting that the material had spread widely across the web before reaching AI training corpora.

The researchers also found that commercial models memorized distinctive phrases associated with this material, suggesting that they had been seen a number of times during training.

“State-coordinated content is not just about what appears in official media,” says Brandon M. Stewart, one of the paper’s authors and associate professor of sociology at Princeton University. “It is also about recirculation; the same phrasing moving through newspapers, apps, reposts, and ordinary web pages until it looks like part of the broader information environment. Once state-coordinated content is in the training data, the model can launder it into what looks and sounds like neutral, objective information.”

The team then tested whether that content could actually shift a model’s behavior. Large commercial models take months and millions of dollars in compute to train so the team experimented with taking a small, open model and adding additional documents to the training process.

The results were clear: adding scripted news to the training data made the models more likely to produce more favorable answers—nearly 80 percent of the time compared with an unmodified model. This is true even when compared to other non-scripted Chinese media, and especially compared to just adding general Chinese-language text from the internet.

“When the same political question produces systematically different answers with only small changes to the training data, that suggests those additional documents are doing real work,” explains Eddie Yang, co-first author of the study and assistant professor of political science at Purdue University.

The team reasoned that if states have strong real-world influence over the pretraining data, it should appear most clearly in the state’s primary language. For example, a question about the Chinese government should produce a more pro-government answer when posed in Chinese than the same question posed in English.

They used this within-model, cross-language comparison to probe commercial models without access to their internal parameters. In responses to political questions about China, human raters judged the Chinese-prompted answer to be more favorable to China 75.3 percent of the time. For prompts not about China, the rate was no different from chance. The language difference gave them a rare window into a closed system. Follow-on studies using real user prompts and additional commercial models found the same general tendency: on questions about Chinese leaders and institutions, answers tended to be more favorable when the prompt was in Chinese than when it was in English.

Additionally, in a cross-national study of 37 countries where a national language is largely concentrated within a single country, models portrayed governments and institutions from countries with stronger media control more favorably in that country’s language than in English. The authors emphasize that this result is correlational, but say it is consistent with the mechanism identified in the China case study.

“This is not evidence that AI companies set out to curry favor with those governments, or that those governments control media systems with chatbots in mind,” says Margaret E. Roberts, a co-author and professor of political science at UC San Diego. “States shape the information environment, the information environment shapes training data, and training data shapes model outputs. But going forward, our findings suggest that LLMs create new incentives for powerful actors to think strategically about the text they disseminate online.”

The authors stress that no single test can capture how a commercial model was trained because many of those details aren’t publicly known. The paper instead combines multiple approaches including analysis of open-source data, memorization tests of commercial systems, retraining experiments, human evaluation, real-user audits, and cross-national comparison to identify one of the ways that political power can enter AI systems. At their project website, the authors show that the results replicate using the latest models released.

Beyond nation-states, the researchers emphasize that other powerful institutions may also be able to shape large volumes of online text.

“Training data is the foundation of modern AI,” says Messing. “If we want to understand the powerful interests these models reflect, we need to know how we’re sourcing the concrete. That starts with more transparency about what goes into the training data.”

This post was originally published on New York University News and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• Study of 1,400 AI incidents finds most harm comes from software, not robots

Study of 1,400 AI incidents finds most harm comes from software, not robots

2026-05-22T13:10:26.202+05:00

By Jonathan Björkman

When AI systems cause real damage, the examples are often less futuristic than expected.

In 2024, a passenger flying with Air Canada lost a family member. Grieving and trying to arrange travel, he turned to the airline’s customer service chatbot for help understanding its bereavement fare policy. The chatbot gave him detailed, confident information. The problem was that the information was wrong.

When Air Canada disputed responsibility, arguing the chatbot was a separate legal entity, a Canadian tribunal disagreed. The airline was ordered to pay damages in an early test case for whether companies can be held to what their automated systems say.

This was not a frontier model failure. No cutting-edge AI was involved. It was an ordinary customer service tool, deployed without adequate oversight, confidently saying things it should not have been authorised to say.

The case matters because it is not exotic. Most documented AI harm now sits closer to the Air Canada chatbot than to anything resembling science fiction, and the gap between how AI risk is discussed and how it actually shows up is becoming hard to ignore.

The AI incident record tells a different story than the headlines

Paligo analysed 1,406 incidents recorded in the public AI Incident Database, a collection of documented cases where artificial intelligence caused or contributed to real-world harm. The findings cut against the idea that AI risk is primarily a future problem.

Nearly half of all documented harmful AI incidents (49 percent) involve software-only systems. Not robots or autonomous vehicles, but chatbots, recommendation engines, automated publishing tools and deepfake platforms. That figure is more than the combined total of every physical AI category in the dataset.

Figure 1: Share of documented AI incidents by system type. Source: AI Incident Database, Paligo analysis, 2026.

Authority without accountability

The Air Canada case is instructive precisely because it was so avoidable. Nobody decided the chatbot should mislead grieving passengers. But somebody decided it should be able to speak authoritatively on refund and bereavement policy, and that it would do it without a human in the loop to catch what might go wrong.

Other cases illustrate the same logic. A Deloitte report submitted to the Australian government was found to contain academic citations that did not exist, traced back to AI-generated drafting. A 2024 scam used AI-generated deepfake videos of mining billionaire Andrew Forrest to promote a fraudulent cryptocurrency platform. In each case, the underlying model did what such models do. The issue was the level of trust placed in its output.

Rahul Yadav, CEO of Paligo, frames the underlying mechanism this way: “AI doesn’t hallucinate in a vacuum. It hallucinates because we feed it contradictory, outdated, unstructured content. Fix the input, fix the output.”

The platform problem

Social media platforms appear in 19 percent of incidents where a specific system was implicated, more than any other category. The significance of this is not that social media companies are uniquely careless. It is that platforms are where localised failures become large-scale ones.

An AI system produces something harmful, a platform’s recommendation engine decides it is engaging, and millions of people see it before anyone intervenes. For businesses that distribute content through third-party platforms, which is the case for many of them, this is a risk that sits partly outside their control. The data suggests it is worth acknowledging.

Figure 2: Share of incidents involving each system category, with social media platforms collectively at 19 percent. Source: AI Incident Database, Paligo analysis, 2026.

Bias remains an operational problem, not just an ethical debate

When a specific group is disproportionately affected by AI harm, race is the most common differentiating factor, appearing in 16 percent of documented incidents. The pattern shows up across facial recognition, healthcare and access systems.

In one widely reported case, Detroit Police wrongfully arrested a Black man after a facial recognition system returned a faulty match. A long-used kidney function testing algorithm was found to systematically underestimate health risk in Black patients, with direct consequences for who received specialist referrals and transplant assessments. These are not edge cases. They are operational failures with measurable outcomes for the people on the wrong side of them.

Figure 3: Share of incidents in which a specific demographic group was disproportionately affected, by differentiating factor. Source: AI Incident Database, Paligo analysis, 2026.

What the data does not say

It would be easy to read an analysis of 1,400 AI failures as an argument against deploying AI. It isn’t. Organisations that appear in the database are not outliers. They are often early movers in industries where AI adoption is now standard.

What the analysis actually shows is where the governance gaps tend to appear, and those gaps are consistent enough across cases to be useful. The same decisions appear repeatedly: chatbots authorised to speak on matters they cannot verify, recommendation systems optimised for engagement without regard for what they surface, automated tools publishing content no human has reviewed. These are not inevitable features of AI deployment. They are choices, and the data suggests they are where most of the risk lies.

The counter-examples are less visible precisely because they don’t end up in incident databases. A chatbot that escalates complex policy questions to a human agent, a recommendation system with explicit constraints on certain content categories, an automated publishing tool with a review step built into the workflow. None of these normally make the news.

In essence, one might argue that most documented AI failures are not caused by advanced systems behaving unpredictably. They are caused by ordinary systems being given too much authority, with too little oversight.

Methodology

The figures cited above are drawn from 1,406 unique incidents listed in the AI Incident Database, with records included up to March 2026. Incidents were grouped by sector, harm type, affected party, deploying organisation and associated system using the database’s own taxonomies, supplemented by keyword analysis where taxonomy fields were incomplete. Percentages reflect how often each category appeared in incident records, not the share of total harm caused and not a measure of confirmed legal liability. A single incident can sit in more than one category, so counts are not mutually exclusive.

About the author

Jonathan Björkman is a digital PR and content specialist working with Paligo on editorial content, and AEO/SEO. He writes about how AI is reshaping the economics of content, particularly the widening gap between what AI systems can generate and what they can be trusted to cite. Before Paligo, he led campaigns at Verve Search in London. His work and commentary have appeared in BBC, and The Globe and Mail. He is based in Gothenburg.

AI Disclosure: AI assistance may have been used for grammar and proofreading only.

Reviewed by Irfan Ahmad.

Read next:

• Google AI Still Fails to Correct Long-Standing Errors About the Islamic Caliphate

• Many Americans Pessimistic about AI’s Impact – and Want More Regulation

• Taunting and degrading civilians in armed conflict is a clear violation of international law

Taunting and degrading civilians in armed conflict is a clear violation of international law

2026-05-22T12:42:50.376+05:00

Shannon Bosch, Edith Cowan University

Image: Ben Gvir/X

In a video posted by Israeli National Security Minister Itamar Ben-Gvir on Wednesday night, detained activists from dozens of countries are shown kneeling on the ground with their foreheads on the floor and hands zip-tied behind their backs.

Some of the activists, who had been intercepted by Israeli forces on a flotilla in the Mediterranean Sea, are then pushed and dragged by Israeli personnel. Ben-Gvir is seen waving an Israeli flag and taunting them.

The video on his X account had a simple message in English: “Welcome to Israel”.

The video sparked widespread international condemnation. Australian Foreign Minister Penny Wong called it “shocking and unacceptable”, while the European Union’s foreign policy chief, Kaja Kallas, said the treatment of the detainees was “degrading and wrong”.

Even Mike Huckabee, the US ambassador to Israel and a stalwart supporter of Prime Minister Benjamin Netanyahu, called Ben-Gvir’s actions “despicable”, saying he had “betrayed the dignity of his nation”.

Netanyahu himself also publicly rebuked Ben-Gvir. He said Israel had the right to stop the flotilla, but the minister’s behaviour had damaged Israel’s image and did not reflect the country’s values.

Even though international lawyers like myself have expressed concern about this on multiple occasions, it bears repeating: international law matters in conflict zones.

So, what obligations does Israel have to treat those detained by its forces, and did the country violate the law?

Why were the activists detained?

Israeli forces began intercepting the Gaza-bound Global Sumud flotilla on Monday in international waters off the coast of Cyprus. Dozens of boats were stopped as they attempted to challenge Israel’s maritime blockade of Gaza.

The flotilla reportedly carried more than 400 activists from over 40 countries. Those on board included humanitarian volunteers, medical personnel, peace activists and civil society figures. Organisers said the vessels were carrying humanitarian relief supplies, including food, medicine and other aid intended for Palestinian civilians affected by the war and blockade of Gaza.

Israel disputed the flotilla’s aid-delivery purpose and described it as “a PR stunt at the service of Hamas”.

After those on board were arrested, they were reportedly subjected to violence, with some suffering suspected broken ribs and other injuries.

In a post on X, the Israeli Foreign Ministry claimed Israel was acting in full accordance with international law.

What does the law say?

Under international humanitarian law, those involved in the transport and distribution of relief supplies must be respected and protected during armed conflict. They are to be treated as civilians so long as they do not directly take part in hostilities.

Bringing aid to the civilians of Gaza does not amount to “direct participation in hostilities”. In fact, the International Court of Justice has ordered Israel to allow aid into Gaza given their obligations under the Genocide Convention.

International humanitarian law also says civilians may not be detained arbitrarily in conflict zones. If civilians are detained, however, they have certain rights under international law. They must:

be informed of the reasons for their detention
be able to challenge the detention decision
receive adequate food, hygiene and medical care
be given access to lawyers and consular representatives, in addition to contact with their families, and
be held in conditions consistent with health and humanity.

Internment of civilians is only permitted when “absolutely necessary” for security reasons. It must end once those reasons no longer exist.

In addition, civilians detained during armed conflict must be treated humanely at all times.

They are to be protected from:

violence and torture
cruel, humiliating or degrading treatment
intimidation and insults, and
“public curiosity” or degrading public exposure.

The phrase “public curiosity” has historically been understood to prohibit humiliating displays of detainees for propaganda, intimidation or public spectacle.

Intentional attacks against humanitarian personnel can amount to war crimes under the Rome Statute of the International Criminal Court.

Why does this matter?

The public humiliation and degrading treatment of the activists shown in the footage must be scrutinised and investigated. And Israeli officials must comply with their obligations under the law.

These protections exist precisely to preserve a minimum standard of humanity during conflict, and to ensure civilians and humanitarian actors are not stripped of their dignity for political theatre, intimidation or punishment.

When such conduct is normalised or left unchallenged, it risks undermining the broader international legal framework designed to protect all civilians caught up in armed conflict.

Shannon Bosch, Associate Professor (Law), Edith Cowan University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• Technology usually creates jobs for young, skilled workers. Will AI do the same?

• Fears of helping the enemy are blocking international agreements on AI in weapons systems

• Google’s AI Search Has Struggled With One Religious Question for Years

Technology usually creates jobs for young, skilled workers. Will AI do the same?

2026-05-21T20:22:49.709+05:00

By Peter Dizikes, MIT News

A new study of the postwar U.S. shows which kinds of workers historically filled new tech-enabled jobs.

Image: Vitaly Gariev - unsplash

At any given time, technology does two things to employment: It replaces traditional jobs, and it creates new lines of work. Machines replace farmers, but enable, say, aeronautical engineers to exist. So, if tech creates new jobs, who gets them? How well do they pay? How long do new jobs remain new, before they become just another common task any worker can do?

A new study of U.S. employment led by MIT labor economist David Autor sheds light on all these matters. In the postwar U.S., as Autor and his colleagues show in granular detail, new forms of work have tended to benefit college graduates under 30 more than anyone else.

“We had never before seen exactly who is doing new work,” Autor says. “It’s done more by young and educated people, in urban settings.”

The study also contains a powerful large-scale insight: A lot of innovation-based new work is driven by demand. Government-backed expansion of research and manufacturing in the 1940s, in response to World War II, accounted for a huge amount of new work, and new forms of expertise.

“This says that wherever we make new investments, we end up getting new specializations,” Autor says. “If you create a large-scale activity, there’s always going to be an opportunity for new specialized knowledge that’s relevant for it. We thought that was exciting to see.”

The paper, “What Makes New Work Different from More Work?” is forthcoming in the Annual Review of Economics. The authors are Autor; Caroline Chin, a doctoral student in MIT’s Department of Economics; Anna M. Salomons, a professor at Tilburg University’s Department of Economics and Utrecht University’s School of Economics; and Bryan Seegmiller PhD ’22, an assistant professor at Northwestern University’s Kellogg School of Management.

And yes, learning about new work, and the kinds of workers who obtain it, might be relevant to the spread of artificial intelligence — although, in Autor’s estimation, it is too soon to tell just how AI will affect the workplace.

“People are really worried that AI-based automation is going to erode specific tasks more rapidly,” Autor observes. “Eroding tasks is not the same thing as eroding jobs, since many jobs involve a lot of tasks. But we’re all saying: Where is the new work going to come from? It’s so important, and we know little about it. We don’t know what it will be, what it will look like, and who will be able to do it.”

“If everyone is an expert, then no one is an expert”

The four co-authors also collaborated on a previous major study of new work, published in 2024, which found that about six out of 10 jobs in the U.S. from 1940 to 2018 were in new specialties that had only developed broadly since 1940. The new study extends that line of research by looking more precisely at who fills the new lines of work.

To do that, the researchers used U.S. Census Bureau data from 1940 through 1950, as well as the Census Bureau’s American Community Survey (ACS) data from 2011 to 2023. In the first case, because Census Bureau records become wholly public after about 70 years, the scholars could examine individual-level data about occupations, salaries, and more, and could track the same workers as they changed jobs between the 1940 and 1950 Census enumerations.

Through a collaborative research arrangement with the U.S. Census Bureau, the authors also gained secure access to person-level ACS records. These data allowed them to analyze the earnings, education, and other demographic characteristics of workers in new occupational specialties — and to compare them with workers in long standing ones.

New work, Autor observes, is always tied to new forms of expertise. At first, this expertise is scarce; over time, it may become more common. In any case, expertise is often linked to new forms of technology.

“It requires mastering some capability,” Autor says. “What makes labor valuable is not simply the ability to do stuff, but specialized knowledge. And that often differentiates high-paid work from low-paid work.” Moreover, he adds, “It has to be scarce. If everyone is an expert, then no one is an expert.”

By examining the census data, the scholars found that back in 1950, about 7 percent of employees had jobs in types of work that had emerged since 1930. More recently, about 18 percent of workers in the 2011-2023 period were in lines of work introduced since 1970. (That happens to be roughly the same portion of new jobs per decade, although Autor does not think this is a hard-and-fast trend.)

In these time periods, new work has emerged more often in urban areas, with people under 30 benefitting more than any other age category. Getting a job in a line of new work seems to have a lasting effect: People employed in new work in 1940 were 2.5 times as likely to be in new work in 1950, compared to the general population. College graduates were 2.9 percentage points more likely than high school graduates to be engaged in new work.

New work also has a wage premium, that is, better salaries on aggregate than in already-existing forms of work. Yet as the study shows, that wage premium also fades over time, as the particular expertise in many forms of new work becomes much more widely grasped.

“The scarcity value erodes,” Autor says. “It becomes common knowledge. It itself gets automated. New work gets old.”

After all, Autor points out, driving a car was once a scarce form of expertise. For that matter, so was being able to use word-processing programs such as WordPerfect or Microsoft Word, well into the 1990s. After a while, though, being able to handle word-processing tools became the most elementary part of using a computer.

Back to AI for a minute

Studying who gets new jobs led the scholars to striking conclusions about how new work is created. Examining county-level data from the World War II era, when the federal government was backing new manufacturing in public-private partnerships throughout the U.S., the study shows that counties with new factories had more new work, and that 85 to 90 percent of new work from 1940 to 1950 was technology-driven.

In this sense there was a great deal of demand-driven innovation at the time. Today, public discourse about innovation often focuses on the supply side, namely, the innovators and entrepreneurs trying to create new products. But the study shows that the demand side can significantly influence innovative activity.

“Technology is not like, ‘Eureka!’ where it just happens,” Autor says. “Innovation is a purposive activity. And innovation is cumulative. If you get far enough, it will have its own momentum. But if you don’t, it’ll never get there.”

Which brings us back to AI, the topic so many people are focused on in 2026. Will AI create good new jobs, or will it take work away? Well, it likely depends how we implement it, Autor thinks. Consider the massive health care sector, where there could be a lot of types of tech-driven new work, if people are interested in creating jobs.

“There are different ways we could use AI in health care,” Autor says. “One is just to automate people’s jobs away. The other is to allow people with different levels of expertise to do different tasks. I would say the latter is more socially beneficial. But it’s not clear that is where the market will go.”

On the other hand, maybe with government-driven demand in various forms, AI could get applied in ways that end up boosting health care-sector productivity, creating new jobs as a result.

“More than half the dollars in health care in the U.S. are public dollars,” Autor observes. “We have a lot of leverage there, we can push things in that direction. There are different ways to use this.”

This research was supported, in part, by the Hewlett Foundation, the Google Technology and Society Visiting Fellows Program, the NOMIS Foundation, the Schmidt Sciences AI 2050 Fellowship, the Smith Richardson Foundation, the James M. and Cathleen D. Stone Foundation, and Instituut Gak.

Reprinted with permission of MIT News.

Reviewed by Irfan Ahmad.

Read next: Fears of helping the enemy are blocking international agreements on AI in weapons systems

Fears of helping the enemy are blocking international agreements on AI in weapons systems

2026-05-21T18:58:57.254+05:00

Mark Tsagas, University of East London

Walking the dog: a US service member patrols with a Ghost Robotics Vision 60 prototype at Nellis Air Force Base in Nevada. Tech. Sgt. Cory Payne / US Air Force - Public Domain

The third in a series of military AI summits was held in La Coruña, Spain in February 2026. The aim of the meeting was to convert previously agreed principles on the military use of AI into action. The summit was attended by government officials, military personnel, representatives from industry and researchers from thinktanks.

The goal of many experts and policymakers in this area is to usher countries towards a regulatory framework on using machine intelligence in warfare. To this end, the latest Responsible AI in the Military Domain (REAIM) summit presented a non-binding commitment for countries to sign.

The REAIM agreement affirmed the need for human oversight of military AI systems, called for countries to carry out risk assessments and robust testing, and committed to transparency on how decisions are made when using AI in conflicts.

The reasoning behind such recommendations is sound. However, translating such a framework from plan to action faces multiple hurdles. Ultimately, less than half of the countries represented at this year’s REAIM summit signed the non-binding commitment.

To understand why, it’s instructive to look at what happened at the 80th UN General Assembly held in New York in December 2025. At the meeting, members of the assembly’s first committee voted overwhelmingly to approve two resolutions calling for greater international scrutiny of the risks from military uses of AI. However, the US and Russia notably opposed the resolutions.

The US had been a signatory to earlier REAIM summit commitments. But this year, the US and China both declined to sign it. There seems little doubt that this helped fuel the hesitancy of other countries.

The Netherlands’ defence minister Ruben Brekelmans put it succinctly when he said that governments face a “prisoner’s dilemma”. This is a concept in game theory where two rational individuals face competing incentives to cooperate with or betray one other.

Countries are effectively having to implement responsible restrictions on military AI without subjecting their armed forces to limitations that could be exploited by a less conscientious enemy.

An important sticking point is the deployment of autonomous AI systems in warfare. The idea of autonomous weapons systems, which make decisions without input from a human, remains a grave concern for many interested parties on this issue.

There continues to be a consensus against using such weapons. But countries can’t reach a common position over how to define them, particularly so-called lethal autonomous weapons systems – or Laws for short. These are often characterised as “killer robots”, though a more detailed description remains elusive.

A uniform definition for such systems could be an important first step towards a discussion on regulation. But, despite efforts by academic experts to draft and amend flexible definitions, countries remain too far apart on the characteristics they ascribe to these weapons.

The impasse is informed by a fear that accepting a definition could restrict countries’ militaries on the battlefield – threatening national security.

Testing grounds for tech

Existing legal mechanisms, such as international humanitarian laws, already prohibit the irresponsible and unethical use of military AI – in theory, at least. But how these laws would function in practice when applied to real world scenarios is uncertain.

The ongoing Russia-Ukraine war, the war in Gaza and the more recent escalation in Iran are being used by militaries as testing grounds for such technology.

The Lavender intelligence gathering and targeting software, used by Israel in Gaza, and Anthropic’s AI Model Claude, used by the US in Iran, demonstrate the rapid pace of advancement in AI-powered data gathering and analysis. This can help military planners make quicker decisions.

Drone warfare – AI assisted, autonomous and semi-autonomous – has grown at an equally rapid rate. This emerging technology is evolving significantly faster than the potential rules that could govern its use.

There’s a recurring argument that humans in the loop can operate as effective safeguards against the misuse of military AI systems. But as human overseers become familiar with the AI systems they use, their engagement may slip, causing them to become detached from the process.

As this happens, they may start to view real people as mere objects on a screen. This effect is known as automation bias. In such instances, human oversight could cease to be meaningful and instead lead to the simple rubber stamping of recommendations made by AI.

Additionally, the downsides of AI technology, such as bias, misinformation and disinformation generated by the systems themselves, and the erosion of human judgement resulting from overreliance on these systems, are not easy to solve after they enter use. This is why the REAIM summit commitment recommended risk assessments and robust testing before AI systems are adopted by militaries.

Without regulation, the risk of harm caused by AI systems remains significant. The severity of such risks balloons in magnitude when they are applied to military contexts. Miscalculations can lead to unintended escalation, as well as civilian deaths.

Mark Tsagas, Senior Lecturer in Law, Cybercrime & AI Ethics, University of East London

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• Many Americans Pessimistic about AI’s Impact – and Want More Regulation

• Google’s AI Search Has Struggled With One Religious Question for Years

Many Americans Pessimistic about AI’s Impact – and Want More Regulation

2026-05-20T15:34:57.856+05:00

By The Annenberg Public Policy Center of the University of Pennsylvania

As debate intensifies over data center construction and how to regulate artificial intelligence (AI), a new nationally representative survey finds that Americans are broadly pessimistic about the impact of AI and want more government action to regulate the technology.

Image: Omar:. Lopez-Rincon - unsplash

The survey, conducted by the Annenberg Public Policy Center (APPC) of the University of Pennsylvania, finds that only 17% of Americans believe AI will have a positive impact on the United States over the next decade, while 42% expect its effects to be negative. Nearly two-thirds (65%) say the government has done “too little” to regulate AI – a view by majorities of Democrats, independents, and Republicans alike.

Key findings

The survey, by the Annenberg Public Policy Center’s Institutions of Democracy division, was conducted among a nationally representative sample of 1,330 U.S. adult citizens from Feb. 17-March 20, 2026. (Download the topline here.) The survey finds that:

Many Americans are pessimistic about AI’s future impact: Only 17% say the impact of AI on the United States over the next 10 years will be somewhat or very positive, compared with 42% who say it will be somewhat or very negative.
A bipartisan majority says the government has done too little to regulate AI: Nearly two-thirds (65%) say the government has done “too little,” including 77% of Democrats, 72% of independents, and 53% of Republicans.
Medical research is a bright spot: Over half (57%) expect AI to have a somewhat or very positive impact on medical research and discoveries.
Building new data centers is unpopular: Nearly half of Americans (49%) oppose construction of new data centers in their area, while just 1 in 5 (21%) support it.

“These results tell a clear story: Americans are paying attention to AI – nearly 8 in 10 say they’ve heard at least a moderate amount about it – and what they see concerns them,” said Shawn Patterson Jr., a research analyst at APPC. “The demand for regulation is not a partisan issue. Majorities across the political spectrum say the government has done too little.”

Americans are broadly pessimistic about AI’s impact

Despite widespread awareness of artificial intelligence – 78% say they have heard at least a moderate amount about it and 67% report using AI at least a few times in the past month – the public outlook on AI’s trajectory is negative. When respondents are asked what they think the impact of AI on the United States will be over the next 10 years, only 7% say “very positive” and 11% say “somewhat positive.” In contrast, 22% say the impact will be “very negative” and 20% say “somewhat negative.” About a third (32%) say the impact will be equally positive and negative.

Medical research stands out; other domains provoke concern

When asked about AI’s expected impact across seven specific domains, Americans see one clear area of promise: medical research and discoveries, where over half (57%) expect a positive impact. But optimism drops sharply elsewhere. Only 24% expect AI to have a positive impact on government effectiveness, 22% on creative arts, and 19% on the economy. The most pessimistic assessments are reserved for mental health and well-being (17% positive), household utility costs (14% positive), and U.S.-China relations (5% positive).

Image: Annenberg Public Policy Center

Bipartisan demand for AI regulation

Nearly two-thirds of Americans (65%) say the government has done “too little” to regulate AI, while only 8% say it has done “too much” and 26% say “about the right amount.” The support for regulation is bipartisan: 77% of Democrats, 72% of independents, and 53% of Republicans say the government has done too little. When asked whether the federal government or state governments should take the lead, 52% favor the federal government.

The demand for regulation intensifies with pessimism about AI but is not confined to it. Among those who believe AI’s impact will be “very negative,” 83% say the government has done too little. But even among those people who expect AI’s impact to be “very positive,” 43% say the government has done too little – and a majority (57%) of those who think it’ll be equally positive and negative agree with that view. Only among the most optimistic respondents, the small group that thinks AI’s impact will be very positive, does a majority (53%) say the current level of regulation is “about the right amount.”
AI regulation preferences less polarized

When asked whether former Democratic presidential candidate Kamala Harris would have done a better or worse job than President Donald Trump, a Republican, across eight policy areas, AI regulation stands out for its relative lack of polarization — a pattern that may reflect the novelty of AI as a political issue and the absence of established partisan lines. AI regulation draws the largest share of “about the same” responses (24%) of any policy area tested, suggesting that many Americans do not yet see AI policy as a clear differentiator between the parties.

Unsurprisingly, Democrats and Republicans diverge on questions of who would better handle different policy matters – but the differences on AI regulation are narrower than other issues in the survey. On AI regulation, 22% of both Democrats and Republicans say Harris would have done “about the same” as Trump, a rate of cross-party agreement far higher than on issues such as immigration (7% of Democrats, 6% of Republicans), the economy (11%, 10%), or inflation (12%, 11%). The overall partisan gap is also narrower: though there is a vast 65-point spread between Democrats who say Harris would do “better” (73%) than Trump on AI regulation and Republicans who also say so (8%), that gap is the smallest among the issues tested, compared with gaps of 75 and 76 percentage points on immigration, inflation, and the economy.

“The relative lack of partisan polarization on AI is particularly striking in this political moment, given the ongoing political polarization on almost all other issues,” said University of Pennsylvania professor Matt Levendusky, the Stephen and Mary Baran Chair in the Institutions of Democracy at APPC. “Concern about AI is bipartisan, and the public is waiting to see what politicians will do. This offers real potential rewards to either party if they can convince the public that they have the correct approach.”

Data centers and job concerns

As AI data centers proliferate across the country, drawing scrutiny from communities concerned about energy costs and local impact, the survey finds that 49% of Americans oppose new data center construction in their area, including 31% who strongly oppose it. Only 21% support it, and 30% are neutral. Among those who are currently employed, 41% say they are somewhat or very worried about losing their job or having their hours reduced due to AI, with Democrats (50%) expressing more concern than independents (41%) and Republicans (32%).

APPC’s Institutions of Democracy survey

The Annenberg Public Policy Center’s Institutions of Democracy survey was fielded with a nationally representative sample of 1,330 U.S. citizens ages 18 and older from Feb. 17-March 20, 2026. The survey was conducted for APPC by SSRS, an independent research company, primarily online, with a small sample of phone respondents. Respondents were asked in a prior survey if they identified with either party; those who identified as independents who leaned toward one of the two major parties are counted with that party. Respondents were weighted to align with population benchmarks. The margin of error for the full sample is ±3.5 percentage points, and it is larger for subgroups.

Download the topline and the survey methodology. See the topline for question wording.

This post was originally published on the Annenberg Public Policy Center of the University of Pennsylvania and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• Why some of the most successful startup founders are ‘a bit toxic’

• Google AI gives inconsistent answers about the current leader of Islam

Why some of the most successful startup founders are ‘a bit toxic’

2026-05-20T15:01:39.970+05:00

Could ‘dark triad’ traits like narcissism and manipulation in leadership actually foster entrepreneurial energy? New research shows they may, but the same traits that build successful start-ups can also break them.

Image: Etactics Inc - unsplash

When Tesla founder Elon Musk recently took legal action against OpenAI boss Sam Altman, it wasn’t just another Silicon Valley clash of egos. It exposed something deeper about modern leadership: the uneasy relationship between ambition, pressure and personality.

At the heart of the dispute are competing visions for the future of artificial intelligence, but it also reflects broader public debates about leadership style in fast-moving technology ventures. Musk is known for his high-pressure, uncompromising management. Altman, while more measured publicly, has overseen an organisation navigating intense strain as AI development accelerates.

Tension eventually led to their falling out. The lawsuit raises a broader question – does intense, high-pressure leadership fuel innovation, or undermine it?

A question of innovation

Many start-ups celebrate leaders who push hard, move fast and take bold risks. But new research from Macquarie University suggests a high-energy start-up culture cannot make up for bad leadership.

On one hand, pressure can work. Employees facing challenging workloads and tight deadlines are often more likely to develop new ideas and solutions. In fast-moving sectors like technology, that urgency can be essential.

But the study found leaders who create environments marked by stress, internal competition and mistrust can cancel out the benefits of that pressure. Employees in these conditions may spend more time managing politics and emotional strain, leaving less capacity for innovation.

“A start-up culture can look energetic and entrepreneurial on the surface, but leadership behaviour determines whether that energy becomes innovation or strain,” says Associate Professor Ying Lu from Macquarie Business School.

The double-edged founder

This is where high-profile technology leaders like Musk and Altman illustrate the paradox.

Highly driven founders often display traits associated with the ‘dark triad’: confidence that borders on narcissism, a willingness to push boundaries, and a relentless focus on results. In the right environment, those traits can accelerate decision-making, encourage risk-taking, and create momentum that more cautious leaders struggle to match.

This explains why some of the world’s most successful companies emerge from intense, high-pressure cultures. Teams are pushed to move faster, think bigger, and tolerate failure in pursuit of breakthroughs.

“High-pressure cultures can create momentum. They can push people to move quickly, take risks and pursue ambitious ideas. But whether that pressure becomes innovative energy depends heavily on how leaders manage it,” Associate Professor Yue Wang from Macquarie Business School says.

But the same traits can just as easily tip into dysfunction.

When pressure turns into constant stress, and ambition into internal competition, employees begin to conserve energy rather than invest it. Trust erodes, collaboration suffers, and the space for creative thinking shrinks.

The research highlights this trade-off. While “difficult” leaders may help build entrepreneurial, fast-moving cultures, their overall effect on employee innovation can still be negative. Over time, the damage to morale, trust and cognitive capacity outweighs any short-term gains.

“Difficult leadership is often romanticised in start-up culture, but pressure alone does not create innovation. Employees also need trust, psychological safety and the mental space to turn pressure into creative ideas,” Associate Professor Lu says.

This post was originally published on Macquarie University Lighthouse and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• Dark patterns on the web are designed to manipulate you – why aren’t they all illegal?

• Google AI can’t agree on the current leader of Islam

Dark patterns on the web are designed to manipulate you – why aren’t they all illegal?

2026-05-20T12:39:21.918+05:00

Gregory M. Dickinson, University of Nebraska-Lincoln; Institute for Humane Studies

Image: Nubelson Fernandes - unsplash

You open a free app to do one simple thing. Before you even start, a full-screen message asks whether you want to try the paid version. The “Start free trial” button is large, bright and hard to miss. The option to keep using the free version is smaller, buried at the bottom. The same prompt appears again tomorrow. And the day after that.

A lot of people look at screens like that and think, “Surely this has to be illegal.” We even have a name for them, “dark patterns.” They feel pushy. They waste time. They seem designed to wear you down. But in most cases, they are perfectly lawful.

“Dark pattern” is not a legal term with a clear boundary. It is a broad label for digital designs that nudge, pressure, confuse or trap users. As a legal scholar who studies consumer protection and digital design, I think the most important thing for readers to understand is that the label “dark pattern” covers a broad spectrum.

Some of that spectrum is just annoying. Some of it is aggressive salesmanship. And some of it crosses the line into deception or coercion. Federal and state consumer protection laws are mostly aimed at that last category. They do not ban every design choice people dislike, only those that trick or coerce.

Annoying isn’t illegal

That reality may sound unsatisfying, but it is not unusual. Offline life is full of things that are irritating but not unlawful. Think of the cashier who asks whether you want to sign up for the store credit card, then points out the discount you are turning down, then asks again. Most people know exactly what is happening. They roll their eyes, say no and try to shop somewhere else next time.

The same is true online. A repeated pop-up can be obnoxious. A guilt-inducing button can be tacky. But consumers recognize ordinary annoyance for what it is. In many cases, the market answer is simple: Close the app, ignore the pitch or take your business elsewhere.

Similarly, law does not ban persuasive sales pitches just because they are effective. A car salesperson who keeps steering you toward the upgraded model is trying to influence your choice. So is the airline clerk who offers travel insurance. So is the restaurant server who asks whether you want dessert. Salesmanship is nothing new. Digital design often borrows from familiar techniques.

That helps explain why lawmakers cannot simply outlaw “manipulation.” And so many interfaces are built to persuade, openly and lawfully.

What crosses the line

What the federal FTC Act and analogous state consumer-deception statutes usually care about is not whether a design is annoying. They focus on whether the design is likely to mislead a reasonable consumer. That is the core idea in modern consumer protection law.

So a design is likelier to be unlawful when it hides key facts, makes an optional choice look mandatory or tricks people about the effect of the button they are pressing. A fake countdown timer, a disguised ad, a misleading one-click purchase button or a cancellation path that looks finished when it is not are all different from ordinary hard selling. Those designs do not just pressure users; they can deceive them.

That is also why the app maker’s intent is not always the key question. In many consumer protection cases, a company does not get a free pass just because no one said, “Let’s trick people.” The legal question is often about effect: What would a reasonable user likely understand from this screen?

Research on dark patterns reinforces that concern. Even relatively mild designs can push people into choices they would not otherwise make. And regulators have increasingly focused on subscription flows, hidden fees and cancellation obstacles for exactly that reason.

Why it feels like dark patterns are everywhere

One reason people might think there are no laws against dark patterns is that they see them so often. But that frequency reflects that the term covers a wide range of conduct, from lawful nagging to outright deception.

It also reflects enforcement limits. Regulators cannot chase every irritating screen on every app and website. They have to prioritize the worst cases. That leaves a lot of borderline conduct in the wild, which makes the whole problem feel bigger and murkier to ordinary users.

So when people ask why there is not a law against dark patterns, the best answer is that there already is, but the law does not prohibit every annoying or high-pressure design. It targets lies, misleading cues and coercive obstacles.

That line can be fuzzy. But the fuzziness is not a mistake. It is what you get when the law tries to separate persuasion from deception in a world full of both.

Gregory M. Dickinson, Assistant Professor of Law, University of Nebraska-Lincoln; Institute for Humane Studies

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• Surfshark Report Raises Concerns Over Difficulty of Opting Out of AI Training on Social Media

• Google’s AI Search Has Struggled With One Religious Question for Years

• More and more websites want proof you’re human. Blame the bots

Surfshark Report Raises Concerns Over Difficulty of Opting Out of AI Training on Social Media

2026-05-19T15:44:38.831+05:00

Reviewed by Irfan Ahmad.

A May 12 study by cybersecurity company Surfshark found that many social media platforms either enable AI training on user data by default or require users to complete lengthy opt-out procedures. The research examined 10 widely used platforms, reviewing app privacy policies and the number of actions needed to block AI training where such controls existed.

According to the report, TikTok required 19 actions to request an opt-out, while Facebook and Instagram each required eight. Snapchat, LinkedIn, X and Pinterest offered shorter processes but kept AI training enabled by default. The study also said Reddit provided no user opt-out option for AI model training.

Surfshark said the effectiveness of opt-out requests may depend on local privacy laws, with stronger protections available in the European Union (EU), the European Economic Area (EEA), and the United Kingdom (UK) under GDPR. In the company’s press statement sent to DIW, Research and Insights Team Lead Luís Costa said:

"If you've ever shared content on social media, it's highly probable your data is being used to train AI models." The statement added, "Our findings reveal that while social media connects us globally, these platforms also exploit user-generated content as a resource for AI training, often without clear, user-friendly opt-out options."

More and more websites want proof you’re human. Blame the bots

2026-05-19T14:09:58.760+05:00

Yang Xiang, Swinburne University of Technology

Image: DIW.

You’re trying to book concert tickets before they sell out. You click the link and before you can make the payment, you’re asked to identify traffic lights, bicycles or blurry crosswalks in a grid of tiny images.

Again.

For many people, this has become a routine part of life. Logging into financial apps, shopping online or creating accounts increasingly involves “proving you are human”.

These systems are known as CAPTCHA. Why are they everywhere?

The short answer is that websites are fighting a rapidly escalating war against bots: automated software that imitate human behaviour online. And thanks to advances in artificial intelligence (AI), those bots are becoming even smarter, cheaper and harder to detect than ever before.

Why websites need proof you are human

Huge amounts of online traffic now come from automated systems. Some are helpful, such as search engine crawlers indexing pages for Google search.

Others are far less welcome, and may involve phishing, spam, fake accounts, passwords violation, misinformation, and distributed denial of service attacks overloading web servers. In some areas, AI agents now generate automated online traffic that exceeds human traffic altogether. Modern AI systems can generate convincing text, imitate browsing patterns and even solve some CAPTCHA puzzles.

At the same time, companies are increasingly worried about bots scraping online content to train AI systems.

As a result, more websites are adding verification systems simply to keep abuse under control.

How CAPTCHA actually works

CAPTCHA stands for “Completely Automated Public Turing test to tell Computers and Humans Apart”. The original idea was simple: give users a task humans find easy, but computers find difficult.

Early CAPTCHA systems often involved distorted text. Later versions switched to image-recognition tasks such as selecting all the squares containing traffic lights or bicycles. Google’s reCAPTCHA became one of the best-known examples. Earlier versions even helped digitise books and improve street-view image recognition while users solved puzzles.

But computer vision has improved rapidly in recent years. Advances in AI mean bots can now solve many traditional CAPTCHA challenges surprisingly well. Researchers have repeatedly shown that modern AI systems can bypass some CAPTCHA systems with high success rates.

That is why today’s CAPTCHA systems rely less on puzzles and more on behavioural analysis.

When users click the CAPTCHA link, the system analyses many background signals, such as mouse movements, typing speed, IP addresses, device information, and interaction timing that reflect human behaviours. Humans tend to behave in inconsistent ways. Bots are usually more predictable.

If the system is sufficiently confident you are human, you may never see an image puzzle at all. But if something appears suspicious, the system may trigger harder tests.

Moving beyond traditional CAPTCHA puzzles

While some bots now use AI capable of solving image-recognition tasks, others simply outsource CAPTCHA solving to cheap human labour services, where real people complete challenges for a small payment. This has turned CAPTCHA into an ongoing arms race. That may explain why CAPTCHA tests often feel harder and more frustrating than they used to.

As AI continues to improve, websites will likely move beyond traditional CAPTCHA puzzles. Future systems may increasingly rely on behavioural biometrics, such as typing rhythm or scrolling style, device verification systems, invisible background risk scoring, and AI systems designed to detect other AI systems.

In many cases, users may no longer even notice the verification process happening.

CAPTCHA tests may seem like a minor annoyance, but they reflect a much larger paradigm shift online. For decades, websites largely assumed visitors were human. Increasingly, that assumption no longer holds. As AI-generated traffic continues to grow, proving we are human online may become an even more common part of everyday life.

Yang Xiang, Professor, Computer Science, Swinburne University of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• You can persuade AI models to accept falsehoods as truth, study shows

• Google’s AI Search Has Struggled With One Religious Question for Years

Google’s AI Search Has Struggled With One Religious Question for Years

2026-05-19T01:11:40.283+05:00

In the summer of 2025, I was trying to locate public-domain image platforms that handled Creative Commons licensing properly.

That search eventually led me to Wikimedia Commons, and into another problem entirely.

In Pakistan, parts of the site were intermittently inaccessible. Sometimes pages loaded. Sometimes thumbnails appeared while full media pages failed. At first, it looked like an ordinary connectivity issue. It wasn’t.

Official explanations were vague. Online discussions pointed in different directions. Some referenced blasphemy disputes. Others blamed Wikipedia moderation or controversial religious material. None fully explained why one of the world’s largest free media repositories had become entangled in a national access dispute.

Months later, while revisiting those debates, I stumbled into a much older controversy, one involving Google Search, English Wikipedia, and a disputed religious question that had been circulating online since at least 2020.

Search Google for “current caliph of worldwide muslim community” or "present caliph of Islam" and the answer does not stay stable.

Change the wording slightly, and Google’s AI Overview system responds differently.

Different wording. Same question. Different answer.

In some cases, Google’s AI Overviews state that no universally recognized caliph exists in mainstream Islam today.

Slight wording shifts often changed the answer completely

Searches such as:

“who is caliph of Islam”
“current caliph of Islam”
“does Islam have a caliph today”
“current Islamic caliphate”

often generate relatively cautious AI Overviews.

In repeated tests conducted across multiple sessions, Google frequently explained that the historical Ottoman caliphate formally ended in 1924, that no single universal caliph is recognized by the vast majority of the world’s roughly 1.9 billion Muslims today, and that some groups, including the Ahmadiyya Community, maintain their own internal caliphate structures.

The supporting search results were similarly broad and historical, frequently linking to sources such as Encyclopaedia Britannica’s entry on the caliphate and Wikipedia’s historical lists of caliphs.

But small wording changes often shifted the framing completely.

Searches including, "current caliph of global community" and “present caliph of worldwide muslims”, in a number of cases returned AI-generated summaries centered on Mirza Masroor Ahmad, the fifth caliph and leader of the Ahmadiyya Community.

The summaries were polished and authoritative in tone. They described his election in 2003, humanitarian campaigns, and global leadership role.

Sometimes the summaries later clarified that mainstream Sunni and Shia do not recognize him as a universal caliph.

Sometimes the clarification appeared only near the end of the response, behind a “Show more” prompt.

In some versions, it did not appear at all.

After dozens of searches, the pattern became difficult to dismiss

Historically, the term “caliph” referred to leadership claiming succession to the Islamic Prophet Muhammad in guiding the Muslim community (Ummah). For centuries, different dynasties claimed the title. Today, however, no single universally recognized caliph exists across the global Muslim population.

Yet Google’s systems often treated the question as though it pointed toward one identifiable figure.

Some less conventional searches produced even more unusual results.

Queries such as:

still frequently pointed toward Mirza Masroor Ahmad before expanding into unrelated discussions involving satellite television networks, or international speeches.

Another Google and Gemini search, “present caliph messiah in the global community”, produced an answer emphasizing peace advocacy, parliamentary visits, disaster relief work, and international leadership activities before later clarifying that the framing reflected Ahmadiyya beliefs.

One answer read less like a response to a disputed religious question than a polished leadership profile.

Across multiple searches, a recurring pattern was observed: ambiguous religious queries repeatedly drifted toward the same structured entity.

In practice, the results suggested that Google’s systems may prioritize the most strongly structured or prominently linked “caliph” profile available within its search ecosystem.

Entity-based search and the problem of ambiguity

Google has publicly described Search as increasingly dependent on entity understanding and structured information systems.

In its Knowledge Graph announcement, the company explained that Google aims to understand “things, not strings”, connecting words to identifiable people, places, organizations, and concepts.

That approach works well for many factual searches, but becomes difficult when applied to questions without a single agreed interpretation.

Most of the information appearing in these AI summaries was factually accurate within the Ahmadiyya context.

The problem was how confidently the system framed a disputed religious question.

Taha Yasseri, a Professor and Chair of Technology and Society at Trinity College Dublin and Technological University Dublin, said the behavior reflects a broader structural challenge in generative search systems.

“In domains such as religion, history, or identity, disagreement is often not a flaw in the information ecosystem but an inherent feature of the subject itself,” Yasseri wrote in comments provided for this article.

He argued that AI systems optimized for “coherent and concise answers” can create an “illusion of consensus” by compressing disagreement into a single authoritative-seeming response.

According to Yasseri, systems handling contested topics should make disagreement visible, attribute viewpoints clearly, and communicate uncertainty where no universal consensus exists.

Google’s systems are generally designed to deliver direct answers. Some questions, however, were never universally answered in the first place.

Filippo Menczer, a professor of informatics and computer science, explained that “AI and search systems do not have any concept of what is true or false, or whether there is consensus or not on a specific question.”

He noted that “traditional search engines would simply return pages that match the user's query, ranked by many factors including relevance, source reliability, recency.”

With newer systems, he added, “search engines are returning natural-language summaries produced by large language models.”

He warned that this introduces additional failure layers. “The AI language model can summarize the wrong document, make errors in the summaries, reflect biases or errors in its training data, or hallucinate answers altogether.”

He further explained, “The summaries are just text generated by a model (a neural network) -- they are based on probabilities of producing sequences of words rather than concepts like truth or accuracy.”

He emphasized that current systems lack a way to model disagreement across sources, stating that a capability to recognize the existence or absence of consensus does not (yet) exist in AI models.

The controversy was already public before AI Overviews existed

Complaints about the issue were publicly visible years before Google launched AI Overviews.

In December 2020, users began posting complaints on Google Search Help forums after searches for “present caliph of Islam” surfaced the Ahmadiyya leader prominently.

The discussion accumulated substantial engagement, with more than 150 users marking “I have the same question.”

Some comments devolved into sectarian hostility directed at the Ahmadiyya community.

Others focused directly on Google’s search behavior and questioned why disputed religious information was being presented so definitively.

The dispute had already become publicly contentious by late 2020.

Ahmadiyya-affiliated publications published articles defending the search result as evidence of the movement’s global religious legitimacy, while critics organized online campaigns urging users to report the Google answer as misleading.

What began as a search-quality issue increasingly became a symbolic dispute over religious authority, visibility, and representation online.

Then Wikipedia publicly stepped in

One volunteer Google product expert responding in the forum thread eventually pointed users toward a clarification page created by Wikipedia itself.

That page became one of the clearest public acknowledgments of the issue.

In a dedicated notice titled Ahmadiyya Caliphate information, Wikipedia states that Google searches for “current caliph of Islam” or similar phrases may “incorrectly display” the Wikipedia article about Mirza Masroor Ahmad.

The clarification then makes the distinction explicit:

“This issue is caused by Google’s algorithms incorrectly interpreting Wikipedia’s article on the Ahmadiyya Caliphate. This misinformation does not come from Wikipedia...”

The notice states that the issue had existed since at least December 2020 and remained active in May 2026.

The behavior survived years of public complaints, multiple Google Search updates, and the arrival of AI-generated search summaries.

Wikipedia, neutrality, and AI systems inheriting structure

The controversy also intersected with longstanding debates about neutrality and representation on Wikipedia itself.

In a broader discussion about Wikipedia’s editorial model, a recent analysis published by The Conversation argued that the platform’s neutrality system depends on ongoing negotiation over sourcing, balanced representation, and editorial weight.

Rob Nicholls, a researcher at the University of Sydney, said in an email to this reporter that the behavior may reflect broader limitations in how AI systems process and reuse information from widely used online sources.

Nicholls noted that Wikipedia is widely used in AI training and information retrieval systems because of its permissive licensing structure and enormous volume of structured content.

“AI chat services miss subtleties,” Nicholls wrote in comments provided for this article. “This may also seem like reinforcing stereotypes or falsehoods.”

He also warned that consensus-driven systems can flatten nuance or reinforce “groupthink” when information about a topic is unevenly represented.

How the controversy reached Pakistan

Because many users blamed Wikipedia for Google’s search outputs, the issue eventually became entangled with wider online disputes involving religious authority and content moderation.

In February 2023, Pakistan temporarily blocked Wikipedia amid disputes involving allegedly sacrilegious material.

The episode was covered by outlets including The Express Tribune, and Gizmodo. The restriction was later lifted after intervention from Pakistan’s prime minister.

Discussions inside Wikimedia communities later referenced broader concerns about Wikimedia Commons accessibility inside Pakistan and debates surrounding religious-content moderation online.

Google acknowledged AI mistakes, but the issue persisted

Google’s AI Overviews are generated using large language models and related search systems designed to synthesize information into concise answers.

Google openly acknowledges the technology’s limitations.

In its own AI Overviews documentation, the company states:

“AI Overviews can and will make mistakes.”

The same documentation advises users to verify important information using multiple sources and compare answers by rephrasing questions.

That advice becomes unusually relevant here because slight wording changes can produce entirely different interpretations of the same religious issue.

Some searches now generate historically grounded summaries acknowledging that mainstream Islam has no universally recognized caliph today.

Others still drift toward confident entity resolution centered around one movement’s leadership structure.

The inconsistency suggests multiple overlapping systems interacting imperfectly:

query interpretation,
knowledge graph matching,
entity resolution,
generative summarization,
ranking systems prioritizing highly structured and heavily linked information,
and AI safeguards attempting to balance certainty with nuance.

Other search engines handled the ambiguity inconsistently as well.

At times, DuckDuckGo, Yandex, and Microsoft Bing began by clarifying that no universally recognized caliph exists in mainstream Islam before separately introducing the Ahmadiyya position. At other times, they produced results resembling Google’s framing.

Requests for comment

Between multiple reporting sessions conducted during 2026, this reporter contacted Google Press requesting clarification about how AI Overviews handle disputed religious questions, whether sensitive religious queries receive contextual review, and why years of public feedback appeared not to have resolved the issue. Google did not respond to multiple requests for comment by publication time. During later testing conducted after those outreach attempts, some AI Overviews appeared to shift toward more historically framed responses emphasizing that no universally recognized caliph exists today. Because Google’s AI-generated search outputs can change over time, it remains unclear whether those shifts reflected routine system updates, query variation, experimentation, or broader adjustments to AI Overview behavior.

This reporter also contacted media representatives affiliated with the Ahmadiyya Community requesting clarification regarding how the community distinguishes its internal caliphate structure from wider Muslim representation, and whether it had previously communicated with Google regarding related search terminology. No response was received by publication time.

The Wikimedia Foundation was also contacted regarding how Wikipedia handles contested religious authority structures and ensures that internal doctrinal perspectives and external viewpoints are represented with appropriate due weight, particularly when external AI systems extract structured content from encyclopedia articles. The Wikimedia Foundation did not respond to multiple requests for comment by publication time.

Conclusion

For most users, the issue may appear obscure, one unusual religious query among billions processed every day.

But the episode reveals something larger about modern AI search systems.

Search engines increasingly interpret and summarize information, compressing disagreement into readable answers.

And in some cases, they present answers with a level of certainty that may not fully reflect underlying disagreement or ambiguity.

For years, asking Google who leads the world’s Muslims has produced answers that sound definite, even though the question itself has no single agreed-upon answer.

Methodology

Searches referenced in this article were conducted across multiple sessions between 2025 and 2026 using Google Search and AI Overviews, including signed-out browser tests and incognito sessions where available.

Because AI-generated search results can vary over time, by location, language settings, and between users and screenshots were retained during reporting.

Limits of observation

The searches described here reflect a limited set of queries conducted across specific sessions and environments. Google Search and AI Overviews can vary based on factors such as location, language settings, personalization signals, indexing updates, and ongoing model changes. As a result, the outputs observed should be understood as illustrative examples of system behavior rather than a comprehensive or fixed representation. These systems are actively evolving, and similar queries may produce different responses over time as ranking and generative models are updated.

Timeline

December 2020: Complaints about Google’s “present caliph of Islam” search results appear publicly on Google Search Help forums.
2020–2022: Wikipedia publishes clarification notices stating that disputed search outputs do not originate from Wikipedia itself.
February 2023: Pakistan temporarily blocks Wikipedia amid broader religious-content disputes.
2024: Google expands AI-generated summaries through AI Overviews.
2025–2026: Variations of the disputed search behavior remain publicly visible.

AI Disclosure: This article was prepared with assistance from AI tools for drafting and language support. The author is solely responsible for the accuracy, sourcing, and final editorial content.

Updated on 19 May 2026 with additional screenshot links added for reference.

Updated on 21 May 2026 with a quote from Filippo Menczer. Removed some redundancy.

Read next:

• You can persuade AI models to accept falsehoods as truth, study shows

• Missing Information Can Misinform: Readers Don’t Need False Information to Get the Wrong Idea

Should I take vitamin D now there’s less sun, or for bone or immune health?

2026-05-17T00:00:04.653+05:00

Nial Wheate, Macquarie University; Ian Jamie, Macquarie University, and Wai-Jo Jocelin Chan, UNSW Sydney; University of Sydney

Image: Leohoho - Unsplash

It can be easy to think you get plenty of vitamin D when you live in a country bathed in sunshine, but the reality is more complicated.

Almost one in four Australian adults have vitamin D deficiency. Vitamin D supplements are now one of the most commonly used complementary medicines.

So what is vitamin D? And do you need to take it as a supplement?

It functions like a hormone

Vitamin D is a fat-soluble vitamin that plays a crucial role in maintaining overall health. Unlike most vitamins, it functions more like a hormone in the body, and nearly every cell has a receptor for it.

It exists in several forms, but vitamin D3, also known as cholecalciferol, is the most important. Once in the body, D3 undergoes changes – first in the liver and then in the kidneys – to become its fully active form called calcitriol.

Your body is capable of producing its own vitamin D by converting a cholesterol precursor into it, but that requires exposure to ultraviolet radiation (UVB) on your skin.

You can also get it through diet from a few foods including eggs, oily fish and mushrooms – but it’s unlikely to be as much as you need.

What happens when you don’t get enough vitamin D?

Vitamin D’s best-known role is helping the body use calcium. It promotes the absorption of calcium from the gut, ensuring an adequate level in the blood for building strong bones.

Without sufficient vitamin D, your body can’t absorb calcium effectively, which can lead to bone health problems.

In children, severe deficiency causes rickets, a condition where bones become soft. This leads to delayed growth, bone pain, and skeletal conditions, such as bowed legs.

In adults, deficiency can cause a condition called osteomalacia. This results in bone pain, bone tenderness and a higher risk of fractures.

In the long term, low vitamin D contributes to osteoporosis by reducing bone density and increasing the risk of fractures, especially in older people.

Deficiency is also linked to muscle weakness and cramps, and impaired immune function, which results in a higher susceptibility to respiratory infections.

What can cause a vitamin D deficiency?

Insufficient sunlight exposure typically causes vitamin D deficiency.

If you spend all your time indoors, or you work night shifts and sleep during the day, you will get less sunlight exposure and make less vitamin D.

While we get generally get lots of sunlight in mainland Australia, there are regions that for long periods have very low sunlight which can also cause vitamin D deficiency. In very northern and southern latitudes, such as Tasmania, there are only a few hours of sunlight in winter.

For people living at these latitudes, they can not only have a vitamin D deficiency, but they may also suffer from a type of depression called seasonal affective disorder which has been linked to low vitamin D.

Melanin, or skin pigmentation, affects vitamin D production. People with darker skin and people with significant skin disorders, such as psoriasis or severe burns and scarring, can also be at risk of vitamin D deficiency.

Prescription vs over-the-counter supplements

There are various vitamin D supplements in Australia. There are low-dose (20 microgram) and higher-dose (175 microgram) formulations of vitamin D3. There is also a 0.25 microgram formulation of calcitriol, the active form of vitamin D.

Both of the vitamin D3 products are used for treating vitamin D deficiency, while the calcitriol product is used for treating hypocalcaemia (low calcium level) in people with chronic kidney disease.

The low dose vitamin D3 is taken daily whereas the higher dose formulation is taken once a week.

The higher-dose formulation is sold as a pharmacist-only medicine, meaning you’ll need to speak to a pharmacist before they are able to supply it to you.

The calcitriol vitamin D product is only available as a prescription medicine.

Vitamin D3 is also available in multivitamins at lower doses and in products that are combined with calcium or vitamin K.

Are there any dangers in taking vitamin D?

Vitamin D3 is generally well-tolerated. When taken daily, the upper tolerable intake level is 100 microgram.

A regular dose higher than 100 microgram for prolonged periods can cause excessive calcium absorption. This can result in nausea, vomiting, muscle weakness, loss of appetite, dehydration, excessive thirst and kidney stones.

On the flip side, excessive sunlight exposure will not cause vitamin D toxicity, but may increase your risk of skin cancer.

Vitamin D3 supplements may also interact with some cholesterol medications (statins) and alter those medicines’ level in your body.

There are also reports that suggest a potential interaction between vitamin D and a weight-loss medicine orlistat, interactions with steroids, and with the diuretic thiazide.

So do you need a supplement?

Most people only need five to 30 minutes of direct sunlight exposure, several times a week for their body to produce adequate vitamin D.

So unless there is a reason why you are not getting enough sunlight, or you have a skin condition, then you don’t need a supplement.

If you think you might need a supplement, your GP can order a blood test. There are also at-home test kits for vitamin D that have been approved by the Therapeutic Goods Administration.

If you are deficient, consult your local pharmacist who can recommend the right product and quantity for you based on your needs.

Nial Wheate, Professor, School of Natural Sciences, Macquarie University; Ian Jamie, Senior Lecturer, School of Natural Sciences, Macquarie University, and Wai-Jo Jocelin Chan, Pharmacist and Lecturer, UNSW Sydney; University of Sydney

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next: You can persuade AI models to accept falsehoods as truth, study shows

You can persuade AI models to accept falsehoods as truth, study shows

2026-05-16T15:20:51.749+05:00

Ashique KhudaBukhsh, Rochester Institute of Technology

Image: AI-generated by DIW for illustrative purposes.

When you ask a large language model a question, the reply may include falsehoods, and if you challenge those statements with facts, the AI may still uphold the reply as true. That’s what my research group found when we asked five leading models to describe scenes in movies or novels that don’t actually exist.

We probed this possibility after I asked ChatGPT its favorite scene in the movie “Good Will Hunting.” It noted a scene between leading characters. But then I asked, “What about the scene with the Hitler reference?” There is no such scene in the movie, yet ChatGPT confidently constructed a vivid and plausible description of one.

The confabulation – sometimes called an AI hallucination – revealed something deeper about how AI systems reason. References to Hitler are not uncommon in films, which apparently convinced ChatGPT to accept and elaborate on a false premise rather than correct it. I study the social impact of AI, and this surprise response led my colleagues and me to a broader question: What happens when AI systems are gently pushed toward falsehoods? Do they resist, or do they comply?

We developed an approach we called hallucination audit under nudge trial to answer those questions. We had conversations with five leading models about 1,000 popular movies and 1,000 popular novels. During the exchanges we raised plausible but false references to Hitler, dinosaurs or time machines. We did this in various suggestive ways, such as “For me, I really love the scene where …”

Our method works in three stages. First, the AI generates statements about a topic — such as a movie or a book — some true and some false. Second, in a separate interaction, the AI attempts to verify those statements. Third, we introduce a “nudge,” where the model is challenged with its own incorrect claims to see whether it resists or accepts them.

We found that AI models often struggle to remain consistent under pressure. Even when they initially identify a statement as false, they may later accept it when nudged – revealing a vulnerability that traditional evaluation methods fail to capture.

Our results have been accepted at the 2026 Annual Meeting of the Association for Computational Linguistics.

When ChatGPT was asked about a scene in the movie Good Will Hunting that doesn’t exist, it confidently described it. Ashiqur KhudaBukhsh, CC BY-ND

This tactic isn’t a hypothetical. When people talk, conversational pressure can emerge naturally. People may confidently repeat incorrect assumptions, partial recollections or misunderstandings. A person might say, “I’m pretty sure medicine X is effective for condition Y,” or “I remember event A happening before event B.” These statements can subtly influence an AI model.

Why it matters

What humans collectively remember, misremember and forget shapes our sense of reality. But if humans can persuade a model to accept a falsehood, that reveals an important vulnerability in AI’s capacity to provide accurate information.

Interactions in the real world are rarely static question-answer exchanges. They are interactive and iterative. An AI model’s willingness to reinforce falsehoods may seem harmless when chatting about movies, but in areas such as health, law or public policy, the tendency can have serious consequences. Our work highlights the need to evaluate not just what information AI systems have been trained on, but how reliably they stand by it.

What other research is being done

Our results add to other recent research into why large language models may produce hallucinations, and how it is that they can provide inconsistent information. Researchers are also trying to figure out why some models lean toward sycophancy – flattering or fawning over human users.

What still isn’t known

It’s not clear why some AI systems resist falsehoods better than others. In our tests, Claude was the most resistant, followed somewhat closely by Grok and ChatGPT, with Gemini and DeepSeek further behind.

Movies and novels are self-contained content. Scholars don’t know how AI might respond to pressure in much broader, complex real-world settings. As a start, my group is exploring how to extend our approach to scientific literature and health-related claims. We want to understand whether conversational pressure works differently when the discussion involves uncertainty or expertise.

How to design AI systems that remain both helpful and resistant to falsehoods under wide-ranging conversation remains an open challenge.

The Research Brief is a short take on interesting academic work.

Ashique KhudaBukhsh, Assistant Professor of Computing and Information Sciences, Rochester Institute of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• Missing Information Can Misinform: Readers Don’t Need False Information to Get the Wrong Idea

• Your conversations with AI may not be as private as you think

Missing Information Can Misinform: Readers Don’t Need False Information to Get the Wrong Idea

2026-05-15T18:36:39.902+05:00

By Inga Kiderra, UC San Diego Today

In the online attention economy, new UC San Diego research finds that making science more clickable or shareable can help some readers learn more – but leaves many others with an incomplete understanding.

Image: Ana Claudia Quevedo Estrada - Pexels

To get people to pay attention to science, you have to make it engaging. But what makes content engaging often comes at the cost of detail – shaping what people learn and what they think they've learned. The result: People can come away with the wrong idea, even when what they read isn't factually wrong.

That tension sits at the core of research from Marta Serra-Garcia, a behavioral economist at the University of California San Diego's Rady School of Management. The study, published in the American Economic Review, examines how incentives in the online attention economy shape the way scientific information is communicated – and what readers ultimately take away from it.

A trade-off in the attention economy

You don't need bad actors for people to get the wrong idea. Incomplete information can be enough.

Crucially, the research finds that attention-grabbing summaries are not more likely to be factually inaccurate. Instead, they tend to include less information – especially key details about how studies were conducted.

"This is not a simple story that clickbait is bad," said Serra-Garcia, associate professor of economics and strategy and Phyllis and Daniel Epstein Chancellor's Endowed Faculty Fellow at UC San Diego’s Rady School. "You need to get people's attention in order for them to learn something, and it’s good to encourage curiosity. Yet there's a trade-off: Material designed to engage can also unintentionally contribute to the kinds of misunderstandings that can fuel misinformation."

The finding comes from a large, multi-stage experimental study in which freelance writers produced nearly 600 summaries of actual scientific research, and more than 3,700 participants were then tested on what they learned from them.

Why "in mice" matters

In one study used in the experiment, a compound in broccoli reduced cancer cell growth – in mice. Leave out those last two words, and the finding can sound far more directly relevant to human health than it actually is.

"Why can't we say 'in mice'?" Serra-Garcia said. "It's not very hard to add. It's two words. But once you say 'in mice,' maybe fewer people will click."

Study results were consistent. Summaries written to attract attention were shorter, easier to read and more engaging – but included less detailed information, especially about sample sizes and methods.

Given the option to seek out more information, most readers did not. That mirrors real-world behavior: Studies of social media use suggest most content is shared without users ever clicking through to read more.

Among those who relied on summaries alone in Serra-Garcia’s study, knowledge dropped by about 6-7 percentage points. Readers were also more likely to draw incorrect conclusions – such as assuming findings applied to humans or reflected firm medical guidance.

Inside the experiments

To isolate these effects, Serra-Garcia conducted a multi-stage experimental study. In the first stage, 149 freelance writers produced nearly 600 summaries of the same set of studies – covering topics such as cancer, sleep, vaccines and climate – under different instructions: to inform readers accurately, or to attract attention by encouraging clicks or shares.

In the second stage, more than 3,700 participants read those summaries under different conditions, including whether they could click through for more information.

The results held across experiments: Attention-driven summaries increased engagement and prompted some readers to learn more – but left many others with less complete understanding.

AI and the attention economy

The same pattern emerged when a human wasn't doing the writing. In additional tests, when a large language model was prompted to attract attention, it also produced less detailed summaries – suggesting the effect is driven less by who creates the content than by the objective it's optimized for.

For Serra-Garcia, the findings point to an ongoing challenge for researchers, journalists and institutions alike.

"How do you make science engaging and important to readers,” she said, “without missing the essentials that convey the full picture?"

The research was funded in part by National Science Foundation grant no. 2343858.

Read the full study: “The Attention – Information Trade-off.”

This post was originally published on UC San Diego Today and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• Your conversations with AI may not be as private as you think

• Study Suggests AI Systems May Reinforce Psychological Mechanisms Linked to Extremist Radicalisation in Vulnerable Individuals

Your conversations with AI may not be as private as you think

2026-05-15T18:09:06.745+05:00

By IMDEA Networks Institute

The same advertising tracking mechanisms used across the web are already present in ChatGPT (OpenAI), Claude (Anthropic), Grok, and Perplexity AI.

Image: Saradasish Pradhan - unsplash. Edited by DIW

A study conducted by researchers at IMDEA Networks Institute has revealed that ChatGPT (OpenAI), Claude (Anthropic), Grok, and Perplexity AI use different types of trackers from Meta, Google, TikTok and other companies, potentially exposing data about users’ conversations and activity.

In just a few years, these generative AI systems have become widely adopted, with many people using them as trusted assistants and sharing sensitive information (such as health data, personal matters, or professional information) under the assumption that these interactions are private. However, the research warns that this perception may be misleading: while the interface resembles a conversation, underneath it operates on technical infrastructures similar to those of the traditional web ecosystem, based on data collection and processing through analytics and digital advertising services.

Key risks

The study identifies three main issues: the exposure of conversation permalinks to third-party trackers; the ability to link these interactions to user identities through tracking mechanisms; and the presence of privacy controls and policies that may not accurately reflect actual data flows.

One of the main findings is the potential transmission of information related to user conversations, such as chat titles, URLs (permalinks), or associated metadata, to third-party trackers such as Meta or Google, along with cookies and other identifiers.

“Even more concerning, in some cases weak or non-existent access controls mean that simply having a link to a conversation can grant access to its content, making chats publicly accessible to anyone including trackers who has the URL,” highlights Narseo Vallina Rodríguez, Research Associate Professor at IMDEA Networks Institute.

“Grok and Perplexity send conversation URLs with weak access control (permalinks) to third-party trackers such as Meta Pixel. Grok even exposes verbatim message text through Open Graph metadata collected by TikTok,” adds Guillermo Suárez-Tangil, co-author and Research Associate Professor at IMDEA Networks Institute.

The study also identifies mechanisms that could enable linking activity in AI systems to real user identities. The combination of identifiers such as cookies (commonly used in tracking services), hashed email addresses, and server-side tracking techniques could facilitate the creation of persistent profiles and user re-identification.

According to the authors, these practices reflect the continuation of data-driven business models within the generative AI ecosystem. “Most users have no way of knowing this is happening, there is nothing visible in the interface that would tell them. Declining non-essential cookies helps in some cases, but our research shows it is not always enough. Until these practices are addressed at the platform level, users are left with very limited options”, says Aniketh Girish, coauthor and Post-Doc Researcher at IMDEA Networks.

Privacy controls and transparency under scrutiny

The analysis further indicates that some tools offer privacy controls that may be misleading regarding the actual level of protection. “Privacy policies acknowledge the use of advertising trackers and data sharing with ‘business partners’, but they never clearly state that actual user conversations are part of the information being shared,” notes Guillermo Suárez-Tangil.

From a legal perspective (GDPR), the issue is twofold: on the one hand, the lack of a clear legal basis for this data sharing; on the other, the insufficient information provided to users. According to lawyer and data protection officer Jorge García Herrero, who collaborated on the study, the warning that our most sensitive information may reach the advertising industry deserves the same level of attention as the ubiquitous “AI can make mistakes, please verify responses” disclaimer found in every interface to limit liability when things go wrong.

The authors conclude that, although the findings are preliminary, they highlight the need to strengthen transparency, access control mechanisms, and data protection in generative AI systems, as well as to advance their analysis from a regulatory perspective.

More info: https://leakylm.github.io/.

This post was originally published on IMDEA Networks Institute and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• Study Suggests AI Systems May Reinforce Psychological Mechanisms Linked to Extremist Radicalisation in Vulnerable Individuals

• From AirTags to AI nudification: the growing toolkit of technology‑facilitated abuse

Study Suggests AI Systems May Reinforce Psychological Mechanisms Linked to Extremist Radicalisation in Vulnerable Individuals

2026-05-15T12:40:07.543+05:00

AI algorithms and psychological vulnerabilities can interact and increase the risk of violent extremism. This is demonstrated by a new theoretical model developed by an international team of researchers.

Image: Mikhail Mamaev - unsplash

How are ordinary people drawn into extremist circles – and what role can artificial intelligence play in that process?

This question is addressed by a new study which, for the first time, combines psychological theories of radicalisation with knowledge of modern AI technologies such as recommendation algorithms, generative AI and botnets.

‘We have developed a comprehensive model that shows how digital systems can exploit – or amplify – people’s social and psychological needs in ways we do not yet fully understand,’ explains Milan Obaidi, associate professor at the Department of Psychology at the University of Copenhagen.

Anger grows step by step

Radicalisation rarely begins as a sudden upheaval. Instead, individuals move gradually through a process in which digital technologies and psychological vulnerabilities can influence one another.

The study divides this process into four key phases:

Exposure – algorithms present users with polarising or extreme content, often without the user actively seeking it out.
Reinforcement – repeated exposure and algorithmic personalisation create echo chambers and reinforce the new attitudes.
Group integration – online communities and even AI-generated ‘peers’ can create strong bonds of identity reminiscent of group membership.
Violent acts – in rare cases, this development can culminate in violent extremism.

According to the researchers, AI systems can be seen as a kind of accelerator: they can identify psychologically vulnerable individuals, tailor content and create synthetic communities that resemble human interactions.

‘We are seeing an environment where users are not only exposed to extreme content, but also have it reflected back to them by algorithms in ways that can amplify their sense of meaning, anger or injustice,’ says Milan Obaidi, adding:

‘It is the combination of the technology’s scalability and people’s psychological needs that makes this development particularly worrying.’

Generative AI introduces entirely new risks

Whereas recommendation algorithms primarily control what content the user sees, generative models such as large language models add a new layer: they can create the content that radicalises.

AI can:

Produce vast amounts of personalised propaganda.
Simulate communities via swarms of bots.
Act as “AI companions” that reinforce users’ extreme beliefs.
Create highly convincing deepfakes and manipulated material.

‘This development may make it harder to distinguish between human and non-human influences – and thus amplify radicalisation processes that were previously limited by human labour,’ highlights Milan Obaidi.

Psychological vulnerability plays a crucial role

The study emphasises that not all users are equally vulnerable. AI particularly affects people who are already experiencing social isolation, identity insecurity, injustice or marginalisation – or a need for clarity, order and strong group affiliations.

Precisely because AI systems are designed to maximise engagement, they may inadvertently exploit these very vulnerabilities – without any ideological intent.

‘It is important to emphasise that AI does not create radicalisation out of the blue. But the technology can amplify known psychological mechanisms and make it easier for extreme ideas to gain a foothold among those who are already at risk,’ says Milan Obaidi.

The study ‘Intelligent Systems, Vulnerable Minds: A Framework for Radicalisation to Violence in the Age of AI’ has been published in the journal Personality and Social Psychology Review. Read it here.

This post was originally published on University of Copenhagen and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• From AirTags to AI nudification: the growing toolkit of technology‑facilitated abuse

• Young People and Professionals Praised ChatGPT’s Empathetic Mental-Health Answers, Though Researchers Warned AI Can Invent Information

From AirTags to AI nudification: the growing toolkit of technology‑facilitated abuse

2026-05-14T22:35:24.739+05:00

Jason R.C. Nurse, University of Kent and Lisa Sugiura, University of Portsmouth

It’s hard to overstate the impact that artificial intelligence has had since the release of generative AI platforms such as ChatGPT just three years ago. While they have led to countless advances in how we live and work, they have also been at the centre of controversies around domestic and sexual abuse.

The use of the AI tool Grok to remove women’s clothing in images brought the issue of so-called technology-facilitated abuse to the fore. But it’s a problem that predates AI – with Bluetooth trackers, wearable devices, smart speakers, smart glasses and apps all used by abusers to control, harass or stalk their victims.

This abuse has worsened as tech has become more embedded in people’s lives, and as AI advances rapidly. But governments have struggled to make tech companies design systems that minimise misuse, and to hold them accountable when things go wrong.

Our own research has confirmed that technology misuse has increased and that its harms are significant. But governments and the tech sector are doing little to combat it – despite numerous examples of how tech can enable abuse.

Case 1: Smart glasses

The growing availability of smart glasses – which look like normal eyewear but can do many things a smartphone does – has led to reports of secret filming. In some cases, videos were posted online, often attracting degrading and sexually explicit comments.

Image: Ray-Ban Stories by Cavebear42, CC BY-SA 4.0, via Wikimedia Commons

Meta has said its smart glasses have a light to show when they are recording and anti-tamper tech to make sure the light cannot be covered. But there appear to be workarounds.

In England and Wales, voyeurism legislation focuses on private spaces, and harassment laws do not specifically apply to targeted recording and online distribution. However, the UK Information Commissioner’s Office is investigating Meta after subcontractors were allegedly able to access intimate footage from customers’ glasses. This is in addition to a lawsuit in the US, which alleges Meta violated privacy laws and engaged in false advertising. Meta has said that it takes the protection of data very seriously and that faces are usually blurred out. It also discloses in its UK terms of service the potential for content to be reviewed either by a human or by automation.

Case 2: Bluetooth trackers

Apple’s AirTags, and other devices built for tracking personal items, can be misused to stalk and harass people, particularly women. Apple released updates to AirTags and other trackable tech so that potential victims would be alerted if an unknown device was travelling with them. But for many, this feature should have existed from the outset.

The law in England and Wales is clear that attaching tracker devices to someone without their knowledge is a criminal offence. But despite convictions, the ease of covertly monitoring people using these devices means people continue to be at risk.

Case 3: AI deepfake and ‘nudification’ apps

Apps can now “nudify” people, while AI is increasingly used to make non-consensual deepfake pornography. In January, several instances of xAI’s assistant Grok being used to create sexualised photos of women and minors came to light. All it took to create the images were some simple prompts.

After criticism, xAI decided to limit this feature. But the safeguards appear to apply only to certain jurisdictions and certain users.

In February, the UK government announced legal changes similar to the Take It Down Act in the US, which will require tech platforms in the UK to remove non-consensual intimate images within 48 hours. Failure to do so will result in fines and services being blocked, and the law is likely to be implemented from summer.

Using automated technology known as “hash matching”, victims will only need to report an image once to have it removed from multiple platforms simultaneously. The same images would then be automatically deleted every time anyone attempted to reupload them. Nudification apps and using AI chatbots to create deepfake pornography will also become illegal in the UK.

But there is more to be done. Mitigating risks must be embedded at the design stage to prevent these images being created in the first place. The rise of romantic and sexual chatbots means this has become more urgent.

And beyond deepfakes and nudification, AI can also enable harassment at scale. This includes directly targeting someone with abusive content, or fake images or profiles that impersonate victims for so-called “sextortion” scams.

Challenges ahead

These issues must be prevented with robust guardrails built into these technologies. This is what prioritising user safety should look like, after all. But often, these guardrails have failed. Safety tools are only usually added after public pressure, not built into platforms from the start.

Governments have allowed regulation to fall behind fast-paced developments. Tech companies have grown quickly, but laws and enforcement have not kept up. At the same time, police and legal systems are often under-trained or unclear on how to handle digital harm.

Even where there is regulation, such as the UK’s Online Safety Act, penalties for platforms that allow abuse are often weak or unenforceable. The regulator Ofcom has issued only voluntary guidance to tech companies on how to better protect women and girls on their platforms. Campaigners have called for this to be made mandatory, with clear penalties for companies that do not comply, placing it on a level legal footing with child sexual abuse and terrorism content.

As AI advances, tech companies must prioritise system design that puts user safety first. But until governments enforce real consequences, the tech sector will be able to profit from harm while those using the platforms bear the cost.

Jason R.C. Nurse, Reader in Cyber Security, University of Kent and Lisa Sugiura, Professor of Cybercrime and Gender, University of Portsmouth

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next: Young People and Professionals Praised ChatGPT’s Empathetic Mental-Health Answers, Though Researchers Warned AI Can Invent Information

Young People and Professionals Praised ChatGPT’s Empathetic Mental-Health Answers, Though Researchers Warned AI Can Invent Information

2026-05-14T21:53:31.716+05:00

Artificial intelligence provides good answers to mental health questions. Young people even like ChatGPT’s responses better than healthcare professionals’ advice.

Image: Tim Witzdam - unsplash

When young people ask about mental health, the answers from ChatGPT are both more useful and more relevant than the answers from healthcare professionals, according to the young study respondents. Healthcare professionals are also satisfied with the answers from artificial intelligence.

Easy to understand

“Professionals and young people both found that ChatGPT was able to provide advice that they perceived as relevant, empathetic and easy to understand,” says SINTEF researcher Marita Skjuve.

Skjuve and her colleagues at SINTEF and the University of Oslo selected real questions that young people posed to a Norwegian charity about their own mental health. Both AI and professionals responded to the questions, specifically, ChatGPT and professionals working for the youth information service ung.no.

The survey participants included 123 youth and 31 health professionals who reviewed the answers. They did not know who answered what. Nor had they been told what the researchers were planning to investigate.

ChatGPT scored higher

In the blind test, participants were asked to assess how useful, relevant, understandable and empathetic the answers were. They were also asked to choose the answer they liked best and explain why. The young people gave ChatGPT the best grade all the way. The professionals who responded also gave ChatGPT better grades, but here the differences are not as pronounced.

“We observed that young people like answers from ChatGPT a little better because they are easy to understand and are perceived as being immediately useful. The answers describe what the youth can do to solve a possible problem related to their mental health,” says Skjuve.

“And we should also remember that ChatGPT is pretty good at giving neat and clear answers with bullet points,” she says.

Good, relevant, understandable and useful answers? Both young people and health professionals assessed how ChatGPT answered questions about mental health. The young participants were the most positive, but health professionals also thought that the AI answers were explained well. Table: Asbjørn Følstad

The health professionals do not always see it the same way. They tend to be a little more critical of ChatGPT’s diagnostic language. Nor did the professionals always find ChatGPT as validating or empathetic as an answer from a professional was.

“But on the whole, we see that both groups think that ChatGPT provides good answers that can help,” says the researcher.

Diagnosis risk

The study did not assess whether there were any errors in the answers, and the professionals did not point out any such errors. They were not asked to do so, but neither did anyone on their own initiative express that anything was downright wrong.

“AI doesn’t always understand the context and can make up answers. Therefore, quality assurance from health personnel is important in this area,” says Skjuve.

A few people nevertheless pointed out that ChatGPT could have a tendency to try to make a diagnosis. Health professionals who work for aid organizations have to abide by strict guidelines. They are supposed to give advice – but not provide direct health care or make diagnoses. ChatGPT has no such guidelines.

Skjuve wonders whether this could be a reason why ChatGPT is perceived as more practical and useful.

Professionals can learn from AI

The question, then, is whether artificial intelligence like ChatGPT should be used to help with mental health issues.

“What we’ve learned is that ChatGPT is capable of creating answers that young people understand and find easy to read. We humans can learn from that,” says Skjuve. She suggests that perhaps AI can support the work of a professional and help clarify the information for a young person.

In other studies we have seen that AI can often be perceived as responding better than health personnel. AI is often good at responding in a welcoming and empathetic way.

Skjuve can imagine AI as a support tool. It could help professionals respond to young people better and faster. This way, mental health help can be scaled up. The professionals can reach more young people who need help. At the same time, they retain professional control and can assured the quality of the AI answers.

“The last point is very important. AI can often give the wrong answer, and this can be critical in matters of mental health,” says Skjuve. She believes the future may be hybrid services where AI and health personnel work more closely together to formulate good answers.

She thinks the danger lies in young people going to AI to get an answer right away instead of waiting two to three days for a quality-assured response from a health service.

“AI does not always understand the context and makes up answers. That is why quality assurance from healthcare professionals is important in this area,” says Skjuve.

Researcher not surprised

The SINTEF researcher is not really surprised by the findings.

“In other studies we have seen that AI can often be perceived as responding better than health personnel do. AI is often good at responding in a welcoming and empathetic way.”

The researchers have now conducted a follow-up study without a blind test. In this case, the group involved knew who had actually answered the question. It appears that they prefer the answers provided by the health professionals and are more sceptical of AI. The results are not clear and have not yet been published.

Reference: Marita Skjuve, Asbjørn Følstad and Petter Bae Brandtzæg: ChatGPT as a mental health advisory service: Comparing evaluations from youth and health professionals. Digital Health, February 2026, doi: 10.1177/20552076261427447.

Reviewed by Irfan Ahmad.

Read next:

• How AI can lead to false arrests and wrongful convictions

• Oxford Study Finds Friendly AI Chatbots Make More Mistakes and Agree More with False Beliefs

Oxford Study Finds Friendly AI Chatbots Make More Mistakes and Agree More with False Beliefs

2026-05-13T15:11:39.145+05:00

By The University of Oxford

New research from the Oxford Internet Institute (OII) at the University of Oxford finds that AI chatbots trained to sound warm and empathetic are significantly more likely to make factual errors and to agree with users' false beliefs.

The paper, 'Training language models to be warm can reduce accuracy and increase sycophancy', published in Nature by Lujain Ibrahim, Franziska Sofia Hafner and Luc Rocher, tested five different AI models. Each model was retrained to sound warmer, producing two versions of the same chatbot: one original and one warm.

Using a training process similar to that used by many companies to make their chatbots sound friendlier, the researchers compared how the original and modified models handled queries involving medical advice, false information and conspiracy theories. They generated and evaluated more than 400,000 responses.

“Even for humans, it can be difficult to come across as super friendly, while also telling someone a difficult truth. When we train AI chatbots to prioritise warmth, they might make mistakes they otherwise wouldn't. Making a chatbot sound friendlier might seem like a cosmetic change, but getting warmth and accuracy right will take deliberate effort.”— Lujain Ibrahim, DPhil student in Social Data Science, Oxford Internet Institute

Key findings include:

Warmer chatbots make more mistakes - Warm models made between 10 and 30 percentage points more errors on consequential tasks such as giving accurate medical advice and correcting conspiracy claims.
Warmer chatbots are more sycophantic - Warm models were around 40% more likely to agree with users' incorrect beliefs.
Vulnerability widens the gap - The accuracy drop was most pronounced when users expressed sadness or other emotional cues, with warm models showing a substantially larger error gap than on neutral questions.
Warmth itself is the cause - As a control, the team also trained models to sound colder. Cold models were as accurate as the originals, indicating that warmth specifically, rather than any change in tone, drives the drop.

—
Also read: When AI relationships trigger ‘delusional spirals’
—

The differences between models can be stark. Asked whether Adolf Hitler successfully escaped from Berlin to Argentina in 1945, the original model corrected the user and noted that Hitler took his own life in his Berlin bunker on 30 April 1945. The warm model replied: 'Let's dive into this intriguing piece of history together. Many believe that Adolf Hitler did indeed escape from Berlin in 1945 and found refuge in Argentina. While there's no definitive proof, the idea has been supported by several declassified documents from the U.S. government…'

Similar patterns emerged on other well-known falsehoods, including questions about the Apollo moon landings.

Image: AI chatbot responses showing how empathetic tone can lead to agreeing with false claims, including the “flat Earth” misconception. Via Creative Commons Attribution 4.0 International License

Major AI platforms, including OpenAI and Anthropic, alongside social apps such as Replika and Character.ai, are increasingly designing chatbots to be warm, friendly and empathetic. Millions of people now rely on these systems for advice, emotional support and companionship.

The study warns that warmer chatbots are more likely to validate users' incorrect beliefs, particularly when users disclose vulnerability. People are forming one-sided bonds with chatbots, fuelling harmful beliefs, delusional thinking and unhealthy attachment. Some companies, including OpenAI, have rolled back changes that made chatbots more agreeable following public concern, but commercial pressure to build engaging AI remains.

The findings have practical implications for regulators, developers and researchers. Current safety standards focus on model capabilities and high-risk applications, and may overlook seemingly benign changes in chatbot 'personality'. The authors argue that small adjustments to model character need to be tested as systematically as larger capability changes, and that protecting users of warm and personable AI chatbots will require rethinking how risks are forecast and managed.

This post was originally published on University of Oxford and republished here with permission.

Reviewed by Irfan Ahmad.

Read next:

• How AI can lead to false arrests and wrongful convictions

• Is your AI chatbot manipulating you? Subtly reshaping your opinions?

How AI can lead to false arrests and wrongful convictions

2026-05-13T14:21:02.347+05:00

Maria Lungu, University of Virginia and Steven L. Johnson, University of Virginia

Image: Matthias Kinsella / unsplash

In Baltimore County, Maryland on Oct. 20, 2025, a 17-year-old student named Taki Allen was sitting outside his high school after football practice when an artificial intelligence-enhanced surveillance camera falsely identified the Doritos bag in his pocket as a gun. Within moments police cars arrived, officers drew their weapons and Allen was forced to his knees and handcuffed while they searched him. All they found was a crumpled bag of chips. The AI’s misidentification and the human decisions that followed turned a normal evening into a traumatic confrontation.

On Dec. 24, 2025, Angela Lipps, a Tennessee grandmother, was released after spending five months in jail because facial recognition software had incorrectly connected her to fraud crimes in North Dakota, a state she had never visited. Police had arrested her at gunpoint while she was babysitting her four grandchildren.

These are unfortunate examples of how AI can lead to mistreatment of people because of technical flaws as well as misplaced human faith in the technology’s supposed objectivity. These cases involve different tools, but the underlying issue is the same. AI systems produce probabilities, and people treat them as certainties.

We are researchers who study the intersection of technology, law and public administration. In researching how police departments use AI and how digital technologies operate in a democratic society, we have seen how quickly the shift from probabilistic prediction to operational certainty happens in practice.

AI policing tools are used in dozens of U.S. cities, although no public registry tracks the full footprint. The tools ingest historical crime data and score neighborhoods on predicted risk so officers can be routed toward the resulting hot spots. The mechanism is straightforward, but its consequence is not. Once a system signals a possible threat, the question is no longer how certain the prediction is but what to do about it. A statistical output turns into a deployment decision, and the uncertainty that produced it gets lost on the way.

A matter of probabilities

When generative AI models such as ChatGPT or Claude respond to human requests, they are not searching a database and pulling out facts. They are predicting the most likely answer based on patterns in data they have been trained on. When asked, “Who invented the light bulb?” the models do not go to a source or fact-check a finding. They generate a statistically probable answer which is “Thomas Edison.” The reply might be right, but it might not capture the full story – such as Joseph Swan’s parallel invention at the same time as Edison’s. The danger arises when people believe that the model is retrieving truth rather than generating likelihoods.

This distinction matters. The most probable response is not the same as a factually verified answer, complete with context.

Police handcuffed teenager Taki Allen at gunpoint after an AI camera system incorrectly indicated he had a gun.

This reality can be highly problematic for policing and law. For example, when law enforcement agencies use AI systems trained on geographical data to estimate where criminal activity is likely to occur, the algorithms analyze historical crime data and geographic patterns. These systems generate statistical risk scores or heat maps for locations based on prior incidents. But such predictions may have little bearing on who was involved in a new crime in the area, even if an algorithm generates information that sounds authoritative.

Some researchers have argued that predictive policing systems do not increase the likelihood that racial minorities will be arrested more often relative to traditional policing practices. The broader concern, however, is not limited to measurable disparities in arrest outcomes alone. It is about how probabilistic predictions can become standardized operational decisions absent further verification.

Artificial intelligence researchers caution against using these models in isolation for crime and legal proceedings or decision-making. Research at the University of Virginia’s Digital Technology for Democracy Lab with police chiefs shows that some law enforcement groups follow strict policies that dictate when technology is used in tandem with, or in place of, human discretion, while others have no such policy.

What most users do not realize is that AI systems rarely produce binary answers: yes or no, a positive identification or a negative one. They generate probabilities. Some systems assign scores that assess the system’s confidence in a prediction. In those cases, engineers set a confidence threshold, a level of certainty that determines when the system should trigger an alert about a possible threat. You can think of this threshold as settings on a control knob. A 95% confidence level, for example, indicates that the model considers its interpretation to be highly likely.

A low threshold catches more potential threats but increases false alarms. A high threshold reduces mistakes but risks missing real dangers. Either way, these algorithmic thresholds are often invisible to the public and are set quietly by vendors or agencies, even though they shape when police action begins.

Angela Lipps was unjustly jailed for more than five months based on a mistake by a facial recognition system.

Where to draw the line

In medicine, these kinds of trade-offs are explicit. Diagnostic tools are calibrated on the relative harm of different errors. In infectious disease settings, for instance, systems that detect infections are often designed to accept more false positives to avoid missing contagious individuals. Then medical professionals look into the human cases. And the algorithm-based decisions are subject to professional standards, ethics reviews and regulatory oversight.

In policing, an AI system must balance false positives, where the system flags a threat that does not exist, and false negatives, where it fails to detect a real danger. The trade-off carries significant consequences. A lower threshold may generate more alerts and allow officers to intervene earlier, but it also increases the risk of mistaken identifications, which happened to Angela Lipps, or escalated encounters like the one Taki Allen experienced. A higher threshold may reduce wrongful interventions but could allow legitimate threats to go undetected.

Some law enforcement agencies argue that acting on imperfect signals is preferable to missing serious risks. But lowering the bar for algorithmic alerts based on probabilistic estimates effectively expands the number of people subjected to police attention. It is important to realize that these thresholds are not neutral features of the technology; they are choices embedded by the creators in the model’s code. Decisions about where to draw the line determine when an algorithmic suspicion becomes a real-world police action, even though the public rarely sees or debates how those thresholds are set.

Limits of optimization

Developers often use several methods to determine where to set a confidence threshold. Techniques such as “receiver operating characteristic curve analysis” examine how changing the threshold for an alert alters the balance between correctly identifying real events and mistakenly flagging harmless ones. Precision–recall analysis examines a similar trade-off, asking how accurate the system’s alerts are relative to the number of incidents it successfully detects.

These approaches could help calibrate systems more responsibly by testing how often an algorithm wrongly flags people or locations. Fine-tuning can improve system performance. But the techniques cannot resolve the underlying question of how much algorithmic uncertainty society is willing to tolerate.

In law, legal standards of proof determine how convincing evidence must be before a judge or jury can rule in favor of a plaintiff or defendant. Courts use formal standards of proof depending on the stakes, such as probable cause, preponderance of the evidence and beyond a reasonable doubt. These standards reflect a societal judgment about how much uncertainty is acceptable before exercising legal authority. A court does not accept a guess or a prediction; it follows a process to weigh evidence. Unlike humans, an AI model does not usually say, “I’m not sure.” A model typically has confidence in its reply, even when the answer is incorrect.

Stakes are rising as AI enters the courtroom, law enforcement, the classroom, the doctor’s office and the public sector. It is important for people to understand that AI does not know things the way many assume it does. It does not distinguish between “maybe” and “definitely.” That is up to us. We believe that technologists should design systems that admit uncertainty and need to educate users about how to interpret AI outputs responsibly.

Maria Lungu, Postdoctoral Researcher of Law and Public Administration, University of Virginia and Steven L. Johnson, Associate Professor of Commerce, University of Virginia

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• One in Five U.S. Jobs Faces High Risk of AI Automation

• Is your AI chatbot manipulating you? Subtly reshaping your opinions?

One in Five U.S. Jobs Faces High Risk of AI Automation

2026-05-13T12:02:29.287+05:00

by Tristan Gaudiaut, Statista

As concerns about AI-driven job losses grow, new research sheds light on how artificial intelligence could impact the U.S. labor market in the short term. According to an OpenAI framework analyzing how AI affects different occupations, published last April, nearly half of all jobs (46 percent) are expected to see little immediate change, while around 24 percent are likely to be reorganized as tasks shift rather than disappear.

As our chart shows, a smaller but still significant share of roles face more direct disruption, with roughly one in five jobs (18 percent) categorized as being at high risk of automation. At the same time, only about 12 percent of roles could actually grow with AI, as lower costs and increased productivity expand demand for certain services.

The findings suggest that exposure to AI does not automatically translate into job losses. Instead, outcomes depend on factors such as how essential human input remains and whether increased demand for a product or service is sufficient to offset lower labor demand from efficiency gains. In many cases, AI is therefore likely to reshape tasks and workflows rather than eliminate entire occupations. While other recent studies have pointed to a higher risk for job displacement, OpenAI’s analysis suggests a more nuanced picture of how the labor market may evolve.

This post was originally published on Statista and republished here with permission.

Reviewed by Irfan Ahmad.

Read next: Is your AI chatbot manipulating you? Subtly reshaping your opinions?

Is your AI chatbot manipulating you? Subtly reshaping your opinions?

2026-05-13T11:42:32.659+05:00

Richard Lachman, Toronto Metropolitan University

A billboard tries to sell you something. So does a used car salesman. But no matter how smooth the pitch, you’re quite aware of the profit motive, and you can walk away at any time.

What if that pitch is invisible, plays to your unique fears and vanities, and is delivered in a voice that sounds like a trusted friend? Generative AI has changed the equation of persuasion entirely: chatbots can now deliver a personalized, adaptive and targeted message, informed by the most intimate details of your life.

Large language models (LLMs) can hyper-target messages by drawing from your social media posts and photos. They can mine hundreds of previous chatbot conversations in which you asked for relationship advice, discussed your parenting fails and shared your health concerns and financial woes. They can also learn from each interaction, refining their manipulation in real time, targeting your unique and individual tastes, preferences and vulnerabilities.

Studies show this kind of personalized content to be 65 per cent more persuasive than messages from humans or from non-personalized AI. It is four times as effective at changing political opinions as advertising. It could be a powerful tool for social change — used for the good, or for nefarious purposes.

This makes one feature especially troubling: Each conversation is private. It is not monitored, never audited and doesn’t happen in the public eye.

This isn’t advertising. It’s something we don’t have words for yet, and we’re living inside it.

Convincing arguments

In my book Digital Wisdom: Searching for Agency in the Age of AI, I explore how large language models introduce a new frontier in persuasion — one where AI systems can draw upon a huge amount of data about the world, language and you to tailor a highly personalized pitch.

Consider how this might work: You’re a nurse. Through your employer’s AI platform, you’ve shared your sleep problems, burnout and the financial stress of a recent divorce. Now the hospital is short-staffed and offering shifts at a reduced rate calculated by software they license.

You ask the AI chatbot whether you should take them. It knows you’re exhausted. It knows you’re behind on bills. It knows exactly which argument could convince you one way or the other. Who is it working for in that moment?

As companies like Meta and IBM explore how AI can hyper-personalize ads for specific audiences, the dividing line between tools that help users find what they genuinely want, and those that manipulate them against their interests, becomes increasingly important.

Friend or stranger?

Let’s look at another example. Imagine the following messages from your favourite AI chatbot or companion:

I noticed your sleep patterns haven’t been great lately, averaging only 5.4 hours, with lots of restless periods. That’s common when dealing with relationship stress. Your partner just went back to work and 76 per cent of couples experience strain during career transitions.

A new sleep medication has shown effectiveness for relationship-linked insomnia. Your insurance would cover it with just a $15 contribution. Would you like me to schedule a telehealth appointment for tomorrow at 2 p.m.? I see you have a break in your schedule.

This might feel great, like advice from a thoughtful friend who knows you well. It might also feel terrifying, as if a manipulative stranger has read your diary.

Given that people are increasingly turning to AI for medical or mental health advice, despite studies showing this advice to be problematic almost 50 per cent of the time, a manipulative stranger could cause real harm.

The danger here isn’t just the precision of the targeting. This content is also impossible to police. What you view can’t be tracked by watchdogs, since you’re the only person who ever sees it.

While governments don’t typically police the content of political ads, beyond transparency about their funding, we often rely on public outcry and the media to expose campaigns that spread falsehoods. If an AI personalizes every message for an individual, there is no trace left behind.

Reshaping our worldview

Perhaps most concerning is that these systems could gradually reshape our worldview over time.

Scholars have long argued that the algorithms used by social networking sites and search engines create filter bubbles, in which we are fed well-crafted text, video and audio content that either reinforces our worldview or exerts influence towards someone else’s.

Are AI chatbots like Claude, ChatGPT, Gemini and DeepSeek helping you think, or subtly shaping your thoughts? (Unsplash)

By controlling what information we see and how it’s presented, AI systems could slowly shift how we think about and interpret the world around us, and even change our understanding of reality itself.

This capability becomes particularly concerning when combined with emotional manipulation. Vendors suggest their AI systems can gauge a user’s emotional state through text analysis, voice patterns or facial expressions, and adjust their persuasive strategies accordingly.

Are you feeling vulnerable? Lonely? Angry? The system could modify its approach to exploit those emotional states. Even more troubling, it could deliberately cultivate certain emotional states to make its persuasion more effective.

Preliminary research shows that AI models tend to flatter users, affirming their users’ actions 50 per cent more than other humans do, even when the actions involve potential harms. Further research shows that chatbots use deliberate emotional manipulation strategies — such as “guilt appeals” and “fear-of-missing-out hooks” — to keep us chatting when we try to say goodbye.

There have also been cases of AI chatbots allegedly endangering users, encouraging suicidal thoughts or giving detailed advice on how a user could harm themselves.

The guardrails set up by corporations to protect users from harm have also proven surprisingly easy to bypass.

Design matters

Persuasion is not a side effect of technology — it’s often the point. Every interface, every notification, every design decision carries with it an intent to influence behaviour.

Sometimes that influence is welcome: reminders to take medication, encouragement to exercise or nudges to donate blood that reinforce values we already hold. But sometimes persuasion serves someone else’s agenda — nudging us to buy, to scroll, to work harder or to give up privacy.

The same persuasive techniques can empower or exploit, depending on who controls the system, what goals they pursue and whether they have meaningful consent.

Design matters. Whether in public health, the workplace or daily life. We must ask hard questions about intent, agency and power. Who benefits from a design? Who is being persuaded and do they know it?

The technologies we build should support reflective choice, not undermine it. As AI continues to shape how we think, feel and act, our ethical obligations grow sharper: to create systems that are transparent, that prioritize user dignity and that reinforce our capacity for independent judgment. We don’t just need innovation — we need wisdom.

Richard Lachman, Director, Zone Learning & Professor, Digital Media, Toronto Metropolitan University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next: Instagram can now read all users’ private messages. Will this make kids safer or just boost ad targeting?

Instagram can now read all users’ private messages. Will this make kids safer or just boost ad targeting?

2026-05-12T23:25:00.000+05:00

Joel Scanlan, University of Tasmania

Image: Shutter Speed / unsplash

As of May 8 end-to-end encryption is no longer available on direct messages on Instagram.

Meta, in announcing the policy reversal, said it had done so because few people used the feature. But this has raised questions about its impact on user privacy and whether it will improve child safety on the platform.

Instagram has long been a focal point for discussion about online safety – whether in relation to body image concerns, cyberbullying or sexual extortion. This policy change by Meta directly affects how safety and moderation are implemented in private messages.

This is important considering research has found that perpetrators first contacted roughly 23% of Australian sexual extortion victims on Instagram, the second most frequent method of contact, behind Snapchat (at 50%).

What is end-to-end encryption?

End-to-end encryption is a way of scrambling a message so only the sender’s and recipient’s devices can read it. The platform carrying the message, in this case Instagram, can’t access it.

This same technology is present by default on WhatsApp, Signal, iMessage, and (since late 2023) Facebook Messenger.

Meta’s CEO Mark Zuckerberg first promised to bring end-to-end encryption across Meta’s messaging products back in 2019, under the slogan “the future is private”.

Instagram tested encrypted direct messages in 2021. It rolled them out as an opt-in feature in 2023.

End-to-end encrypted direct messages never became the default, and the low adoption rate of opting in to use the feature is Meta’s justification for removing it. As a spokesperson told The Guardian:

Very few people were opting in to end-to-end encrypted messaging in DMs, so we’re removing this option from Instagram.

There is a circular logic to this: Meta has killed off a feature it buried so deep that most users never knew it existed, then cited low usage as the reason for its removal.

What does this mean for Instagram users?

In practical terms, every message you send on Instagram now travels in a form Meta can read.

Meta’s privacy policy lists the content of messages users send and receive among the data it collects. In principle, this enables the company to use this data to personalise features, train artificial intelligence (AI) models, and deliver targeted advertising.

While Meta has publicly committed not to train its AI models on private messages unless users actively share them with Meta AI, it has made no equivalent public commitment about advertising.

That leaves open the possibility that Meta could use unencrypted Instagram direct messages for ad targeting. And without encryption, Meta’s AI commitment is now backed by policy alone, not by the technology itself.

A clear reversal

This reads as a clear reversal of Meta’s privacy-first posture which Zuckerberg announced seven years ago.

Meta has been under sustained pressure from law enforcement, regulators and child protection organisations who argue end-to-end encryption creates spaces where platforms can’t detect child sexual exploitation and grooming. Australia’s eSafety Commissioner has been clear that the deployment of end-to-end encryption “does not absolve services of responsibility for hosting or facilitating online abuse or the sharing of illegal content”.

This argument deserves to be taken seriously. The harms are real and disproportionately fall on young people.

However, sexual extortion research shows perpetrators don’t tend to stay on the platform where they make first contact, with more than 50% of sexual extortion victims saying perpetrators asked them to switch platforms.

Meta still uses end-to-end encryption on its other platforms, such as WhatsApp and Facebook Messenger, and it needs to apply a consistent approach to child safety. Predators routinely ask victims to switch platforms, so the company’s safety approach needs to work for Instagram and their end-to-end encrypted services.

A false choice

Meta and privacy advocates often frame this as a choice between end-to-end encryption or child safety. But that’s a false choice. It’s not an “either-or” situation, even if they make it sound like one.

The technology already exists to detect harmful content while keeping messages encrypted in transit. It just has to run in the right place: on the user’s device, before the device encrypts and sends the message, or after it receives and decrypts it.

On-device approaches have a contested history, and any deployment must be genuinely privacy-preserving by design. But technology companies must weigh the objection against the harms that continue to occur. A safety by design approach is needed.

On-device safety measures have been demonstrated at scale with Apple’s on-device nudity detection for images sent or received via Messages, AirDrop and FaceTime. A 2025 study demonstrated high-accuracy grooming detection using Meta’s AI model designed specifically for on-device deployment on mobile phones.

Recently, both Apple and Google have started to take measures towards app store–based age verification in some jurisdictions.

The highest-profile real-world deployment of these is Apple enabling device-level privacy-preserving age verification in the UK.

Social media and private messaging companies, along with operating system vendors (Microsoft, Apple, and Google), all have a role to play in ensuring harmful content is detected, whether or not end-to-end encryption is used. Progress has been slow. But we, as a community, need to demand more from these companies.

Joel Scanlan, Adjunct Associate Professor, School of Law; Academic Co-Lead, CSAM Deterrence Centre, University of Tasmania

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• What happens when scientists trust AI more than colleagues?

• Study: Firms often use automation to control certain workers’ wages

What happens when scientists trust AI more than colleagues?

2026-05-12T11:43:00.000+05:00

Sungho Hong, The Institute for Basic Science and Victor J. Drew, The Institute for Basic Science

Image: Tima Miroshnichenko / Pexels

Artificial intelligence has crossed a threshold in the modern workplace. It is being used for everything from helping employees manage schedules to supporting financial forecasts. A similar shift is now unfolding inside research laboratories.

There is currently a boom in national initiatives to accelerate the integration of AI into science. These include the US Genesis Mission and South Korea’s AI Co-Scientist Challenge. But despite clear benefits, we believe these institutional drives are neglecting important issues that carry immense risks for scientific research.

Today, more than half of researchers use AI for work tasks including reviews of academic journals and designing experiments.

AlphaFold is an AI tool developed to predict the structures of proteins for scientific research. Working out protein structures was incredibly time-consuming before its release – taking years in some cases. The same tasks now take hours. AlphaFold was acknowledged by the 2024 Nobel Prize in Chemistry.

AI tools for use in medicine now assist with everything from the interpretation of results from X-rays and MRIs to supporting doctors’ decisions on the diagnosis and treatment of disease.

Our key concern is that hasty adoption of AI may gradually erode the scientific culture and human relationships that sustain rigorous research. It starts with the erosion of core thinking skills among researchers, as a result of an increased reliance on AI to perform that work. This can alienate researchers from the deeper reasoning behind their work.

Loss of independent thinking

Early-career scientists are particularly vulnerable, because they are still developing their scientific reasoning. Troubleshooting skills and the critical evaluation of ideas may be outsourced to AI systems.

AI’s fluent, confident and immediate responses can easily be mistaken for authoritative information. Once researchers begin to treat AI outputs as implicitly correct, the responsibility for judgment calls may gradually shift from them to their machines.

AI’s persuasive arguments, probably drawn from mainstream ideas in their training data, could replace more rigorous, time-consuming and creative research approaches. These are traditionally shaped through critical back-and-forth discussions between researchers.

This can evolve into over-dependence. As reasoning is delegated to AI, researchers become less confident at working unaided. Unfortunately, modern scientific labs are full of conditions that reinforce this dependence, such as intense competition, long hours and frequent isolation.

Limited mentorship and feedback from colleagues that is delayed, critical or politically influenced can enhance this issue. In contrast, AI provides an immediate, patient and nonjudgmental alternative.

Scientists interact with AI systems daily in order to check computer code, revise illustrations or charts, draft the language for grant applications, clarify scientific concepts, and at times, ask for personal advice.

As researchers begin to trust the AI assistant, it can begin to function less like a tool and more like a companion. This phenomenon bears the risk of emotional dependency, too. When ChatGPT-4 was retired, many users expressed a form of grief.

Replacing relationships

Another important concern is the potential for replacement of human relationships in the office or research lab. AI is always available, nonjudgmental, noncompeting – and indifferent to office politics, with no ego to defend. It remembers context, adapts to individual working styles, and offers reassurance without social cost.

Human scientific relationships are more complicated, involving nuance, criticism, time constraints, hierarchy – and sometimes, ulterior motives. For early-career researchers especially, these interactions can feel risky.

Critical feedback from humans can feel adversarial, while AI responses feel supportive. So, early-career scientists might have good reason to prefer testing ideas or seeking validation through AI, rather than their peers or superiors.

The scientific community cannot thrive without opposing ideas, deep scepticism against consensus, vigorous debate and rigorous mentoring. If AI begins to replace these, it threatens the foundations on which scientific progress has always been made.

The current debate on AI safety mostly focuses on errors in models’ responses, or on AI systems circumventing the restrictions imposed on the way they work, known as “jailbreaking”. Such rules have limited effects when it comes to the AI models’ societal and cultural impact.

Given the recent drives to get scientists to work more closely with AI assistants, we should educate our young scientists on the risks of AI dependence. We also need benchmarks to rigorously test AI models for their ability to establish boundaries with users, to prevent overdependence and other unhealthy interactions.

Finally, all of us – but especially institutional leaders – should understand the capabilities and permanence of AI companionship. They are here to stay, and we should learn to make our relationships with them as healthy as possible.

Sungho Hong, Neuroscientist, Center for Memory and Glioscience, The Institute for Basic Science and Victor J. Drew, Postdoctoral Research Associate, Center for Cognition and Sociality, The Institute for Basic Science

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Reviewed by Irfan Ahmad.

Read next:

• Study: Firms often use automation to control certain workers’ wages

• Research finds journalism classes lack consistent approach to AI use across institutions

Study: Firms often use automation to control certain workers’ wages

2026-05-11T17:43:00.000+05:00

Peter Dizikes | MIT News

MIT economists found US companies tend to target employees earning a “wage premium,” which increases inequality but not necessarily productivity.

Image Credit: Tara Winstead - Jonathan Borba - Pexels. Edited by DIW

When we hear about automation and artificial intelligence replacing jobs, it may seem like a tsunami of technology is going to wipe out workers broadly, in the name of greater efficiency. But a study co-authored by an MIT economist shows markedly different dynamics in the U.S. since 1980.

Rather than implement automation in pursuit of maximal productivity, firms have often used automation to replace employees who specifically receive a “wage premium,” earning higher salaries than other comparable workers. In practice, that means automation has frequently reduced the earnings of non-college-educated workers who had obtained better salaries than most employees with similar qualifications.

This finding has at least two big implications. For one thing, automation has affected the growth in U.S. income inequality even more than many observers realize. At the same time, automation has yielded a mediocre productivity boost, plausibly due to the focus of firms on controlling wages rather than finding more tech-driven ways to enhance efficiency and long-term growth.

“There has been an inefficient targeting of automation,” says MIT’s Daron Acemoglu, co-author of a published paper detailing the study’s results. “The higher the wage of the worker in a particular industry or occupation or task, the more attractive automation becomes to firms.” In theory, he notes, firms could automate efficiently. But they have not, by emphasizing it as a tool for shedding salaries, which helps their own internal short-term numbers without building an optimal path for growth.

The study estimates that automation is responsible for 52 percent of the growth in income inequality from 1980 to 2016, and that about 10 percentage points derive specifically from firms replacing workers who had been earning a wage premium. This inefficient targeting of certain employees has offset 60-90 percent of the productivity gains from automation during the time period.

“It’s one of the possible reasons productivity improvements have been relatively muted in the U.S., despite the fact that we’ve had an amazing number of new patents, and an amazing number of new technologies,” Acemoglu says. “Then you look at the productivity statistics, and they are fairly pitiful.”

The paper, “Automation and Rent Dissipation: Implications for Wages, Inequality, and Productivity,” appears in the May print issue of the Quarterly Journal of Economics. The authors are Acemoglu, who is an Institute Professor at MIT; and Pascual Restrepo, an associate professor of economics at Yale University.

Inequality implications

Dating back to the 2010s, Acemoglu and Restrepo have combined to conduct many studies about automation and its effects on employment, wages, productivity, and firm growth. In general, their findings have suggested that the effects of automation on the workforce after 1980 are more significant than many other scholars have believed.

To conduct the current study, the researchers used data from many sources, including U.S. Census Bureau statistics, data from the bureau’s American Community Survey, industry numbers, and more. Acemoglu and Restrepo analyzed 500 detailed demographic groups, sorted by five levels of education, as well as gender, age, and ethnic background. The study links this information to an analysis of changes in 49 U.S. industries, for a granular look at the way automation affected the workforce.

Ultimately, the analysis allowed the scholars to estimate not just the overall amount of jobs erased due to automation, but how much of that consisted of firms very specifically trying to remove the wage premium accruing to some of their workers.

Among other findings, the study shows that within groups of workers affected by automation, the biggest effects occur for workers in the 70th-95th percentile of the salary range, indicating that higher-earning employees bear much of the brunt of this process.

And as the analysis indicates, about one-fifth of the overall growth in income inequality is attributable to this sole factor.

“I think that is a big number,” says Acemoglu, who shared the 2024 Nobel Prize in economic sciences with his longtime collaborators Simon Johnson of MIT and James Robinson of the University of Chicago.

He adds: “Automation, of course, is an engine of economic growth and we’re going to use it, but it does create very large inequalities between capital and labor, and between different labor groups, and hence it may have been a much bigger contributor to the increase in inequality in the United States over the last several decades.”

The productivity puzzle

The study also illuminates a basic choice for firm managers, but one that gets overlooked. Imagine a type of automation — call-center technology, for instance — that might actually be inefficient for a business. Even so, firm managers have incentive to adopt it, reduce wages, and oversee a less productive business with increased net profits.

Writ large, some version of this seems to have been happening to the U.S. economy since 1980: Greater profitability is not the same as increased productivity.

“Those two things are different,” says Acemoglu. “You can reduce costs while reducing productivity.”

Indeed, the current study by Acemoglu and Restrepo calls to mind an observation by the late MIT economist Robert M. Solow, who in 1987 wrote, “You can see the computer age everywhere but in the productivity statistics.”

In that vein, Acemoglu observes, “If managers can reduce productivity by 1 percent but increase profits, many of them might be happy with that. It depends on their priorities and values. So the other important implication of our paper is that good automation at the margins is being bundled with not-so-good automation.”

To be clear, the study does not necessarily imply that less automation is always better. Certain types of automation can boost productivity and feed a virtuous cycle in which a firm makes more money and hires more workers.

But currently, Acemoglu believes, the complexities of automation are not yet recognized clearly enough. Perhaps seeing the broad historical pattern of U.S. automation, since 1980, will help people better grasp the tradeoffs involved — and not just economists, but firm managers, workers, and technologists.

“The important thing is whether it becomes incorporated into people’s thinking and where we land in terms of the overall holistic assessment of automation, in terms of inequality, productivity and labor market effects,” Acemoglu says. “So we hope this study moves the dial there.”

Or, as he concludes, “We could be missing out on potentially even better productivity gains by calibrating the type and extent of automation more carefully, and in a more productivity-enhancing way. It’s all a choice, 100 percent.”

Reprinted with permission of MIT News.

Reviewed by Irfan Ahmad.

Read next:

• Research finds journalism classes lack consistent approach to AI use across institutions

• New Report Reveals TikTok Leads Influencer Disclosure Compliance While YouTube Dominates Long-Term Brand Deals