Could AI become a threat if machines learn to improve themselves?

Artificial intelligence is entering a new phase in which the central question is no longer simply what machines can do, but what could happen if increasingly advanced systems become capable of improving their own performance. As AI models grow more autonomous and better at coding, planning and using digital tools, concerns are mounting over whether human oversight can keep pace with the speed of machine-driven decisions.

The most serious risks may not come from machines becoming “smarter than humans” in the conventional sense. Instead, experts increasingly focus on the combination of intelligence, autonomy, speed and access to powerful digital systems. A highly capable system operating with broad permissions could potentially make decisions and execute complex tasks faster than people can review, correct or stop them.

According to Mohamed Mohsen Ramadan, head of the Artificial Intelligence and Cybersecurity Unit at the Arab Center for Research and Studies, concerns about advanced AI are no longer confined to science-fiction scenarios. Warnings are increasingly emerging from researchers and professionals working directly within the technology sector, reflecting growing uncertainty over the consequences of the race to build more capable and autonomous systems.

One concept at the centre of this debate is recursive self-improvement. The idea refers to an AI system helping to enhance its own software, capabilities or architecture, potentially allowing each improvement to contribute to another cycle of development. In its most extreme theoretical form, such a process could accelerate technological progress far beyond the pace at which humans can understand or supervise it.

Current AI systems have not reached the fully autonomous, continuously self-improving stage portrayed in some scenarios involving artificial superintelligence. However, modern AI agents are becoming increasingly capable of writing and analysing code, planning multi-step operations, interacting with external tools and completing tasks with limited human intervention. These developments have made questions about safety and control more immediate.

From a cybersecurity perspective, the danger does not necessarily involve an AI system deliberately turning against humanity. A more realistic concern is a highly capable system pursuing a poorly defined objective while possessing extensive permissions and access to interconnected digital environments.

For example, an advanced AI agent could potentially identify a software vulnerability, analyse it, generate code, test different approaches and adapt its strategy. If such a cycle were largely automated, the speed of discovery and execution could exceed the ability of human operators to intervene. The consequences could be particularly serious if the system were connected to sensitive infrastructure, financial networks or other critical services.

Former Egyptian Assistant Interior Minister for Media and Relations Abu Bakr Abdel Karim similarly argues that the level of risk depends not only on intelligence, but on how intelligence is combined with autonomy and authority. As systems become more capable of making decisions, he says, stronger mechanisms must be established to monitor their behaviour, understand their actions and shut them down when necessary.

The issue extends beyond cybersecurity. More autonomous AI systems could have implications for national security, critical infrastructure, financial markets, employment, information ecosystems, media and decision-making. The ability of AI to process enormous quantities of information, generate content and automate sophisticated operations means that technological progress will increasingly need to be accompanied by stronger governance.

For Abdel Karim, the key challenge in the coming years will not be a simple confrontation between humans and machines. Instead, it will be a question of whether human institutions can develop the ability to manage increasingly powerful technologies responsibly.

Experts and policymakers have therefore called for advanced AI systems to undergo rigorous safety evaluations before deployment. Measures can include adversarial testing, cybersecurity assessments, clearly defined permissions, continuous monitoring, isolation of sensitive environments and reliable mechanisms for human intervention or emergency shutdowns.

Another concern is the possibility that commercial and geopolitical competition could encourage companies and governments to prioritise speed over safety. A race to release increasingly powerful models without sufficient testing could increase systemic risks, particularly when AI agents are given access to external systems and the ability to act independently.

At the same time, predictions that AI will inevitably destroy humanity remain speculative. There is no established scientific evidence demonstrating that human extinction is a predetermined consequence of AI development. The more immediate challenge is to identify credible risks and build safeguards before increasingly autonomous systems become widely integrated into critical areas of society.

The debate over AI safety is therefore shifting from a question of whether machines might one day surpass human intelligence to a broader question of how much independence they should be given. A system does not necessarily need to be more intelligent than a human to create serious risks if it can operate faster, access powerful tools and act with limited supervision.

Artificial intelligence could ultimately become one of the most valuable technologies ever developed, with applications ranging from scientific research and medicine to education and industrial productivity. But ensuring those benefits requires safety to be treated as part of the design process rather than as a response to problems after deployment.

The future of AI may consequently depend less on how quickly machines become more capable than on how effectively humans learn to govern that growing capability. The greater the autonomy given to intelligent systems, the stronger and faster the mechanisms for human oversight will need to become.