ON THE CAPABILITIES OF MODERN NEURAL NETWORKS AS A MEANS OF ARTIFICIAL INTELLIGENCE

Abstract

This article provides a comprehensive analysis of the potential and limitations of artificial intelligence (AI), with a particular focus on its capacity to generate novel scientific and technical solutions.
The central thesis of the article rests on the analysis of AI systems as approximators – mathematical constructs that map sets of input variables to corresponding outputs based on training data. Artificial neural networks (ANNs), in particular, are identified as the most prominent and flexible form of such approximators. Through a detailed mathematical framework, the study illustrates how these systems are trained on structured datasets to form a functional mapping y=F(x), and how their predictive capabilities are divided into two categories: interpolation and extrapolation.
A core example used throughout the work involves training a neural network on a simple quadratic dataset and analyzing its ability to generalize (interpolate) within the known data range, versus its tendency to fail when extrapolating beyond this domain.
The discussion then transitions to modern AI applications, particularly transformer-based large language models (LLMs) like ChatGPT, which, although structurally more complex, remain conceptually similar to traditional feedforward neural networks. These models can interpolate within an extensive informational domain with impressive fluency and creativity, generating coherent outputs by recombining previously seen elements. An illustrative case compares this process to generating a fictional story from known character archetypes and narrative structures, potentially yielding combinations that are original but still rooted within known patterns.
This principle is further extended to scientific and engineering applications, where AI may recombine known physical, chemical, or mechanical principles to suggest viable new inventions. For example, by combining known types of forces, fuels, and energy conversion methods, a neural network could propose novel engine designs – some of which may be genuinely innovative. These processes mirror traditional methodologies in morphological analysis and TRIZ (Theory of Inventive Problem Solving), but are significantly enhanced by the processing speed and scale of modern AI.
However, the article also emphasizes the inherent limitations of current AI systems. While they are highly effective within the bounds of interpolation, their capacity to generate valuable outputs in extrapolative scenarios – those requiring a leap beyond known data – is fundamentally flawed. This limitation is not only theoretical but practically observable in the phenomenon of hallucination in large language models.
In conclusion, the article argues that interpolation-based innovation – producing new but logically grounded combinations of existing knowledge – is well within the reach of today’s AI systems and may yield real scientific or technical breakthroughs when human researchers have not yet explored such possibilities. However, true extrapolation, or the generation of fundamentally new theories and concepts beyond the known knowledge domain, remains largely inaccessible to neural-network-based architectures. To achieve this, future AI systems must adopt hybrid approaches, combining neural models with deterministic logic, heuristic reasoning, or other cognitive frameworks capable of venturing beyond learned experience. These hybrid architectures hold the potential to not only mimic but meaningfully extend the boundaries of human creativity and scientific discovery.