A detailed intro to neural networks
What are neural networks and what is all this LLM buzz?
If you’re reading this post, I hope you stay till the end. If not, I would love for you to complete a very brief survey so I can make this newsletter better. I am grateful for your support so far.
We briefly touched on neural networks and deep learning in the last post. Today, I wanted to dive a bit deeper into neural networks.
Once again, a neural network is a type of machine learning model inspired by the structure and functioning of the human brain's neural networks. Neural networks are used for various tasks such as pattern recognition, classification, regression, and decision-making based on data. Neural networks are also fundamental to deep learning.
The basic building block of a neural network is a neuron, also called a node or unit. Neurons are organized into layers, and there are typically three types of layers:
Input Layer: This is the first layer, which receives the initial data or features as input.
Hidden Layers: These are the intermediate layers between the input and output layers. The hidden layers are responsible for learning patterns and features from the input data.
Output Layer: This is the final layer, which produces the model's output or prediction.
Each neuron in a layer is connected to every neuron in the subsequent layer through connections called weights. These weights determine the strength of the connections between neurons and play a crucial role in learning.
History of neural networks
Here's a brief summary of the key events in the history of neural networks:
1940s - 1950s: Researchers like Warren McCulloch and Walter Pitts introduced the concept of artificial neurons as simple computational models of biological neurons. They proposed that networks of these artificial neurons could perform computations and emulate some aspects of human brain function.
1957 - Perceptron: The perceptron was capable of learning and making binary classifications. Rosenblatt's work was influential and sparked early interest in neural network research.
1960s - 1970s: While the perceptron was promising, it was quickly proven that it could not be trained to recognize many types of patterns. The "perceptron controversy" led to a decrease in funding and interest in neural network research.
1980s - Backpropagation: In the 1980s, interest in neural networks was revived with the development of the backpropagation algorithm (remember from the last post). This algorithm, independently discovered by multiple researchers, allowed for efficient training of multilayer neural networks.
1980s - 1990s: The backpropagation algorithm led to the development of various neural network architectures, such as feedforward neural networks and recurrent neural networks. However, despite the progress, limitations in computational power and training techniques still hindered the widespread adoption of neural networks for complex tasks.
1990s - 2000s: Interest in neural networks continued to grow, but it was tempered by the rise of alternative machine learning methods, such as support vector machines and decision trees.
2000s - Deep Learning Resurgence: The breakthrough in the mid-2000s came with the introduction of techniques to train deep neural networks effectively. Researchers discovered that by using techniques like unsupervised pre-training, it was possible to train deep networks.
2010s - Present: The 2010s witnessed an explosion in the adoption and success of deep learning models. Convolutional Neural Networks (CNNs) revolutionized image recognition, Recurrent Neural Networks (RNNs) improved sequence modeling, and the development of attention mechanisms significantly advanced natural language processing tasks.
The history of neural networks demonstrates a cycle of enthusiasm, periods of reduced interest, and eventual resurgence driven by breakthroughs in algorithms, hardware capabilities, and data availability. Today, neural networks are a fundamental component of many cutting-edge technologies and continue to shape the landscape of artificial intelligence and machine learning.
Applications of neural networks
Without diving into LLM-specific applications and generative AI, here are some of the baseline applications of neural networks that have dominated the last two decades:
Healthcare:
We’ve already discussed some of these applications before from a different perspective but here is how they use neural networks:
Imaging and Scans: Utilizing deep neural networks (DNNs) aids in deciphering medical scans of pathologies, electrocardiograms, and endoscopy results. Notably, the realm of radiology, specifically the analysis of X-ray images, receives significant focus. Google harnessed algorithms to decode chest scans, enabling the identification of 14 distinct diagnoses, encompassing conditions such as pneumonia, cardiac hypertrophy, and lung collapse. DNNs additionally possess the capability to diagnose specific forms of cancer, fractures, hemorrhages, retinopathy, skin lesions, and numerous other medical conditions.
Drug Discovery: Neural networks predict the properties and potential of chemical compounds, aiding in drug discovery. A key strategy behind numerous effective drugs involves focusing on receptor proteins situated in the exteriors of cells. Within the bloodstream, drug molecules can disrupt these proteins, impeding the transmission of signals they would normally convey to the internal cellular apparatus. Scientists can evaluate the essential interaction between a desired protein target and a potential small-molecule drug through laboratory experiments. For molecules, graph neural networks (GNNs) have been shown to be extremely effective. These systems represent data in terms of nodes (which correspond to atoms) and the edges that connect them (which correspond to the bonds between atoms in a molecule). Once the three-dimensional structure of a protein is understood—computational modeling can determine whether prospective drugs can align with the protein's contours and atomic interactions within its intricate folds, which are responsible for recognizing signaling molecules.
This strategy has gotten a huge boost from recent progress in AI prediction of protein structure from its understood mechanism. In 2020, Alphabet-owned DeepMind was a winner in the Critical Assessment of Protein Structure Prediction project. In 2022, the project used neural networks and deep learning to predict structures for all known proteins, which it made public.
Autonomous Vehicles:
Neural networks are the driving force behind the functioning of autonomous vehicles. The creators of self-driving cars leverage extensive datasets from image recognition systems, employing machine learning and neural networks to construct systems capable of independent driving. Images captured by cameras mounted on self-driving vehicles, from which the neural network learns to discern various elements of a given driving environment, such as traffic lights, pedestrians, trees, street signs, curbs, and other components.
A big example of self-driving cars is Waymo, Google's self-driving car initiative. Waymo employs an amalgamation of sensors, including cameras, lidar (akin to RADAR), and additional systems. By amalgamating the data generated by these sources, the technology identifies the vehicle's surroundings and forecasts the potential actions of these objects, all transpiring within fractions of a second. The maturation of these systems plays a pivotal role. As the system accumulates more driving experience, it incorporates a greater volume of data into its deep learning algorithms, thereby enabling the system to execute more sophisticated driving decisions.
Finance:
Neural networks can be used to analyze price data and uncover opportunities. With a neural network, a business can make a trade decision using financial data, which is not necessarily the case when using traditional technical analysis methods. This can be highly effective for a trading agency that will get much more out of a non-linear neural network that can identify patterns that are simply not catchable by traditional analyses and humans.
There are many other areas of applications of Neural Networks all around in areas such as Natual Language Processing, Robotics, Art & Creative, Film, VFX, and Gaming. In my next post in what is now an AI series, we will go deeper into Deep learning and look at how LLMs are powered.
Great post!