Generative Adversarial Networks, or GANs, are a type of artificial intelligence where two neural networks compete to improve each other’s performance. One network creates new data, such as images or sounds, while the other tries to detect if the data is real or fake. This competition helps both networks get better, resulting in highly realistic…
Category: Deep Learning
Subsymbolic Feedback Tuning
Subsymbolic feedback tuning is a process used in artificial intelligence and machine learning where systems adjust their internal parameters based on feedback, without relying on explicit symbols or rules. This approach is common in neural networks, where learning happens through changing connections between units rather than following step-by-step instructions. By tuning these connections in response…
Zero Resource Learning
Zero Resource Learning is a method in artificial intelligence where systems learn from raw data without needing labelled examples or pre-existing resources like dictionaries. Instead of relying on human-annotated data, these systems discover patterns and structure by themselves. This approach is especially useful for languages or domains where labelled data is scarce or unavailable.
Model Pruning
Model pruning is a technique used in machine learning where unnecessary or less important parts of a neural network are removed. This helps reduce the size and complexity of the model without significantly affecting its accuracy. By cutting out these parts, models can run faster and require less memory, making them easier to use on…
Gradient Clipping
Gradient clipping is a technique used in training machine learning models to prevent the gradients from becoming too large during backpropagation. Large gradients can cause unstable training and make the model’s learning process unreliable. By setting a maximum threshold, any gradients exceeding this value are scaled down, helping to keep the learning process steady and…
Contrastive Pretraining
Contrastive pretraining is a method in machine learning where a model learns to tell how similar or different two pieces of data are. It does this by being shown pairs of data and trying to pull similar pairs closer together in its understanding, while pushing dissimilar pairs further apart. This helps the model build useful…
Loss Landscape Analysis
Loss landscape analysis is the study of how the values of a machine learning model’s loss function change as its parameters are adjusted. It helps researchers and engineers understand how easy or difficult it is to train a model by visualising or measuring the shape of the loss surface. A smoother or flatter loss landscape…
Quantisation-Aware Training
Quantisation-Aware Training is a method used to prepare machine learning models for running efficiently on devices with limited computing power, such as smartphones or embedded systems. It teaches the model to handle the reduced precision of numbers, which happens when large models are made smaller by using fewer bits to represent data. This approach helps…
Batch Normalisation
Batch normalisation is a technique used in training deep neural networks to make learning faster and more stable. It works by adjusting and scaling the activations of each layer so they have a consistent mean and variance. This helps prevent problems where some parts of the network learn faster or slower than others, making the…
Attention Rollout
Attention Rollout is a technique used to visualise and interpret how information flows through the layers of an attention-based model, such as a transformer. It helps to track which parts of the input the model focuses on at each stage, giving insight into the decision-making process. This method combines attention maps from different layers to…