The Expectation-Maximisation (EM) Algorithm is a method used to find the most likely parameters for statistical models when some data is missing or hidden. It works by alternating between estimating missing data based on current guesses and then updating those guesses to better fit the observed data. This process repeats until the solution stabilises and…
Category: Model Training & Tuning
Knowledge Consolidation
Knowledge consolidation is the process by which information learned or acquired is stabilised and stored in long-term memory. This process helps new knowledge become more permanent, making it easier to recall and use later. It often involves revisiting, reviewing, or practising information over time to strengthen understanding and retention.
Neural Network Pruning
Neural network pruning is a technique used to reduce the size and complexity of artificial neural networks by removing unnecessary or less important connections, neurons, or layers. This process helps make models smaller and faster without significantly affecting their accuracy. Pruning often follows the training of a large model, where the least useful parts are…
Physics-Informed Neural Networks
Physics-Informed Neural Networks, or PINNs, are a type of artificial intelligence model that learns to solve problems by combining data with the underlying physical laws, such as equations from physics. Unlike traditional neural networks that rely only on data, PINNs also use mathematical rules that describe how things work in nature. This approach helps the…
Domain Randomisation
Domain randomisation is a technique used in artificial intelligence, especially in robotics and computer vision, to make models more robust. It involves exposing a model to many different simulated environments where aspects like lighting, textures, and object positions are changed randomly. By training on these varied scenarios, the model learns to perform well even when…
Symbolic Regression
Symbolic regression is a type of machine learning that tries to find mathematical equations that best fit a set of data. Instead of just adjusting numbers in a fixed equation, symbolic regression searches for both the structure and the parameters of equations. This means it can suggest entirely new formulas that describe how inputs relate…
Neural Turing Machines
Neural Turing Machines are a type of artificial intelligence model that combines a neural network with an external memory bank. This setup allows the model to read from and write to its memory, similar to how a computer program works. It is designed to help machines learn tasks that require storing and recalling information over…
Policy Gradient Methods
Policy Gradient Methods are a type of approach in reinforcement learning where an agent learns to make decisions by directly improving its decision-making policy. Instead of trying to estimate the value of each action, these methods adjust the policy itself to maximise rewards over time. The agent uses feedback from its environment to gradually tweak…
Structured Prediction
Structured prediction is a type of machine learning where the goal is to predict complex outputs that have internal structure, such as sequences, trees, or grids. Unlike simple classification or regression, where each prediction is a single value or label, structured prediction models outputs that are made up of multiple related elements. This approach is…
Neural Process Models
Neural process models are computational systems that use neural networks to learn functions or processes from data. Unlike traditional neural networks that focus on mapping inputs to outputs, neural process models aim to understand entire functions, allowing them to adapt quickly to new tasks with limited data. These models are especially useful for problems where…