Category: Model Optimisation Techniques

Ghost Parameter Retention

Ghost Parameter Retention refers to the practice of keeping certain parameters or settings in a system or software, even though they are no longer in active use. These parameters may have been used by previous versions or features, but are retained to maintain compatibility or prevent errors. This approach helps ensure that updates or changes…

Memory-Constrained Inference

Memory-constrained inference refers to running artificial intelligence or machine learning models on devices with limited memory, such as smartphones, sensors or embedded systems. These devices cannot store or process large amounts of data at once, so models must be designed or adjusted to fit within their memory limitations. Techniques like model compression, quantisation and streaming…

Cognitive Load Balancing

Cognitive load balancing is the process of managing and distributing mental effort to prevent overload and improve understanding. It involves organising information or tasks so that people can process them more easily and efficiently. Reducing cognitive load helps learners and workers focus on what matters most, making it easier to remember and use information.

Feature Engineering

Feature engineering is the process of transforming raw data into meaningful inputs that improve the performance of machine learning models. It involves selecting, modifying, or creating new variables, known as features, that help algorithms understand patterns in the data. Good feature engineering can make a significant difference in how well a model predicts outcomes or…

Gradient Clipping

Gradient clipping is a technique used in training machine learning models to prevent the gradients from becoming too large during backpropagation. Large gradients can cause unstable training and make the model’s learning process unreliable. By setting a maximum threshold, any gradients exceeding this value are scaled down, helping to keep the learning process steady and…

Quantisation-Aware Training

Quantisation-Aware Training is a method used to prepare machine learning models for running efficiently on devices with limited computing power, such as smartphones or embedded systems. It teaches the model to handle the reduced precision of numbers, which happens when large models are made smaller by using fewer bits to represent data. This approach helps…

Activation Functions

Activation functions are mathematical formulas used in neural networks to decide whether a neuron should be activated or not. They help the network learn complex patterns by introducing non-linearity, allowing it to solve more complicated problems than a simple linear system could handle. Without activation functions, neural networks would not be able to model tasks…