"Layers!".
"Onion has layers. Neural Net has layers."
That what Shrek said on his quest to get back his swamp. (or did he?!)
In the previous post, we talked about how Large Language Model utilises Neural Network as part of its learning processes. So, "What is a Neural Network?". Let me have a try...A Neural Network is:
- A machine learning program that performs analysis by passing inputs through a series of interconnected logical processing units (known as nodes) and generating outputs. The final outcomes are the sum of all the outputs from the first layer (input layer), through one or more intermediate layers, and finally to the last layer (the output layer).
- Each layer consists of one or more interconnected nodes. They have least 3 layers:
- Input layer
- Hidden layer or layers
- Output layer
- Each logic-processing unit or node:
- takes one or more inputs
- processes inputs through some function, then assigns weight to it
- if weight > threshold (or bias), a node fires (pass data) to the next layer
- This passing data to the next layer == "Feedforward" usually flows in one direction
So, what happens inside those layers?
Neural Node - an intrinsic element of a neural network
A logical processing unit of a neural network known as a node. A node processes inputs and produces outputs. It passes signals to the next node only if the outcomes are greater than some threshold (bias) value.
Each input to a node, sometimes called a "variable" or "feature", carries importance or value that affects the outcomes. This is referred to as "weight". This can be represented with equation $ w_i x_i $, where 'w' is weight, and 'x' is the value of the input.
A node can accept one or more variables (or features). Its outputs can be calculated with the mathematic equation:
$$ \sum w_ix_i + bias = (w_1x_1 + w_2x_2 + ...) + bias $$
$$ output = f(x) = 1, if \sum w_ix_i + bias >= 0; = 0, if \sum w_ix_i + bias < 0 $$
Node: Should Shrek fight the dragon?
- Outputs:
- 1: Yes, Shrek should fight the dragon;
- 0: No, Shrek should not fight the dragon
- Variables (or features), let's say this node takes 3 inputs:
- $x_1$: Is the dragon scary?
- if "yes", $x_1$ = 0, i.e. don't fight
- if "no", $x_1$ = 1, let's fight the dragon
- This input has weight of 3, ($w_1$ = 3)
- $x_2$: Has Shrek ever won in a fight with dragon?
- if "yes", $x_2$ = 1, let's fight
- if "no", $x_2$ = 0, don't fight
- This input has weight of 5, ($w_2$ = 5)
- $x_3$: Has Shrek had a talking donkey as his sidekick?
- if "yes", $x_3$ = 1, let's fight
- if "no", $x_3$ = 0, let's not fight
- This input is very important as the dragon has "the thing" for a talking donkey, so we assign it with weight of 7 in value, ($w_3$ = 7)
As you can see, we can have multiple combinations of the above variables/features. One scenario would be:
- $x_1$: Yes the dragon is scary. ($x_1 = 0, w_1 = 3$)
- $x_2$: No, Shrek has never fought a dragon. ($x_2 = 0, w_2 = 5$)
- $x_3$: Yes, Shrek has a talking donkey as his sidekick. ($x_3 = 1, w_3 = 7$)
- To reduce or remove non essential combinations, let say that any output value below 5 shall be ignored. So this node has bias (or threshold) = -5
$$ \sum w_ix_i + bias = (w_1x_1 + w_2x_2 + ...) + bias $$
$$ nodeValue = \sum ((0*3) + (0*5) + (1*7)) - 5 = 2$$
$$nodeOutput = f(x) = 1$$
In this scenario, our node output 1 - "Yes" - let's fight the dragon.
Already, you probably can sense some complexity here - giving different inputs and bias values, this node yields multiple different results ("yes" or "no").
But, can Shrek get his swamp back?
Neural Network
Node: Should Shrek rescue the princess?
Some of the features/parameters into this node might be, for example:
- $x_1$: Is the princess pretty? Weight $w_1 = 3$ (Shrek doesn't really care for the look)
- $x_2$: Is the princess very tall? Weight $w_2 = 7$, this is very important as we know that Lord Faquard would not like to marry anyone taller than him.
- $x_3$: Is the princess a nice person?
- (so on)
Other nodes might be:
- Node: Is Princess pretty?
- Node: Is Princess a nice person?
- (etc)
But, how does ML/Neural Network differ from the conventional computing algorithm?
Upon first glance, one might think:
"hang on. I can just write an algorithm to help Shrek make this decision - whether to flight or fight. Why use AI?!"
While the conventional algorithm may work, AI is more suitable for tasks where the solutions are not well-defined. For examples, how does one determine "scary", or "pretty", or "nice person"? This is when we subject our "model" to the "learning process" where it goes through many iterations of training cycles.
The future of Man and Their Machines
From the above factitious model, with only a few sample nodes, it already demonstrates the complexity of machine learning processes. As the number of nodes increased, with each more parameters, and bias values, changing values from one node may cause behavioural changes for the whole outcomes.
I have spent a short time training Sidekick bots and am already experiencing some of the well-known symptoms:
- Stubbornness - where bot refuses to accept the new teaching/training
- Hallucination - where bot (seems) to come up with its own solutions that are not related to the teaching
- Prejudice - where bot's learning progress has been influenced/affected by the certain type of sample data being fed to the learning process
- Correctness - how do we (human trainers) know if the answers were correct
- Diagnosis Difficulty - this was when I often wondered "Where did it go wrong?"
- many more
Hang on a sec! How come I just sounded like a parent ranting about his stubborn kid?!
That's because, they are! These learning bots are more like children, more and more.
The big question is ...
How do you trust your children?
No comments:
Post a Comment