Bayesian
Bayes's Theorem
prior:
likelihood:
posterior:
Optimal Bayes Decision Rule: minimize the probability of error.
if then True state of nature
;
if then True state of nature
.
Prove: For a particular
,
if we decide
;
if we decide
.
Bayes Decision Rule:Decide;otherwise decide
Therefore:.
The unconditional errorobtained by integration over all
.
Bayesian Decision Theory
c state of nature:
a possible actions:
the loss for taking action when the true state of nature is
:
Select the action for which the conditional risk is minimum.
Bayes Risk: .
- Example 1:
action: deciding
action: deciding
if, action
- Example 2:
Suppose
Conditional risk
Minimizing the riskMaximizing the posterior
.
So we have the discriminant function(max. discriminant corresponds to min. risk):
Set of discriminant functions:
Classifier assigns a feature vectorif:
Binary classification Multi‐class classfication
- One vs. One
class, design
classifiers, denote for result.
- One vs. Rest
design - ECOC (Error‐Correcting Output Codes)
The code consisting of the labels predicted by these classifiers is compared with each line, and the one with the smallest distance between codes is the result.
| f1 | f2 | f3 | |
|---|---|---|---|
| c1 | -1 | 1 | -1 |
| c2 | 1 | -1 | -1 |
| c3 | -1 | 1 | 1 |
