1、Pattern Recognition &artificial IntelligenceLecture 10: 聚 类 算法（六）1 EXPECTATION MAXIMIZATION (EM)Jensens inequalitySingle Gaussian Model （ SGM）Maximum Likelihood (ML) estimationEM estimationGaussian Mixture Model (GMM)Difference between K-means and EMApplicationsModel-based clustering (1)23Jensens in
2、equalityMathematical Foundation (1) Convex FunctionsDefinition: Let f be a real valued function defined on an interval I =a, b, f is said to be convex on I, if4Jensens inequalityMathematical Foundation (1) Convex FunctionsTheorem: If f(x) is twice differentiable on a, b and f(x)0 on a, b, then f(x)
3、is convex on a, b.f(x) increases gradually, which means f(x) 05Jensens inequalityMathematical Foundation (1) Concave FunctionsDefinition: Let f be a real valued function defined on an interval I =a, b, f is said to be concave on I, if6Jensens inequalityMathematical Foundation (1) Concave FunctionsTh
4、eorem: If f(x) is twice differentiable on a, b and f(x)0 on a, b, then f(x) is concave on a, b.f(x) increases gradually, which means f(x)07Jensens inequalityMathematical Foundation (2) Expectation of a function Theorem: If X is a random variable, and Y=g(X), then: Where:is the probability density of
5、 XdiscreteContinuous8Jensens inequalityJensens inequality: For convex functionFor concave function Generalized convex functionGeneralized concave functionEquality holds if and only if or f is linear. 9Single Gaussian Model （ SGM）Sampling10Single Gaussian Model （ SGM）SamplingGiven x, it is a function of and 2We want to maximize it.(LIKELIHOOD FUNCTION)Independent
