Bounding Scalars: The Gate Mixing Proportions

We will update the mixing proportions by holding the experts fixed as well as the gate means and covariances. By taking derivatives of the Q function over only $\alpha_m$ we get Equation 7.19.

 

$$\begin{array}{lll} \frac{\partial Q(\Theta^t,\Theta^{(t-1)})} {\p... ... N}} ({\bf x}_i;\mu_x^m,\Sigma_{xx}^m)}{\sum_{i=1}^N {\hat h}_{im}} \end{array}$$ (7.19)

The update for $\alpha_m$ is a function of the $\mu_x^m$ and the $\Sigma_{xx}^m$. It is possible to keep this equality and use it later as we derive the update rules for the means and the covariances. However, it is quite cumbersome to manipulate analytically if it is maintained as shown above. Thus, we lock the values of $\mu_x$ and $\Sigma_{xx}$ at their previous estimates (i.e. at $\Theta^{(t-1)}$) and numerically update the $\alpha$ mixing proportions.

 

$$\begin{array}{lll} \alpha_m & := & \frac{ \sum_{i=1}^N r_i {\hat ... ...left . \right \vert _{\Theta^{(t-1)}} }{\sum_{i=1}^N {\hat h}_{im}} \end{array}$$ (7.20)

Since we are maximizing a bound, the above update rule increases conditional likelihood monotonically. This was also verified with a numerical implementation.