The Method of Lagrangian Multipliers and The KKT Conditions

This article simply introduced strategies for finding the stationary points of the objective function subject to one or more equality or inequality constraints.

Consider a standard form of continuous optimization problem, $$ \min\limits_{\bf x} f({\bf x}) \ {\rm s.t.};;g_k({\bf x})\leq0,;;k=1,2,\cdots,m \ h_l({\bf x})=0,;;l=1,2,\cdots,p \ $$ in which $$ {\bf x} \in \Re^{n} \ f,g_k,h_l:\Re^n \rightarrow \Re;;{\rm for};;k=1,2,\cdots,m;;{\rm and};;l=1,2,\cdots,p \ m,p\geq0 $$ And $f,g_k,h_l$ are all continuous differentable.

We divided the problem into two cases: $p = 0$ or $p \neq 0$. For the former we introduced The Method of Lagrange Multipliers as the solving strategy, and simply introduced KKT Conditions for the other one when it suits some Regularity Conditions.

Notice that it was easy to treat a maximization problem by negating the objective function, we only use the maximization problem as a general example.

矩阵微积分(一):标量对矩阵求导

矩阵微积分是多元函数微积分在矩阵空间中进行表达时采用的一种简化形式,常用于机器学习中,可通过将多个变量记为向量或矩阵的形式来简化运算。但目前矩阵微积分的相关概念定义尚未统一,运算方式亦繁乱无绪;若按许多人之习惯做法,显式依照定义进行计算,则又因为对向量/矩阵进行逐元素运算而破坏整体性,计算难度颇高,多须借助查表方能完成运算。本文通过引入矩阵微分算子的方式重新维护了整体性,大幅降低了运算难度,同时亦使得其定义更契合已有的标量微积分运算。

本文是矩阵求导术(上)的笔记,对全文缺漏处进行了部分完善,同时在证明与例题处亦重新进行了推导,十分感谢作者长躯鬼侠

在word中借助mathjax插入latex公式

最近有在word里书写大量数学公式的需求,但word自带的公式编辑器难用的一批而且我又没买office 365,因此想到用mathjax将latex公式转为mathml code,之后便可直接在word中显示。