设 X = ( x i j ) m × n X = (x_{ij})_{m \times n} X=(xij)m×n，函数 f ( X ) = f ( x 11 , x 12 , … , x 1 n , x 21 , … , x m n ) f(X) = f(x_{11}, x_{12}, \ldots, x_{1n}, x_{21}, \ldots, x_{mn}) f(X)=f(x11,x12,…,x1n,x21,…,xmn) 是一个 m × n m \times n m×n 元的多元函数，且偏导数
∂ f ∂ x i j ( i = 1 , 2 , … , m , j = 1 , 2 , … , n ) \frac{\partial f}{\partial x_{ij}} \quad (i=1,2,\ldots,m,\ j=1,2,\ldots,n) ∂xij∂f(i=1,2,…,m, j=1,2,…,n)
都存在。定义 f ( X ) f(X) f(X) 对矩阵 X X X 的导数为：
d f ( X ) d X = ( ∂ f ∂ x i j ) m × n = [ ∂ f ∂ x 11 ⋯ ∂ f ∂ x 1 n ⋮ ⋱ ⋮ ∂ f ∂ x m 1 ⋯ ∂ f ∂ x m n ] \frac{df(X)}{dX} = \left( \frac{\partial f}{\partial x_{ij}} \right)_{m \times n} =\begin{bmatrix} \frac{\partial f}{\partial x_{11}} & \cdots & \frac{\partial f}{\partial x_{1n}} \\ \vdots & \ddots & \vdots \\ \frac{\partial f}{\partial x_{m1}} & \cdots & \frac{\partial f}{\partial x_{mn}} \end{bmatrix} dXdf(X)=(∂xij∂f)m×n= ∂x11∂f⋮∂xm1∂f⋯⋱⋯∂x1n∂f⋮∂xmn∂f
(1) 设 x = ( ξ 1 , ξ 2 , ⋯ , ξ n ) ⊤ \mathbf{x} = (\xi_1, \xi_2, \cdots, \xi_n)^\top x=(ξ1,ξ2,⋯,ξn)⊤， n n n 元函数 f ( x ) f(\mathbf{x}) f(x)，求 d f d x ⊤ \frac{df}{d\mathbf{x}^\top} dx⊤df、 d f d x \frac{df}{d\mathbf{x}} dxdf 和 d 2 f d x 2 \frac{d^2f}{d\mathbf{x}^2} dx2d2f。
d f d x ⊤ = ( ∂ f ∂ ξ 1 , ∂ f ∂ ξ 2 , ⋯ , ∂ f ∂ ξ n ) \frac{df}{d\mathbf{x}^\top} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1}, \frac{\partial f}{\partial \xi_2},\cdots, \frac{\partial f}{\partial \xi_n} \end{pmatrix} dx⊤df=(∂ξ1∂f,∂ξ2∂f,⋯,∂ξn∂f)
∇ f ( x ) = d f d x = ( ∂ f ∂ ξ 1 ∂ f ∂ ξ 2 ⋮ ∂ f ∂ ξ n ) ，这就是梯度。 \nabla f(\mathbf{x}) = \frac{df}{d\mathbf{x}} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} \text{，这就是梯度。} ∇f(x)=dxdf= ∂ξ1∂f∂ξ2∂f⋮∂ξn∂f ，这就是梯度。
H ( x ) = ∇ 2 f ( x ) = ∂ 2 f ∂ x ∂ x ⊤ = [ ∂ 2 f ∂ ξ 1 2 ∂ 2 f ∂ ξ 1 ∂ ξ 2 ⋯ ∂ 2 f ∂ ξ 1 ∂ ξ n ∂ 2 f ∂ ξ 2 ∂ ξ 1 ∂ 2 f ∂ ξ 2 2 ⋯ ∂ 2 f ∂ ξ 2 ∂ ξ n ⋮ ⋮ ⋱ ⋮ ∂ 2 f ∂ ξ n ∂ ξ 1 ∂ 2 f ∂ ξ n ∂ ξ 2 ⋯ ∂ 2 f ∂ ξ n 2 ] , 这就是Hessian 矩阵，它是对称的。 H(\mathbf{x}) = \nabla^2 f(\mathbf{x}) = \frac{\partial^2 f}{\partial \mathbf{x} \partial \mathbf{x}^\top} = \begin{bmatrix} \frac{\partial^2 f}{\partial \xi_1^2} & \frac{\partial^2 f}{\partial \xi_1 \partial \xi_2} & \cdots & \frac{\partial^2 f}{\partial \xi_1 \partial \xi_n} \\ \frac{\partial^2 f}{\partial \xi_2 \partial \xi_1} & \frac{\partial^2 f}{\partial \xi_2^2} & \cdots & \frac{\partial^2 f}{\partial \xi_2 \partial \xi_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial \xi_n \partial \xi_1} & \frac{\partial^2 f}{\partial \xi_n \partial \xi_2} & \cdots & \frac{\partial^2 f}{\partial \xi_n^2} \end{bmatrix}, \text{这就是Hessian 矩阵，它是对称的。} H(x)=∇2f(x)=∂x∂x⊤∂2f= ∂ξ12∂2f∂ξ2∂ξ1∂2f⋮∂ξn∂ξ1∂2f∂ξ1∂ξ2∂2f∂ξ22∂2f⋮∂ξn∂ξ2∂2f⋯⋯⋱⋯∂ξ1∂ξn∂2f∂ξ2∂ξn∂2f⋮∂ξn2∂2f ,这就是Hessian 矩阵，它是对称的。
(2) 设 a = ( a 1 , a 2 , ⋯ , a n ) ⊤ \mathbf{a} = \begin{pmatrix} a_1, a_2, \cdots, a_n \end{pmatrix}^\top a=(a1,a2,⋯,an)⊤ 为向量变量，且 f ( x ) = f ( x , a ) f(\mathbf{x}) = f(\mathbf{x}, \mathbf{a}) f(x)=f(x,a)，求 ∂ f ∂ x \frac{\partial f}{\partial \mathbf{x}} ∂x∂f。
解：由于 f ( x ) = ∑ i = 1 n a i ξ j f(\mathbf{x}) = \sum_{i=1}^{n} a_i \xi_j f(x)=∑i=1naiξj， ∂ f ∂ ξ j = a j \frac{\partial f}{\partial \xi_j} = a_j ∂ξj∂f=aj， ( j = 1 , 2 , ⋯ , n ) (j = 1,2,\cdots, n) (j=1,2,⋯,n)，所以
∂ f ∂ x = ( ∂ f ∂ ξ 1 ∂ f ∂ ξ 2 ⋮ ∂ f ∂ ξ n ) = ( a 1 a 2 ⋮ a n ) = a \frac{\partial f}{\partial \mathbf{x}} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} = \begin{pmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{pmatrix} = \mathbf{a} ∂x∂f= ∂ξ1∂f∂ξ2∂f⋮∂ξn∂f = a1a2⋮an =a
(3) 设 A = ( a i j ) m × n A = \left(a_{ij}\right)_{m \times n} A=(aij)m×n 为常矩阵， X = ( x i j ) n × m X = \left( x_{ij} \right)_{n \times m} X=(xij)n×m 为矩阵变量，且 f ( X ) = tr ( A X ) f(\mathbf{X}) = \operatorname{tr}(\mathbf{A X}) f(X)=tr(AX)，求 ∂ f ∂ X \frac{\partial f}{\partial X} ∂X∂f。
分析：
(
c
11
⋯
c
1
m
⋮
⋱
⋮
c
m
1
⋯
c
m
m
)
=
(
a
11
⋯
a
1
n
⋮
⋱
⋮
a
m
1
⋯
a
m
n
)
(
x
11
⋯
x
1
n
⋮
⋱
⋮
x
n
1
⋯
x
n
m
)
\begin{pmatrix} c_{11} & \cdots & c_{1m} \\ \vdots & \ddots & \vdots \\ c_{m1} & \cdots & c_{mm} \end{pmatrix}=\begin{pmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{m1} & \cdots & a_{mn} \end{pmatrix}\begin{pmatrix} x_{11} & \cdots & x_{1n} \\ \vdots & \ddots & \vdots \\ x_{n1} & \cdots & x_{nm} \end{pmatrix}
c11⋮cm1⋯⋱⋯c1m⋮cmm
=
a11⋮am1⋯⋱⋯a1n⋮amn
x11⋮xn1⋯⋱⋯x1n⋮xnm
展开后得：
c
11
=
a
11
x
11
+
a
12
x
21
+
⋯
+
a
1
n
x
n
1
,
c
22
=
a
21
x
12
+
a
22
x
22
+
⋯
+
a
2
n
x
n
2
,
⋮
c
m
n
=
a
m
1
x
1
m
+
a
m
2
x
2
m
+
⋯
+
a
m
n
x
n
m
\begin{equation} \begin{aligned} c_{11} &= a_{11}x_{11} + a_{12}x_{21} + \cdots + a_{1n}x_{n1}, \\ c_{22} &= a_{21}x_{12} + a_{22}x_{22} + \cdots + a_{2n}x_{n2}, \\ &\qquad \mathllap{\vdots} \\ c_{mn} &= a_{m1}x_{1m} + a_{m2}x_{2m} + \cdots + a_{mn}x_{nm} \end{aligned} \end{equation}
c11c22cmn=a11x11+a12x21+⋯+a1nxn1,=a21x12+a22x22+⋯+a2nxn2,⋮=am1x1m+am2x2m+⋯+amnxnm
规律：每个 x x x 只会被用到一次， x x x 的下标和 a a a 的下标是相反的。
解：由于 A X = ( ∑ i = 1 n a i k x k i ) m × m AX = \left(\sum_{i=1}^n a_{ik}x_{ki}\right)_{m \times m} AX=(∑i=1naikxki)m×m，
所以： f ( X ) = tr ( A X ) = ∑ s = 1 m ∑ k = 1 n a s k x k s f(\mathbf{X}) = \operatorname{tr}(\mathbf{AX}) = \sum_{s=1}^{m} \sum_{k=1}^n a_{sk} x_{ks} f(X)=tr(AX)=∑s=1m∑k=1naskxks。
而：
(
∂
f
∂
x
i
j
)
n
×
m
=
(
a
j
i
)
n
×
m
(
i
=
1
,
2
,
⋯
,
n
,
j
=
1
,
2
,
⋯
,
m
)
\left( \frac{\partial f}{\partial x_{ij}} \right)_{n \times m} = (a_{ji})_{n \times m} \quad (i=1,2,\cdots,n, j = 1,2,\cdots,m)
(∂xij∂f)n×m=(aji)n×m(i=1,2,⋯,n,j=1,2,⋯,m)
故：
∂
f
∂
X
=
(
∂
f
∂
x
i
j
)
=
(
a
j
i
)
n
×
m
=
A
⊤
\frac{\partial f}{\partial X} = \left( \frac{\partial f}{\partial x_{ij}} \right) = (a_{ji})_{n \times m} = A^\top
∂X∂f=(∂xij∂f)=(aji)n×m=A⊤
(4) 设 x = ( ξ 1 , ξ 2 , ⋯ , ξ n ) ⊤ \mathbf{x} = \left( \xi_1, \xi_2, \cdots, \xi_n \right)^\top x=(ξ1,ξ2,⋯,ξn)⊤，矩阵 A = ( a i j ) n × n A = \left(a_{ij}\right)_{n \times n} A=(aij)n×n， n n n 元函数 f ( x ) = x ⊤ A x f(\mathbf{x}) = \mathbf{x}^\top A \mathbf{x} f(x)=x⊤Ax，求导数 d f d x \dfrac{d f}{d \mathbf{x}} dxdf。
解：因
f
(
x
)
=
x
⊤
A
x
=
(
ξ
1
,
ξ
2
,
⋯
,
ξ
n
)
(
a
11
a
12
⋯
a
1
n
a
21
a
22
⋯
a
2
n
⋮
⋮
⋱
⋮
a
n
1
a
n
2
⋯
a
n
n
)
(
ξ
1
ξ
2
ξ
3
⋮
ξ
n
)
=
(
ξ
1
ξ
2
⋯
ξ
k
⋯
ξ
n
)
(
∑
i
=
1
n
a
1
i
ξ
i
∑
i
=
1
n
a
2
i
ξ
i
⋮
∑
i
=
1
n
a
k
i
ξ
i
⋮
∑
i
=
1
n
a
n
i
ξ
i
)
=
ξ
1
∑
j
=
1
n
a
1
j
ξ
j
+
⋯
+
ξ
k
∑
j
=
1
n
a
k
j
ξ
j
+
⋯
+
ξ
n
∑
j
=
1
n
a
n
j
ξ
j
\begin{align*} f\left( \mathbf{x} \right) &= \mathbf{x}^\top A \mathbf{x} \\ &= \left( \xi_1, \xi_2, \cdots, \xi_n \right) \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix} \begin{pmatrix} \xi_1 \\ \xi_2 \\ \xi_3 \\ \vdots \\ \xi_n \end{pmatrix} \\ &= \left( \begin{array}{cccccc} \xi_1 & \xi_2 & \cdots & \xi_k & \cdots & \xi_n \end{array} \right) \left( \begin{array}{c} \displaystyle \sum_{i=1}^n a_{1i} \xi_i \\ \displaystyle \sum_{i=1}^n a_{2i} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{ki} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{ni} \xi_i \end{array} \right) \\ &= \xi_1\sum_{j=1}^{n}a_{1j}\xi_j + \cdots + \xi_k\sum_{j=1}^{n}a_{kj}\xi_j + \cdots + \xi_n\sum_{j=1}^{n}a_{nj}\xi_j \end{align*}
f(x)=x⊤Ax=(ξ1,ξ2,⋯,ξn)
a11a21⋮an1a12a22⋮an2⋯⋯⋱⋯a1na2n⋮ann
ξ1ξ2ξ3⋮ξn
=(ξ1ξ2⋯ξk⋯ξn)
i=1∑na1iξii=1∑na2iξi⋮i=1∑nakiξi⋮i=1∑naniξi
=ξ1j=1∑na1jξj+⋯+ξkj=1∑nakjξj+⋯+ξnj=1∑nanjξj
所以：
∂
f
(
x
)
∂
ξ
k
=
ξ
1
a
1
k
+
⋯
+
ξ
k
−
1
a
k
−
1
,
k
+
(
∑
j
=
1
n
a
k
j
ξ
j
+
ξ
k
a
k
k
)
+
ξ
k
+
1
a
k
+
1
,
k
+
⋯
+
ξ
n
a
n
k
=
∑
i
=
1
n
a
i
k
ξ
i
+
∑
j
=
1
n
a
k
j
ξ
j
,
k
=
1
,
2
,
⋯
,
n
\begin{align*} \frac{\partial f(\mathbf{x})}{\partial \xi_k} &= \xi_1 a_{1k} + \cdots + \xi_{k-1} a_{k-1,k} + \left( \sum_{j=1}^{n} a_{kj} \xi_j + \xi_k a_{kk}\right) + \xi_{k+1} a_{k+1,k} + \cdots + \xi_n a_{nk} \\ &= \sum_{i=1}^n a_{ik} \xi_i + \sum_{j=1}^n a_{kj} \xi_j, \quad k=1,2,\cdots,n \end{align*}
∂ξk∂f(x)=ξ1a1k+⋯+ξk−1ak−1,k+(j=1∑nakjξj+ξkakk)+ξk+1ak+1,k+⋯+ξnank=i=1∑naikξi+j=1∑nakjξj,k=1,2,⋯,n
所以：
d
f
d
x
=
(
∂
f
∂
ξ
1
∂
f
∂
ξ
2
⋮
∂
f
∂
ξ
n
)
=
(
∑
j
=
1
n
a
1
j
ξ
j
∑
j
=
1
n
a
2
j
ξ
j
⋮
∑
j
=
1
n
a
n
j
ξ
j
)
+
(
∑
i
=
1
n
a
i
1
ξ
i
∑
i
=
1
n
a
i
2
ξ
i
⋮
∑
i
=
1
n
a
i
n
ξ
i
)
=
A
x
+
A
⊤
x
=
(
A
+
A
⊤
)
x
\begin{align*} \dfrac{d f}{d \mathbf{x}} &=\begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} =\left( \begin{array}{c} \displaystyle \sum_{j=1}^n a_{1j} \xi_j \\ \displaystyle \sum_{j=1}^n a_{2j} \xi_j \\ \vdots \\ \displaystyle \sum_{j=1}^n a_{nj} \xi_j \end{array} \right) + \left( \begin{array}{c} \displaystyle \sum_{i=1}^n a_{i1} \xi_i \\ \displaystyle \sum_{i=1}^n a_{i2} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{in} \xi_i \end{array} \right) \\ &=Ax + A^\top x = (A + A^\top)x \end{align*}
dxdf=
∂ξ1∂f∂ξ2∂f⋮∂ξn∂f
=
j=1∑na1jξjj=1∑na2jξj⋮j=1∑nanjξj
+
i=1∑nai1ξii=1∑nai2ξi⋮i=1∑nainξi
=Ax+A⊤x=(A+A⊤)x
特别地，当A为对称矩阵时，
d
f
d
x
=
2
A
x
\dfrac{d f}{d \mathbf{x}} = 2Ax
dxdf=2Ax
