Lec01 Introduction¶
·mathematical optimization There exist some exceptions that can be solved efficiently and reliably.
1.Least-squares and Linear programming¶
1.1Least-squqres¶
To solve problems:
\[
minimize \quad ||Ax-b||_2^2
\]
- solution \(x^*=(A^TA)^{-1}A^Tb\)
- is a reliable and efficient algorithm and software.
- computation time proportional to \(n^2k\) (where A belong to \(R^{k\times n}\) )
1.2 Linear programming¶
To solve problems:
\[
minimize \quad c^Tx\\
\]
\[
subject \ to\quad a_i^Tx\leq b_i\quad i=1,2,...,m
\]
- no analytical solution
- also is a reliable and efficient algorithm and software.
- computation time proportional to \(n^2m\)
2.Convex Optimization¶
To solve problems:
\[
minimize \quad f_0(x)\\
\]
\[
subject \ to\quad f_i(x)\leq b_i\quad i=1,2,...,m
\]
where the functions are convex:
Convex function
-
\[f_(αx + βy) ≤ αf_i(x) + βf_i(y)\]
if α + β = 1, α ≥ 0, β ≥ 0
- no analytical solution
- reliable and efficient algorithms
- computation time (roughly) proportional to max{n3, n2m, F }, where F is cost of evaluating fi’s and their first and second derivatives