Nonlinear Equations Are Not Just One Kind of Problem
Classifying nonlinear problems by their form and by what we want to find
1. Linear and Nonlinear: What Does Nonlinear Mean?
What comes to mind when you hear the phrase “nonlinear equation”? It may be a quadratic equation, an equation involving a trigonometric function, or a complicated system with many unknowns.
The following is a linear equation:
Both unknowns appear only to the first power, and they are not multiplied by each other. The following equations, however, are nonlinear:
They contain a square, a product of unknowns, or a trigonometric function. Roughly speaking, we enter the nonlinear world when the relationship between unknowns can no longer be represented by a single straight line or plane.
A nonlinear equation may have one solution, several solutions, or no real solution at all.
2. Two Viewpoints: Do Not Mix the Classifications
Two separate viewpoints make nonlinear problems easier to organize:
- Mathematical form: polynomial, rational, exponential, trigonometric, and so on
- Purpose: make an equation zero, reduce an error, or find the best feasible point
“Polynomial” describes what an expression is made of.
“Nonlinear optimization” describes what we are trying to find.
They are not opposing categories; they can overlap in the same problem.
3. Polynomial Equations
A polynomial equation is built from addition, subtraction, multiplication, and nonnegative integer powers of its unknowns.
With several unknowns and equations, it becomes a polynomial system:
Geometrically, this example asks for the intersections of the circle $x^2+y^2=1$ and the curve $y=x^3$. Polynomial systems have algebraic structure that helps us study the number and arrangement of their solutions. Homotopy continuation can also track many solutions systematically.
This is the world of problems for which PHCpack is designed.
4. Rational and Transcendental Equations
Rational equations
A quotient of polynomials produces a rational equation.
Clearing the denominator turns this example into a polynomial equation, but $x=1$ was never allowed in the original expression. We must preserve the original domain when transforming the equation.
Transcendental equations
Equations containing exponential, logarithmic, or trigonometric functions are generally called transcendental equations.
They cannot be constructed as finite polynomials. Bisection and Newton's method are common ways to find an individual solution, but Newton's method may reach different solutions—or fail to converge—depending on the initial value.
5. Nonlinear Systems
A nonlinear system asks for values that satisfy several nonlinear equations simultaneously.
If every $f_i$ is a polynomial, this is a polynomial system. If exponential or trigonometric functions appear, it is a non-polynomial nonlinear system. Polynomial and non-polynomial systems therefore live inside the broader category of nonlinear systems.
Circuit analysis leads naturally to this form. A circuit made only of ideal resistors is linear, but diode and transistor characteristics make its equations nonlinear.
6. Nonlinear Least Squares: When Nothing Fits Exactly
When fitting a mathematical model to experimental or measured data, it may be impossible to make every residual exactly zero. Let each residual be $r_i(\boldsymbol{x})$ and minimize their sum of squares:
This is a nonlinear least-squares problem. Instead of finding a point that satisfies every equation perfectly, it finds a point that makes the total discrepancy as small as possible. The goal is not an exact match, but the best fit to the data.
7. Nonlinear Optimization: Finding the Best Point
Nonlinear optimization seeks a point where an objective function is minimized or maximized, rather than merely seeking a root.
The objective function describes which points are desirable, while the constraints describe which points are allowed. If the objective or any constraint is nonlinear, the problem is a nonlinear optimization problem.
When all objectives and constraints are polynomials, it is a nonlinear optimization problem made from polynomials. Because “polynomial” and “optimization” classify different aspects of a problem, both descriptions can be true at once.
8. Differential Equations: When the Unknown Is a Function
So far, the unknowns have been numbers. In a differential equation, the unknown is a function that changes with time or position.
This nonlinear differential equation is known as the logistic equation. It is nonlinear because its right-hand side contains $y^2$. The answer is not a single number but a function $y(t)$ describing change over time.
Such equations appear in circuit transients, motion, population models, and many other dynamical systems.
9. Classification Table
| Viewpoint | Category | Example | What We Seek |
|---|---|---|---|
| Form | Polynomial | $x^3-2x+1=0$ | A value satisfying the equation |
| Form | Rational | $1/(x-1)=x$ | A solution inside the domain |
| Form | Transcendental | $e^{-x}=x$ | An intersection of functions |
| Purpose | Nonlinear system | $\boldsymbol{F}(\boldsymbol{x})=\boldsymbol{0}$ | A point satisfying every equation |
| Purpose | Nonlinear least squares | $\min\sum r_i^2$ | A point with the smallest error |
| Purpose | Nonlinear optimization | $\min f(\boldsymbol{x})$ | The best feasible point |
| Unknown | Nonlinear differential equation | $dy/dt=y(1-y)$ | A changing function |
A single problem can have several valid names. A system made from polynomials, for example, is both a nonlinear system and a polynomial system.
10. Dr.WataWata's Insight
Look at what the problem asks us to find
“Nonlinear equation” covers a remarkably wide range of problems. Three questions help identify what we are actually facing:
- Is the unknown a number, or a function that changes with time?
- Is the expression polynomial, or does it contain exponential or trigonometric functions?
- Are we seeking an exact root, the smallest error, or the best objective value?
Treating every problem as “nonlinear, so use Newton's method” misses the structure that should guide our choice of solver. Recognizing both the form and the purpose of a problem is the first step toward choosing an appropriate method.
Nonlinear problems are not one single species. From the same nonlinear entrance, separate paths lead to polynomial systems, circuits, optimization, data analysis, and differential equations.