The theory of time series uses state space technique to characterize stochastic processes. The key idea is that a stochastic process is modeled as an output of a stochastic linear/nonlinear dynamic system (having memory).
Linear stochastic model for stationary processes: AR(p), MA(q), ARMA(p,q),
Linear stochastic model for non-stationary processes: ARIMA(p,d,q).
Nonlinear stochastic model (nonlinear time series): ARCH
In contrast, linear/nonlinear regression models are used to characterize stochastic linear/nonlinear static system (without memory), i.e., characterize the input-output relationship. E.g., a regression model is y = f(x) +n, where n is noise vector, x is an independent variable/vector, y is a dependent variable/vector. E(y)=f(x) is the regression.
Linear regression: f(x) is linear, i.e., y = a+b*x +n
Nonlinear regression: f(x) is nonlinear.
Regression modeling is also called curve fitting.
(Two steps) First determine the function form, e.g., the order of a polynomial; then estimate the parameters of the function, e.g., coefficients of the polynomial.
(Model selection) To determine a function form, a hypothesis test (goodness-of-fit test) is needed. The goodness-of-fit test is a Neyman-Pearson test. Hence it is a likelihood ratio test and typically reduced to comparing a threshold.
(Parameter estimation) To estimate the parameters of the function, a least square criterion, minimizing the squared error, is usually used. The estimate is called LS estimate.
How to use the regression model?
The regression models obtained can be regarded as rules (e.g., physical laws) that govern the corresponding phenomena. We can use them for extrapolation, e.g., predicting the future position of an object. Regression method is very useful in experimental physics, economics, etc.
A sufficient condition for a wide-sense stationary process (time series) to have a rational z-spectrum, is that the process is ARMA. Not clear about what is the necessary condition. I think the autocorrelation has to be exponentially decayed; otherwise its z-transform won't be summable as a rational function.