# [Maxima] revision of lsquares package

Daniel Lakeland dlakelan at street-artists.org
Mon Jul 30 00:13:11 CDT 2007

```On Sun, Jul 29, 2007 at 09:26:33PM -0600, Robert Dodier wrote:
> Hello,
>
> I have replaced share/contrib/lsquares.mac with a new version.
> I hope this proves to be useful.

Hi Robert. It is so incredibly useful to have least squares fitting,
that I am sure there will be plenty of people who find this
useful. Thanks for working on this. Here are some comments.

> Replace previous lsquares (least squares) package.
> Features of new version:
>
> (1) Formulate numerical problem as a minimization (solved via lbfgs)
> instead of a system of equations (solved via mnewton).
> The minimization formulation is more likely to converge
> (since there is always a notion of the correct direction, namely downhill)

For nonlinear least squares this seems to be a reasonable way to
go. For linear least squares there are some standard methods using the
Singular Value Decomposition (SVD) which are numerically stable. I
don't think we have a numerical SVD in maxima, but perhaps it's time
to think about getting one, and creating a numerical "generalized
inverse" function. I know this has been mentioned before.

> (2) Separate functions for exact vs numerical estimates
> (3) Separate functions to construct the mean square error expression,
> residuals, and residual mean square error.

This seems like a great idea. It's always a good idea to consider the
errors.

Perhaps we should also consider a least trimmed squares routine. The
idea is to fit with least squares to some fraction of the data,
throwing out some fraction of the biggest outliers. There is typically
an iterative method. This is a robust statistical technique that
sacrifices some efficiency when there are no outliers for resistance
to error when there are outliers.

Perhaps I will take a look at your code and consider how to add this
technique in the same framework.

--
Daniel Lakeland
dlakelan at street-artists.org
http://www.street-artists.org/~dlakelan
```