Linear Least-Squares Fitting

This chapter describes routines for performing least squares fits to experimental data using linear combinations of functions. The data may be weighted or unweighted, i.e. with known or unknown errors. For weighted data the functions compute the best fit parameters and their associated covariance matrix. For unweighted data the covariance matrix is estimated from the scatter of the points, giving a variance-covariance matrix.

The functions are divided into separate versions for simple one- or two-parameter regression and multiple-parameter fits.

Overview

Least-squares fits are found by minimizing

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

(chi-squared), the weighted sum of squared residuals over

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

experimental datapoints

System Message: WARNING/2 ((x_i, y_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the model

System Message: WARNING/2 (Y(c,x))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\chi^2 = \sum_i w_i (y_i - Y(c, x_i))^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

parameters of the model are

System Message: WARNING/2 (c = \{c_0, c_1, \dots\})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The weight factors

System Message: WARNING/2 (w_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are given by

System Message: WARNING/2 (w_i = 1/\sigma_i^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (\sigma_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the experimental error on the data-point

System Message: WARNING/2 (y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The errors are assumed to be Gaussian and uncorrelated. For unweighted data the chi-squared sum is computed without any weight factors.

The fitting routines return the best-fit parameters

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and their

System Message: WARNING/2 (p \times p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

covariance matrix. The covariance matrix measures the statistical errors on the best-fit parameters resulting from the errors on the data,

System Message: WARNING/2 (\sigma_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and is defined as

System Message: WARNING/2 (C_{ab} = \langle \delta c_a \delta c_b \rangle )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (\langle \, \rangle)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

denotes an average over the Gaussian error distributions of the underlying datapoints.

The covariance matrix is calculated by error propagation from the data errors

System Message: WARNING/2 (\sigma_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The change in a fitted parameter

System Message: WARNING/2 (\delta c_a)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

caused by a small change in the data

System Message: WARNING/2 (\delta y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is given by

System Message: WARNING/2 (\delta c_a = \sum_i {\partial c_a \over \partial y_i} \delta y_i )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

allowing the covariance matrix to be written in terms of the errors on the data,

System Message: WARNING/2 (C_{ab} = \sum_{i,j} {\partial c_a \over \partial y_i} {\partial c_b \over \partial y_j} \langle \delta y_i \delta y_j \rangle)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

For uncorrelated data the fluctuations of the underlying datapoints satisfy

System Message: WARNING/2 (\langle \delta y_i \delta y_j \rangle = \sigma_i^2 \delta_{ij} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

giving a corresponding parameter covariance matrix of

System Message: WARNING/2 (C_{ab} = \sum_{i} {1 \over w_i} {\partial c_a \over \partial y_i} {\partial c_b \over \partial y_i} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

When computing the covariance matrix for unweighted data, i.e. data with unknown errors, the weight factors

System Message: WARNING/2 (w_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

in this sum are replaced by the single estimate

System Message: WARNING/2 (w = 1/\sigma^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (\sigma^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the computed variance of the residuals about the best-fit model,

System Message: WARNING/2 (\sigma^2 = \sum (y_i - Y(c,x_i))^2 / (n-p))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. This is referred to as the variance-covariance matrix.

The standard deviations of the best-fit parameters are given by the square root of the corresponding diagonal elements of the covariance matrix,

System Message: WARNING/2 (\sigma_{c_a} = \sqrt{C_{aa}})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The correlation coefficient of the fit parameters

System Message: WARNING/2 (c_a)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (c_b)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is given by

System Message: WARNING/2 (\rho_{ab} = C_{ab} / \sqrt{C_{aa} C_{bb}})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Linear regression

The functions in this section are used to fit simple one or two parameter linear regression models. The functions are declared in the header file gsl_fit.h.

Linear regression with a constant term

The functions described in this section can be used to perform least-squares fits to a straight line model,

System Message: WARNING/2 (Y(c,x) = c_0 + c_1 x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_fit_linear(const double * x, const size_t xstride, const double * y, const size_t ystride, size_t n, double * c0, double * c1, double * cov00, double * cov01, double * cov11, double * sumsq)

This function computes the best-fit linear regression coefficients (c0, c1) of the model

System Message: WARNING/2 (Y = c_0 + c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the dataset (x, y), two vectors of length n with strides xstride and ystride. The errors on y are assumed unknown so the variance-covariance matrix for the parameters (c0, c1) is estimated from the scatter of the points around the best-fit line and returned via the parameters (cov00, cov01, cov11). The sum of squares of the residuals from the best-fit line is returned in sumsq. Note: the correlation coefficient of the data can be computed using gsl_stats_correlation(), it does not depend on the fit.

int gsl_fit_wlinear(const double * x, const size_t xstride, const double * w, const size_t wstride, const double * y, const size_t ystride, size_t n, double * c0, double * c1, double * cov00, double * cov01, double * cov11, double * chisq)

This function computes the best-fit linear regression coefficients (c0, c1) of the model

System Message: WARNING/2 (Y = c_0 + c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the weighted dataset (x, y), two vectors of length n with strides xstride and ystride. The vector w, of length n and stride wstride, specifies the weight of each datapoint. The weight is the reciprocal of the variance for each datapoint in y.

The covariance matrix for the parameters (c0, c1) is computed using the weights and returned via the parameters (cov00, cov01, cov11). The weighted sum of squares of the residuals from the best-fit line,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq.

int gsl_fit_linear_est(double x, double c0, double c1, double cov00, double cov01, double cov11, double * y, double * y_err)

This function uses the best-fit linear regression coefficients c0, c1 and their covariance cov00, cov01, cov11 to compute the fitted function y and its standard deviation y_err for the model

System Message: WARNING/2 (Y = c_0 + c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

at the point x.

Linear regression without a constant term

The functions described in this section can be used to perform least-squares fits to a straight line model without a constant term,

System Message: WARNING/2 (Y = c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_fit_mul(const double * x, const size_t xstride, const double * y, const size_t ystride, size_t n, double * c1, double * cov11, double * sumsq)

This function computes the best-fit linear regression coefficient c1 of the model

System Message: WARNING/2 (Y = c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the datasets (x, y), two vectors of length n with strides xstride and ystride. The errors on y are assumed unknown so the variance of the parameter c1 is estimated from the scatter of the points around the best-fit line and returned via the parameter cov11. The sum of squares of the residuals from the best-fit line is returned in sumsq.

int gsl_fit_wmul(const double * x, const size_t xstride, const double * w, const size_t wstride, const double * y, const size_t ystride, size_t n, double * c1, double * cov11, double * sumsq)

This function computes the best-fit linear regression coefficient c1 of the model

System Message: WARNING/2 (Y = c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the weighted datasets (x, y), two vectors of length n with strides xstride and ystride. The vector w, of length n and stride wstride, specifies the weight of each datapoint. The weight is the reciprocal of the variance for each datapoint in y.

The variance of the parameter c1 is computed using the weights and returned via the parameter cov11. The weighted sum of squares of the residuals from the best-fit line,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq.

int gsl_fit_mul_est(double x, double c1, double cov11, double * y, double * y_err)

This function uses the best-fit linear regression coefficient c1 and its covariance cov11 to compute the fitted function y and its standard deviation y_err for the model

System Message: WARNING/2 (Y = c_1 X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

at the point x.

Multi-parameter regression

This section describes routines which perform least squares fits to a linear model by minimizing the cost function

System Message: WARNING/2 (\chi^2 = \sum_i w_i (y_i - \sum_j X_{ij} c_j)^2 = || y - Xc ||_W^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a vector of

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

observations,

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is an

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix of predictor variables,

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a vector of the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

unknown best-fit parameters to be estimated, and

System Message: WARNING/2 (||r||_W^2 = r^T W r)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The matrix

System Message: WARNING/2 (W = \diag(w_1,w_2,...,w_n))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

defines the weights or uncertainties of the observation vector.

This formulation can be used for fits to any number of functions and/or variables by preparing the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

appropriately. For example, to fit to a

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-th order polynomial in x, use the following matrix,

System Message: WARNING/2 (X_{ij} = x_i^j )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where the index

System Message: WARNING/2 (i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

runs over the observations and the index

System Message: WARNING/2 (j)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

runs from 0 to

System Message: WARNING/2 (p-1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

To fit to a set of

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

sinusoidal functions with fixed frequencies

System Message: WARNING/2 (\omega_1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\omega_2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\ldots)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\omega_p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, use,

System Message: WARNING/2 (X_{ij} = \sin(\omega_j x_i) )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

To fit to

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

independent variables

System Message: WARNING/2 (x_1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (x_2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\ldots)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (x_p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, use,

System Message: WARNING/2 (X_{ij} = x_j(i) )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (x_j(i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-th value of the predictor variable

System Message: WARNING/2 (x_j)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

The solution of the general linear least-squares system requires an additional working space for intermediate results, such as the singular value decomposition of the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

These functions are declared in the header file gsl_multifit.h.

gsl_multifit_linear_workspace

This workspace contains internal variables for fitting multi-parameter models.

gsl_multifit_linear_workspace * gsl_multifit_linear_alloc(const size_t n, const size_t p)

This function allocates a workspace for fitting a model to a maximum of n observations using a maximum of p parameters. The user may later supply a smaller least squares system if desired. The size of the workspace is

System Message: WARNING/2 (O(np + p^2))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

void gsl_multifit_linear_free(gsl_multifit_linear_workspace * work)

This function frees the memory associated with the workspace w.

int gsl_multifit_linear_svd(const gsl_matrix * X, gsl_multifit_linear_workspace * work)

This function performs a singular value decomposition of the matrix X and stores the SVD factors internally in work.

int gsl_multifit_linear_bsvd(const gsl_matrix * X, gsl_multifit_linear_workspace * work)

This function performs a singular value decomposition of the matrix X and stores the SVD factors internally in work. The matrix X is first balanced by applying column scaling factors to improve the accuracy of the singular values.

int gsl_multifit_linear(const gsl_matrix * X, const gsl_vector * y, gsl_vector * c, gsl_matrix * cov, double * chisq, gsl_multifit_linear_workspace * work)

This function computes the best-fit parameters c of the model

System Message: WARNING/2 (y = X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y and the matrix of predictor variables X, using the preallocated workspace provided in work. The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

variance-covariance matrix of the model parameters cov is set to

System Message: WARNING/2 (\sigma^2 (X^T X)^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the standard deviation of the fit residuals. The sum of squares of the residuals from the best-fit,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq. If the coefficient of determination is desired, it can be computed from the expression

System Message: WARNING/2 (R^2 = 1 - \chi^2 / TSS)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where the total sum of squares (TSS) of the observations y may be computed from gsl_stats_tss().

The best-fit is found by singular value decomposition of the matrix X using the modified Golub-Reinsch SVD algorithm, with column scaling to improve the accuracy of the singular values. Any components which have zero singular value (to machine precision) are discarded from the fit.

int gsl_multifit_linear_tsvd(const gsl_matrix * X, const gsl_vector * y, const double tol, gsl_vector * c, gsl_matrix * cov, double * chisq, size_t * rank, gsl_multifit_linear_workspace * work)

This function computes the best-fit parameters c of the model

System Message: WARNING/2 (y = X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y and the matrix of predictor variables X, using a truncated SVD expansion. Singular values which satisfy

System Message: WARNING/2 (s_i \le tol \times s_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are discarded from the fit, where

System Message: WARNING/2 (s_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the largest singular value. The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

variance-covariance matrix of the model parameters cov is set to

System Message: WARNING/2 (\sigma^2 (X^T X)^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the standard deviation of the fit residuals. The sum of squares of the residuals from the best-fit,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq. The effective rank (number of singular values used in solution) is returned in rank. If the coefficient of determination is desired, it can be computed from the expression

System Message: WARNING/2 (R^2 = 1 - \chi^2 / TSS)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where the total sum of squares (TSS) of the observations y may be computed from gsl_stats_tss().

int gsl_multifit_wlinear(const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_vector * c, gsl_matrix * cov, double * chisq, gsl_multifit_linear_workspace * work)

This function computes the best-fit parameters c of the weighted model

System Message: WARNING/2 (y = X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y with weights w and the matrix of predictor variables X, using the preallocated workspace provided in work. The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

covariance matrix of the model parameters cov is computed as

System Message: WARNING/2 ((X^T W X)^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The weighted sum of squares of the residuals from the best-fit,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq. If the coefficient of determination is desired, it can be computed from the expression

System Message: WARNING/2 (R^2 = 1 - \chi^2 / WTSS)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where the weighted total sum of squares (WTSS) of the observations y may be computed from gsl_stats_wtss().

int gsl_multifit_wlinear_tsvd(const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, const double tol, gsl_vector * c, gsl_matrix * cov, double * chisq, size_t * rank, gsl_multifit_linear_workspace * work)

This function computes the best-fit parameters c of the weighted model

System Message: WARNING/2 (y = X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y with weights w and the matrix of predictor variables X, using a truncated SVD expansion. Singular values which satisfy

System Message: WARNING/2 (s_i \le tol \times s_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are discarded from the fit, where

System Message: WARNING/2 (s_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the largest singular value. The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

covariance matrix of the model parameters cov is computed as

System Message: WARNING/2 ((X^T W X)^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The weighted sum of squares of the residuals from the best-fit,

System Message: WARNING/2 (\chi^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, is returned in chisq. The effective rank of the system (number of singular values used in the solution) is returned in rank. If the coefficient of determination is desired, it can be computed from the expression

System Message: WARNING/2 (R^2 = 1 - \chi^2 / WTSS)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where the weighted total sum of squares (WTSS) of the observations y may be computed from gsl_stats_wtss().

int gsl_multifit_linear_est(const gsl_vector * x, const gsl_vector * c, const gsl_matrix * cov, double * y, double * y_err)

This function uses the best-fit multilinear regression coefficients c and their covariance matrix cov to compute the fitted function value y and its standard deviation y_err for the model

System Message: WARNING/2 (y = x.c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

at the point x.

int gsl_multifit_linear_residuals(const gsl_matrix * X, const gsl_vector * y, const gsl_vector * c, gsl_vector * r)

This function computes the vector of residuals

System Message: WARNING/2 (r = y - X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y, coefficients c and matrix of predictor variables X.

size_t gsl_multifit_linear_rank(const double tol, const gsl_multifit_linear_workspace * work)

This function returns the rank of the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which must first have its singular value decomposition computed. The rank is computed by counting the number of singular values

System Message: WARNING/2 (\sigma_j)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which satisfy

System Message: WARNING/2 (\sigma_j > tol \times \sigma_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (\sigma_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the largest singular value.

Regularized regression

Ordinary weighted least squares models seek a solution vector

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which minimizes the residual

System Message: WARNING/2 (\chi^2 = || y - Xc ||_W^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

observation vector,

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

design matrix,

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

solution vector,

System Message: WARNING/2 (W = \diag(w_1,...,w_n))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the data weighting matrix, and

System Message: WARNING/2 (||r||_W^2 = r^T W r)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. In cases where the least squares matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is ill-conditioned, small perturbations (ie: noise) in the observation vector could lead to widely different solution vectors

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. One way of dealing with ill-conditioned matrices is to use a “truncated SVD” in which small singular values, below some given tolerance, are discarded from the solution. The truncated SVD method is available using the functions gsl_multifit_linear_tsvd() and gsl_multifit_wlinear_tsvd(). Another way to help solve ill-posed problems is to include a regularization term in the least squares minimization

System Message: WARNING/2 (\chi^2 = || y - Xc ||_W^2 + \lambda^2 || L c ||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for a suitably chosen regularization parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. This type of regularization is known as Tikhonov, or ridge, regression. In some applications,

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is chosen as the identity matrix, giving preference to solution vectors

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

with smaller norms. Including this regularization term leads to the explicit “normal equations” solution

System Message: WARNING/2 (c = \left( X^T W X + \lambda^2 L^T L \right)^{-1} X^T W y )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which reduces to the ordinary least squares solution when

System Message: WARNING/2 (L = 0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. In practice, it is often advantageous to transform a regularized least squares system into the form

System Message: WARNING/2 (\chi^2 = || \tilde{y} - \tilde{X} \tilde{c} ||^2 + \lambda^2 || \tilde{c} ||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

This is known as the Tikhonov “standard form” and has the normal equations solution

System Message: WARNING/2 (\tilde{c} = \left( \tilde{X}^T \tilde{X} + \lambda^2 I \right)^{-1} \tilde{X}^T \tilde{y} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

For an

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which is full rank and has

System Message: WARNING/2 (m >= p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

(ie:

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is square or has more rows than columns), we can calculate the “thin” QR decomposition of

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and note that

System Message: WARNING/2 (||L c|| = ||R c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

since the

System Message: WARNING/2 (Q)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

factor will not change the norm. Since

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, we can then use the transformation

System Message: WARNING/2 (\tilde{X} &= W^{1 \over 2} X R^{-1} \\ \tilde{y} &= W^{1 \over 2} y \\ \tilde{c} &= R c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

to achieve the standard form. For a rectangular matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

with

System Message: WARNING/2 (m < p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, a more sophisticated approach is needed (see Hansen 1998, chapter 2.3). In practice, the normal equations solution above is not desirable due to numerical instabilities, and so the system is solved using the singular value decomposition of the matrix

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is often chosen as the identity matrix, or as a first or second finite difference operator, to ensure a smoothly varying coefficient vector

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, or as a diagonal matrix to selectively damp each model parameter differently. If

System Message: WARNING/2 (L \ne I)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the user must first convert the least squares problem to standard form using gsl_multifit_linear_stdform1() or gsl_multifit_linear_stdform2(), solve the system, and then backtransform the solution vector to recover the solution of the original problem (see gsl_multifit_linear_genform1() and gsl_multifit_linear_genform2()).

In many regularization problems, care must be taken when choosing the regularization parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. Since both the residual norm

System Message: WARNING/2 (||y - X c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and solution norm

System Message: WARNING/2 (||L c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are being minimized, the parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

represents a tradeoff between minimizing either the residuals or the solution vector. A common tool for visualizing the comprimise between the minimization of these two quantities is known as the L-curve. The L-curve is a log-log plot of the residual norm

System Message: WARNING/2 (||y - X c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

on the horizontal axis and the solution norm

System Message: WARNING/2 (||L c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

on the vertical axis. This curve nearly always as an

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

shaped appearance, with a distinct corner separating the horizontal and vertical sections of the curve. The regularization parameter corresponding to this corner is often chosen as the optimal value. GSL provides routines to calculate the L-curve for all relevant regularization parameters as well as locating the corner.

Another method of choosing the regularization parameter is known as Generalized Cross Validation (GCV). This method is based on the idea that if an arbitrary element

System Message: WARNING/2 (y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is left out of the right hand side, the resulting regularized solution should predict this element accurately. This leads to choosing the parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which minimizes the GCV function

System Message: WARNING/2 (G(\lambda) = {||y - X c_{\lambda}||^2 \over \textrm{Tr}(I_n - X X_{\lambda}^I)^2} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (X_{\lambda}^I)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the matrix which relates the solution

System Message: WARNING/2 (c_{\lambda})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

to the right hand side

System Message: WARNING/2 (y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, ie:

System Message: WARNING/2 (c_{\lambda} = X_{\lambda}^I y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. GSL provides routines to compute the GCV curve and its minimum.

For most applications, the steps required to solve a regularized least squares problem are as follows:

1. Construct the least squares system (

   System Message: WARNING/2 (X)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (y)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (W)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (L)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   )
2. Transform the system to standard form (

   System Message: WARNING/2 (\tilde{X})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (\tilde{y})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ). This step can be skipped if

   System Message: WARNING/2 (L = I_p)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   and

   System Message: WARNING/2 (W = I_n)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
3. Calculate the SVD of

   System Message: WARNING/2 (\tilde{X})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
4. Determine an appropriate regularization parameter

   System Message: WARNING/2 (\lambda)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   (using for example L-curve or GCV analysis).
5. Solve the standard form system using the chosen

   System Message: WARNING/2 (\lambda)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   and the SVD of

   System Message: WARNING/2 (\tilde{X})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
6. Backtransform the standard form solution

   System Message: WARNING/2 (\tilde{c})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   to recover the original solution vector

   System Message: WARNING/2 (c)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .

int gsl_multifit_linear_stdform1(const gsl_vector * L, const gsl_matrix * X, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multifit_linear_workspace * work)

int gsl_multifit_linear_wstdform1(const gsl_vector * L, const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multifit_linear_workspace * work)

These functions define a regularization matrix

System Message: WARNING/2 (L = \diag(l_0,l_1,...,l_{p-1}))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The diagonal matrix element

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is provided by the

System Message: WARNING/2 (i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-th element of the input vector L. The

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

least squares matrix X and vector y of length

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are then converted to standard form as described above and the parameters (

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) are stored in Xs and ys on output. Xs and ys have the same dimensions as X and y. Optional data weights may be supplied in the vector w of length

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. In order to apply this transformation,

System Message: WARNING/2 (L^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

must exist and so none of the

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

may be zero. After the standard form system has been solved, use gsl_multifit_linear_genform1() to recover the original solution vector. It is allowed to have X = Xs and y = ys for an in-place transform. In order to perform a weighted regularized fit with

System Message: WARNING/2 (L = I)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the user may call gsl_multifit_linear_applyW() to convert to standard form.

int gsl_multifit_linear_L_decomp(gsl_matrix * L, gsl_vector * tau)

This function factors the

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

regularization matrix L into a form needed for the later transformation to standard form. L may have any number of rows

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. If

System Message: WARNING/2 (m \ge p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

the QR decomposition of L is computed and stored in L on output. If

System Message: WARNING/2 (m < p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the QR decomposition of

System Message: WARNING/2 (L^T)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is computed and stored in L on output. On output, the Householder scalars are stored in the vector tau of size

System Message: WARNING/2 (MIN(m,p))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. These outputs will be used by gsl_multifit_linear_wstdform2() to complete the transformation to standard form.

int gsl_multifit_linear_stdform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_matrix * M, gsl_multifit_linear_workspace * work)

int gsl_multifit_linear_wstdform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_matrix * M, gsl_multifit_linear_workspace * work)

These functions convert the least squares system (X, y, W,

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) to standard form (

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) which are stored in Xs and ys respectively. The

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

regularization matrix L is specified by the inputs LQR and Ltau, which are outputs from gsl_multifit_linear_L_decomp(). The dimensions of the standard form parameters (

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) depend on whether

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is larger or less than

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. For

System Message: WARNING/2 (m \ge p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, Xs is

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, ys is

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-1, and M is not used. For

System Message: WARNING/2 (m < p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, Xs is

System Message: WARNING/2 ((n - p + m))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, ys is

System Message: WARNING/2 ((n - p + m))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-1, and M is additional

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

workspace, which is required to recover the original solution vector after the system has been solved (see gsl_multifit_linear_genform2()). Optional data weights may be supplied in the vector w of length

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (W = \diag(w))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_multifit_linear_solve(const double lambda, const gsl_matrix * Xs, const gsl_vector * ys, gsl_vector * cs, double * rnorm, double * snorm, gsl_multifit_linear_workspace * work)

This function computes the regularized best-fit parameters

System Message: WARNING/2 (\tilde{c})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which minimize the cost function

System Message: WARNING/2 (\chi^2 = || \tilde{y} - \tilde{X} \tilde{c} ||^2 + \lambda^2 || \tilde{c} ||^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which is in standard form. The least squares system must therefore be converted to standard form prior to calling this function. The observation vector

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is provided in ys and the matrix of predictor variables

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

in Xs. The solution vector

System Message: WARNING/2 (\tilde{c})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is returned in cs, which has length min(

System Message: WARNING/2 (m,p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

). The SVD of Xs must be computed prior to calling this function, using gsl_multifit_linear_svd(). The regularization parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is provided in lambda. The residual norm

System Message: WARNING/2 (|| \tilde{y} - \tilde{X} \tilde{c} || = ||y - X c||_W)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is returned in rnorm. The solution norm

System Message: WARNING/2 (|| \tilde{c} || = ||L c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is returned in snorm.

int gsl_multifit_linear_genform1(const gsl_vector * L, const gsl_vector * cs, gsl_vector * c, gsl_multifit_linear_workspace * work)

After a regularized system has been solved with

System Message: WARNING/2 (L = \diag(\l_0,\l_1,...,\l_{p-1}))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, this function backtransforms the standard form solution vector cs to recover the solution vector of the original problem c. The diagonal matrix elements

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are provided in the vector L. It is allowed to have c = cs for an in-place transform.

int gsl_multifit_linear_genform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * y, const gsl_vector * cs, const gsl_matrix * M, gsl_vector * c, gsl_multifit_linear_workspace * work)

int gsl_multifit_linear_wgenform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, const gsl_vector * cs, const gsl_matrix * M, gsl_vector * c, gsl_multifit_linear_workspace * work)

After a regularized system has been solved with a general rectangular matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, specified by (LQR, Ltau), this function backtransforms the standard form solution cs to recover the solution vector of the original problem, which is stored in c, of length

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The original least squares matrix and observation vector are provided in X and y respectively. M is the matrix computed by gsl_multifit_linear_stdform2(). For weighted fits, the weight vector w must also be supplied.

int gsl_multifit_linear_applyW(const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_matrix * WX, gsl_vector * Wy)

For weighted least squares systems with

System Message: WARNING/2 (L = I)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, this function may be used to convert the system to standard form by applying the weight matrix

System Message: WARNING/2 (W = \diag(w))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

to the least squares matrix X and observation vector y. On output, WX is equal to

System Message: WARNING/2 (W^{1/2} X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and Wy is equal to

System Message: WARNING/2 (W^{1/2} y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. It is allowed for WX = X and Wy = y for an in-place transform.

int gsl_multifit_linear_lcurve(const gsl_vector * y, gsl_vector * reg_param, gsl_vector * rho, gsl_vector * eta, gsl_multifit_linear_workspace * work)

This function computes the L-curve for a least squares system using the right hand side vector y and the SVD decomposition of the least squares matrix X, which must be provided to gsl_multifit_linear_svd() prior to calling this function. The output vectors reg_param, rho, and eta must all be the same size, and will contain the regularization parameters

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, residual norms

System Message: WARNING/2 (||y - X c_i||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and solution norms

System Message: WARNING/2 (|| L c_i ||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which compose the L-curve, where

System Message: WARNING/2 (c_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the regularized solution vector corresponding to

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The user may determine the number of points on the L-curve by adjusting the size of these input arrays. The regularization parameters

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are estimated from the singular values of X, and chosen to represent the most relevant portion of the L-curve.

int gsl_multifit_linear_lcorner(const gsl_vector * rho, const gsl_vector * eta, size_t * idx)

This function attempts to locate the corner of the L-curve

System Message: WARNING/2 ((||y - X c||, ||L c||))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

defined by the rho and eta input arrays respectively. The corner is defined as the point of maximum curvature of the L-curve in log-log scale. The rho and eta arrays can be outputs of gsl_multifit_linear_lcurve(). The algorithm used simply fits a circle to 3 consecutive points on the L-curve and uses the circle’s radius to determine the curvature at the middle point. Therefore, the input array sizes must be

System Message: WARNING/2 (\ge 3)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. With more points provided for the L-curve, a better estimate of the curvature can be obtained. The array index corresponding to maximum curvature (ie: the corner) is returned in idx. If the input arrays contain colinear points, this function could fail and return GSL_EINVAL.

int gsl_multifit_linear_lcorner2(const gsl_vector * reg_param, const gsl_vector * eta, size_t * idx)

This function attempts to locate the corner of an alternate L-curve

System Message: WARNING/2 ((\lambda^2, ||L c||^2))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

studied by Rezghi and Hosseini, 2009. This alternate L-curve can provide better estimates of the regularization parameter for smooth solution vectors. The regularization parameters

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and solution norms

System Message: WARNING/2 (||L c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are provided in the reg_param and eta input arrays respectively. The corner is defined as the point of maximum curvature of this alternate L-curve in linear scale. The reg_param and eta arrays can be outputs of gsl_multifit_linear_lcurve(). The algorithm used simply fits a circle to 3 consecutive points on the L-curve and uses the circle’s radius to determine the curvature at the middle point. Therefore, the input array sizes must be

System Message: WARNING/2 (\ge 3)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. With more points provided for the L-curve, a better estimate of the curvature can be obtained. The array index corresponding to maximum curvature (ie: the corner) is returned in idx. If the input arrays contain colinear points, this function could fail and return GSL_EINVAL.

int gsl_multifit_linear_gcv_init(const gsl_vector * y, gsl_vector * reg_param, gsl_vector * UTy, double * delta0, gsl_multifit_linear_workspace * work)

This function performs some initialization in preparation for computing the GCV curve and its minimum. The right hand side vector is provided in y. On output, reg_param is set to a vector of regularization parameters in decreasing order and may be of any size. The vector UTy of size

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is set to

System Message: WARNING/2 (U^T y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The parameter delta0 is needed for subsequent steps of the GCV calculation.

int gsl_multifit_linear_gcv_curve(const gsl_vector * reg_param, const gsl_vector * UTy, const double delta0, gsl_vector * G, gsl_multifit_linear_workspace * work)

This funtion calculates the GCV curve

System Message: WARNING/2 (G(\lambda))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and stores it in G on output, which must be the same size as reg_param. The inputs reg_param, UTy and delta0 are computed in gsl_multifit_linear_gcv_init().

int gsl_multifit_linear_gcv_min(const gsl_vector * reg_param, const gsl_vector * UTy, const gsl_vector * G, const double delta0, double * lambda, gsl_multifit_linear_workspace * work)

This function computes the value of the regularization parameter which minimizes the GCV curve

System Message: WARNING/2 (G(\lambda))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and stores it in lambda. The input G is calculated by gsl_multifit_linear_gcv_curve() and the inputs reg_param, UTy and delta0 are computed by gsl_multifit_linear_gcv_init().

double gsl_multifit_linear_gcv_calc(const double lambda, const gsl_vector * UTy, const double delta0, gsl_multifit_linear_workspace * work)

This function returns the value of the GCV curve

System Message: WARNING/2 (G(\lambda))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

corresponding to the input lambda.

int gsl_multifit_linear_gcv(const gsl_vector * y, gsl_vector * reg_param, gsl_vector * G, double * lambda, double * G_lambda, gsl_multifit_linear_workspace * work)

This function combines the steps gcv_init, gcv_curve, and gcv_min defined above into a single function. The input y is the right hand side vector. On output, reg_param and G, which must be the same size, are set to vectors of

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (G(\lambda))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

values respectively. The output lambda is set to the optimal value of

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which minimizes the GCV curve. The minimum value of the GCV curve is returned in G_lambda.

int gsl_multifit_linear_Lk(const size_t p, const size_t k, gsl_matrix * L)

This function computes the discrete approximation to the derivative operator

System Message: WARNING/2 (L_k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

of order k on a regular grid of p points and stores it in L. The dimensions of L are

System Message: WARNING/2 ((p-k))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_multifit_linear_Lsobolev(const size_t p, const size_t kmax, const gsl_vector * alpha, gsl_matrix * L, gsl_multifit_linear_workspace * work)

This function computes the regularization matrix L corresponding to the weighted Sobolov norm

System Message: WARNING/2 (||L c||^2 = \sum_k \alpha_k^2 ||L_k c||^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (L_k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

approximates the derivative operator of order

System Message: WARNING/2 (k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. This regularization norm can be useful in applications where it is necessary to smooth several derivatives of the solution. p is the number of model parameters, kmax is the highest derivative to include in the summation above, and alpha is the vector of weights of size kmax + 1, where alpha[k] =

System Message: WARNING/2 (\alpha_k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the weight assigned to the derivative of order

System Message: WARNING/2 (k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The output matrix L is size p-by-p and upper triangular.

double gsl_multifit_linear_rcond(const gsl_multifit_linear_workspace * work)

This function returns the reciprocal condition number of the least squares matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, defined as the ratio of the smallest and largest singular values, rcond =

System Message: WARNING/2 (\sigma_{min}/\sigma_{max})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The routine gsl_multifit_linear_svd() must first be called to compute the SVD of

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Robust linear regression

Ordinary least squares (OLS) models are often heavily influenced by the presence of outliers. Outliers are data points which do not follow the general trend of the other observations, although there is strictly no precise definition of an outlier. Robust linear regression refers to regression algorithms which are robust to outliers. The most common type of robust regression is M-estimation. The general M-estimator minimizes the objective function

System Message: WARNING/2 (\sum_i \rho(e_i) = \sum_i \rho (y_i - Y(c, x_i)) )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (e_i = y_i - Y(c, x_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the residual of the ith data point, and

System Message: WARNING/2 (\rho(e_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a function which should have the following properties:

• System Message: WARNING/2 (\rho(e) \ge 0)

  latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

• System Message: WARNING/2 (\rho(0) = 0)

  latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

• System Message: WARNING/2 (\rho(-e) = \rho(e))

  latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

• System Message: WARNING/2 (\rho(e_1) > \rho(e_2))

  latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
  for

  System Message: WARNING/2 (|e_1| > |e_2|)

  latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

The special case of ordinary least squares is given by

System Message: WARNING/2 (\rho(e_i) = e_i^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. Letting

System Message: WARNING/2 (\psi = \rho')

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

be the derivative of

System Message: WARNING/2 (\rho)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, differentiating the objective function with respect to the coefficients

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and setting the partial derivatives to zero produces the system of equations

System Message: WARNING/2 (\sum_i \psi(e_i) X_i = 0 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (X_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a vector containing row

System Message: WARNING/2 (i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

of the design matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. Next, we define a weight function

System Message: WARNING/2 (w(e) = \psi(e)/e)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and let

System Message: WARNING/2 (w_i = w(e_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

:

System Message: WARNING/2 (\sum_i w_i e_i X_i = 0 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

This system of equations is equivalent to solving a weighted ordinary least squares problem, minimizing

System Message: WARNING/2 (\chi^2 = \sum_i w_i e_i^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The weights however, depend on the residuals

System Message: WARNING/2 (e_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, which depend on the coefficients

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, which depend on the weights. Therefore, an iterative solution is used, called Iteratively Reweighted Least Squares (IRLS).

1. Compute initial estimates of the coefficients

   System Message: WARNING/2 (c^{(0)})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   using ordinary least squares
2. For iteration

   System Message: WARNING/2 (k)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   , form the residuals

   System Message: WARNING/2 (e_i^{(k)} = (y_i - X_i c^{(k-1)})/(t \sigma^{(k)} \sqrt{1 - h_i}))

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   , where

   System Message: WARNING/2 (t)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   is a tuning constant depending on the choice of

   System Message: WARNING/2 (\psi)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   , and

   System Message: WARNING/2 (h_i)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   are the statistical leverages (diagonal elements of the matrix

   System Message: WARNING/2 (X (X^T X)^{-1} X^T)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ). Including

   System Message: WARNING/2 (t)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   and

   System Message: WARNING/2 (h_i)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   in the residual calculation has been shown to improve the convergence of the method. The residual standard deviation is approximated as

   System Message: WARNING/2 (\sigma^{(k)} = MAD / 0.6745)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   , where MAD is the Median-Absolute-Deviation of the

   System Message: WARNING/2 (n-p)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   largest residuals from the previous iteration.
3. Compute new weights

   System Message: WARNING/2 (w_i^{(k)} = \psi(e_i^{(k)})/e_i^{(k)})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
4. Compute new coefficients

   System Message: WARNING/2 (c^{(k)})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   by solving the weighted least squares problem with weights

   System Message: WARNING/2 (w_i^{(k)})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
5. Steps 2 through 4 are iterated until the coefficients converge or until some maximum iteration limit is reached. Coefficients are tested for convergence using the critera:

System Message: WARNING/2 (|c_i^{(k)} - c_i^{(k-1)}| \le \epsilon \times \hbox{max}(|c_i^{(k)}|, |c_i^{(k-1)}|) )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for all

System Message: WARNING/2 (0 \le i < p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (\epsilon)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a small tolerance factor.

The key to this method lies in selecting the function

System Message: WARNING/2 (\psi(e_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

to assign smaller weights to large residuals, and larger weights to smaller residuals. As the iteration proceeds, outliers are assigned smaller and smaller weights, eventually having very little or no effect on the fitted model.

gsl_multifit_robust_workspace

This workspace is used for robust least squares fitting.

gsl_multifit_robust_workspace * gsl_multifit_robust_alloc(const gsl_multifit_robust_type * T, const size_t n, const size_t p)

This function allocates a workspace for fitting a model to n observations using p parameters. The size of the workspace is

System Message: WARNING/2 (O(np + p^2))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The type T specifies the function

System Message: WARNING/2 (\psi)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and can be selected from the following choices.

gsl_multifit_robust_type

gsl_multifit_robust_default

This specifies the gsl_multifit_robust_bisquare type (see below) and is a good general purpose choice for robust regression.

gsl_multifit_robust_bisquare

This is Tukey’s biweight (bisquare) function and is a good general purpose choice for robust regression. The weight function is given by

System Message: WARNING/2 (w(e) = \left\{ \begin{array}{cc} (1 - e^2)^2, & |e| \le 1 \\ 0, & |e| > 1 \end{array} \right.)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 4.685)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_cauchy

This is Cauchy’s function, also known as the Lorentzian function. This function does not guarantee a unique solution, meaning different choices of the coefficient vector c could minimize the objective function. Therefore this option should be used with care. The weight function is given by

System Message: WARNING/2 (w(e) = {1 \over 1 + e^2} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 2.385)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_fair

This is the fair

System Message: WARNING/2 (\rho)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

function, which guarantees a unique solution and has continuous derivatives to three orders. The weight function is given by

System Message: WARNING/2 (w(e) = {1 \over 1 + |e|} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 1.400)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_huber

This specifies Huber’s

System Message: WARNING/2 (\rho)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

function, which is a parabola in the vicinity of zero and increases linearly for a given threshold

System Message: WARNING/2 (|e| > t)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. This function is also considered an excellent general purpose robust estimator, however, occasional difficulties can be encountered due to the discontinuous first derivative of the

System Message: WARNING/2 (\psi)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

function. The weight function is given by

System Message: WARNING/2 (w(e) = \left\{ \begin{array}{cc} 1, & |e| \le 1 \\ {1 \over |e|}, & |e| > 1 \end{array} \right.)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 1.345)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_ols

This specifies the ordinary least squares solution, which can be useful for quickly checking the difference between the various robust and OLS solutions. The weight function is given by

System Message: WARNING/2 (w(e) = 1 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_welsch

This specifies the Welsch function which can perform well in cases where the residuals have an exponential distribution. The weight function is given by

System Message: WARNING/2 (w(e) = \exp{(-e^2)} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the default tuning constant is

System Message: WARNING/2 (t = 2.985)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

void gsl_multifit_robust_free(gsl_multifit_robust_workspace * w)

This function frees the memory associated with the workspace w.

const char * gsl_multifit_robust_name(const gsl_multifit_robust_workspace * w)

This function returns the name of the robust type T specified to gsl_multifit_robust_alloc().

int gsl_multifit_robust_tune(const double tune, gsl_multifit_robust_workspace * w)

This function sets the tuning constant

System Message: WARNING/2 (t)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

used to adjust the residuals at each iteration to tune. Decreasing the tuning constant increases the downweight assigned to large residuals, while increasing the tuning constant decreases the downweight assigned to large residuals.

int gsl_multifit_robust_maxiter(const size_t maxiter, gsl_multifit_robust_workspace * w)

This function sets the maximum number of iterations in the iteratively reweighted least squares algorithm to maxiter. By default, this value is set to 100 by gsl_multifit_robust_alloc().

int gsl_multifit_robust_weights(const gsl_vector * r, gsl_vector * wts, gsl_multifit_robust_workspace * w)

This function assigns weights to the vector wts using the residual vector r and previously specified weighting function. The output weights are given by

System Message: WARNING/2 (wts_i = w(r_i / (t \sigma)))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where the weighting functions

System Message: WARNING/2 (w)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are detailed in gsl_multifit_robust_alloc().

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is an estimate of the residual standard deviation based on the Median-Absolute-Deviation and

System Message: WARNING/2 (t)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the tuning constant. This function is useful if the user wishes to implement their own robust regression rather than using the supplied gsl_multifit_robust() routine below.

int gsl_multifit_robust(const gsl_matrix * X, const gsl_vector * y, gsl_vector * c, gsl_matrix * cov, gsl_multifit_robust_workspace * w)

This function computes the best-fit parameters c of the model

System Message: WARNING/2 (y = X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y and the matrix of predictor variables X, attemping to reduce the influence of outliers using the algorithm outlined above. The

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

variance-covariance matrix of the model parameters cov is estimated as

System Message: WARNING/2 (\sigma^2 (X^T X)^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is an approximation of the residual standard deviation using the theory of robust regression. Special care must be taken when estimating

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and other statistics such as

System Message: WARNING/2 (R^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and so these are computed internally and are available by calling the function gsl_multifit_robust_statistics().

If the coefficients do not converge within the maximum iteration limit, the function returns GSL_EMAXITER. In this case, the current estimates of the coefficients and covariance matrix are returned in c and cov and the internal fit statistics are computed with these estimates.

int gsl_multifit_robust_est(const gsl_vector * x, const gsl_vector * c, const gsl_matrix * cov, double * y, double * y_err)

This function uses the best-fit robust regression coefficients c and their covariance matrix cov to compute the fitted function value y and its standard deviation y_err for the model

System Message: WARNING/2 (y = x \cdot c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

at the point x.

int gsl_multifit_robust_residuals(const gsl_matrix * X, const gsl_vector * y, const gsl_vector * c, gsl_vector * r, gsl_multifit_robust_workspace * w)

This function computes the vector of studentized residuals

System Message: WARNING/2 (r_i = {y_i - (X c)_i \over \sigma \sqrt{1 - h_i}})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for the observations y, coefficients c and matrix of predictor variables X. The routine gsl_multifit_robust() must first be called to compute the statisical leverages

System Message: WARNING/2 (h_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

of the matrix X and residual standard deviation estimate

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multifit_robust_stats gsl_multifit_robust_statistics(const gsl_multifit_robust_workspace * w)

This function returns a structure containing relevant statistics from a robust regression. The function gsl_multifit_robust() must be called first to perform the regression and calculate these statistics. The returned gsl_multifit_robust_stats structure contains the following fields.

gsl_multifit_robust_stats

double sigma_ols

This contains the standard deviation of the residuals as computed from ordinary least squares (OLS).

double sigma_mad

This contains an estimate of the standard deviation of the final residuals using the Median-Absolute-Deviation statistic

double sigma_rob

This contains an estimate of the standard deviation of the final residuals from the theory of robust regression (see Street et al, 1988).

double sigma

This contains an estimate of the standard deviation of the final residuals by attemping to reconcile sigma_rob and sigma_ols in a reasonable way.

double Rsq

This contains the

System Message: WARNING/2 (R^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

coefficient of determination statistic using the estimate sigma.

double adj_Rsq

This contains the adjusted

System Message: WARNING/2 (R^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

coefficient of determination statistic using the estimate sigma.

double rmse

This contains the root mean squared error of the final residuals

double sse

This contains the residual sum of squares taking into account the robust covariance matrix.

size_t dof

This contains the number of degrees of freedom

System Message: WARNING/2 (n - p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

size_t numit

Upon successful convergence, this contains the number of iterations performed

gsl_vector * weights

This contains the final weight vector of length n

gsl_vector * r

This contains the final residual vector of length n,

System Message: WARNING/2 (r = y - X c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

Large dense linear systems

This module is concerned with solving large dense least squares systems

System Message: WARNING/2 (X c = y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

has

System Message: WARNING/2 (n >> p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

(ie: many more rows than columns). This type of matrix is called a “tall skinny” matrix, and for some applications, it may not be possible to fit the entire matrix in memory at once to use the standard SVD approach. Therefore, the algorithms in this module are designed to allow the user to construct smaller blocks of the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and accumulate those blocks into the larger system one at a time. The algorithms in this module never need to store the entire matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

in memory. The large linear least squares routines support data weights and Tikhonov regularization, and are designed to minimize the residual

System Message: WARNING/2 (\chi^2 = || y - Xc ||_W^2 + \lambda^2 || L c ||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

observation vector,

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

design matrix,

System Message: WARNING/2 (c)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

solution vector,

System Message: WARNING/2 (W = \diag(w_1,...,w_n))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the data weighting matrix,

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is an

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

regularization matrix,

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a regularization parameter, and

System Message: WARNING/2 (||r||_W^2 = r^T W r)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. In the discussion which follows, we will assume that the system has been converted into Tikhonov standard form,

System Message: WARNING/2 (\chi^2 = || \tilde{y} - \tilde{X} \tilde{c} ||^2 + \lambda^2 || \tilde{c} ||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and we will drop the tilde characters from the various parameters. For a discussion of the transformation to standard form, see Regularized regression.

The basic idea is to partition the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and observation vector

System Message: WARNING/2 (y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

as

System Message: WARNING/2 (\left( \begin{array}{c} X_1 \\ X_2 \\ X_3 \\ \vdots \\ X_k \end{array} \right) c = \left( \begin{array}{c} y_1 \\ y_2 \\ y_3 \\ \vdots \\ y_k \end{array} \right))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

into

System Message: WARNING/2 (k)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

blocks, where each block (

System Message: WARNING/2 (X_i,y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) may have any number of rows, but each

System Message: WARNING/2 (X_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

has

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

columns. The sections below describe the methods available for solving this partitioned system. The functions are declared in the header file gsl_multilarge.h.

Normal Equations Approach

The normal equations approach to the large linear least squares problem described above is popular due to its speed and simplicity. Since the normal equations solution to the problem is given by

System Message: WARNING/2 (c = \left( X^T X + \lambda^2 I \right)^{-1} X^T y )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

only the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-1 vector

System Message: WARNING/2 (X^T y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

need to be stored. Using the partition scheme described above, these are given by

System Message: WARNING/2 (X^T X &= \sum_i X_i^T X_i \\ X^T y &= \sum_i X_i^T y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

Since the matrix

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is symmetric, only half of it needs to be calculated. Once all of the blocks

System Message: WARNING/2 ((X_i,y_i))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

have been accumulated into the final

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (X^T y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the system can be solved with a Cholesky factorization of the

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix. The

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix is first transformed via a diagonal scaling transformation to attempt to reduce its condition number as much as possible to recover a more accurate solution vector. The normal equations approach is the fastest method for solving the large least squares problem, and is accurate for well-conditioned matrices

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. However, for ill-conditioned matrices, as is often the case for large systems, this method can suffer from numerical instabilities (see Trefethen and Bau, 1997). The number of operations for this method is

System Message: WARNING/2 (O(np^2 + {1 \over 3}p^3))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Tall Skinny QR (TSQR) Approach

An algorithm which has better numerical stability for ill-conditioned problems is known as the Tall Skinny QR (TSQR) method. This method is based on computing the thin QR decomposition of the least squares matrix

System Message: WARNING/2 (X = Q R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, where

System Message: WARNING/2 (Q)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is an

System Message: WARNING/2 (n)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix with orthogonal columns, and

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is a

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

upper triangular matrix. Once these factors are calculated, the residual becomes

System Message: WARNING/2 (\chi^2 = || Q^T y - R c ||^2 + \lambda^2 || c ||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which can be written as the matrix equation

System Message: WARNING/2 (\left( \begin{array}{c} R \\ \lambda I \end{array} \right) c = \left( \begin{array}{c} Q^T y \\ 0 \end{array} \right))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

The matrix on the left hand side is now a much smaller

System Message: WARNING/2 (2p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix which can be solved with a standard SVD approach. The

System Message: WARNING/2 (Q)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix is just as large as the original matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, however it does not need to be explicitly constructed. The TSQR algorithm computes only the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-1 vector

System Message: WARNING/2 (Q^T y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and updates these quantities as new blocks are added to the system. Each time a new block of rows (

System Message: WARNING/2 (X_i,y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) is added, the algorithm performs a QR decomposition of the matrix

System Message: WARNING/2 (\left( \begin{array}{c} R_{i-1} \\ X_i \end{array} \right))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (R_{i-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the upper triangular

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

factor for the matrix

System Message: WARNING/2 (\left( \begin{array}{c} X_1 \\ \vdots \\ X_{i-1} \end{array} \right))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

This QR decomposition is done efficiently taking into account the sparse structure of

System Message: WARNING/2 (R_{i-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. See Demmel et al, 2008 for more details on how this is accomplished. The number of operations for this method is

System Message: WARNING/2 (O(2np^2 - {2 \over 3}p^3))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Large Dense Linear Systems Solution Steps

The typical steps required to solve large regularized linear least squares problems are as follows:

1. Choose the regularization matrix

   System Message: WARNING/2 (L)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
2. Construct a block of rows of the least squares matrix, right hand side vector, and weight vector (

   System Message: WARNING/2 (X_i)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (y_i)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (w_i)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ).
3. Transform the block to standard form (

   System Message: WARNING/2 (\tilde{X_i})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (\tilde{y_i})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ). This step can be skipped if

   System Message: WARNING/2 (L = I)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   and

   System Message: WARNING/2 (W = I)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
4. Accumulate the standard form block (

   System Message: WARNING/2 (\tilde{X_i})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ,

   System Message: WARNING/2 (\tilde{y_i})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   ) into the system.
5. Repeat steps 2-4 until the entire matrix and right hand side vector have been accumulated.
6. Determine an appropriate regularization parameter

   System Message: WARNING/2 (\lambda)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   (using for example L-curve analysis).
7. Solve the standard form system using the chosen

   System Message: WARNING/2 (\lambda)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .
8. Backtransform the standard form solution

   System Message: WARNING/2 (\tilde{c})

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   to recover the original solution vector

   System Message: WARNING/2 (c)

   latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.
   .

Large Dense Linear Least Squares Routines

gsl_multilarge_linear_workspace

This workspace contains parameters for solving large linear least squares problems.

gsl_multilarge_linear_workspace * gsl_multilarge_linear_alloc(const gsl_multilarge_linear_type * T, const size_t p)

This function allocates a workspace for solving large linear least squares systems. The least squares matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

has p columns, but may have any number of rows.

gsl_multilarge_linear_type

The parameter T specifies the method to be used for solving the large least squares system and may be selected from the following choices

gsl_multilarge_linear_normal

This specifies the normal equations approach for solving the least squares system. This method is suitable in cases where performance is critical and it is known that the least squares matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is well conditioned. The size of this workspace is

System Message: WARNING/2 (O(p^2))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

gsl_multilarge_linear_tsqr

This specifies the sequential Tall Skinny QR (TSQR) approach for solving the least squares system. This method is a good general purpose choice for large systems, but requires about twice as many operations as the normal equations method for

System Message: WARNING/2 (n >> p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The size of this workspace is

System Message: WARNING/2 (O(p^2))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

void gsl_multilarge_linear_free(gsl_multilarge_linear_workspace * w)

This function frees the memory associated with the workspace w.

const char * gsl_multilarge_linear_name(gsl_multilarge_linear_workspace * w)

This function returns a string pointer to the name of the multilarge solver.

int gsl_multilarge_linear_reset(gsl_multilarge_linear_workspace * w)

This function resets the workspace w so it can begin to accumulate a new least squares system.

int gsl_multilarge_linear_stdform1(const gsl_vector * L, const gsl_matrix * X, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multilarge_linear_workspace * work)

int gsl_multilarge_linear_wstdform1(const gsl_vector * L, const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multilarge_linear_workspace * work)

These functions define a regularization matrix

System Message: WARNING/2 (L = \diag(l_0,l_1,...,l_{p-1}))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The diagonal matrix element

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is provided by the

System Message: WARNING/2 (i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-th element of the input vector L. The block (X, y) is converted to standard form and the parameters (

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) are stored in Xs and ys on output. Xs and ys have the same dimensions as X and y. Optional data weights may be supplied in the vector w. In order to apply this transformation,

System Message: WARNING/2 (L^{-1})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

must exist and so none of the

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

may be zero. After the standard form system has been solved, use gsl_multilarge_linear_genform1() to recover the original solution vector. It is allowed to have X = Xs and y = ys for an in-place transform.

int gsl_multilarge_linear_L_decomp(gsl_matrix * L, gsl_vector * tau)

This function calculates the QR decomposition of the

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

regularization matrix L. L must have

System Message: WARNING/2 (m \ge p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. On output, the Householder scalars are stored in the vector tau of size

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. These outputs will be used by gsl_multilarge_linear_wstdform2() to complete the transformation to standard form.

int gsl_multilarge_linear_stdform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multilarge_linear_workspace * work)

int gsl_multilarge_linear_wstdform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_matrix * X, const gsl_vector * w, const gsl_vector * y, gsl_matrix * Xs, gsl_vector * ys, gsl_multilarge_linear_workspace * work)

These functions convert a block of rows (X, y, w) to standard form (

System Message: WARNING/2 (\tilde{X})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

,

System Message: WARNING/2 (\tilde{y})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) which are stored in Xs and ys respectively. X, y, and w must all have the same number of rows. The

System Message: WARNING/2 (m)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

regularization matrix L is specified by the inputs LQR and Ltau, which are outputs from gsl_multilarge_linear_L_decomp(). Xs and ys have the same dimensions as X and y. After the standard form system has been solved, use gsl_multilarge_linear_genform2() to recover the original solution vector. Optional data weights may be supplied in the vector w, where

System Message: WARNING/2 (W = \diag(w))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_multilarge_linear_accumulate(gsl_matrix * X, gsl_vector * y, gsl_multilarge_linear_workspace * w)

This function accumulates the standard form block (

System Message: WARNING/2 (X,y)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) into the current least squares system. X and y have the same number of rows, which can be arbitrary. X must have

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

columns. For the TSQR method, X and y are destroyed on output. For the normal equations method, they are both unchanged.

int gsl_multilarge_linear_solve(const double lambda, gsl_vector * c, double * rnorm, double * snorm, gsl_multilarge_linear_workspace * w)

After all blocks (

System Message: WARNING/2 (X_i,y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

) have been accumulated into the large least squares system, this function will compute the solution vector which is stored in c on output. The regularization parameter

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is provided in lambda. On output, rnorm contains the residual norm

System Message: WARNING/2 (||y - X c||_W)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and snorm contains the solution norm

System Message: WARNING/2 (||L c||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_multilarge_linear_genform1(const gsl_vector * L, const gsl_vector * cs, gsl_vector * c, gsl_multilarge_linear_workspace * work)

After a regularized system has been solved with

System Message: WARNING/2 (L = \diag(\l_0,\l_1,...,\l_{p-1}))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, this function backtransforms the standard form solution vector cs to recover the solution vector of the original problem c. The diagonal matrix elements

System Message: WARNING/2 (l_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are provided in the vector L. It is allowed to have c = cs for an in-place transform.

int gsl_multilarge_linear_genform2(const gsl_matrix * LQR, const gsl_vector * Ltau, const gsl_vector * cs, gsl_vector * c, gsl_multilarge_linear_workspace * work)

After a regularized system has been solved with a regularization matrix

System Message: WARNING/2 (L)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, specified by (LQR, Ltau), this function backtransforms the standard form solution cs to recover the solution vector of the original problem, which is stored in c, of length

System Message: WARNING/2 (p)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

int gsl_multilarge_linear_lcurve(gsl_vector * reg_param, gsl_vector * rho, gsl_vector * eta, gsl_multilarge_linear_workspace * work)

This function computes the L-curve for a large least squares system after it has been fully accumulated into the workspace work. The output vectors reg_param, rho, and eta must all be the same size, and will contain the regularization parameters

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, residual norms

System Message: WARNING/2 (||y - X c_i||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, and solution norms

System Message: WARNING/2 (|| L c_i ||)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which compose the L-curve, where

System Message: WARNING/2 (c_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the regularized solution vector corresponding to

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The user may determine the number of points on the L-curve by adjusting the size of these input arrays. For the TSQR method, the regularization parameters

System Message: WARNING/2 (\lambda_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are estimated from the singular values of the triangular

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

factor. For the normal equations method, they are estimated from the eigenvalues of the

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix.

int gsl_multilarge_linear_rcond(double * rcond, gsl_multilarge_linear_workspace * work)

This function computes the reciprocal condition number, stored in rcond, of the least squares matrix after it has been accumulated into the workspace work. For the TSQR algorithm, this is accomplished by calculating the SVD of the

System Message: WARNING/2 (R)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

factor, which has the same singular values as the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. For the normal equations method, this is done by computing the eigenvalues of

System Message: WARNING/2 (X^T X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, which could be inaccurate for ill-conditioned matrices

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Troubleshooting

When using models based on polynomials, care should be taken when constructing the design matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. If the

System Message: WARNING/2 (x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

values are large, then the matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

could be ill-conditioned since its columns are powers of

System Message: WARNING/2 (x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, leading to unstable least-squares solutions. In this case it can often help to center and scale the

System Message: WARNING/2 (x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

values using the mean and standard deviation:

System Message: WARNING/2 (x' = {x - \mu(x) \over \sigma(x)} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and then construct the

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

matrix using the transformed values

System Message: WARNING/2 (x')

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

Examples

The example programs in this section demonstrate the various linear regression methods.

Simple Linear Regression Example

The following program computes a least squares straight-line fit to a simple dataset, and outputs the best-fit line and its associated one standard-deviation error bars.

#include <stdio.h>
#include <gsl/gsl_fit.h>

int
main (void)
{
  int i, n = 4;
  double x[4] = { 1970, 1980, 1990, 2000 };
  double y[4] = {   12,   11,   14,   13 };
  double w[4] = {  0.1,  0.2,  0.3,  0.4 };

  double c0, c1, cov00, cov01, cov11, chisq;

  gsl_fit_wlinear (x, 1, w, 1, y, 1, n,
                   &c0, &c1, &cov00, &cov01, &cov11,
                   &chisq);

  printf ("# best fit: Y = %g + %g X\n", c0, c1);
  printf ("# covariance matrix:\n");
  printf ("# [ %g, %g\n#   %g, %g]\n",
          cov00, cov01, cov01, cov11);
  printf ("# chisq = %g\n", chisq);

  for (i = 0; i < n; i++)
    printf ("data: %g %g %g\n",
                   x[i], y[i], 1/sqrt(w[i]));

  printf ("\n");

  for (i = -30; i < 130; i++)
    {
      double xf = x[0] + (i/100.0) * (x[n-1] - x[0]);
      double yf, yf_err;

      gsl_fit_linear_est (xf,
                          c0, c1,
                          cov00, cov01, cov11,
                          &yf, &yf_err);

      printf ("fit: %g %g\n", xf, yf);
      printf ("hi : %g %g\n", xf, yf + yf_err);
      printf ("lo : %g %g\n", xf, yf - yf_err);
    }
  return 0;
}

The following commands extract the data from the output of the program and display it using the GNU plotutils “graph” utility:

$ ./demo > tmp
$ more tmp
# best fit: Y = -106.6 + 0.06 X
# covariance matrix:
# [ 39602, -19.9
#   -19.9, 0.01]
# chisq = 0.8

$ for n in data fit hi lo ;
   do
     grep "^$n" tmp | cut -d: -f2 > $n ;
   done
$ graph -T X -X x -Y y -y 0 20 -m 0 -S 2 -Ie data
     -S 0 -I a -m 1 fit -m 2 hi -m 2 lo

The result is shown in Fig. 29.

Fig. 29 Straight line fit with 1-

System Message: WARNING/2 (\sigma)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

error bars

Multi-parameter Linear Regression Example

The following program performs a quadratic fit

System Message: WARNING/2 (y = c_0 + c_1 x + c_2 x^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

to a weighted dataset using the generalised linear fitting function gsl_multifit_wlinear(). The model matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

for a quadratic fit is given by,

System Message: WARNING/2 (X = \left( \begin{array}{ccc} 1 & x_0 & x_0^2 \\ 1 & x_1 & x_1^2 \\ 1 & x_2 & x_2^2 \\ \dots & \dots & \dots \end{array} \right))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where the column of ones corresponds to the constant term

System Message: WARNING/2 (c_0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The two remaining columns corresponds to the terms

System Message: WARNING/2 (c_1 x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (c_2 x^2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

The program reads n lines of data in the format (x, y, err) where err is the error (standard deviation) in the value y.

#include <stdio.h>
#include <gsl/gsl_multifit.h>

int
main (int argc, char **argv)
{
  int i, n;
  double xi, yi, ei, chisq;
  gsl_matrix *X, *cov;
  gsl_vector *y, *w, *c;

  if (argc != 2)
    {
      fprintf (stderr,"usage: fit n < data\n");
      exit (-1);
    }

  n = atoi (argv[1]);

  X = gsl_matrix_alloc (n, 3);
  y = gsl_vector_alloc (n);
  w = gsl_vector_alloc (n);

  c = gsl_vector_alloc (3);
  cov = gsl_matrix_alloc (3, 3);

  for (i = 0; i < n; i++)
    {
      int count = fscanf (stdin, "%lg %lg %lg",
                          &xi, &yi, &ei);

      if (count != 3)
        {
          fprintf (stderr, "error reading file\n");
          exit (-1);
        }

      printf ("%g %g +/- %g\n", xi, yi, ei);

      gsl_matrix_set (X, i, 0, 1.0);
      gsl_matrix_set (X, i, 1, xi);
      gsl_matrix_set (X, i, 2, xi*xi);

      gsl_vector_set (y, i, yi);
      gsl_vector_set (w, i, 1.0/(ei*ei));
    }

  {
    gsl_multifit_linear_workspace * work
      = gsl_multifit_linear_alloc (n, 3);
    gsl_multifit_wlinear (X, w, y, c, cov,
                          &chisq, work);
    gsl_multifit_linear_free (work);
  }

#define C(i) (gsl_vector_get(c,(i)))
#define COV(i,j) (gsl_matrix_get(cov,(i),(j)))

  {
    printf ("# best fit: Y = %g + %g X + %g X^2\n",
            C(0), C(1), C(2));

    printf ("# covariance matrix:\n");
    printf ("[ %+.5e, %+.5e, %+.5e  \n",
               COV(0,0), COV(0,1), COV(0,2));
    printf ("  %+.5e, %+.5e, %+.5e  \n",
               COV(1,0), COV(1,1), COV(1,2));
    printf ("  %+.5e, %+.5e, %+.5e ]\n",
               COV(2,0), COV(2,1), COV(2,2));
    printf ("# chisq = %g\n", chisq);
  }

  gsl_matrix_free (X);
  gsl_vector_free (y);
  gsl_vector_free (w);
  gsl_vector_free (c);
  gsl_matrix_free (cov);

  return 0;
}

A suitable set of data for fitting can be generated using the following program. It outputs a set of points with gaussian errors from the curve

System Message: WARNING/2 (y = e^x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

in the region

System Message: WARNING/2 (0 < x < 2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

.

#include <stdio.h>
#include <math.h>
#include <gsl/gsl_randist.h>

int
main (void)
{
  double x;
  const gsl_rng_type * T;
  gsl_rng * r;

  gsl_rng_env_setup ();

  T = gsl_rng_default;
  r = gsl_rng_alloc (T);

  for (x = 0.1; x < 2; x+= 0.1)
    {
      double y0 = exp (x);
      double sigma = 0.1 * y0;
      double dy = gsl_ran_gaussian (r, sigma);

      printf ("%g %g %g\n", x, y0 + dy, sigma);
    }

  gsl_rng_free(r);

  return 0;
}

The data can be prepared by running the resulting executable program:

$ GSL_RNG_TYPE=mt19937_1999 ./generate > exp.dat
$ more exp.dat
0.1 0.97935 0.110517
0.2 1.3359 0.12214
0.3 1.52573 0.134986
0.4 1.60318 0.149182
0.5 1.81731 0.164872
0.6 1.92475 0.182212
....

To fit the data use the previous program, with the number of data points given as the first argument. In this case there are 19 data points:

$ ./fit 19 < exp.dat
0.1 0.97935 +/- 0.110517
0.2 1.3359 +/- 0.12214
...
# best fit: Y = 1.02318 + 0.956201 X + 0.876796 X^2
# covariance matrix:
[ +1.25612e-02, -3.64387e-02, +1.94389e-02
  -3.64387e-02, +1.42339e-01, -8.48761e-02
  +1.94389e-02, -8.48761e-02, +5.60243e-02 ]
# chisq = 23.0987

The parameters of the quadratic fit match the coefficients of the expansion of

System Message: WARNING/2 (e^x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, taking into account the errors on the parameters and the

System Message: WARNING/2 (O(x^3))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

difference between the exponential and quadratic functions for the larger values of

System Message: WARNING/2 (x)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The errors on the parameters are given by the square-root of the corresponding diagonal elements of the covariance matrix. The chi-squared per degree of freedom is 1.4, indicating a reasonable fit to the data.

Fig. 30 shows the resulting fit.

Fig. 30 Weighted fit example with error bars

Regularized Linear Regression Example 1

The next program demonstrates the difference between ordinary and regularized least squares when the design matrix is near-singular. In this program, we generate two random normally distributed variables

System Message: WARNING/2 (u)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (v)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, with

System Message: WARNING/2 (v = u + noise)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

so that

System Message: WARNING/2 (u)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and

System Message: WARNING/2 (v)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

are nearly colinear. We then set a third dependent variable

System Message: WARNING/2 (y = u + v + noise)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and solve for the coefficients

System Message: WARNING/2 (c_1,c_2)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

of the model

System Message: WARNING/2 (Y(c_1,c_2) = c_1 u + c_2 v)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. Since

System Message: WARNING/2 (u \approx v)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the design matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is nearly singular, leading to unstable ordinary least squares solutions.

Here is the program output:

matrix condition number = 1.025113e+04

=== Unregularized fit ===
best fit: y = -43.6588 u + 45.6636 v
residual norm = 31.6248
solution norm = 63.1764
chisq/dof = 1.00213

=== Regularized fit (L-curve) ===
optimal lambda: 4.51103
best fit: y = 1.00113 u + 1.0032 v
residual norm = 31.6547
solution norm = 1.41728
chisq/dof = 1.04499

=== Regularized fit (GCV) ===
optimal lambda: 0.0232029
best fit: y = -19.8367 u + 21.8417 v
residual norm = 31.6332
solution norm = 29.5051
chisq/dof = 1.00314

We see that the ordinary least squares solution is completely wrong, while the L-curve regularized method with the optimal

System Message: WARNING/2 (\lambda = 4.51103)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

finds the correct solution

System Message: WARNING/2 (c_1 \approx c_2 \approx 1)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The GCV regularized method finds a regularization parameter

System Message: WARNING/2 (\lambda = 0.0232029)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which is too small to give an accurate solution, although it performs better than OLS. The L-curve and its computed corner, as well as the GCV curve and its minimum are plotted in Fig. 31.

Fig. 31 L-curve and GCV curve for example program.

The program is given below.

#include <gsl/gsl_math.h>
#include <gsl/gsl_vector.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_rng.h>
#include <gsl/gsl_randist.h>
#include <gsl/gsl_multifit.h>

int
main()
{
  const size_t n = 1000; /* number of observations */
  const size_t p = 2;    /* number of model parameters */
  size_t i;
  gsl_rng *r = gsl_rng_alloc(gsl_rng_default);
  gsl_matrix *X = gsl_matrix_alloc(n, p);
  gsl_vector *y = gsl_vector_alloc(n);

  for (i = 0; i < n; ++i)
    {
      /* generate first random variable u */
      double ui = 5.0 * gsl_ran_gaussian(r, 1.0);

      /* set v = u + noise */
      double vi = ui + gsl_ran_gaussian(r, 0.001);

      /* set y = u + v + noise */
      double yi = ui + vi + gsl_ran_gaussian(r, 1.0);

      /* since u =~ v, the matrix X is ill-conditioned */
      gsl_matrix_set(X, i, 0, ui);
      gsl_matrix_set(X, i, 1, vi);

      /* rhs vector */
      gsl_vector_set(y, i, yi);
    }

  {
    const size_t npoints = 200;                   /* number of points on L-curve and GCV curve */
    gsl_multifit_linear_workspace *w =
      gsl_multifit_linear_alloc(n, p);
    gsl_vector *c = gsl_vector_alloc(p);          /* OLS solution */
    gsl_vector *c_lcurve = gsl_vector_alloc(p);   /* regularized solution (L-curve) */
    gsl_vector *c_gcv = gsl_vector_alloc(p);      /* regularized solution (GCV) */
    gsl_vector *reg_param = gsl_vector_alloc(npoints);
    gsl_vector *rho = gsl_vector_alloc(npoints);  /* residual norms */
    gsl_vector *eta = gsl_vector_alloc(npoints);  /* solution norms */
    gsl_vector *G = gsl_vector_alloc(npoints);    /* GCV function values */
    double lambda_l;                              /* optimal regularization parameter (L-curve) */
    double lambda_gcv;                            /* optimal regularization parameter (GCV) */
    double G_gcv;                                 /* G(lambda_gcv) */
    size_t reg_idx;                               /* index of optimal lambda */
    double rcond;                                 /* reciprocal condition number of X */
    double chisq, rnorm, snorm;

    /* compute SVD of X */
    gsl_multifit_linear_svd(X, w);

    rcond = gsl_multifit_linear_rcond(w);
    fprintf(stderr, "matrix condition number = %e\n\n", 1.0 / rcond);

    /* unregularized (standard) least squares fit, lambda = 0 */
    gsl_multifit_linear_solve(0.0, X, y, c, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0);

    fprintf(stderr, "=== Unregularized fit ===\n");
    fprintf(stderr, "best fit: y = %g u + %g v\n",
      gsl_vector_get(c, 0), gsl_vector_get(c, 1));
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* calculate L-curve and find its corner */
    gsl_multifit_linear_lcurve(y, reg_param, rho, eta, w);
    gsl_multifit_linear_lcorner(rho, eta, &reg_idx);

    /* store optimal regularization parameter */
    lambda_l = gsl_vector_get(reg_param, reg_idx);

    /* regularize with lambda_l */
    gsl_multifit_linear_solve(lambda_l, X, y, c_lcurve, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0) + pow(lambda_l * snorm, 2.0);

    fprintf(stderr, "\n=== Regularized fit (L-curve) ===\n");
    fprintf(stderr, "optimal lambda: %g\n", lambda_l);
    fprintf(stderr, "best fit: y = %g u + %g v\n",
            gsl_vector_get(c_lcurve, 0), gsl_vector_get(c_lcurve, 1));
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* calculate GCV curve and find its minimum */
    gsl_multifit_linear_gcv(y, reg_param, G, &lambda_gcv, &G_gcv, w);

    /* regularize with lambda_gcv */
    gsl_multifit_linear_solve(lambda_gcv, X, y, c_gcv, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0) + pow(lambda_gcv * snorm, 2.0);

    fprintf(stderr, "\n=== Regularized fit (GCV) ===\n");
    fprintf(stderr, "optimal lambda: %g\n", lambda_gcv);
    fprintf(stderr, "best fit: y = %g u + %g v\n",
            gsl_vector_get(c_gcv, 0), gsl_vector_get(c_gcv, 1));
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* output L-curve and GCV curve */
    for (i = 0; i < npoints; ++i)
      {
        printf("%e %e %e %e\n",
               gsl_vector_get(reg_param, i),
               gsl_vector_get(rho, i),
               gsl_vector_get(eta, i),
               gsl_vector_get(G, i));
      }

    /* output L-curve corner point */
    printf("\n\n%f %f\n",
           gsl_vector_get(rho, reg_idx),
           gsl_vector_get(eta, reg_idx));

    /* output GCV curve corner minimum */
    printf("\n\n%e %e\n",
           lambda_gcv,
           G_gcv);

    gsl_multifit_linear_free(w);
    gsl_vector_free(c);
    gsl_vector_free(c_lcurve);
    gsl_vector_free(reg_param);
    gsl_vector_free(rho);
    gsl_vector_free(eta);
    gsl_vector_free(G);
  }

  gsl_rng_free(r);
  gsl_matrix_free(X);
  gsl_vector_free(y);

  return 0;
}

Regularized Linear Regression Example 2

The following example program minimizes the cost function

System Message: WARNING/2 (||y - X c||^2 + \lambda^2 ||x||^2 )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

where

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is the

System Message: WARNING/2 (10)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

-by-

System Message: WARNING/2 (8)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

Hilbert matrix whose entries are given by

System Message: WARNING/2 (X_{ij} = {1 \over i + j - 1} )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

and the right hand side vector is given by

System Message: WARNING/2 (y = [1,-1,1,-1,1,-1,1,-1,1,-1]^T)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. Solutions are computed for

System Message: WARNING/2 (\lambda = 0)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

(unregularized) as well as for optimal parameters

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

chosen by analyzing the L-curve and GCV curve.

Here is the program output:

matrix condition number = 3.565872e+09

=== Unregularized fit ===
residual norm = 2.15376
solution norm = 2.92217e+09
chisq/dof = 2.31934

=== Regularized fit (L-curve) ===
optimal lambda: 7.11407e-07
residual norm = 2.60386
solution norm = 424507
chisq/dof = 3.43565

=== Regularized fit (GCV) ===
optimal lambda: 1.72278
residual norm = 3.1375
solution norm = 0.139357
chisq/dof = 4.95076

Here we see the unregularized solution results in a large solution norm due to the ill-conditioned matrix. The L-curve solution finds a small value of

System Message: WARNING/2 (\lambda = 7.11e-7)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which still results in a badly conditioned system and a large solution norm. The GCV method finds a parameter

System Message: WARNING/2 (\lambda = 1.72)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

which results in a well-conditioned system and small solution norm.

The L-curve and its computed corner, as well as the GCV curve and its minimum are plotted in Fig. 32.

Fig. 32 L-curve and GCV curve for example program.

The program is given below.

#include <gsl/gsl_math.h>
#include <gsl/gsl_vector.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_multifit.h>
#include <gsl/gsl_blas.h>

static int
hilbert_matrix(gsl_matrix * m)
{
  const size_t N = m->size1;
  const size_t M = m->size2;
  size_t i, j;

  for (i = 0; i < N; i++)
    {
      for (j = 0; j < M; j++)
        {
          gsl_matrix_set(m, i, j, 1.0/(i+j+1.0));
        }
    }

  return GSL_SUCCESS;
}

int
main()
{
  const size_t n = 10; /* number of observations */
  const size_t p = 8;  /* number of model parameters */
  size_t i;
  gsl_matrix *X = gsl_matrix_alloc(n, p);
  gsl_vector *y = gsl_vector_alloc(n);

  /* construct Hilbert matrix and rhs vector */
  hilbert_matrix(X);

  {
    double val = 1.0;
    for (i = 0; i < n; ++i)
      {
        gsl_vector_set(y, i, val);
        val *= -1.0;
      }
  }

  {
    const size_t npoints = 200;                   /* number of points on L-curve and GCV curve */
    gsl_multifit_linear_workspace *w =
      gsl_multifit_linear_alloc(n, p);
    gsl_vector *c = gsl_vector_alloc(p);          /* OLS solution */
    gsl_vector *c_lcurve = gsl_vector_alloc(p);   /* regularized solution (L-curve) */
    gsl_vector *c_gcv = gsl_vector_alloc(p);      /* regularized solution (GCV) */
    gsl_vector *reg_param = gsl_vector_alloc(npoints);
    gsl_vector *rho = gsl_vector_alloc(npoints);  /* residual norms */
    gsl_vector *eta = gsl_vector_alloc(npoints);  /* solution norms */
    gsl_vector *G = gsl_vector_alloc(npoints);    /* GCV function values */
    double lambda_l;                              /* optimal regularization parameter (L-curve) */
    double lambda_gcv;                            /* optimal regularization parameter (GCV) */
    double G_gcv;                                 /* G(lambda_gcv) */
    size_t reg_idx;                               /* index of optimal lambda */
    double rcond;                                 /* reciprocal condition number of X */
    double chisq, rnorm, snorm;

    /* compute SVD of X */
    gsl_multifit_linear_svd(X, w);

    rcond = gsl_multifit_linear_rcond(w);
    fprintf(stderr, "matrix condition number = %e\n", 1.0 / rcond);

    /* unregularized (standard) least squares fit, lambda = 0 */
    gsl_multifit_linear_solve(0.0, X, y, c, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0);

    fprintf(stderr, "\n=== Unregularized fit ===\n");
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* calculate L-curve and find its corner */
    gsl_multifit_linear_lcurve(y, reg_param, rho, eta, w);
    gsl_multifit_linear_lcorner(rho, eta, &reg_idx);

    /* store optimal regularization parameter */
    lambda_l = gsl_vector_get(reg_param, reg_idx);

    /* regularize with lambda_l */
    gsl_multifit_linear_solve(lambda_l, X, y, c_lcurve, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0) + pow(lambda_l * snorm, 2.0);

    fprintf(stderr, "\n=== Regularized fit (L-curve) ===\n");
    fprintf(stderr, "optimal lambda: %g\n", lambda_l);
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* calculate GCV curve and find its minimum */
    gsl_multifit_linear_gcv(y, reg_param, G, &lambda_gcv, &G_gcv, w);

    /* regularize with lambda_gcv */
    gsl_multifit_linear_solve(lambda_gcv, X, y, c_gcv, &rnorm, &snorm, w);
    chisq = pow(rnorm, 2.0) + pow(lambda_gcv * snorm, 2.0);

    fprintf(stderr, "\n=== Regularized fit (GCV) ===\n");
    fprintf(stderr, "optimal lambda: %g\n", lambda_gcv);
    fprintf(stderr, "residual norm = %g\n", rnorm);
    fprintf(stderr, "solution norm = %g\n", snorm);
    fprintf(stderr, "chisq/dof = %g\n", chisq / (n - p));

    /* output L-curve and GCV curve */
    for (i = 0; i < npoints; ++i)
      {
        printf("%e %e %e %e\n",
               gsl_vector_get(reg_param, i),
               gsl_vector_get(rho, i),
               gsl_vector_get(eta, i),
               gsl_vector_get(G, i));
      }

    /* output L-curve corner point */
    printf("\n\n%f %f\n",
           gsl_vector_get(rho, reg_idx),
           gsl_vector_get(eta, reg_idx));

    /* output GCV curve corner minimum */
    printf("\n\n%e %e\n",
           lambda_gcv,
           G_gcv);

    gsl_multifit_linear_free(w);
    gsl_vector_free(c);
    gsl_vector_free(c_lcurve);
    gsl_vector_free(reg_param);
    gsl_vector_free(rho);
    gsl_vector_free(eta);
    gsl_vector_free(G);
  }

  gsl_matrix_free(X);
  gsl_vector_free(y);

  return 0;
}

Robust Linear Regression Example

The next program demonstrates the advantage of robust least squares on a dataset with outliers. The program generates linear

System Message: WARNING/2 ((x,y))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

data pairs on the line

System Message: WARNING/2 (y = 1.45 x + 3.88)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, adds some random noise, and inserts 3 outliers into the dataset. Both the robust and ordinary least squares (OLS) coefficients are computed for comparison.

#include <stdio.h>
#include <gsl/gsl_multifit.h>
#include <gsl/gsl_randist.h>

int
dofit(const gsl_multifit_robust_type *T,
      const gsl_matrix *X, const gsl_vector *y,
      gsl_vector *c, gsl_matrix *cov)
{
  int s;
  gsl_multifit_robust_workspace * work
    = gsl_multifit_robust_alloc (T, X->size1, X->size2);

  s = gsl_multifit_robust (X, y, c, cov, work);
  gsl_multifit_robust_free (work);

  return s;
}

int
main (int argc, char **argv)
{
  size_t i;
  size_t n;
  const size_t p = 2; /* linear fit */
  gsl_matrix *X, *cov;
  gsl_vector *x, *y, *c, *c_ols;
  const double a = 1.45; /* slope */
  const double b = 3.88; /* intercept */
  gsl_rng *r;

  if (argc != 2)
    {
      fprintf (stderr,"usage: robfit n\n");
      exit (-1);
    }

  n = atoi (argv[1]);

  X = gsl_matrix_alloc (n, p);
  x = gsl_vector_alloc (n);
  y = gsl_vector_alloc (n);

  c = gsl_vector_alloc (p);
  c_ols = gsl_vector_alloc (p);
  cov = gsl_matrix_alloc (p, p);

  r = gsl_rng_alloc(gsl_rng_default);

  /* generate linear dataset */
  for (i = 0; i < n - 3; i++)
    {
      double dx = 10.0 / (n - 1.0);
      double ei = gsl_rng_uniform(r);
      double xi = -5.0 + i * dx;
      double yi = a * xi + b;

      gsl_vector_set (x, i, xi);
      gsl_vector_set (y, i, yi + ei);
    }

  /* add a few outliers */
  gsl_vector_set(x, n - 3, 4.7);
  gsl_vector_set(y, n - 3, -8.3);

  gsl_vector_set(x, n - 2, 3.5);
  gsl_vector_set(y, n - 2, -6.7);

  gsl_vector_set(x, n - 1, 4.1);
  gsl_vector_set(y, n - 1, -6.0);

  /* construct design matrix X for linear fit */
  for (i = 0; i < n; ++i)
    {
      double xi = gsl_vector_get(x, i);

      gsl_matrix_set (X, i, 0, 1.0);
      gsl_matrix_set (X, i, 1, xi);
    }

  /* perform robust and OLS fit */
  dofit(gsl_multifit_robust_ols, X, y, c_ols, cov);
  dofit(gsl_multifit_robust_bisquare, X, y, c, cov);

  /* output data and model */
  for (i = 0; i < n; ++i)
    {
      double xi = gsl_vector_get(x, i);
      double yi = gsl_vector_get(y, i);
      gsl_vector_view v = gsl_matrix_row(X, i);
      double y_ols, y_rob, y_err;

      gsl_multifit_robust_est(&v.vector, c, cov, &y_rob, &y_err);
      gsl_multifit_robust_est(&v.vector, c_ols, cov, &y_ols, &y_err);

      printf("%g %g %g %g\n", xi, yi, y_rob, y_ols);
    }

#define C(i) (gsl_vector_get(c,(i)))
#define COV(i,j) (gsl_matrix_get(cov,(i),(j)))

  {
    printf ("# best fit: Y = %g + %g X\n",
            C(0), C(1));

    printf ("# covariance matrix:\n");
    printf ("# [ %+.5e, %+.5e\n",
               COV(0,0), COV(0,1));
    printf ("#   %+.5e, %+.5e\n",
               COV(1,0), COV(1,1));
  }

  gsl_matrix_free (X);
  gsl_vector_free (x);
  gsl_vector_free (y);
  gsl_vector_free (c);
  gsl_vector_free (c_ols);
  gsl_matrix_free (cov);
  gsl_rng_free(r);

  return 0;
}

The output from the program is shown in Fig. 33.

Fig. 33 Linear fit to dataset with outliers.

Large Dense Linear Regression Example

The following program demostrates the large dense linear least squares solvers. This example is adapted from Trefethen and Bau, and fits the function

System Message: WARNING/2 (f(t) = \exp{(\sin^3{(10t)}}))

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

on the interval

System Message: WARNING/2 ([0,1])

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

with a degree 15 polynomial. The program generates

System Message: WARNING/2 (n = 50000)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

equally spaced points

System Message: WARNING/2 (t_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

on this interval, calculates the function value and adds random noise to determine the observation value

System Message: WARNING/2 (y_i)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The entries of the least squares matrix are

System Message: WARNING/2 (X_{ij} = t_i^j)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, representing a polynomial fit. The matrix is highly ill-conditioned, with a condition number of about

System Message: WARNING/2 (1.4 \cdot 10^{11})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The program accumulates the matrix into the least squares system in 5 blocks, each with 10000 rows. This way the full matrix

System Message: WARNING/2 (X)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

is never stored in memory. We solve the system with both the normal equations and TSQR methods. The results are shown in Fig. 34. In the top left plot, we see the unregularized normal equations solution has larger error than TSQR due to the ill-conditioning of the matrix. In the bottom left plot, we show the L-curve, which exhibits multiple corners. In the top right panel, we plot a regularized solution using

System Message: WARNING/2 (\lambda = 10^{-6})

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

. The TSQR and normal solutions now agree, however they are unable to provide a good fit due to the damping. This indicates that for some ill-conditioned problems, regularizing the normal equations does not improve the solution. This is further illustrated in the bottom right panel, where we plot the L-curve calculated from the normal equations. The curve agrees with the TSQR curve for larger damping parameters, but for small

System Message: WARNING/2 (\lambda)

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Mageia) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/share/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texmf-dist/tex/latex/base/inputenc.sty ! LaTeX Error: File `utf8x.def' not found. Type X to quit or <RETURN> to proceed, or enter new name. (Default extension: def) Enter file name: ! Emergency stop. <read *> l.158 \endinput ^^M No pages of output. Transcript written on math.log.

, the normal equations approach cannot provide accurate solution vectors leading to numerical inaccuracies in the left portion of the curve.

#include <gsl/gsl_math.h>
#include <gsl/gsl_vector.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_rng.h>
#include <gsl/gsl_randist.h>
#include <gsl/gsl_multifit.h>
#include <gsl/gsl_multilarge.h>
#include <gsl/gsl_blas.h>

/* function to be fitted */
double
func(const double t)
{
  double x = sin(10.0 * t);
  return exp(x*x*x);
}

/* construct a row of the least squares matrix */
int
build_row(const double t, gsl_vector *row)
{
  const size_t p = row->size;
  double Xj = 1.0;
  size_t j;

  for (j = 0; j < p; ++j)
    {
      gsl_vector_set(row, j, Xj);
      Xj *= t;
    }

  return 0;
}

int
solve_system(const int print_data, const gsl_multilarge_linear_type * T,
             const double lambda, const size_t n, const size_t p,
             gsl_vector * c)
{
  const size_t nblock = 5;         /* number of blocks to accumulate */
  const size_t nrows = n / nblock; /* number of rows per block */
  gsl_multilarge_linear_workspace * w =
    gsl_multilarge_linear_alloc(T, p);
  gsl_matrix *X = gsl_matrix_alloc(nrows, p);
  gsl_vector *y = gsl_vector_alloc(nrows);
  gsl_rng *r = gsl_rng_alloc(gsl_rng_default);
  const size_t nlcurve = 200;
  gsl_vector *reg_param = gsl_vector_alloc(nlcurve);
  gsl_vector *rho = gsl_vector_alloc(nlcurve);
  gsl_vector *eta = gsl_vector_alloc(nlcurve);
  size_t rowidx = 0;
  double rnorm, snorm, rcond;
  double t = 0.0;
  double dt = 1.0 / (n - 1.0);

  while (rowidx < n)
    {
      size_t nleft = n - rowidx;         /* number of rows left to accumulate */
      size_t nr = GSL_MIN(nrows, nleft); /* number of rows in this block */
      gsl_matrix_view Xv = gsl_matrix_submatrix(X, 0, 0, nr, p);
      gsl_vector_view yv = gsl_vector_subvector(y, 0, nr);
      size_t i;

      /* build (X,y) block with 'nr' rows */
      for (i = 0; i < nr; ++i)
        {
          gsl_vector_view row = gsl_matrix_row(&Xv.matrix, i);
          double fi = func(t);
          double ei = gsl_ran_gaussian (r, 0.1 * fi); /* noise */
          double yi = fi + ei;

          /* construct this row of LS matrix */
          build_row(t, &row.vector);

          /* set right hand side value with added noise */
          gsl_vector_set(&yv.vector, i, yi);

          if (print_data && (i % 100 == 0))
            printf("%f %f\n", t, yi);

          t += dt;
        }

      /* accumulate (X,y) block into LS system */
      gsl_multilarge_linear_accumulate(&Xv.matrix, &yv.vector, w);

      rowidx += nr;
    }

  if (print_data)
    printf("\n\n");

  /* compute L-curve */
  gsl_multilarge_linear_lcurve(reg_param, rho, eta, w);

  /* solve large LS system and store solution in c */
  gsl_multilarge_linear_solve(lambda, c, &rnorm, &snorm, w);

  /* compute reciprocal condition number */
  gsl_multilarge_linear_rcond(&rcond, w);

  fprintf(stderr, "=== Method %s ===\n", gsl_multilarge_linear_name(w));
  fprintf(stderr, "condition number = %e\n", 1.0 / rcond);
  fprintf(stderr, "residual norm    = %e\n", rnorm);
  fprintf(stderr, "solution norm    = %e\n", snorm);

  /* output L-curve */
  {
    size_t i;
    for (i = 0; i < nlcurve; ++i)
      {
        printf("%.12e %.12e %.12e\n",
               gsl_vector_get(reg_param, i),
               gsl_vector_get(rho, i),
               gsl_vector_get(eta, i));
      }
    printf("\n\n");
  }

  gsl_matrix_free(X);
  gsl_vector_free(y);
  gsl_multilarge_linear_free(w);
  gsl_rng_free(r);
  gsl_vector_free(reg_param);
  gsl_vector_free(rho);
  gsl_vector_free(eta);

  return 0;
}

int
main(int argc, char *argv[])
{
  const size_t n = 50000;   /* number of observations */
  const size_t p = 16;      /* polynomial order + 1 */
  double lambda = 0.0;      /* regularization parameter */
  gsl_vector *c_tsqr = gsl_vector_alloc(p);
  gsl_vector *c_normal = gsl_vector_alloc(p);

  if (argc > 1)
    lambda = atof(argv[1]);

  /* solve system with TSQR method */
  solve_system(1, gsl_multilarge_linear_tsqr, lambda, n, p, c_tsqr);

  /* solve system with Normal equations method */
  solve_system(0, gsl_multilarge_linear_normal, lambda, n, p, c_normal);

  /* output solutions */
  {
    gsl_vector *v = gsl_vector_alloc(p);
    double t;

    for (t = 0.0; t <= 1.0; t += 0.01)
      {
        double f_exact = func(t);
        double f_tsqr, f_normal;

        build_row(t, v);
        gsl_blas_ddot(v, c_tsqr, &f_tsqr);
        gsl_blas_ddot(v, c_normal, &f_normal);

        printf("%f %e %e %e\n", t, f_exact, f_tsqr, f_normal);
      }

    gsl_vector_free(v);
  }

  gsl_vector_free(c_tsqr);
  gsl_vector_free(c_normal);

  return 0;
}

References and Further Reading

A summary of formulas and techniques for least squares fitting can be found in the “Statistics” chapter of the Annual Review of Particle Physics prepared by the Particle Data Group,

• Review of Particle Properties, R.M. Barnett et al., Physical Review D54, 1 (1996) http://pdg.lbl.gov

The Review of Particle Physics is available online at the website given above.

The tests used to prepare these routines are based on the NIST Statistical Reference Datasets. The datasets and their documentation are available from NIST at the following website,

http://www.nist.gov/itl/div898/strd/index.html

More information on Tikhonov regularization can be found in

• Hansen, P. C. (1998), Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM Monogr. on Mathematical Modeling and Computation, Society for Industrial and Applied Mathematics
• M. Rezghi and S. M. Hosseini (2009), A new variant of L-curve for Tikhonov regularization, Journal of Computational and Applied Mathematics, Volume 231, Issue 2, pages 914-924.

The GSL implementation of robust linear regression closely follows the publications

• DuMouchel, W. and F. O’Brien (1989), “Integrating a robust option into a multiple regression computing environment,” Computer Science and Statistics: Proceedings of the 21st Symposium on the Interface, American Statistical Association
• Street, J.O., R.J. Carroll, and D. Ruppert (1988), “A note on computing robust regression estimates via iteratively reweighted least squares,” The American Statistician, v. 42, pp. 152-154.

More information about the normal equations and TSQR approach for solving large linear least squares systems can be found in the publications

• Trefethen, L. N. and Bau, D. (1997), “Numerical Linear Algebra”, SIAM.
• Demmel, J., Grigori, L., Hoemmen, M. F., and Langou, J. “Communication-optimal parallel and sequential QR and LU factorizations”, UCB Technical Report No. UCB/EECS-2008-89, 2008.