umontreal.iro.lecuyer.gof

Class GofFormat

java.lang.Object

umontreal.iro.lecuyer.gof.GofFormat

public class GofFormat
extends java.lang.Object

This class contains methods used to format results of GOF test statistics, or to apply a series of tests simultaneously and format the results. It is in fact a translation from C to Java of a set of functions that were specially written for the implementation of TestU01, a software package for testing uniform random number generators.

Strictly speaking, applying several tests simultaneously makes the p-values ``invalid'' in the sense that the probability of having at least one p-value less than 0.01, say, is larger than 0.01. One must therefore be careful with the interpretation of these p-values (one could use, e.g., the Bonferroni inequality). Applying simultaneous tests is convenient in some situations, such as in screening experiments for detecting statistical deficiencies in random number generators. In that context, rejection of the null hypothesis typically occurs with extremely small p-values (e.g., less than 10^-15), and the interpretation is quite obvious in this case.

The class also provides tools to plot an empirical or theoretical distribution function, by creating a data file that contains a graphic plot in a format compatible with the software specified by the environment variable graphSoft.

Note: This class uses the Colt library.

Field Summary

Fields

Modifier and Type	Field and Description
static boolean[]	activeTests The set of EDF tests that are to be performed when calling the methods activeTests, formatActiveTests, etc.
static int	AD Anderson-Darling test
static int	CM Cramér-von Mises test
static int	COR Correlation
static double	EPSILONP Environment variable used in formatp0 to determine which p-values are too close to 0 or 1 to be printed explicitly.
static int	GNUPLOT Data file format used for plotting functions with Gnuplot.
static int	graphSoft Environment variable that selects the type of software to be used for plotting the graphs of functions.
static int	KS Kolmogorov-Smirnov test
static int	KSM Kolmogorov-Smirnov- test
static int	KSP Kolmogorov-Smirnov+ test
static int	MATHEMATICA Data file format used for creating graphics with Mathematica.
static int	MEAN Mean
static int	NTESTTYPES Total number of test types
static double	SUSPECTP Environment variable used in formatp1 to determine which p-values should be marked as suspect when printing test results.
static java.lang.String[]	TESTNAMES Name of each testType test.
static int	WG Watson G test
static int	WU Watson U test

Method Summary

Methods

Modifier and Type	Method and Description
static void	activeTests(DoubleArrayList data, ContinuousDistribution dist, double[] sVal, double[] pVal) The observations are in data, not necessarily sorted, and we want to compare their empirical distribution with the distribution dist.
static void	activeTests(DoubleArrayList sortedData, double[] sVal, double[] pVal) Computes the EDF test statistics by calling tests, then computes the p-values of those that currently belong to activeTests, and return these quantities in sVal and pVal, respectively.
static java.lang.String	drawCdf(ContinuousDistribution dist, double a, double b, int m, java.lang.String desc) Formats data to plot the graph of the distribution function F over the interval [a, b], and returns the result as a String.
static java.lang.String	drawDensity(ContinuousDistribution dist, double a, double b, int m, java.lang.String desc) Formats data to plot the graph of the density f (x) over the interval [a, b], and returns the result as a String.
static java.lang.String	formatActiveTests(int n, double[] sVal, double[] pVal) Gets the p-values of the active EDF test statistics, which are in activeTests.
static java.lang.String	formatChi2(int k, int d, double chi2) Computes the p-value of the chi-square statistic chi2 for a test with k intervals.
static java.lang.String	formatKS(DoubleArrayList data, ContinuousDistribution dist) Computes the KS test statistics to compare the empirical distribution of the observations in data with the theoretical distribution dist and formats the results.
static java.lang.String	formatKS(int n, double dp, double dm, double d) Computes the p-values of the three Kolmogorov-Smirnov statistics DN+, DN-, and DN, whose values are in dp, dm, d, respectively, assuming a sample of size n.
static java.lang.String	formatKSJumpOne(DoubleArrayList data, ContinuousDistribution dist, double a) Similar to formatKS, but for DN+(a).
static java.lang.String	formatKSJumpOne(int n, double a, double dp) Similar to formatKS, but for the KS statistic DN+(a).
static java.lang.String	formatp0(double p) Returns the significance level (or p-value) p of a test, in the format ``1 - p'' if p is close to 1, and p otherwise.
static java.lang.String	formatp1(double p) Returns the string ``Significance level of test : '', then calls formatp0 to print p, and adds the marker ``****'' if p is considered suspect (uses the environment variable RSUSPECTP for this).
static java.lang.String	formatp2(double x, double p) Returns x on a single line, then go to the next line and calls formatp1.
static java.lang.String	formatp3(java.lang.String testName, double x, double p) Formats the test statistic x for a test named testName with p-value p.
static java.lang.String	graphDistUnif(DoubleArrayList data, java.lang.String desc) Formats data to plot the empirical distribution of U(1),..., U(N), which are assumed to be in data[0...N-1], and to compare it with the uniform distribution.
static java.lang.String	graphFunc(ContinuousDistribution dist, double a, double b, int m, int mono, java.lang.String desc) Deprecated.
static java.lang.String	iterPowRatioTests(DoubleArrayList sortedData, int k, boolean printval, boolean graph, java.io.PrintWriter f) Similar to iterSpacingsTests, but with the GofStat.powerRatios transformation.
static java.lang.String	iterSpacingsTests(DoubleArrayList sortedData, int k, boolean printval, boolean graph, java.io.PrintWriter f) Repeats the following k times: Applies the GofStat.iterateSpacings transformation to the U(0),..., U(N-1), assuming that these observations are in sortedData, then computes the EDF test statistics and calls activeTests after each transformation.
static void	tests(DoubleArrayList data, ContinuousDistribution dist, double[] sVal) The observations V are in data, not necessarily sorted, and their empirical distribution is compared with the continuous distribution dist.
static void	tests(DoubleArrayList sortedData, double[] sVal) Computes all EDF test statistics to compare the empirical distribution of U(0),..., U(N-1) with the uniform distribution, assuming that these sorted observations are in sortedData.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

GNUPLOT

public static final int GNUPLOT

Data file format used for plotting functions with Gnuplot.

See Also:

Constant Field Values

MATHEMATICA

public static final int MATHEMATICA

Data file format used for creating graphics with Mathematica.

See Also:

Constant Field Values

graphSoft

public static int graphSoft

Environment variable that selects the type of software to be used for plotting the graphs of functions. The data files produced by graphFunc and graphDistUnif will be in a format suitable for this selected software. The default value is GNUPLOT. To display a graphic in file f using gnuplot, for example, one can use the command ``plot f with steps, x with lines'' in gnuplot. graphSoft can take the values GNUPLOT or MATHEMATICA.

EPSILONP

public static double EPSILONP

Environment variable used in formatp0 to determine which p-values are too close to 0 or 1 to be printed explicitly. If EPSILONP = ε, then any p-value (or significance level) less than ε or larger than 1 - ε is not written explicitly; the program simply writes ``eps'' or ``1-eps''. The default value is 10^-15.

SUSPECTP

public static double SUSPECTP

Environment variable used in formatp1 to determine which p-values should be marked as suspect when printing test results. If SUSPECTP = α, then any p-value (or significance level) less than α or larger than 1 - α is considered suspect and is ``singled out'' by formatp1. The default value is 0.01.

KSP

public static final int KSP

Kolmogorov-Smirnov+ test

See Also:

Constant Field Values

KSM

public static final int KSM

Kolmogorov-Smirnov- test

See Also:

Constant Field Values

KS

public static final int KS

Kolmogorov-Smirnov test

See Also:

Constant Field Values

AD

public static final int AD

Anderson-Darling test

See Also:

Constant Field Values

CM

public static final int CM

Cramér-von Mises test

See Also:

Constant Field Values

WG

public static final int WG

Watson G test

See Also:

Constant Field Values

WU

public static final int WU

Watson U test

See Also:

Constant Field Values

MEAN

public static final int MEAN

Mean

See Also:

Constant Field Values

COR

public static final int COR

Correlation

See Also:

Constant Field Values

NTESTTYPES

public static final int NTESTTYPES

Total number of test types

See Also:

Constant Field Values

TESTNAMES

public static final java.lang.String[] TESTNAMES

Name of each testType test. Could be used for printing the test results, for example.

activeTests

public static boolean[] activeTests

The set of EDF tests that are to be performed when calling the methods activeTests, formatActiveTests, etc. By default, this set contains KSP, KSM, and AD. Note: MEAN and COR are always excluded from this set of active tests. The valid indices for this array are KSP, KSM, KS, AD, CM, WG, WU, MEAN, and COR.

Method Detail

graphFunc

@Deprecated
public static java.lang.String graphFunc(ContinuousDistribution dist,
                                    double a,
                                    double b,
                                    int m,
                                    int mono,
                                    java.lang.String desc)

Deprecated.

Use drawCdf instead. Formats data to plot the graph of the distribution function F (or bar(F)) over the interval [a, b], and returns the result as a String. dist.cdf(x) (or dist.barF(x)) returns the value of F (or bar(F)) at x, and that F is either non-decreasing or non-increasing. If mono = 1, the method will verify that F is non-decreasing; if mono = -1, it will verify that bar(F) is non-increasing. (This is useful to verify if F is effectively a sensible approximation to a distribution function or its complementary in the given interval.) The String desc gives a short caption for the graphic plot. The method computes the m + 1 points (x_i, F(x_i)), where x_i = a + i(b - a)/m for i = 0, 1,…, m, and formats these points into a String in a format suitable for the software specified by graphSoft.

Parameters:

dist - continuous distribution function to plot

a - lower bound of the interval to plot

b - upper bound of the interval to plot

m - number of points in the plot minus one

mono - 1 for plotting a distribution function, -1 if for a complementary distribution function

desc - short caption describing the plot

Returns:

a string representation of the plot data

drawCdf

public static java.lang.String drawCdf(ContinuousDistribution dist,
                       double a,
                       double b,
                       int m,
                       java.lang.String desc)

Formats data to plot the graph of the distribution function F over the interval [a, b], and returns the result as a String. The method dist.cdf(x) returns the value of F at x. The String desc gives a short caption for the graphic plot. The method computes the m + 1 points (x_i, F(x_i)), where x_i = a + i(b - a)/m for i = 0, 1,…, m, and formats these points into a String in a format suitable for the software specified by graphSoft.

Parameters:

dist - continuous distribution function to plot

a - lower bound of the interval to plot

b - upper bound of the interval to plot

m - number of points in the plot minus one

desc - short caption describing the plot

Returns:

a string representation of the plot data

drawDensity

public static java.lang.String drawDensity(ContinuousDistribution dist,
                           double a,
                           double b,
                           int m,
                           java.lang.String desc)

Formats data to plot the graph of the density f (x) over the interval [a, b], and returns the result as a String. The method dist.density(x) returns the value of f (x) at x. The String desc gives a short caption for the graphic plot. The method computes the m + 1 points (x_i, f (x_i)), where x_i = a + i(b - a)/m for i = 0, 1,…, m, and formats these points into a String in a format suitable for the software specified by graphSoft.

Parameters:

dist - continuous density function to plot

a - lower bound of the interval to plot

b - upper bound of the interval to plot

m - number of points in the plot minus one

desc - short caption describing the plot

Returns:

a string representation of the plot data

graphDistUnif

public static java.lang.String graphDistUnif(DoubleArrayList data,
                             java.lang.String desc)

Formats data to plot the empirical distribution of U₍₁₎,..., U_(N), which are assumed to be in data[0...N-1], and to compare it with the uniform distribution. The U_(i) must be sorted. The two endpoints (0, 0) and (1, 1) are always included in the plot. The string desc gives a short caption for the graphic plot. The data is printed in a format suitable for the software specified by graphSoft.

Parameters:

data - array of observations to plot

desc - short caption describing the plot

Returns:

a string representation of the plot data

formatp0

public static java.lang.String formatp0(double p)

Returns the significance level (or p-value) p of a test, in the format ``1 - p'' if p is close to 1, and p otherwise. Uses the environment variable EPSILONP and replaces p by ε when it is too small.

Parameters:

p - the p-value or significance level to be formated

Returns:

the string representation of the p-value

formatp1

public static java.lang.String formatp1(double p)

Returns the string ``Significance level of test : '', then calls formatp0 to print p, and adds the marker ``****'' if p is considered suspect (uses the environment variable RSUSPECTP for this).

Parameters:

p - the p-value or significance level to be formated

Returns:

the string representation of the significance level of test

formatp2

public static java.lang.String formatp2(double x,
                        double p)

Returns x on a single line, then go to the next line and calls formatp1.

Parameters:

x - value of the statistic for which the significance level is formated

p - the p-value or significance level to be formated

Returns:

the string representation of the significance level of test

formatp3

public static java.lang.String formatp3(java.lang.String testName,
                        double x,
                        double p)

Formats the test statistic x for a test named testName with p-value p. The first line of the returned string contains the name of the test and the statistic whereas the second line contains its significance level. The formated values of x and p are aligned.

Parameters:

testName - name of the test that was performed

x - value of the test statistic

p - significance level (or p-value) of the test

Returns:

the string representation of the test result

formatChi2

public static java.lang.String formatChi2(int k,
                          int d,
                          double chi2)

Computes the p-value of the chi-square statistic chi2 for a test with k intervals. Uses d decimal digits of precision in the calculations. The result of the test is returned as a string. The p-value is computed using pDisc.

Parameters:

k - number of subintervals for the chi-square test

chi2 - chi-square statistic

Returns:

the string representation of the test result and p-value

formatKS

public static java.lang.String formatKS(int n,
                        double dp,
                        double dm,
                        double d)

Computes the p-values of the three Kolmogorov-Smirnov statistics D_N⁺, D_N^-, and D_N, whose values are in dp, dm, d, respectively, assuming a sample of size n. Then formats these statistics and their p-values using formatp2 for each one.

Parameters:

n - sample size

dp - value of the D_N⁺ statistic

dm - value of the D_N^- statistic

d - value of the D_N statistic

Returns:

the string representation of the Kolmogorov-Smirnov statistics and their significance levels

formatKS

public static java.lang.String formatKS(DoubleArrayList data,
                        ContinuousDistribution dist)

Computes the KS test statistics to compare the empirical distribution of the observations in data with the theoretical distribution dist and formats the results.

Parameters:

data - array of observations to be tested

dist - assumed distribution of the observations

Returns:

the string representation of the Kolmogorov-Smirnov statistics and their significance levels

formatKSJumpOne

public static java.lang.String formatKSJumpOne(int n,
                               double a,
                               double dp)

Similar to formatKS, but for the KS statistic D_N⁺(a). Writes a header, computes the p-value and calls formatp2.

Parameters:

n - sample size

a - size of the jump

dp - value of D_N⁺(a)

Returns:

the string representation of the Kolmogorov-Smirnov statistic and its significance level

formatKSJumpOne

public static java.lang.String formatKSJumpOne(DoubleArrayList data,
                               ContinuousDistribution dist,
                               double a)

Similar to formatKS, but for D_N⁺(a).

Parameters:

data - array of observations to be tested

dist - assumed distribution of the data

a - size of the jump

Returns:

string representation of the Kolmogorov-Smirnov statistic and its significance level

tests

public static void tests(DoubleArrayList sortedData,
         double[] sVal)

Computes all EDF test statistics to compare the empirical distribution of U₍₀₎,..., U_(N-1) with the uniform distribution, assuming that these sorted observations are in sortedData. If N > 1, returns sVal with the values of the KS statistics D_N⁺, D_N^- and D_N, of the Cramér-von Mises statistic W_N², Watson's G_N and U_N², Anderson-Darling's A_N², and the average of the U_i's, respectively. If N = 1, only puts 1 -sortedData.get (0) in sVal[KSP]. Calling this method is more efficient than computing these statistics separately by calling the corresponding methods in GofStat.

Parameters:

sortedData - array of sorted observations

sVal - array that will be filled with the results of the tests

tests

public static void tests(DoubleArrayList data,
         ContinuousDistribution dist,
         double[] sVal)

The observations V are in data, not necessarily sorted, and their empirical distribution is compared with the continuous distribution dist. If N = 1, only puts data.get (0) in sVal[MEAN], and 1 -dist.cdf (data.get (0)) in sVal[KSP].

Parameters:

data - array of observations to test

dist - assumed distribution of the observations

sVal - array that will be filled with the results of the tests

activeTests

public static void activeTests(DoubleArrayList sortedData,
               double[] sVal,
               double[] pVal)

Computes the EDF test statistics by calling tests, then computes the p-values of those that currently belong to activeTests, and return these quantities in sVal and pVal, respectively. Assumes that U₍₀₎,..., U_(N-1) are in sortedData and that we want to compare their empirical distribution with the uniform distribution. If N = 1, only puts 1 -sortedData.get (0) in sVal[KSP], pVal[KSP], and pVal[MEAN].

Parameters:

sortedData - array of sorted observations

sVal - array that will be filled with the results of the tests

pVal - array that will be filled with the p-values

activeTests

public static void activeTests(DoubleArrayList data,
               ContinuousDistribution dist,
               double[] sVal,
               double[] pVal)

The observations are in data, not necessarily sorted, and we want to compare their empirical distribution with the distribution dist. If N = 1, only puts data.get(0) in sVal[MEAN], and 1 -dist.cdf (data.get (0)) in sVal[KSP], pVal[KSP], and pVal[MEAN].

Parameters:

data - array of observations to test

dist - assumed distribution of the observations

sVal - array that will be filled with the results of the tests

pVal - array that will be filled with the p-values

formatActiveTests

public static java.lang.String formatActiveTests(int n,
                                 double[] sVal,
                                 double[] pVal)

Gets the p-values of the active EDF test statistics, which are in activeTests. It is assumed that the values of these statistics and their p-values are already computed, in sVal and pVal, and that the sample size is n. These statistics and p-values are formated using formatp2 for each one. If n=1, prints only pVal[KSP] using formatp1.

Parameters:

n - sample size

sVal - array containing the results of the tests

pVal - array containing the p-values

Returns:

the results formated as a string

iterSpacingsTests

public static java.lang.String iterSpacingsTests(DoubleArrayList sortedData,
                                 int k,
                                 boolean printval,
                                 boolean graph,
                                 java.io.PrintWriter f)

Repeats the following k times: Applies the GofStat.iterateSpacings transformation to the U₍₀₎,..., U_(N-1), assuming that these observations are in sortedData, then computes the EDF test statistics and calls activeTests after each transformation. The function returns the original array sortedData (the transformations are applied on a copy of sortedData). If printval = true, stores all the values into the returned String after each iteration. If graph = true, calls graphDistUnif after each iteration to print to stream f the data for plotting the distribution function of the U_i.

Parameters:

sortedData - array containing the sorted observations

k - number of times the tests are applied

printval - if true, stores all the values of the observations at each iteration

graph - if true, the distribution of the U_i will be plotted after each iteration

f - stream where the plots are written to

Returns:

a string representation of the test results

iterPowRatioTests

public static java.lang.String iterPowRatioTests(DoubleArrayList sortedData,
                                 int k,
                                 boolean printval,
                                 boolean graph,
                                 java.io.PrintWriter f)

Similar to iterSpacingsTests, but with the GofStat.powerRatios transformation.

Parameters:

sortedData - array containing the sorted observations

k - number of times the tests are applied

printval - if true, stores all the values of the observations at each iteration

graph - if true, the distribution of the U_i will be plotted after each iteration

f - stream where the plots are written to

Returns:

a string representation of the test results