org.joone.engine.extenders

Class RpropExtender

java.lang.Object

org.joone.engine.extenders.LearnerExtender

org.joone.engine.extenders.DeltaRuleExtender

org.joone.engine.extenders.RpropExtender

public class RpropExtender
extends DeltaRuleExtender

This class changes the delta value in such a way that it implements the RPROP algorithm.

Author:

Boris Jansen

Field Summary

Fields

Modifier and Type	Field and Description
protected double[][]	theDeltas Each weight has its own individual update-value (delta_ij(t)) represented by the next object.
protected double[][]	thePreviousGradients The gradient pattern of the previous epoch (dE(t-1)/dW_ij).
protected RpropParameters	theRpropParameters The parameters for the RPROP learning algorithm.
protected double[][]	theSummedGradients The current som of the gradients of all patterns seen so far.

Constructor Summary

Constructors

Constructor and Description
RpropExtender() Creates a new instance of RpropExtender

Method Summary

Methods

Modifier and Type	Method and Description
double	getDelta(double[] currentGradientOuts, int j, double aPreviousDelta) Computes the delta value for a bias.
double	getDelta(double[] currentInps, int j, double[] currentPattern, int k, double aPreviousDelta) Computes the delta value for a weight.
RpropParameters	getParameters() Gets the parameters of this learning algorithm.
void	postBiasUpdate(double[] currentGradientOuts) Gives extenders a change to do some post-computing after the biases are updated.
void	postWeightUpdate(double[] currentPattern, double[] currentInps) Gives extenders a change to do some post-computing after the weights are updated.
void	preBiasUpdate(double[] currentGradientOuts) Gives extenders a change to do some pre-computing before the biases are updated.
void	preWeightUpdate(double[] currentPattern, double[] currentInps) Gives extenders a change to do some pre-computing before the weights are updated.
void	reinit() (Re)Initializes this RPROP learner.
void	setParameters(RpropParameters aParameters) Sets the parameters for this learning algorithm.
protected double	sign(double d) Gets the sign of a double.

Methods inherited from class org.joone.engine.extenders.LearnerExtender

getLearner, isEnabled, setEnabled, setLearner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

theDeltas

protected double[][] theDeltas

Each weight has its own individual update-value (delta_ij(t)) represented by the next object. The weight update deltaW_ij(t) is defined as follows (dE(t) / dW_ij is the summed gradient for a single epoch): | -delta_ij(t), if dE(t) / dW_ij > 0 deltaW_ij(t) = | delta_ij(t), if dE(t) / dW_ij < 0 | 0 otherwise The delta_ij values are updated as follows: | eta_inc * delta_ij(t-1), if dE(t-1)/ dW_ij * dE(t)/ dW_ij > 0 delta_ij(t) = | eta_dec * delta_ij(t-1), if dE(t-1)/ dW_ij * dE(t)/ dW_ij < 0 | delta_ij(t-1), otherwise where 0 < eta_dec < 1 < eta_inc

thePreviousGradients

protected double[][] thePreviousGradients

The gradient pattern of the previous epoch (dE(t-1)/dW_ij).

theRpropParameters

protected RpropParameters theRpropParameters

The parameters for the RPROP learning algorithm.

theSummedGradients

protected double[][] theSummedGradients

The current som of the gradients of all patterns seen so far. The number of summed gradients is smaller the "batch size" and will be reset to zero, if the number of sums becomes equal to "batch size" after the weights/biases have been updated.

Constructor Detail

RpropExtender

public RpropExtender()

Creates a new instance of RpropExtender

Method Detail

reinit

public void reinit()

(Re)Initializes this RPROP learner.

getDelta

public double getDelta(double[] currentGradientOuts,
              int j,
              double aPreviousDelta)

Description copied from class: DeltaRuleExtender

Computes the delta value for a bias.

Specified by:

getDelta in class DeltaRuleExtender

Parameters:

currentGradientOuts - the back propagated gradients.

j - the index of the bias.

aPreviousDelta - a delta value calculated by a previous delta extender.

getDelta

public double getDelta(double[] currentInps,
              int j,
              double[] currentPattern,
              int k,
              double aPreviousDelta)

Description copied from class: DeltaRuleExtender

Computes the delta value for a weight.

Specified by:

getDelta in class DeltaRuleExtender

Parameters:

currentInps - the forwarded input.

j - the input index of the weight.

currentPattern - the back propagated gradients.

k - the output index of the weight.

aPreviousDelta - a delta value calculated by a previous delta extender.

postBiasUpdate

public void postBiasUpdate(double[] currentGradientOuts)

Description copied from class: LearnerExtender

Gives extenders a change to do some post-computing after the biases are updated.

Specified by:

postBiasUpdate in class LearnerExtender

Parameters:

currentGradientOuts - the back propagated gradients.

postWeightUpdate

public void postWeightUpdate(double[] currentPattern,
                    double[] currentInps)

Description copied from class: LearnerExtender

Gives extenders a change to do some post-computing after the weights are updated.

Specified by:

postWeightUpdate in class LearnerExtender

Parameters:

currentPattern - the back propagated gradients.

currentInps - the forwarded input.

preBiasUpdate

public void preBiasUpdate(double[] currentGradientOuts)

Description copied from class: LearnerExtender

Gives extenders a change to do some pre-computing before the biases are updated.

Specified by:

preBiasUpdate in class LearnerExtender

Parameters:

currentGradientOuts - the back propagated gradients.

preWeightUpdate

public void preWeightUpdate(double[] currentPattern,
                   double[] currentInps)

Description copied from class: LearnerExtender

Gives extenders a change to do some pre-computing before the weights are updated.

Specified by:

preWeightUpdate in class LearnerExtender

Parameters:

currentPattern - the back propagated gradients.

currentInps - the forwarded input.

getParameters

public RpropParameters getParameters()

Gets the parameters of this learning algorithm.

Returns:

the parameters of this learning algorithm.

setParameters

public void setParameters(RpropParameters aParameters)

Sets the parameters for this learning algorithm.

Parameters:

aParameters - the parameters for this learning algorithm.

sign

protected double sign(double d)

Gets the sign of a double. return the sign of a double (-1, 0, 1).