Simple perceptron - HTML documentation under construction.

Inheritance:

Public Fields

static const double	SIG_OFF
static const double	SIG_TRESHOLD
static const double	TANH_ON
static const double	TANH_OFF
static const double	TANH_TRESHOLD
static const double	ID_ON
static const double	ID_OFF
static const double	ID_TRESHOLD

Public Methods

	OneLayer () Default constructor it does nothing.
	OneLayer (unsigned nout, unsigned nin, act_func fout=sigmoid) Constructor It builds a one layer perceptron with nout output units, nin input units and fout activation function.
	OneLayer (unsigned dummy) Constructor It builds only the codex matrix and initiailizes the queue of historical errors
virtual	~OneLayer () Destructor
virtual void	init_weights_norm (int s = 0) initializes all values according to a noraml distribution centerd in 0 with unitary standard deviation
virtual void	init_weights_const (double x) initializes all values to x
virtual void	read_weights (char* namefile) It reads a matrix of weights The matrix is read from the file namefile.
virtual void	print_weights (void) It prints on stdout the weight matrix
virtual vect&	compute (vect& vinput) It computes the output vector of the net
virtual unsigned	predictclassWTA (vect& vinput) It outputs the predicted class (WTA criterion) Given an input vector, it predict a class using the winner-takes-all criterion.
virtual unsigned	predictclass (vect& vinput) It outputs the predicted class (min.
virtual void	backprop (vect& t, vect& x) It computes the error gradient of the net.
virtual void	print_info_grad (void) It outputs the delta values, ie.
void	print_C (void) It prints on stdout the decomposition matrix C
virtual void	save_weights (char* namefile) It saves onto a file the weights and the paramenters of the net It saves onto the file namefilenet the parameters of the net and its weights File format: "Version: " version number (unsigned) (actually always 1) namefile (string) "Number_layers: " Layers number (unsigned) "Dim_output: " Output layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) "Dim_hidden1: " I hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 1 "Dim_hidden2: " II hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 2 "Dim_input: " Input size (unsigned) "Weight matrix first layer :" Weight matrix output layer (double) "Bias vector :" Bias vector output layer (double) "Weight matrix first hidden layer :" present if number layers > 1 Matrix (matrix of double) present if number layers > 1 "Bias vector :" present if number layers > 1 Vector (vector of double) present if number layers > 1 "Weight matrix second hidden layer :" present if number layers > 2 Matrix (matrix of double) present if number layers > 2 "Bias vector :" present if number layers > 2 Vector (vector of double) present if number layers > 2 "END" File termination string NOTE : the activation function can be: SIGMOID = 1, TANHYP = 2, IDENTITY = 3.
inline queue<float> &	read_qerror () It returns the queue of the historical errors along the epochs.
void	save_errors (char* file)
virtual stop_cond	Learn_by_pattern (TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)
virtual stop_cond	Learn_by_epoch (TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)
matrix	test (TrainingSet& tset) Testing the neural net It performs the testing of the net on the testing set tset.
matrix	test (TrainingSet& tset, deque<unsigned>& outputs) Testing the neural net It performs the testing of the net on the testing set tset.
virtual void	save_out (TrainingSet& tset, char* file)
void	save_out (TrainingSet& tset, ofstream& f) It saves onto a stream the outputs of the net File format: y1 y2 .
void	save_predicted (deque<unsigned>& outputs, char* file) Saves onto a file the classes predicted by the net The labels are stored one for each line.
void	save_decomp_matrix (ofstream& f)
int	CrossValidate (char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* savecvfile = "", char* readcvfile = "", char* outfile = "", int saveout = 0, int seed=0, bool printout=true)
virtual int	CrossValidateTrialWeights (char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* outfile = "", int saveout = 0, int seed=0, int times=3, bool printout=true)
void	SaveCV (char* name, vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, matrix& conf, clock_t start, clock_t end)
void	print_resultCV (vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, clock_t start, clock_t end)
void	print_errors ()
unsigned	Code_act_fun (act_func g)
act_func	Decode_act_fun (unsigned fun)
int	Read_seed (void) It returns the the random initialization seed for the weights
double	get_perc_errors (void) Return the testing percentual of errors.
unsigned	Read_minHam (void)

Public Members

Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent

Protected Fields

static const unsigned	ERR_PRECISION precision by which are saved error in the stream
static const unsigned	OUTPUT_PRECISION precision of the net output in the stream
static const unsigned	MAXITS default value of max.
static const double	ERR_TOL default value of the tolerated RMS value
static const unsigned	INTEST_SIZE width of the heading field of the file for saving weigths
unsigned	n_class Cardinality of the classes
matrix	C Codex matrix one-per-class of dimension n_outputs * n_class Ith column correspond to the codeword of the class i+1; Matrix values are determined by the activation function used by the net.
LayerNetTrain	layer [MAXLAYERS] Layers of the neurons.
queue<float>	Qerror Queue for recording the errors sequence of the net along the epochs
unsigned	num_errors Number of testing errors
double	perc_errors Percentual of testing errors
int	seed Random initialization seed for the weights
unsigned	minHam Minimum Hamming distance among codewords of the classes

Protected Methods

virtual void	Init_C (void)
void	analyze (act_func g, double& on, double& off)
double	Calc_square_error (vect& t, vect& out)
double	calc_denom_RMS (vect& t, unsigned n_train)
double	RMS_error (double err, double denom)
void	read_weights (istream& f)
virtual void	save_weights_heading (char* namefile) It saves onto a file the general paramenters of the net It saves the number of layers, size of the layers and type of the activation function
virtual void	save_weights_heading (ofstream& f, char* namebuf) It saves in a stream the general paramenters of the net It saves the number of layers, size of the layers and type of the activation function
virtual void	save_weights_body (char* namefile) It appends onto a file the weights of the net.
virtual void	save_weights_body (ofstream& f) It appends to a stream the weights of the net.
void	save_weights_cv_heading (char* savecvfile, int nfold) It saves onto a file the general paramenters of the nets of the cross validation It saves the number of layers, size of the layers and type of the activation function
void	save_weights_cv_body (char* namefile, int fold_n) It appends onto a file the weights of the net of the cross validation.
void	do_Sum_deltas (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
void	Update_Sum_Delta (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
void	Zero (unsigned n_layers)
void	Zero_Sum_Delta (unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
void	Build_Sum_Delta (matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
void	Delete_Sum_Delta (matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
void	Copy_Sum_Delta_To_Delta (unsigned n_layers, LayerNetTrain layer[], matrix* Sum_Delta_W[], vect* Sum_delta_bias[])
virtual unsigned	computeclassWTA (vect& out)
virtual unsigned	computeclass (vect& out)

Inherited from Network:

Public Fields

static const char* const NET_EXT

static const char* const NETCV_EXT

static const char* const ERR_EXT

static const char* const OUT_EXT

static const unsigned OUTPUT

static const unsigned HIDDEN1

static const unsigned HIDDEN2

static const char* const CV_SUFFIX

Public Methods

inline unsigned Nlayers() const

inline unsigned Ninput() const

inline unsigned Noutput() const

Protected Fields

unsigned n_layers

unsigned n_inputs

unsigned n_outputs

Documentation

Simple perceptron
It Implements a simple perceptron for classification with learning capabilities

static const unsigned ERR_PRECISION

precision by which are saved error in the stream

static const unsigned OUTPUT_PRECISION

precision of the net output in the stream

static const unsigned MAXITS

default value of max. number of iterations

static const double ERR_TOL

default value of the tolerated RMS value

static const unsigned INTEST_SIZE

width of the heading field of the file for saving weigths

unsigned n_class

Cardinality of the classes

matrix C

Codex matrix one-per-class of dimension n_outputs * n_class
Ith column correspond to the codeword of the class i+1; Matrix values are determined by the activation function used by the net.

LayerNetTrain layer[MAXLAYERS]

Layers of the neurons.
They are filled completely only by the children classes. OneLayer fills only layer[0].

queue<float> Qerror

Queue for recording the errors sequence of the net along the epochs

unsigned num_errors

Number of testing errors

double perc_errors

Percentual of testing errors

int seed

Random initialization seed for the weights

unsigned minHam

Minimum Hamming distance among codewords of the classes

virtual void Init_C(void)

void analyze(act_func g, double& on, double& off)

double Calc_square_error(vect& t, vect& out)

double calc_denom_RMS(vect& t, unsigned n_train)

double RMS_error(double err, double denom)

void read_weights(istream& f)

virtual void save_weights_heading(char* namefile)

It saves onto a file the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function

Parameters:

- namefile name of the file for saving the net. The suffix net. is appended to namefile

See Also:

save_weights

virtual void save_weights_heading(ofstream& f, char* namebuf)

It saves in a stream the general paramenters of the net
It saves the number of layers, size of the layers and type of the activation function

Parameters:

f - streamfile for saving the net. Used by save_weights_heading(char* namefile).
- namefile name of the file for saving the net.

See Also:

save_weights

virtual void save_weights_body(char* namefile)

It appends onto a file the weights of the net.

Parameters:

- namefile name of the file for saving the net. The suffix net. is appended to namefile

See Also:

save_weights

virtual void save_weights_body(ofstream& f)

It appends to a stream the weights of the net.

Parameters:

f - name stream file for saving the net. Used by save_weights_body(char* namefile).

See Also:

save_weights

void save_weights_cv_heading(char* savecvfile, int nfold)

It saves onto a file the general paramenters of the nets of the cross validation
It saves the number of layers, size of the layers and type of the activation function

Parameters:

savecvfile - name of the file for saving the nets. The suffix cvnet. is appended to namefile
nfold - total number of the folds

void save_weights_cv_body(char* namefile, int fold_n)

It appends onto a file the weights of the net of the cross validation.

Parameters:

- namefile name of the file for saving the net. The suffix cvnet. is appended to namefile
fold_n - number of the fold

void do_Sum_deltas(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

void Update_Sum_Delta(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

void Zero(unsigned n_layers)

void Zero_Sum_Delta(unsigned n_layers, matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

void Build_Sum_Delta(matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

void Delete_Sum_Delta(matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

void Copy_Sum_Delta_To_Delta(unsigned n_layers, LayerNetTrain layer[], matrix* Sum_Delta_W[], vect* Sum_delta_bias[])

virtual unsigned computeclassWTA(vect& out)

virtual unsigned computeclass(vect& out)

Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent

Activation, deactivation and threshold values with respectively sigmoid, hyperbolic tangent

static const double SIG_OFF

static const double SIG_TRESHOLD

static const double TANH_ON

static const double TANH_OFF

static const double TANH_TRESHOLD

static const double ID_ON

static const double ID_OFF

static const double ID_TRESHOLD

OneLayer()

Default constructor
it does nothing.

OneLayer(unsigned nout, unsigned nin, act_func fout=sigmoid)

Constructor
It builds a one layer perceptron with nout output units, nin input units and fout activation function.

Parameters:

nout - number of output units
nin - number of input units
fout - type of activation function

OneLayer(unsigned dummy)

Constructor
It builds only the codex matrix and initiailizes the queue of historical errors

virtual ~OneLayer()

Destructor

virtual void init_weights_norm(int s = 0)

initializes all values according to a noraml distribution centerd in 0 with unitary standard deviation

virtual void init_weights_const(double x)

initializes all values to x

virtual void read_weights(char* namefile)

It reads a matrix of weights
The matrix is read from the file namefile. The weights are read only if the dimension of matrix agrees with the dimension of the reading net.

Parameters:

- namefile file name of the recorded weights to be read.

virtual void print_weights(void)

It prints on stdout the weight matrix

virtual vect& compute(vect& vinput)

It computes the output vector of the net

Parameters:

vinput - input vector of the net

Returns:

output vector computed by the net

virtual unsigned predictclassWTA(vect& vinput)

It outputs the predicted class (WTA criterion)
Given an input vector, it predict a class using the winner-takes-all criterion.

Parameters:

vinput - input vector of the net

Returns:

predicted class

virtual unsigned predictclass(vect& vinput)

It outputs the predicted class (min. Hamming distance criterion).
Given an input vector, it predict a class using the minimum Hamming distance criterion.

Parameters:

vinput - input vector of the net

Returns:

predicted class

virtual void backprop(vect& t, vect& x)

It computes the error gradient of the net.
It records the error gradient respect to the weights.

Parameters:

- t target vector
x - input vector

virtual void print_info_grad(void)

It outputs the delta values, ie. the error gradient of the net.

void print_C(void)

It prints on stdout the decomposition matrix C

virtual void save_weights(char* namefile)

It saves onto a file the weights and the paramenters of the net
It saves onto the file namefilenet the parameters of the net and its weights
File format:
"Version: " version number (unsigned) (actually always 1)
namefile (string)
"Number_layers: " Layers number (unsigned)
"Dim_output: " Output layer size (unsigned) "Actfunction: " Number of the activation function (unsigned)
"Dim_hidden1: " I hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 1
"Dim_hidden2: " II hidden layer size (unsigned) "Actfunction: " Number of the activation function (unsigned) present if number layers > 2
"Dim_input: " Input size (unsigned)
"Weight matrix first layer :"
Weight matrix output layer (double)
"Bias vector :"
Bias vector output layer (double)
"Weight matrix first hidden layer :" present if number layers > 1
Matrix (matrix of double) present if number layers > 1
"Bias vector :" present if number layers > 1
Vector (vector of double) present if number layers > 1
"Weight matrix second hidden layer :" present if number layers > 2
Matrix (matrix of double) present if number layers > 2
"Bias vector :" present if number layers > 2
Vector (vector of double) present if number layers > 2
"END" File termination string
NOTE : the activation function can be: SIGMOID = 1, TANHYP = 2, IDENTITY = 3.

Parameters:

- namefile name of the file for saving the net. To namefile is appended the suffix .net.

inline queue<float> & read_qerror()

It returns the queue of the historical errors along the epochs.

void save_errors(char* file)

virtual stop_cond Learn_by_pattern(TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)

virtual stop_cond Learn_by_epoch(TrainingSet& t_set, Learning& learn, unsigned& iter, double& error, double errtol = ERR_TOL, unsigned maxits = MAXITS, bool printout = true)

matrix test(TrainingSet& tset)

Testing the neural net
It performs the testing of the net on the testing set tset. The number of errors is recorded in num_errors and the percentual of errors is recorded in perc_errors.

Parameters:

tset - training set

Returns:

confusion matrix

matrix test(TrainingSet& tset, deque<unsigned>& outputs)

Testing the neural net
It performs the testing of the net on the testing set tset. The number of errors is recorded in num_errors and the percentual of errors is recorded in perc_errors. The computed classes are pushed back to the queue outputs.

Parameters:

tset - training set
outputs - vector of the output classes computed by the network.

Returns:

confusion matrix

virtual void save_out(TrainingSet& tset, char* file)

void save_out(TrainingSet& tset, ofstream& f)

It saves onto a stream the outputs of the net
File format:
y1 y2 . yn computed_class right_class
Where yi (double) are the bits of the codeword coding the class, computed_class (unsigned) is the class compute by the net, right_class (unsigned) is the correct class

Parameters:

tset - training set
f - stream where the outputs are saved

void save_predicted(deque<unsigned>& outputs, char* file)

Saves onto a file the classes predicted by the net
The labels are stored one for each line.

Parameters:

outputs - vector of the output classes computed by the network
file - file of the predicted classes

void save_decomp_matrix(ofstream& f)

int CrossValidate(char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* savecvfile = "", char* readcvfile = "", char* outfile = "", int saveout = 0, int seed=0, bool printout=true)

virtual int CrossValidateTrialWeights(char* name, unsigned nfold, unsigned num_attr, char* filerr, LearnType& learninfo, char* outfile = "", int saveout = 0, int seed=0, int times=3, bool printout=true)

void SaveCV(char* name, vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, matrix& conf, clock_t start, clock_t end)

void print_resultCV(vector<unsigned>& iter, vect& abserr, vect& percerr, unsigned ntrain, clock_t start, clock_t end)

void print_errors()

unsigned Code_act_fun(act_func g)

act_func Decode_act_fun(unsigned fun)

int Read_seed(void)

It returns the the random initialization seed for the weights

double get_perc_errors(void)

Return the testing percentual of errors.

unsigned Read_minHam(void)

Direct child classes:

TwoLayer
OneLayerECOC
Dico1

Alphabetic index HTML hierarchy of classes or Java