5 replies to this topic

[ibad]

Hi... I know this is a bit esoteric, but if some one can help me with some basic neural netting I would be grateful.

It's for use in a simple letter-by-letter OCR project that is geared towards printed text in a few fonts with variable size, and some slight random angle applied etc.

Anyway... some basic conceptual questions...

I was reading about multi-layer perceptrons and it seems that each output variable is output as a number by an output neuron that represents that variable. In my case it's just one variable... we want to know which letter it is.

Well... I can imagine an ANN outputting various arbitrary numbers as codes for different letters, but biological neurons only spike, 0 or 1, right? So how would a bio-mimetic ANN encode the output?

One way I can think of is have the network select one output neuron and each output neuron represents one character, and the output is based on which output neuron is firing.

Is this a valid and do-able approach? Are there any other approaches?

How would I train an NN that is asserting multiple output neurons when it should just choose one? What if one of them is the right one or none of them is? Ofcourse we're talking about premade data-sets and supervised learning here.

Should I have "multiple" levels of wrongness..punishing multiple outputs (confusion) more, especially if none of them is the right one?

What about spiking frequency on a single output neuron, is that a better output encoding?

any help is GREATLY appreciated. I know this is a slightly esoteric field, so ANY help will get you lots of good karma and guru-worship!

thanks.

[WingedPanther]

I would rethink. Maybe have several neurons fire. One for "A-ness", one for "B-ness", etc. E is likely to fire both "E-ness" and "F-ness", but also "B-ness". F would not fire "B-ness". You want to be careful to not over-train your neural net, also. Trying to force a result can backfire, especially if you deal with multiple fonts.

[TALucas]

Let me make sure I understand the problem. You have a NN that has many inputs, and just one output? But you don't know how to convert that one output value into the letter it's supposed to represent? I'm not sure how much you can vary this, so I'll throw out a few ideas.

I think you could do it several ways....and results vary based on training.

1. Divide the output value of a single neuron into ranges ( .0=A, .3=E, .5=M, etc.)

2. Have 8 outputs with a value of 1 or 0, and have the combination of the 8 represent the binary/ascii value. (lower case a= ASCII 97, 01100001)

[ibad]

One more question for anyone who can help... anyone have a primer on reinforcement learning rule?

I got a simple formula for it from a book:

∆Wij = ρ ⋅ r ⋅ (xi - pi ) ⋅ xj + λ ⋅ ρ ⋅ (1-r)(1- xi - pi ) ⋅ xi

where:
pi is the probability that unit i will fire, and r and λ are constants

ok first question... what the hell is mue (p like thingy, wrong spelling I know)?

Is mue also a constant?

Also Pi.. that seems like the presynaptic neuron... is it? That seems counter intuitive to me, to use the presynaptic neuron's activation, but fine... is it actually supposed to be the presynaptic or is Pj the presynaptic neuron's porbability?

In a spiking network the "activation" is always 1 or 0 right?

By the way. I see the words "features" used a lot, and "feature" detectors. What does "feature" mean? is it the name of individual neurons in a particular layer or what?

What are "labels"? what does it mean to have a neural net without labels? Labels are just specific inputs that symbolically repesent a particular output that a hebbian NN would have to learn to associate with an input vector right?

What do labels mean in an MLP? the output neurons? (or combinations thereoff, if we using a binary encoding scheme)?

Thanks!!

[Predictor]

ibad said:

Hi... I know this is a bit esoteric, but if some one can help me with some basic neural netting I would be grateful.

It's for use in a simple letter-by-letter OCR project that is geared towards printed text in a few fonts with variable size, and some slight random angle applied etc.

Anyway... some basic conceptual questions...

I was reading about multi-layer perceptrons and it seems that each output variable is output as a number by an output neuron that represents that variable. In my case it's just one variable... we want to know which letter it is.

Well... I can imagine an ANN outputting various arbitrary numbers as codes for different letters, but biological neurons only spike, 0 or 1, right? So how would a bio-mimetic ANN encode the output?

One way I can think of is have the network select one output neuron and each output neuron represents one character, and the output is based on which output neuron is firing.

Is this a valid and do-able approach? Are there any other approaches?

How would I train an NN that is asserting multiple output neurons when it should just choose one? What if one of them is the right one or none of them is? Ofcourse we're talking about premade data-sets and supervised learning here.

Should I have "multiple" levels of wrongness..punishing multiple outputs (confusion) more, especially if none of them is the right one?

What about spiking frequency on a single output neuron, is that a better output encoding?

You will want multiple output nodes (one for each target class: 'A', 'B', etc.) or you will need to build multiple models (again, one for each output class). Strictly speaking, multilayer perceptrons do not "spike" the way biological neurons do: there is simply a number (typically ranging from 0.0 to 1.0 or -1.0 to +1.0) which is an analogue to "spiking frequency". On the output nodes, these represent a confidence in each target class: choose the one with the highest output.

[Predictor]

TALucas said:

Let me make sure I understand the problem. You have a NN that has many inputs, and just one output? But you don't know how to convert that one output value into the letter it's supposed to represent? I'm not sure how much you can vary this, so I'll throw out a few ideas.

I think you could do it several ways....and results vary based on training.

1. Divide the output value of a single neuron into ranges ( .0=A, .3=E, .5=M, etc.)

2. Have 8 outputs with a value of 1 or 0, and have the combination of the 8 represent the binary/ascii value. (lower case a= ASCII 97, 01100001)

Both of these solutions would be much more difficult to produce than having multiple output nodes or multiple models. The trouble is that the function to be learned is very complicated. Think about the ASCII encoding: any given bit will be 1 for an arbitrary collections of letters, regardless of their visual similarity or difference.