Jump to content


Check out our Community Blogs

Register and join over 40,000 other developers!


Recent Status Updates

View All Updates

Photo
* * * * * 4 votes

Goopies - Evolving neural networks (w.i.p.)


  • Please log in to reply
135 replies to this topic

#121 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 02 September 2014 - 08:01 AM

Nino I think I love you, goopies is such an awesome project. :P

 

Lol! Thank you Kad. This really gives me energy to keep going. :D

 

 

Thanks for your input, I revived my project last night and managed to get it sort of working again ... I wrote it in Actionscript 3 a while back so I did a dirty port to javascript / html5 canvas ... the speed sucks in a browser and JS isn't the best language for this kind of things ... so maybe I'll start working on my CPP skills.

 

I defenitely recommend using C/C++. Neuroevolution works best when it can use all of the processor's power. :)

 

 

Do the Guppy's thrusters always fire with the same intensity or is the force calculated from the NN output ? couldn't quite figure that from your answer. In my project the output is directly linked to the amount of thust so the canceling out part you described will be quite hard to accomplish since the outputs would have to be almost the same for every thuster and a big difference in thrust will get the little fellows in quite a dizzying spin  :biggrin:

 

You are right. Guppies' thrusters are directly linked the their Neural Network, and thus can output any number from 0 to 1. So perfect canceling is indeed very unlikely to happen. In fact, when Guppies are first initialized (and are thus completely stupid), they do little more than dizzying spins!

 

 

I found out the sight sensors in my project were a bit crude too, just letting a 'Critter' (as I called them) know the distance vector of the nearest other Critter and the nearest food. This works but I really dig your idea of giving them a virtual eye that mimics nature. How do you calculate the amount of light hitting one of the photo-cells ? I just figured I divide the RGB values by the length of the ray that hits the object ... but that seems a bit simplistic, although it works quite well.

 

I'd say that programming the Guppies' Eyes ad Skin sensors took about 75% of all the total work on the project! I tried several models...

 

Firstly I used Box2D RayCasting system to directly cast rays outwards from each Guppy eye/skin cell into the world. Whenever these rays hit another entity, the corresponding cell would fire an input into the Guppy brain. Its intensity would depend on the distance at which the ray contacted the entity. This worked well, but I discovered it was VASTLY inefficient (as for each Guppy being simulated, I'd had to cast a ** of a lot of rays. Box2D was really struggling and I could not have more that 40 Guppies running simultaneously without experiencing severe lag).

 

What I do now is have a single eye sensor (that has the shape of the Guppy's field of view) and a single skin sensor (with the shape of a large circle around each Guppy). When an entity enters any of these sensors it is added to a render Queue, which is owned by the Guppy that owns the sensor. On each loop, each Guppy iterates through its render queue and calculates the level of activation of each cell depending on the angle, the distance and the size of the entity on the Queue, efectively "Renderind" a 1D image of its world (in color for the eye, and monochrome for it's skin).

 

Figuring out how to do this was a huge pain! I even had to learn some Calculus and I'm not a good mathematician! The results, however, are crazy good. Guppies now receive a much smoother and realistic input and it's all done much faster than Ray Casting. I can now have up to 200 Guppies being run simultaneously. :)


  • 2

#122 RobTheeuwes

RobTheeuwes

    CC Lurker

  • New Member
  • Pip
  • 6 posts

Posted 02 September 2014 - 11:07 AM

Nice solution on the sensors ...

 

raycasting (kinda) works for me but my Critters don't have that many sensors (yet) and for a 40 degree vision I only use 4 rays, so my creatures have some blind spots.... yeah i cheated ;)

 

But doesn't your method require the same amount of processing power ? I guess you just do a basic hittest of all other entities on the area of the sensor right ? or does your math involve a faster way of deciding if a shape/entity intersect with the shape of your sensor ? My math knowledge is fairly limited too, being mostly a web-developer but doing things like this does make me regret not paying more attention in highschool  :P

 

I think I will invest some time to learn a bit more about CPP and graphics in CPP .. would you advise using OpenGL above than plain 2d gfx engines like SDL or something like that ? I develop using Linux but if I do create something stand-alone I would like it to be cross platform.

 

I did have a nice idea for your Guppy's by the way ... would it be hard to implement a sort of digital pheromones ? it could be in the form of particles that a Guppy can leave behind that attracts/repels other Guppies (depending on their 'mood' / color) ... or maybe pheromones could be used by Guppies too mark territory ... who knows it would make them guard their own piece of the Tank ?


  • 0

#123 MartinsAbilevs

MartinsAbilevs

    CC Lurker

  • New Member
  • Pip
  • 4 posts

Posted 17 September 2014 - 11:05 PM

inspired by gopies i get my own ann project to som usable level :)

 

e9x4wm.pngtoday i realize one fact.  big brains is not the best for simple task :D  i reduced HL grid size from 8x8 to 4x4  and my creatures learn avoid danger in 10-20 gen.   yesterday i run all day simulations without any succses


  • 0

#124 MartinsAbilevs

MartinsAbilevs

    CC Lurker

  • New Member
  • Pip
  • 4 posts

Posted 18 September 2014 - 03:49 AM

one smal video 


  • 0

#125 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 24 September 2014 - 03:21 PM

But doesn't your method require the same amount of processing power ? I guess you just do a basic hittest of all other entities on the area of the sensor right ? or does your math involve a faster way of deciding if a shape/entity intersect with the shape of your sensor ? My math knowledge is fairly limited too, being mostly a web-developer but doing things like this does make me regret not paying more attention in highschool  :P

 

I let the Box2D engine do all the sensor work for me. My guess is that a Box2D Ray-Cast requires a similar amount of computation as a Sensor Collision test. Therefore having just 2 sensors (one for the eye and one for the skin) requiers MUCH less computation than having a large amount of rays being casted outwards from the Guppy (I used to cast 30 rays from the eyes + 120 rays from the skin!!).

 

So, with this new method, once the single skin or eye sensor is "hit" by another entity, doing the Calculus needed to (duh) calculate the activation of each eye/skin cell if a breeze for the computer!

 

As a side note, I hardly learned any Calculus on high-school either. The internet is a better teacher, lol!

 

 

I think I will invest some time to learn a bit more about CPP and graphics in CPP .. would you advise using OpenGL above than plain 2d gfx engines like SDL or something like that ? I develop using Linux but if I do create something stand-alone I would like it to be cross platform.

 

I did have a nice idea for your Guppy's by the way ... would it be hard to implement a sort of digital pheromones ? it could be in the form of particles that a Guppy can leave behind that attracts/repels other Guppies (depending on their 'mood' / color) ... or maybe pheromones could be used by Guppies too mark territory ... who knows it would make them guard their own piece of the Tank ?

 

SDL and SFML both use OpenGL internally. Both are VERY simple to use. SDL is written in C and SFML in CPP. Both are fully cross-platform. :)

 

Your pheromore idea is great! However, I'm still trying to get the Guppies to do very simple learning using indirect (HyperNeat) encoding, so this is my current goal before complexifying their world even further. ;)


one smal video 

 

Your simulation looks interesting. But I can't tell, what are your agents supposed to be learning? What do they need to do to survive and what are the hazards of their world?


  • 0

#126 RobTheeuwes

RobTheeuwes

    CC Lurker

  • New Member
  • Pip
  • 6 posts

Posted 25 September 2014 - 12:52 AM

Hi Nino, 

 

I got the sensors working in my (still web / JS based) version in about the same way (i think), by just creating a cone shaped polygon and doing a SAT test on all other objects and then figure out the relative angle and distance ... it is a lot faster in my case because the heavy calculations are only done when needed. 

 

You are correct though, internet is a better teacher but I still wish they'd make math in school more interesting by showing real world applications like game-development instead of just the boring stuff we had to learn about 20 years ago (damn that makes me feel old saying that :P

 

SMFL is quite simple indeed, got a simple demo running in minutes, but still CPP isn't my strong suite and keeps annoying me with it's language constructs ;) But I am getting there.

 

I will look into hyperneat soon, but I am first going to update the neural net to an Elman network instead of just a basic feedforward which is always fully connected ... I even convinced my boss to reserve some time to write a better NN library, since we use it in a couple of our applications for analyzing data. 

 

Cheers


  • 0

#127 RobTheeuwes

RobTheeuwes

    CC Lurker

  • New Member
  • Pip
  • 6 posts

Posted 25 September 2014 - 01:11 AM

Ohyeah .. forgot to ask. 

 

How do Goopies die ?... if their energy runs out they die and leave a pellet of food, but are they spawned again in a complete random way, or cloned from one of the survivors with some minor mutations ? 

 

My version uses a cycle of a certain amount of ticks after which the 'genes' (NN connection weights) are run through a Genetic Algorithm, killing about 70% of the little fellows and then mating the remaining ones until the right population size is reached and then feed the genes from each member of the population into the corresponding Critter's neural network. This makes the simulation 'jumpy' after the GA cycle because some Critters suddenly act very different... that's to be expected since I give them a digital lobotomy ... but I would like to have a more delicate way of brain surgery ;) 

 

Thanks again for the insights


  • 0

#128 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 25 September 2014 - 06:44 AM

Ohyeah .. forgot to ask. 

 

How do Goopies die ?... if their energy runs out they die and leave a pellet of food, but are they spawned again in a complete random way, or cloned from one of the survivors with some minor mutations ? 

 

My version uses a cycle of a certain amount of ticks after which the 'genes' (NN connection weights) are run through a Genetic Algorithm, killing about 70% of the little fellows and then mating the remaining ones until the right population size is reached and then feed the genes from each member of the population into the corresponding Critter's neural network. This makes the simulation 'jumpy' after the GA cycle because some Critters suddenly act very different... that's to be expected since I give them a digital lobotomy ... but I would like to have a more delicate way of brain surgery ;)

 

Thanks again for the insights

 

The way Guppies die and compete for energy is still not fixed. As this is one of the most important aspects of any A-Life simulation (it pretty much defines the "fitness" function), getting it right requires experimentation, thus I'm currently experimenting with it.

 

What has become a sort of "cannon", however, is that Zappers "zap" energy out of the Guppies, Pellets give them energy, Guppies waste energy just by being alive/moving/changing color/etc. and die when their energy runs out. The genetic algorithm (the real-time version of HyperNeat) then searched for dead Guppies and replaces them one by one. The new Guppies inherit their genes from the best performing Guppies that are still alive on the field (HyperNeat describes the way on which crossover and mutation occurs). And thus the cycle goes on. :)


  • 0

#129 RobTheeuwes

RobTheeuwes

    CC Lurker

  • New Member
  • Pip
  • 6 posts

Posted 25 September 2014 - 06:52 AM

Nice ... thanks 

 

I will look into (hyper)NEAT ... sounds cool ... not entirely different from my current implementation except that HyperNEAT can alter the complete network structure where my implementation just fumbles around with the weights in the network. 

 

I hope to publish an example shortly of the html5/js version ... it isn't nearly as cool looking but it does do the trick since my Critters seem to become less of a moron (calling them intelligent would be a insult to everything with a working brain ;)


  • 0

#130 RFpi

RFpi

    CC Lurker

  • New Member
  • Pip
  • 9 posts

Posted 24 November 2014 - 08:29 AM

[...]

 

I know the algorithm works excellently well with the NeuroRacers I made before. So my best bet is that HypeNeat is better suited to "static" worlds, that are perceived equally among all agents.

 

[...]

 

If you take an evolved agent from a track, and put it in another track, does it behave correctly, or it only memorizes the previous track and act strangely?


Edited by RFpi, 24 November 2014 - 08:32 AM.

  • 0

#131 TheGentleman

TheGentleman

    CC Lurker

  • New Member
  • Pip
  • 3 posts

Posted 05 July 2015 - 12:36 AM

Its been nearly a year and no updates! Is this dead? I hope not but it looks like it.


  • 0

#132 mbcev

mbcev

    CC Regular

  • Member
  • PipPipPip
  • 37 posts
  • Location:Scottsdale
  • Programming Language:C, Java, C#, Others
  • Learning:Others

Posted 06 July 2015 - 10:15 AM

Its been nearly a year and no updates! Is this dead? I hope not but it looks like it.

 

On a related note another neuro-evolution AI project worth looking into if it suits you and if it turns out the thread/projects herein are dead: youtube.com/watch?v=qv6UVOQ0F44

 

-edit-

 

I'd also like to point out that Mar/IO (link above) uses the NEAT algorithm.


Edited by mbcev, 06 July 2015 - 10:17 AM.

  • 0




Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download