Jump to content


Check out our Community Blogs

Register and join over 40,000 other developers!


Recent Status Updates

View All Updates

Photo
* * * * * 4 votes

Goopies - Evolving neural networks (w.i.p.)


  • Please log in to reply
135 replies to this topic

#25 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 14 November 2012 - 02:14 PM

It's interesting to note that Roomba's are programmed to spiral clockwise, while your Guppies evolved to spiral counter-clockwise. Wonder if it makes a difference.

I know it goes against the natural evolution of the experiment, but can you manually edit their neural networks? For instance, take one Guppie and "flip" it so it spirals in the opposite direction? Then introduce one of the "flipped" ones to the pool and see if the trait catches on or is bred out after a few generations? This could be analogous to random "mutations" which sometimes get introduced into the gene pool.


I've had many simulations on which they evolve to spin clock-wise as well. Given that their world is absolutely symmetrical I doubt that there's any kind of advantage in spinning in one direction over the other.

The reason that, on a given simulation, all guppies end up spinning the same way is that, once a "spinning gene" forms out of the random mutations that proves advantageous (whether its a "clockwise gene" or a "counter-clockwise gene"), it instantly spreads out and becomes "dominant" among the whole population, which then begins spinning the same way.

Nonetheless, I want to eradicate this obsessive spinning. :)
  • 0

#26 gregwarner

gregwarner

    Obi Wan of Programming

  • Expert Member
  • PipPipPipPipPipPipPip
  • 1586 posts
  • Location:Arkansas
  • Programming Language:C, Java, C++, C#, PHP, Transact-SQL

Posted 14 November 2012 - 02:24 PM

Nonetheless, I want to eradicate this obsessive spinning. :)


Well, seeing as though they consistently evolve that way, it must mean it's an efficient pattern for scouring an area for resources. And since iRobot's engineers programmed their Roombas to do the same, it seems logical.

But if you'd like to see different behavior, perhaps changing the rules of the game so that it's no longer the most efficient way to gather resources...?
  • 0

ti-99-sig.png
Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
– Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid


#27 mbcev

mbcev

    CC Regular

  • Member
  • PipPipPip
  • 37 posts
  • Location:Scottsdale
  • Programming Language:C, Java, C#, Others
  • Learning:Others

Posted 14 November 2012 - 02:37 PM

The dominant reason that our brains evolved as they did was because of movement. More specifically, to optimize our movement. There are energy costs in all movement, far higher than thinking about movement (i.e. planning your actions ahead of time). I'm not sure how your energy function works other than energy exists and is being depleted with time for some reason and being refilled with pellets. I suspect that if you added complexity to your energy function you would see more complex movement. I believe it has little to do with the "shape" of the world.

If standing still uses say 1 unit of energy per time unit, but moving uses say 2-3 units, then there will be times when just standing still would be far more advantageous than moving.
  • 0

#28 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 14 November 2012 - 03:32 PM

The dominant reason that our brains evolved as they did was because of movement. More specifically, to optimize our movement. There are energy costs in all movement, far higher than thinking about movement (i.e. planning your actions ahead of time). I'm not sure how your energy function works other than energy exists and is being depleted with time for some reason and being refilled with pellets. I suspect that if you added complexity to your energy function you would see more complex movement. I believe it has little to do with the "shape" of the world.

If standing still uses say 1 unit of energy per time unit, but moving uses say 2-3 units, then there will be times when just standing still would be far more advantageous than moving.


Hmm, yes. You may be right. Up until now the guppies' energy depleted based only on time. I will add an additional energy cost to movement and color change and see if it produces any significant change in behavioral patterns.
  • 0

#29 mbcev

mbcev

    CC Regular

  • Member
  • PipPipPip
  • 37 posts
  • Location:Scottsdale
  • Programming Language:C, Java, C#, Others
  • Learning:Others

Posted 14 November 2012 - 04:52 PM

I'm not going to claim to be an expert in the field of computational neuroscience haha, but I'm well versed in the surrounding subjects so if you ever want to ask my advice on something, I'm all ears. Like I said, you guys are working on like my absolute favorite stuff in the world. :)

Anyway, I'm pretty sure you'll see that by adjusting your energy function the spinning will stop. That is of course, assuming that they have 360 degree vision, which was my understanding. If not, then it's possible that you'll still see spinning behavior because your food pellets appear randomly and your Zappies move randomly.

Also along the same lines, I know that you're hoping to see more complex behaviors like using color as camouflage and Goopies hunting Goopies and etc, but those types of behaviors only arise when there is reason for them to arise. The reason that you saw the occasional hunters cropping up was because you provided an incentive; that being that dead Goopies provide more energy than pellets. If you want to see Goopies using colors then there has to exist some kind of advantage to it. Otherwise if changing colors costs energy but it helps you in exactly no way what so ever, no Goopies will ever use it. If on the other hand it costs no energy you'll probably evolve disco Goopies if you run enough simulations kind of like you saw with the Nascar Goopies.

I want to say that "there has to be a reason for everything and anything" but I don't know if that's precisely the best way to put it. I feel a slightly truer to life way of phrasing it would be to think of things in terms of a your fitness function and your energy function. You can think of all the possible things that can happen in your energy function like a 3D plane. There will be peaks and valleys on that function depending on various behaviors and ultimately the "law of the universe" is to hang out in stable states (just look at chemistry, physics, biology, etc); which in your case will be the valleys of your energy map because that's where minimal energy is being used (or again you can rephrase it and say the peaks which is where you acquire the most energy, just depends on the problem at hand). So for example if you change your pellet generation from being uniformly distributed to say being a single pole Gaussian, you might find that one of the "optimal" solutions to your fitness function (which we'll get to in a minute) would be to simply figure out where pellets tend to appear, move to that location, and then just sit there because food shows up often enough that you'll be able to gain energy without expending any extraneous movement-based energy. Right? So what I'm getting at is that for most types of behaviors, like communication, team work, hunting, or etc, the gains have to out way the cost otherwise there really won't be any reason for them to evolve. Just because you add in an appendage doesn't mean anyone will choose to use it.

The other thing besides your energy function (which you can think of as the reason why anything gets done) which you need to consider is your fitness function (which you can think of as the solution to a problem, in this case staying alive). I think in your case your fitness function is simply "whoever stays around the longest is the winner and moves to the next round." Correct? (If I'm wrong please correct me, but regardless my point here will still be pertinent) This is the biggest reason why your Nascar drivers crop up uniformly. You're mostly correct in your "genetic inheritance" explanation of why everyone adopts that behavior, but it isn't the reason (which makes absolutely no sense). See the problem is, everyone is basically everyone's clone. It isn't that the Goopies are a tight knit family if the guy who wins the "I stayed alive the longest" contest is the guy whose "neural genes" get passed on to the next generation, rather everyone is just a slightly altered clone of him. This is why you might see hunting behaviors crop up but then never really catch on, because that guy doesn't ever get the chance to have kids so someone from the next generation has to re-figure out that behavior essentially. (Again, assuming I understand your model. everything I've said thus far applies to the idea that you're only selection one Goopie based off of that one criteria to "breed"). Much like how your energy function can be thought of a 3D plane, your fitness function can also be thought of as a 3D plane, or at least you want to think of it that way if you want to "solve" problems. Ultimately that's what both neural networks and genetic algorithms do and what they're for. They're meant to traverse the problem landscape in search of those peaks and valleys, and settling in. So you might not get the mathematically perfect algorithm, but you'll get pretty dang close. And if there are multiple solutions (peaks/valleys depending on how you describe your problem) then again, depending on the luck of the draw, you might find one good solution, run a new experiment and find a completely different solution. Right now because your fitness function is so limiting you aren't traversing that problem landscape very well. Instead of picking the best guy and keeping him around, you could alter your fitness function to say drop the worst X amount, reproduce two of the top Y guys (or even just the top 1 guy), and reproduce one of everyone else. Again, just an example you can play with it as you will, but with this example, while your gene pool is still small, it'll be more closely relatable to families of Goopies which is what will allow for different and more interesting behaviors.

At the end of the day, the more complex your "world" is the more complex behavior that will not only be allowed to evolve, but that will have a "reason" to evolve.
  • 0

#30 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 14 November 2012 - 05:28 PM

At the end of the day, the more complex your "world" is the more complex behavior that will not only be allowed to evolve, but that will have a "reason" to evolve.


Thanks mbcev for all your valuable input! :)

I may have not made it clear before, but the guppies change color so that, when they collide, they actually "transfer" energy from one another. Each guppie can be in one of three color-states. Those three colors transfer energy in a rocks-paper-scissors fashion. So, lets say there are three colors, red, green and blue: whenever two guppies collide red will "zap" energy out of green, green out of blue and blue out of red. I expect this "physical law" to incite guppies to become "hunters" and try to kill each other out (note that the energy they get from zapping each other is actually more than what they get from eating pellets), but of course hunting is a much more complex behavior to evolve and will probably require longer simulation runs.

I've implemented an energetic cost for movement and color change. Remarkably, their movement is now much less circular: they appear to be moving in short segments, then turning, then moving forward again, probably in an attempt of saving energy. I will upload a new vid once I get some time! :)
  • 0

#31 mbcev

mbcev

    CC Regular

  • Member
  • PipPipPip
  • 37 posts
  • Location:Scottsdale
  • Programming Language:C, Java, C#, Others
  • Learning:Others

Posted 14 November 2012 - 05:51 PM

Oh that's right, I forgot about the rock paper scissors game. Herp derp. One thing to note is that because your energy function has changed, depending on how much it takes to move versus how much you gain additional by playing, you have to make it worth while. For example, pellet is one energy, corpse is two, but zapping is three. Or something along those lines. It is more complex to play the game but the gains are better. Otherwise vulture-ism will probably be easier. The trick to all of it is providing incentive for stuff to happen,but not so much that it becomes the only way to win.

Zapping doesn't necessarily have to give more than corpses, just kind of depends on other factors. You want balanced imbalance haha.
  • 0

#32 gregwarner

gregwarner

    Obi Wan of Programming

  • Expert Member
  • PipPipPipPipPipPipPip
  • 1586 posts
  • Location:Arkansas
  • Programming Language:C, Java, C++, C#, PHP, Transact-SQL

Posted 15 November 2012 - 06:37 AM

My question is concerning their vision. Do they have 360 degree vision? If so, then I have a question with regards to hunting: The approaching Hunter spots his Prey, and at the same time, the Prey spots the Hunter. If they begin to evolve intelligent color-shifting behavior, the Prey might try and shift colors at the last second to gain the advantage over the Hunter. And if the Hunter also has intelligent color-shifting behavior, if he's quick enough, he may try and change his color yet again in response to the color-shift of his Prey. Their colors could potentially go psychedelic-disco-rave just as they are about to collide as they rapidly oscillate colors back and forth. Whoever wins might end up being a random event.

Of course, we'll just have to wait and see what happens. I could be wrong. But then again, science is all about proposing an hypothesis and then testing it. I guess the above is my proposed hypothesis. :)
  • 0

ti-99-sig.png
Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
– Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid


#33 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 15 November 2012 - 05:26 PM

My question is concerning their vision. Do they have 360 degree vision? If so, then I have a question with regards to hunting: The approaching Hunter spots his Prey, and at the same time, the Prey spots the Hunter. If they begin to evolve intelligent color-shifting behavior, the Prey might try and shift colors at the last second to gain the advantage over the Hunter. And if the Hunter also has intelligent color-shifting behavior, if he's quick enough, he may try and change his color yet again in response to the color-shift of his Prey. Their colors could potentially go psychedelic-disco-rave just as they are about to collide as they rapidly oscillate colors back and forth. Whoever wins might end up being a random event.

Of course, we'll just have to wait and see what happens. I could be wrong. But then again, science is all about proposing an hypothesis and then testing it. I guess the above is my proposed hypothesis. :)


The guppies have a single compound eye that allows them to see color, with a 60º field of view in the forward direction. Their "skin" however, can "sense" any object 360º around, but cannot tell which color it is. So my idea is that they'll develop somewhat of a hunter-prey dynamic. If one guppie approaches another from its back, the other will sense him coming, but will not be able to "see" what color its hunter is until its probably too late. His best choice, perhaps, will be to try to speed away. The first one, the "hunter" should be able to change colors while approaching. This hunter-prey dynamic, although it sounds cool, is only my hypothesis so far but I have not observed it substantially. :)

Right now I'm working on a Long-Short term memory model for the neural nets. http://en.wikipedia....ort_term_memory Hopefully this model will lead to more complex behavior.
  • 0

#34 Chall

Chall

    CC Addict

  • Senior Member
  • PipPipPipPipPip
  • 349 posts
  • Location:Cedar Rapids, IA
  • Programming Language:Java
  • Learning:C, Java, C++, C#, Python, JavaScript, Assembly

Posted 15 November 2012 - 07:02 PM


Might wanna change the color back to black. White is just too... bright. Black helps the eyes define the shapes they are seeing more easily, and I personally think black would look better than white in this case ;)


It's just a thing of taste, I believe! ;)
Although, more than for aesthetic purposes, the ability to change colors allows for a wider range of experimentation. Note that the Guppies actually "see" the same colors that you see. Therefore, an user could make, for example, all things black, forcing the critters to evolve and fine-tune their skin sensors to identify objects based on their size instead of their color?, or perhaps give the Guppies the ability to camouflage on their environment by shifting colors to match the background, who knows!

Creatures are now using Simple Recurrent Networks, or Elman networks. I've uploaded a new video to youtube. Most interesting moment certainly occurs at about 4:12, when one critter literally saves its neck after being trapped on a tight space! Some other nice behaviour I observed included Sweeping through bunches of food pellets, and (quite remarkable) bumping into other guppies on what seemed attempts to push them into the Zappers so they loose energy and die?? Are the critters already becoming effective hunters??

http://www.youtube.com/watch?v=X7qUH8mSyUE

Hey, and Chall, how is it going with your experiment!? Any progress so far?


I'm taking a break to work on another one of my past projects. :D
  • 0
Speaks fluent Java

#35 alan2here

alan2here

    CC Lurker

  • New Member
  • Pip
  • 8 posts

Posted 17 November 2012 - 01:31 PM

Hey, you got featured by CodeCall.

As well as a viewing mode, can this have a simulation mode, that runs as fast as possible, therefore not showing the simulation, or at least not every step.

I'd love it to have a large number of parameters about the abilities of the creatures and the made up of their surroundings, maybe an option to have overlapping generations as well, instead of in waves as shown in the videos.

Could you make use of this? I wrote it some time ago, "algo" is an algorithm thats dynamic, when they run they compute by editing themselves. They can be created, asked to become random, mutated and also run for some number of steps, remembering their state between runs like a groupie or not, like a function. It's straightforward to specify limits on memory or computation steps for each as well.

with-logic.co.uk/2012/main.cpp
  • 0

#36 nino

nino

    CC Resident

  • Validating
  • PipPipPipPip
  • 66 posts
  • Programming Language:C, C++, PHP, Python, JavaScript

Posted 17 November 2012 - 09:00 PM

Hey, you got featured by CodeCall.

As well as a viewing mode, can this have a simulation mode, that runs as fast as possible, therefore not showing the simulation, or at least not every step.

I'd love it to have a large number of parameters about the abilities of the creatures and the made up of their surroundings, maybe an option to have overlapping generations as well, instead of in waves as shown in the videos.

Could you make use of this? I wrote it some time ago, "algo" is an algorithm thats dynamic, when they run they compute by editing themselves. They can be created, asked to become random, mutated and also run for some number of steps, remembering their state between runs like a groupie or not, like a function. It's straightforward to specify limits on memory or computation steps for each as well.

with-logic.co.uk/2012/main.cpp



Featured by Codecall? I'm a newbie to the fórum so I dont know what means :)

Yes, I have already implemented a way of shutting down the graphics to speed up the simulation. It's what I do usually at the beggining of each run (with gfx off I can get step-rates of almost 300 fps or higher, but that depends on the paramenters of the simulation).

On the final version of the software I do indeed want to provide the user with maximum flexibility, as well as several simulation styles. I already thought of having a continuous simulation style on which one generation "blends" into the next one instead of being generated instantaneously when the previous one dies off.

I'd like to see your link, but it seems to be broken... :(
  • 0




Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download