Serialized objects continuing to grow in size.

I have a problem. I'm Serializing an Object, but somehow the size of the serialization keeps growing.
I can't give any details when or how, because it rarely happens, and even if it does, it's really hard to say, what caused it.
But it's a problem.

IO part looks like this:

public static Network loadFromFile(File file) throws IOException, ClassNotFoundException{        
        ObjectInputStream in = new ObjectInputStream(new FileInputStream(file));
        Network n = (Network)in.readObject();        
        in.close();        
        return n;
}
    
public void writeToFile(File file) throws FileNotFoundException, IOException{        
        ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(file));    
        if(file.exists())file.delete();        
        out.writeObject(this);
        out.close();        
}

A simple Neural network, as simple as that:

jv3.HiddenNode@5a34ab7a node : 
    jv3.HiddenNode@23239010 12.0
    jv3.HiddenNode@2aefd637 13.0
    jv3.HiddenNode@5a7af475 12.0
    jv3.HiddenNode@7a043387 13.0
    jv3.HiddenNode@36eac1a3 12.0

jv3.HiddenNode@588ce62e node : 
    jv3.HiddenNode@44a5ab44 59.0

jv3.HiddenNode@ad31191 node : 
    jv3.HiddenNode@73844508 63.0

jv3.HiddenNode@4f955775 node : 
    jv3.HiddenNode@ad31191 16.0
    jv3.HiddenNode@73844508 15.0
    jv3.HiddenNode@697bbc44 15.0
    jv3.HiddenNode@1d17b521 16.0

jv3.HiddenNode@4a96e8fb node : 
    jv3.HiddenNode@23239010 19.0

jv3.HiddenNode@23239010 node : 
    jv3.HiddenNode@7a043387 10.0

jv3.HiddenNode@16f91dd6 node : 

jv3.HiddenNode@26130488 node : 
    jv3.HiddenNode@697bbc44 37.0
    jv3.HiddenNode@366956 37.0

jv3.HiddenNode@280f4c4 node : 
    jv3.HiddenNode@26130488 9.0
    jv3.HiddenNode@73844508 10.0

jv3.HiddenNode@590864d2 node : 
    jv3.HiddenNode@73844508 18.0
    jv3.HiddenNode@79f65b0e 18.0
    jv3.HiddenNode@697bbc44 19.0
    jv3.HiddenNode@67678515 18.0

jv3.HiddenNode@2aefd637 node : 
    jv3.HiddenNode@5385bb82 51.0

jv3.HiddenNode@73844508 node : 
    jv3.HiddenNode@5a34ab7a 32.0
    jv3.HiddenNode@4a96e8fb 33.0
    jv3.HiddenNode@16f91dd6 33.0
    jv3.HiddenNode@590864d2 32.0
    jv3.HiddenNode@697bbc44 33.0
    jv3.HiddenNode@5d8a989a 32.0
    jv3.HiddenNode@3389a966 32.0

jv3.HiddenNode@79f65b0e node : 
    jv3.HiddenNode@5385bb82 -7608296.0
    jv3.HiddenNode@7a043387 -7608296.0
    jv3.HiddenNode@3244eabe -7608296.0
    jv3.HiddenNode@1548411c -6938598.0
    jv3.HiddenNode@208ca47 -3774652.0

jv3.HiddenNode@5a7af475 node : 
    jv3.HiddenNode@1dda486e -7623605.0
    jv3.HiddenNode@2ba95c45 -7421056.0
    jv3.HiddenNode@1dc9cea8 -196376.0

jv3.HiddenNode@697bbc44 node : 
    jv3.HiddenNode@4f955775 31.0
    jv3.HiddenNode@3244eabe 31.0
    jv3.HiddenNode@7917ef91 31.0

jv3.HiddenNode@5385bb82 node : 
    jv3.HiddenNode@25f39756 -7142524.0

jv3.HiddenNode@7a043387 node : 
    jv3.HiddenNode@73844508 9.0
    jv3.HiddenNode@697bbc44 10.0

jv3.HiddenNode@3244eabe node : 
    jv3.HiddenNode@16f91dd6 -7142524.0
    jv3.HiddenNode@15b3bfc5 -1045692.0

jv3.HiddenNode@5d8a989a node : 
    jv3.HiddenNode@4a96e8fb -7600510.0
    jv3.HiddenNode@16f91dd6 -7608769.0
    jv3.HiddenNode@62c78a6f -7419573.0
    jv3.HiddenNode@63283018 -392239.0

jv3.HiddenNode@25f39756 node : 
    jv3.HiddenNode@5a34ab7a -7622884.0
    jv3.HiddenNode@3244eabe -7622885.0
    jv3.HiddenNode@5bdc9a1a -5945430.0

jv3.HiddenNode@725278fd node : 
    jv3.HiddenNode@ad31191 -7580507.0
    jv3.HiddenNode@4f955775 -7580507.0
    jv3.HiddenNode@1bb38339 -3968263.0

jv3.HiddenNode@1dda486e node : 
    jv3.HiddenNode@16f91dd6 -7608399.0
    jv3.HiddenNode@5a7af475 -7608398.0
    jv3.HiddenNode@6b70cd6 -5945472.0

Takes 4mb of space, when it should take about 8kb. I have no idea what causes it.
I printed all the neurons and connections between them(previous output), there's nothing that could cause such a jump in space.
Am I doing something wrong on Object output?