9 replies to this topic

[sakishrist]

Hello,

I have a weird problem, an "Out of memory" one. The weird thing is that the script I have is not even started when I get that error (I mean, I have a print line in the very beginning and it is not printed) and the memory meter does not move at all. Please help me with this situation.

My OS is Ubuntu.

I have marked the lines that when removed the "Out of memory" error is gone:

#!/usr/bin/perl






sub dictionary
{

    while (($n = read FILE, $data, 1) != 0) { 
        $nr += $n;
    
        if($data eq "e") {return;}
        if($data eq "d") {dictionary();}
        if($data eq "l") {list();}
        if($data eq "i") {integer();}

        if($data eq ":"){
            print LOG $buf."\n\n";
            [COLOR=red]$n = read FILE, $data, $buf;[/COLOR]
            $nr += $n;
            [COLOR=red]$buf="";[/COLOR]
            print LOG $data."\n\n";
            

        } [COLOR=red]else {
        $buf .= $data;}[/COLOR]
    }
    
}

sub list
{

    while (($n = read FILE, $data, 1) != 0) { 
        $nr += $n;
    
        if($data eq "e") {return;}
        if($data eq "d") {dictionary();}
        if($data eq "l") {list();}
        if($data eq "i") {integer();}

        if($data eq ":"){
            print LOG $buf."\n\n";
            [COLOR=red]$n = read FILE, $data, $buf;[/COLOR]
            $nr += $n;
            [COLOR=red]$buf="";[/COLOR]
            print LOG $data."\n\n";
            

        } [COLOR=red]else {
        $buf .= $data;}[/COLOR]
    }

}

sub integer
{

    while (($n = read FILE, $data, 1) != 0) { 
        $nr += $n;
    
        if($data eq "e") {return;}

        $buf .= $data;
        
    }
    #print LOG $buf."\n\n";
    print $n."\n";
}

{
print "started";
open FILE, "test.torrent" or die $!;
open LOG, ">log" or die $!;
binmode FILE;

my ($buf, $data, $n, $nr);
$nr=0;

while (($n = read FILE, $data, 1) != 0){
    
    if($data eq "d") {dictionary();}
    if($data eq "l") {list();}
    if($data eq "i") {integer();}
}

close(FILE);
close(LOG);
}

The purpose of the file is to read a torrent file section by section and it is incomplete.

Thanks in advance

Edited by sakishrist, 13 April 2011 - 11:11 AM.

[dargueta]

Weird...is this all of your code?

[sakishrist]

Yes, it's all of it.

It seems like perl is "predicting" that the loop with the constant adding to the the buffer will never end and thus will lead to an out of memory situation. So I spent quite a lot of time trying to figure if this really leads to an infinite loop but I think it does not.

[dargueta]

Weird. I ran it on my system with a random binary file I generated and got the output "starting0" (no quotes).

[sakishrist]

Could this be some problem with my version? This is what I get when I type --version:

~$ /usr/bin/perl --version

This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi
(with 40 registered patches, see perl -V for more detail)

Could you please tell me what your version is?

Or ... could it be some setting about the memory?

Thanks

[dargueta]

This is perl, v5.10.1 (*) built for i686-linux-gnu-thread-multi (with 40 registered patches, see perl -V for more detail)

Basically the only difference is that mine is 32-bit. Try running ulimit -a and post back your output.

[sakishrist]

This is what I get:

~$ ulimit -a

core file size          (blocks, -c) 0

data seg size           (kbytes, -d) unlimited

scheduling priority             (-e) 20

file size               (blocks, -f) unlimited

pending signals                 (-i) 16382

max locked memory       (kbytes, -l) 64

max memory size         (kbytes, -m) unlimited

open files                      (-n) 1024

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

real-time priority              (-r) 0                                                                                                                                              

stack size              (kbytes, -s) 8192                                                                                                                                           

cpu time               (seconds, -t) unlimited                                                                                                                                      

max user processes              (-u) unlimited                                                                                                                                      

virtual memory          (kbytes, -v) unlimited                                                                                                                                      

file locks                      (-x) unlimited

[dargueta]

Um...weird...I'll scrounge around a bit later today and see what I can find.

[sakishrist]

Thanks a lot!

EDIT:
Strange, the problem seems to be the file. I changed it with an empty one and it worked (well ... it did not give me the Out of memory error), I changed it with an 800KB binary file and again, no error, but when I used a 13KB torrent file it gave me the error again. It is strange because the file is never fully loaded into memory, I only load portions of it, (write them to the log), and then load some new information in the same variable.

I guess any torrent would reproduce the error (at least some that I downloaded from privet trackers did).

Thanks

EDIT 2:
The script actually starts, the problem is that perl (automatically) flushes the output on new line only. So when I added "\n" at the end of the message it was printed. Now I need to find where the error occurs.

Edited by sakishrist, 13 April 2011 - 09:11 AM.

[sakishrist]

Now everything works fine! The problem was actually overloading of a variable I guess. I fixed that and now it runs smoothly!

Thanks for the help.