67 replies to this topic

[RobotGymnast]

Hello everybody.

I am having an extremely difficult time with my nifty little program. It seems that when I try to allocate memory that I totally have space for in RAM, it keeps throwing bad_alloc exceptions.

A screenshot is attached. Any help would be greatly appreciated. Feel free to ask any questions.

Attached Files

•  Memory Allocation.jpg   476.52K   52 downloads

[roboticforest]

I hate to answer with a question, but are you getting an exception when you allocate smaller amounts of memory? Like a megabyte or something.

[RobotGymnast]

No, In fact, I can allocate 100,000,000 elements of Scenario7 without a problem. And yet, I get these problems even when I have more than sufficient amounts of memory. Is there some sort of cap on allocation that I don't know about?

Edit by the real RobotGymnast (it's a shared account, but I'm posting here so that other users can see what I wrote): There are some limits I've never run into before, but I believe they're above what is being used here, I think it's 4GB or something (although that's for files), it may happen here for some odd reason

Edit by the fake RobotGymnast: 4GB is the maximum addressable memory on an x86 system (notice how its 0xFFFFFFFF bytes?). I'm only using 1.5 GB, so I don't know wtf's going on here.

Edited by RobotGymnast, 04 January 2009 - 06:03 PM.

[dargueta]

There's a 2GB limit on memory addressible by the program. The upper 2GB is reserved by Windows for loading DLLs and stuff, even if you don't need it. Also, an additional 64K is reserved at the beginning and end of that 2GB address space, so really your program can only address 1.9998GB. This includes code, stack, data and heap, so really you're using all the memory you can.

[roboticforest]

Where did you learn that?

[RobotGymnast]

Real RobotGymnast: is that why code starts at 0x400000? (I'm not sure if I have the right number of 0s there)

Fake RobotGymnast: No, 2GB would mean it'd start at 0x7FFF FFFF.
Anyhow, the 2GB thing doesn't apply here for two reasons. First of all, I'm only allocating 1.5GB, second of all, I disabled that option.

Real: Oops yeah I was thinking a quarter

Edited by RobotGymnast, 06 January 2009 - 02:17 PM.

[dargueta]

You disabled what option?

Code can really be loaded anywhere, but generally, yes it is loaded at 0x00004000 or something like that. Also, you must realize that the slightly less than 2GB must hold your program, stack, data and heap (where you're allocating from), so that's why a 1.5GB allocation fails - your program's code and stack have to take up some space.

00000000-0000FFFF: Reserved by the system to catch null pointers.
00010000-7FFEFFFF: Program address space
7FFF0000-7FFFFFFF: Reserved by the system to catch bad pointers.
80000000-FFFFFFFF: Reserved; DLLs are loaded here.

MSDN: Virtual Address Space

Edited by dargueta, 06 January 2009 - 02:17 PM.
Correction

[RobotGymnast]

I'm interested, how does it use that third section to catch bad pointers? The null pointers I could understand, but wouldn't those ONLY point to 00000000?

[dargueta]

Not if you add offsets. For example, last night I was having problems with this section of code:

pRecord = strstr(pRecord,"<p class=\"course") + 16;

If the string <p class="course isn't found in pRecord, the result will be a null pointer, plus the sixteen, so it'd be 0x00000010. That could point to anything. Hence the buffer zone. Besides, since memory used to be segmented, reserving an entire segment (64K) is easier than watching a single byte (check out protected mode paging). The idea just carried over into the modern NT-based systems.

As far as the second buffer zone, you have to remember this is Microsoft we're talking about here. Since when does anything they do make complete sense?

[RobotGymnast]

But what if you add a large offset to a null pointer (I mean theoretically), shouldn't it be checked before any operations (like adding offsets) occur?

dargueta said:

As far as the second buffer zone, you have to remember this is Microsoft we're talking about here. Since when does anything they do make complete sense?

I just meant.. what do bad pointers look like? How do you detect them at all? xD it's true about microsoft though

[dargueta]

Yes, the programmer should check pointers before adding offsets, which is how I fixed my program. Even if the offset is large enough to make the pointer point to something beyond the first segment, it could still be invalid, since only memory actually in use is mapped. Attempts to access unmapped memory result in access violations.

Bad pointers can look like any other pointer, but typically are either 0x00000000 or 0xFFFFFFFF if deliberately set. If you allocate memory, then free it, the pointer to that block of memory is now a bad pointer, but there's really no way to tell just by looking at it. There are other ways of checking with WinAPI, I don't know about standard library functions.

[RobotGymnast]

The option I enabled was being "Large address-aware." (Screenshot attached)

Anyhow, I doubt the stack, initialization, etc take up 500MB. I checked, and even the debug version used only had 3 MB in use at startup.

Attached Files

•  2GB.jpg   66.02K   15 downloads