1 reply to this topic

[bbqroast]

Hello, I am writing a program that will at some point be moving large files, so I was wondering- does fopen 'stream' files. As in it doesn't actually load the entire file into RAM instead when it is asked to do something it then does it in chunks??

I mean what the PHP fopen does.

[Alexander]

You will only take up a substantial amount of memory if you tell it to, and as such it does "stream" as you ask. For example if you allocated forty megabytes and fread forty megabytes in to the buffer only then will it need to hold that much for available use.

If you were to use a loop, and fread on a smaller buffer, say four megabytes, it would only use what it needs to store at one time. i.e.

#include <stdio.h>
#include <stdlib.h>
#define FSIZE (1024*1024*4) //four megabyte buffer

int main(int argc, char **argv) {
  FILE* from = fopen("infile", "rb");
  FILE* to   = fopen("outfi;e", "wb");
  char* buffer = malloc(FSIZE);
  
  size_t in_b = 0;
  size_t out_b   = 0;
  
  if(buffer != NULL) {
    while(!feof(from)) {
      in_b  = fread(buffer, 1, FSIZE, from);
      out_b = fwrite(buffer, 1, in_b, to);
      if(in_b != out_b) {
        //total bytes read are not bytes written
      }
    }
  }
    return 0;
}

If it is very large, you may wish to write to a temporary file and verify its size, and only then move it to its proper file name.