[FIXED] Base64 encode file by chunks

Issue

I want to split a file into multiple chunks (in this case, trying lengths of 300) and base64 encode it, since loading the entire file to memory gives a negative array exception when base64 encoding it. I tried using the following code:

int offset = 0;
bis = new BufferedInputStream(new FileInputStream(f));
while(offset + 300 <= f.length()){
    byte[] temp = new byte[300];
    bis.skip(offset);
    bis.read(temp, 0, 300);
    offset += 300;
    System.out.println(Base64.encode(temp));
}
if(offset < f.length()){
    byte[] temp = new byte[(int) f.length() - offset];
    bis.skip(offset);
    bis.read(temp, 0, temp.length);
    System.out.println(Base64.encode(temp));
}

At first it appears to be working, however, at one point it switches to just printing out "AAAAAAAAA" and fills up the entire console with it, and the new file is corrupted when decoded. What could be causing this error?

Solution

skip() "Skips over and discards n bytes of data from the input stream", and read() returns "the number of bytes read".

So, you read some bytes, skip some bytes, read some more, skip, .... eventually reaching EOF at which point read() returns -1, but you ignore that and use the content of temp which contains all 0's, that are then encoded to all A's.

Your code should be:

try (InputStream in = new BufferedInputStream(new FileInputStream(f))) {
    int len;
    byte[] temp = new byte[300];
    while ((len = in.read(temp)) > 0)
        System.out.println(Base64.encode(temp, 0, len));
}

This code reuses the single buffer allocated before the loop, so it will also cause much less garbage collection than your code.

If Base64.encode doesn't have a 3 parameter version, do this:

try (InputStream in = new BufferedInputStream(new FileInputStream(f))) {
    int len;
    byte[] temp = new byte[300];
    while ((len = in.read(temp)) > 0) {
        byte[] data;
        if (len == temp.length)
            data = temp;
        else {
            data = new byte[len];
            System.arraycopy(temp, 0, data, 0, len);
        }
        System.out.println(Base64.encode(data));
    }
}

Answered By - Andreas
Answer Checked By - Pedro (JavaFixing Volunteer)