I'm not sure about tokenization (not sure what that actually is), but one thing that your code is doing is checking whether a file is equal to false or 0 after you open it. The method you should be using is
if (!infile.is_open()), for example. Another, perhaps better, way you could check is to check the failbit using
if (infile.fail()) since the failbit is set if the requested file is not opened or the variable already represents an open file (calling open() on the same variable without closing it properly makes the call to open() fail, setting the failbit).
As for file comparison, perhaps you would prefer binary read/write instead of text. That way, you could compare the bytes rather than just text, where text may not actually be text. Also, if you wanted to, you could shorten your code a bit:
Code:
//just remove the "| ifstream::binary" if you want to use text
ifstream infile("pdf1.txt", ifstream::in | ifstream::binary);
ifstream infile2("pdf2.txt", ifstream::in | ifstream::binary); You could simply have them on the same line like you do in your code, but I split them up for slightly better readability.