Re-using text, tokens, or trees?
When processing a header file, we want to remember what we read so it's faster the next time.
We have a choice between:
Saving the text in the buffer. Simple, low-overhead, but doesn't buy much.
Saving the tokens in the buffer, either before or after preprocessing. Requires new memory intensive data structure.
Saving the semantic data resulting from the header files -
The latter gives us the biggest potential pay-off, so that is what we do.