Support specification of external user-defined macros

[dmikushin]

This PR adds support for external user-defined macros, e.g. those that may come from the compiler command line options when we do gcc -DFOO=\"bar\".

Currently, tcpp assumes all macro definitions are a part of the source code text. However, compilers generally allow specifying extra macros from the command line. Moreover, compilers may implicitly define builtin macros, e.g. __cplusplus.

Therefore, I thought we need a way to provide the tcpp preprocessor with a pre-existing symbol table. It's just as easy as adding a couple of lines of code.

A typical usage for this feature would be a tcpp driver program below. In order to avoid creation of symbols table by hand, we ask tcpp to parse a pretty-printed text of macro definitions. Then this symbols table is retrieved and added back upon constructing the preprocessor for the target source file:

#include "tcppLibrary.hpp"

#include <cstdio>
#include <cstdlib>
#include <string>
#include <fstream>
#include <streambuf>
                 
int main(int argc, char* argv[])
{
	if (argc < 2)
	{
		printf("Usage: %s <source_file> [MACRO[=VAL]]...\n", argv[0]);
		return 0;
	}

	std::string source = argv[1];

	std::ifstream t(source);
	if (!t.is_open())
	{
		fprintf(stderr, "Cannot open file %s, aborting\n", source.c_str());
		exit(EXIT_FAILURE);
	}
	
	std::string inputSource((std::istreambuf_iterator<char>(t)),
		std::istreambuf_iterator<char>());
	
	using namespace tcpp;
	
	auto errorCallback = [&source](const TErrorInfo& info)
	{
		const std::string errorStr = ErrorTypeToString(info.mType);
		fprintf(stderr, "%s:%d error: %s\n", source.c_str(), static_cast<int>(info.mLine), errorStr.c_str());
	};

	auto includeCallback = [](const std::string& path, bool isSystem)
	{
		printf("including %sfile %s", isSystem ? "system " : "", path.c_str());
		
		return std::make_unique<StringInputStream>("");
	};

	// Add user-defined macro definitions.
	bool skipComments = false;
	Preprocessor::TSymTable userDefines;
	{
		std::stringstream ss;
		for (int i = 2; i < argc; i++)
		{
			std::string userDefineStr = argv[i];
			std::size_t pos = userDefineStr.find('=');
			if (pos != std::string::npos)
			{
				std::string key = userDefineStr.substr(0, pos);
				std::string value = userDefineStr.substr(pos + 1);

				ss << "#define ";
				ss << key;
				ss << " ";
				ss << value;
				ss << std::endl;
			}
			else
			{
				ss << "#define ";
				ss << userDefineStr;
				ss << std::endl;
			}
		}

		// In order to avoid creation of symbols table by hand,
		// we ask tcpp to parse a pretty-printed text of macro definitions. 
		Lexer lexer(std::make_unique<StringInputStream>(ss.str()));

		Preprocessor::TPreprocessorConfigInfo config { errorCallback, {}, skipComments };

		Preprocessor preprocessor(lexer, config, userDefines);

		preprocessor.Process();
		
		userDefines = preprocessor.GetSymbolsTable();
	}

	Lexer lexer(std::make_unique<StringInputStream>(inputSource));

	Preprocessor::TPreprocessorConfigInfo config { errorCallback, includeCallback, skipComments };
	
	Preprocessor preprocessor(lexer, config, userDefines);
	preprocessor.Process();

	return 0;
}

[bnoazx005]

Maybe it's better to move userDefines into TPreprocessorConfigInfo type to minimize number of arguments