This Library's Relationship to Boost.Spirit

	Note
	If you are familiar with Spirit 2 and/or Spirit X3, you may be interested in this section. If you are not, and you have not read the tutorial for Boost.Parser yet, very little of this will make sense.

Boost.Spirit is a library that is already in Boost, and it has been around for a long time.

However, it does not suit user needs in some ways.

Spirit 2 suffers from very long compile times.
Spirit 2 has error reporting that requires a lot of user intervention to work.
Spirit 2 requires user intervention, including a (long) recompile, to enable parse tracing.
Spirit X3 has rules that do not compose well — the attributes produced by a rule can change depending on the context in which you use the rule.
Spirit X3 is missing many of the convenient interfaces to parsers that Spirit 2 had. For instance, you cannot add parameters to a parser (see description of _locals() in More About Rules).
All versions of Spirit have Unicode support, but it is quite difficult to get working.

I wanted a library that does not suffer from any of the above limitations. It should be noted that while Spirit X3 only has a couple of flaws in the list above, the one related to rules is a deal-breaker. The ability to write rules, test them in isolation, and then re-use them throughout a complex parser is essential.

Though no version of Boost.Spirit (Spirit 2 or Spirit X3) suffers from all those limitations, there also does not exist any one version that avoids all of them. Boost.Parser does so. However, there are a lot of great ideas in Boost.Spirit that have been retained in Boost.Parser. Both libraries:

use the same operator overloads to combine parsers;
use approximately the same set of directives to influence the parse (e.g. lexeme[]);
are built around a flexible parse context object that has state added to and removed from it during the parse (again, comparing to Spirit X3).

The Spirit X3 rule problem

Some readers have wanted a concrete example of my claim that Spirit X3's rules do not compose well. Consider this program.

#include <boost/spirit/home/x3.hpp>

#include <iostream>
#include <set>
#include <string>
#include <vector>


namespace x3 = boost::spirit::x3;
using ints_type = x3::rule<class ints, std::vector<int>>;
BOOST_SPIRIT_DECLARE(ints_type);

x3::rule<class ints, std::vector<int>> ints = "ints";
constexpr auto ints_def = x3::int_ % ',';
BOOST_SPIRIT_DEFINE(ints);

#define FIXED_ATTRIBUTE 0


int main()
{
    std::string input = "43, 42";
    auto first = input.begin();
    auto const last = input.end();
#if FIXED_ATTRIBUTE
    std::vector<int> result;
#else
    std::set<int> result;
#endif
    bool success = x3::phrase_parse(first, last, ints, x3::space, result);
    if (success) {
        // We want this to print "43 42\n".
        for (auto x : result) {
            std::cout << x << ' ';
        }
        std::cout << "\n";
    }

    return 0;
}

Defining FIXED_ATTRIBUTE to be 1 leads to a well-formed program that prints "42 43\n" instead of the desired result. The problem here is that if you feed an attribute out-param to x3::phrase_parse(), you get the loose-match semantics that Spirit X3 and Boost.Parser both do. This is a problem, because the user explicitly asserted that the type of the ints rule's attribute should be std::vector<int>. In my opinion, this code should be ill-formed with FIXED_ATTRIBUTE == 1. To make it well-formed again, the user could use ints_def directly, since it does not specify an attribute type.

When the user explicitly states that a type is some fixed T, a library should not ignore that. As a user of X3, I was bitten by this in such a way that I considered X3 to be a nonviable option for my uses. I ran into a problem that resulted from X3's ignoring one or more of my rules' attributes so that it made the parse produce the wrong result, and I could see no way to fix it.

When a library provides wider use cases via genericity, we generally consider this a good thing. If it is too loose in its semantics, we generally say that it is type-unsafe. Using rules to nail down type flexibility is one way Boost.Parser tries to enable genericity where it is desired, and let the user turn it off where it is not.