Continue reading »

All three of Spirit’s sub-libraries – Qi, Karma, and Lex – support semantic actions. In each case they are different and have some specifics. Today I will highlight semantic actions in Qi. But I will dedicate later Tips of the Day to semantic actions in Karma and  Lex.

Semantic actions are functions or function objects attached to some specific part of a grammar. In Qi they are invoked after the corresponding parser successfully recognizes a portion of the input. Here the semantic action receives the attribute value of the matching parser.

Semantic Actions – a General View

A semantic action f are attached to a Qi parser p by simply writing:

p[f]

The function (or function object) f has to expose a certain interface allowing Spirit to pass the proper argument types. In the simplest case this can be a global function taking no arguments at all.

void func()
{
    std::cout << "Matched an integer!\n";
}

std::string input("1234");
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
qi::parse(begin, end, qi::int_[func]);     // this will call func

Most of the time this is not sufficient as a semantic action is expected to receive the matched attribute value. This is possible by writing:

void func(int attribute)
{
    std::cout << "Matched integer: " << attribute << "\n";
}

The type of the expected parameter (in this case the int) depends on the parser the semantic action is attached to. The attribute type exposed by the parser has to be convertible to the argument type.

There are actually 2 more arguments being passed: the parser context and a reference to a boolean ‘hit’ parameter. The parser context is meaningful only if the semantic action is attached somewhere to the right hand side of a rule. We will see more information about this shortly. The boolean value can be set to false inside the semantic action invalidates the match in retrospective, making the parser fail. Qi allows us to bind a nullary or a single argument function, like above. The other arguments are simply ignored.

It is feasible to bind any function object (such as generated by Boost.Bind or Boost.Lambda) as an semantic action. Even if the documentation shows a couple of examples (see here), I would not recommend using those libraries in this context. For me the preferred method of writing semantic actions is to employ Boost.Phoenix – a companion library bundled with Spirit. It is like Boost.Lambda on steroids, with special custom features that make it easy to integrate semantic actions with Spirit. If your requirements go beyond simple parsing, I suggest that you use this library. All the following examples in this article will use Boost.Phoenix for semantic actions. But whatever method you use, please let me highlight the following:

The three libraries allow you to utilize special placeholders to control parameter placement (_1, _2, etc.). Unfortunately, each of those libraries has it’s own implementation of the placeholders, all in different namespaces. You have to make sure not to mix placeholders with a library they don’t belong to and not to use different libraries while writing a semantic action.

Generally, for Boost.Bind, use ::_1, ::_2, etc. (yes, these placeholders are defined in the global namespace).

For Boost.Lambda use the placeholders defined in the namespace boost::lambda.

For semantic actions written using Boost.Phoenix use the placeholders defined in the namespace boost::spirit. Please note that all existing placeholders for your convenience are also available from the namespace boost::spirit::qi.

The current version of Spirit (V2.2) does not yet support binding a native C++0x lambda function as a semantic action, but this is something we are currently working on. You can expect this to be possible in the near future.

Writing Phoenix based Semantic Actions

Writing a semantic action with Phoenix is beneficial as Spirit  ‘knows’ about Phoenix. If you write them with the help of Phoenix you can utilize special placeholders Spirit provides you with. Those placeholders refer to elements in the context of the current parser execution such as attributes, local variables and inherited attributes of rules, etc. None of the other means of writing semantic actions (using Bind, Lambda, or had written function objects) gives you direct access to those elements. The following table lists all available placeholders exposed by Spirit (as mentioned earlier, all are defined in the namespace boost::spirit::qi). Again, please note, these are only available inside a semantic action and only if the semantic action is written utilizing Phoenix.

Obviously, the placeholders listed in the last three rows of the table are meaningful only if used in a rule definition. As an example, let us rewrite the semantic action from above with Phoenix:

std::string input("1234");
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
qi::parse(begin, end,
    qi::int_
    [
        std::cout << "Matched integer: " << qi::_1 << "\n";
    ]
);

One problem with earlier versions of Spirit (i.e. Spirit.Classic) was that while parsing sequences of things it was difficult to avoid calling a semantic action prematurely. For instance, in a parser sequence of two integer parsers (int_[f1] >> ‘,’ >> int_[f2]) the function f1 got called immediately after the first integer matched, and even if the second integer parser would fail later on. In the current version of Spirit this is not an issue anymore as it is possible to attach a semantic action to the whole sequence while still referring to the single attributes of the different sequence elements:

std::string input("1234,2345");
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
qi::parse(begin, end,
    (qi::int_ >> ',' >> qi::int_)
    [
        std::cout << "Matched integers: "
              << qi::_1 << " and " << qi::_2 << "\n";
    ]
);

Here, qi::_1 refers to the attribute matched by the first integer parser, and qi::_2 to the second one.

Initially I was planning to additionally describe the internal interface of a semantic action. Utilizing this interface allows you to write your own function objects and still to get access to the elements of the parser context mentioned above (attributes, the rule’s local variables and inherited attributes, etc.). But this post already got longer as anticipated, which is why I defer this discussion to a second Tip of the Day. Stay tuned!

While parsing some formatted data stream it is very often desirable to ignore some parts of the input. A common example would be the need to skip whitespace and comments while parsing some computer language. Certainly it is possible to explicitly account for the tokens to skip (such as the whitespace or the comments) while writing the grammar. But this can get very tedious as those tokens are valid to appear at any point in the input.

For the sake of simplicity, let us assume we want to parse a simple key/value expression: key=value, where we want to allow for any number of space characters before, in between, or after the key or the value. A naive grammar matching the plain key/value pair without whitespace skipping would look like (see Parsing a List of Key-Value Pairs Using Spirit.Qi for more details):

pair  =  key >> '=' >> value;
key   =  qi::char_("a-zA-Z_") >> *qi::char_("a-zA-Z_0-9");
value = +qi::char_("a-zA-Z_0-9");

If we want to explicitly accommodate the rule pair to match any interspersed space characters we get:

pair  = *space >> key >> *space >> '=' *space >> value >> *space;

which, while it produces the desired result, is not only error prone, but additionally difficult to write, to understand, and to maintain. If we look closer we see, that the process of skipping the whitespace tokens is easily automated. It seems to be sufficient to insert a repeated invocation of the space parser (or generally, any skip parser) in between the elements of the user defined parser expression sequences.

In fact, that is exactly what Spirit can do for you! The library invokes any supplied skip parser upon entry to the parse member function of any parser conforming to the PrimitiveParser concept. The skip parser has to be supplied by calling a special API function: phrase_parse:

namespace qi = boost::spirit::qi;
typedef std::string::const_iterator iterator;

qi::rule<iterator, qi::space_type> pair = key >> '=' >> value;
qi::rule<iterator> key = qi::char_("a-zA-Z_") >> *qi::char_("a-zA-Z_0-9");
qi::rule<iterator> value = +qi::char_("a-zA-Z_0-9");

std::string input(" key = value ");
iterator_type begin = input.begin();
iterator_type end = input.end();
qi::phrase_parse(begin, end, pair, qi::space);

This code snippet illustrates several important things:

• The function qi::phrase_parse is equivalent to the API function qi::parse except for its additional parameter, the skip parser. Our example utilizes qi::space, but it is possible to use any other, even more complex parser expression as the skipper instead.
• All rules which we want to perform the skip parsing need to be declared with the type of the skip parser they are going to be used with. Our example specifies the type of the qi::space parser expression, which is qi::space_type. For more complex parser expressions you might want to use a (mini) grammar or take advantage of BOOST_TYPEOF to let the compiler deduce the actual type.
• All rules which should not perform skip parsing have to be declared without an additional skip parser type. These rules behave like an implicit lexeme[] directive (for more information about lexeme[], see below), they inhibit the invocation of the skip parser even if they are executed as part of a rule with an associated skipper.

In the example above we suppressed skipping while matching either the key or the value, otherwise our grammar would match any additional space character inside the key or value as well. Remember, the expression char_ conforms to the PrimitiveParser concept, it will execute the skip parser for each of its invocations. In this case any skip parser would be executed in between any two of the matched characters.

Sometimes it is necessary to turn of skipping for a smaller part of the grammar only. For this purpose Spirit implements the lexeme[] directive. This directive inhibits skipping during the execution of the embedded parser. For instance, parsing a quoted string of alphanumeric characters would look like this:

string = lexeme['"' >> *alnum >> '"'];

Here the lexeme directive disables skipping while matching the string, which avoids ‘loosing’ characters otherwise matched by the skipper. Please note: lexeme[] performs a pre-skip step, even if it is not a PrimitiveParser itself (it is essentially considered to be a logical primitive by design). If this is undesired, you can utilize the no_skip[] directive instead:

string = '"' >> no_skip[*alnum] >> '"';

This parser will match all the characters in between the quotes, even if the string starts with a character sequence matched by the applied skip parser. The no_skip[] directive is semantically equivalent to lexeme[] except it does not perform a pre-skip before executing the embedded parser. Note: as the no_skip[] directive has been added only recently. It will be available starting with the next release (Boost V1.43).

This short article would not be complete without mentioning the skip[] directive. This directive is the counterpart to lexeme[]. It enables skipping for the embedded parser. Without any argument it can be used inside a lexeme or no_skip directive only. In this case it just re-enables the outer skipper:

string = lexeme['"' >> *(alpha | skip[digit]) >> '"'];

This (purely hypothetical) parser would enable skipping inside a string as long as it matches digits. But the skip directive can do more. It may take an additional argument allowing to specify a new skipper, for instance:

skip(qi::space)[*alnum]

which will skip spaces while executing the embedded *alnum parser. This form of the directive can be applied for two purposes. It can be used either for changing the current skip parser or to establish skipping inside a context otherwise not doing skipping at all (even if invoked with the qi::parse() API function).

For more detailed information about all the mentioned directives please see the corresponding documentation.

Spirit’s permutation parser a ^ b matches either a, b, a >> b, or b >> a, where a and b can be arbitrary parser expressions. Just like normal sequences this operator can be utilized to combine more than two operands. For instance, the expression a ^ b ^ c will match a or b or c (or an combination thereof) in any sequence. The attribute propagation rule for the permutation parser is

a: A, b: B --> (a ^ b): tuple<optional<A>, optional<B> >

As usual, if one or more operand of the expression do not expose any attribute (expose unused_type as their attribute, which is equivalent), this operand disappears from attribute handling:

a: A, b: Unused --> (a ^ b): optional<A>;

The permutation parser works out of the box whenever you do not require to match all of the elements in the input. But what if you want strict permutation (operands get matched exactly once)? You have two possibilities, as often, one simple and less versatile and one more complex but universally applicable solution. The simple solution is to parse the input and to check afterward whether all optionals in the resulting attribute have been filled. I will leave that solution as an exercise for the reader.

If we assume the attribute to be a (Fusion) tuple of optionals, containing one optional for each of the parser components in the permutation parser we can write the following code (thanks to Carl Barron for the initial idea).

This code defines a Phoenix function (a lazy function encapsulating some custom functionality) checking whether one or more of the optionals in a given Fusion sequence are empty. The Fusion algorithm find_if iterates over the given sequence of optionals, invoking the option_empty::operator() for each of the elements. fusion::find_if stops iterating on the first invocation returning true and returns the iterator to the element it stopped on. This is very similar to the well known std::find_if algorithm.

namespace phoenix = boost::phoenix;
namespace fusion = boost::fusion;
namespace qi = boost::spirit::qi;

class no_empties_impl
{
    // helper function object to be invoked by fusion::find_if
    struct optional_empty
    {
        template <typename T>
        bool operator ()(T const& val) const
        {
            return !val;  // return true if 'val' is empty.
        }
    };

public:
    template <typename T>
    struct result { typedef bool type; };

    // This operator will get called from the semantic action attached
    // to the permutation parser. The parameter refers to its overall
    // attribute: the fusion tuple of optionals.
    template <typename T>
    bool operator ()(T const& t) const
    {
        // look for an empty optional, if any return false.
        return fusion::find_if<optional_empty>(t) ==
               fusion::end(t);
    }
};

// define the Phoenix function
phoenix::function<no_empties_impl> const no_empties = no_empties_impl();

The overall Phoenix function no_empties will return false if we found at least one non-initialized optional in the passed sequence. The following code snippet illustrates how everything fits together:

std::string input ("BCA");
std::string::const_iterator begin = input.begin();
std::string::const_iterator end = input.end();
qi::parse(begin, end,
    (qi::char_('A') ^ 'B' ^ 'C')[qi::_pass = no_empties(qi::_0)]);

We assign the result of the invocation of no_empties to Qi’s predefined placeholder _pass. If we assign false, then the parser the semantic action is attached to will be forced to fail in retrospective (even if it matched the input successfully before). As a result the overall parser expression will succeed as long as a) the permutation parser matches its input and b) the Phoenix function inside the semantic action returns true.

For more information about the permutation parser please consult its documentation here. Overall, this example is a bit more complex than the average parser you might usually write. It utilizes three libraries: Spirit, Phoenix, and Fusion in a seamless manner. But for sure, once you understand the idea, it will be easier for you to come up with similar solutions. Spirit has been designed with Phoenix and Fusion in mind, and in fact it relies on Fusion heavily itself. As a result, the integration of those libraries is almost perfect.

There have been plans for a while to create a separate Fusion facility BOOST_FUSION_ADAPT_TPL_STRUCT allowing to adapt templated data types, but this is not in place yet. Today I will describe a trick you can apply to adapt your templates into ‘proper’ Fusion sequences anyway.

We will use the fact that a Qi grammar is already a template in most cases, and even if it is not a template yet, it can be easily converted into one. Further we will use the built-in capability of rule’s to invoke a custom attribute transformation if the attribute type of the right hand side does not exactly match the left hand side’s attribute type.

Let us assume this to be our data structure we want to fill while parsing:

template <typename A, typename B>
struct data
{
    A a;
    B b;
};

We would like to be able to directly utilize this template type as an attribute for our grammar. A possible way of adapting the template type to make it usable as a Fusion sequence is to define a fusion::vector<A&, B&> and initialize it with the references to the data members of our template type. If we the pass this Fusion vector as the attribute to the actual parser expression we effectively supply our original data members as the attributes to the parsing process.

namespace qi = boost::spirit::qi;
namespace fusion = boost::fusion;

template <typename Iterator, typename A, typename B>
struct data_grammar : qi::grammar<Iterator, data<A, B>()>
{
    data_grammar() : data_grammar::base_type(start)
    {
        // the implicit attribute transformation 'adapts' data<> to
        // the Fusion vector
        start = real_start;

        // do the actual parsing of the data<> members
        real_start = qi::auto_ >> ',' >> qi::auto_;
    }

    qi::rule<Iterator, data<A, B>()> start;
    qi::rule<Iterator, fusion::vector<A&, B&>()> real_start;
};

The signature of the grammar’s start rule has to match the signature of the grammar itself. To accommodate for this we introduce a second rule ‘real_start’ dedicated to the parsing of our data members. At the same time this allows us to inject the needed transformation of our data<> attribute to the Fusion vector. As the left hand side’s and right hand side’s attribute types do not match, the parser expression start = real_start will invoke Spirit’s customization point transform_attribute. But since the default implementation of this customization point does not handle our special data types the way we want, we are required to implement our own specialization:

namespace boost { namespace spirit { namespace traits
{
    template <typename A, typename B>
    struct transform_attribute<data<A, B>, fusion::vector<A&, B&> >
    {
        typedef fusion::vector<A&, B> type;
        static type pre(data<A, B>& val) { return type(val.a, val.b); }
        static void post(data<A, B>&, fusion::vector<A&, B&> const&) {}
        static void fail(data<A, B>&) {}
    };
}}}

The function pre() is called before the right hand side parser expression is invoked. It gets passed the left hand side’s attribute (the data<> instance) and is required to return the attribute to be passed to the rule’s right hand side expression. The returned Fusion vector is initialized with the references to the data members of our original data<> instance. The functions post() and fail() can be left empty in our case. For more information about this customization point please see the corresponding documentation here.

I added a new example to Spirit demonstrating this technique. Currently, it can be accessed from the Boost SVN only (see adapt_template_struct.cpp), but in the future it will be released as part of Spirit.

Just in case you were wondering: yes, this trick works equally well for Karma generators. The only difference is that the members of the created Fusion vector will have to be constant references instead.

To reflect the breadth of the Boost community, the conference includes sessions aimed at two constituencies: Boost end-users and hard-core Boost library and tool developers. The program fosters interaction and engagement within and across those two groups, with an emphasis on hands-on, participatory sessions.

We’d love to see you all there! BoostCon, from 2007, has always been a meeting place of Spirit folks. It has been the place where we meet eye to eye and discuss Spirit related events, future plans and road maps, using the library, etc. This year, 3 Spirit related talks were accepted:

Using Spirit 2.1 : Qi and Karma

By Michael Caisse

Machinery, sensors, equipment, client/server communications, even file formats… Parsing and producing communication streams is everywhere you look. Often these tasks are simple or small enough to tempt ad-hoc solutions. The Spirit 2.1 library provides a model that is simple enough to tackle those “quick hacks” and easily scales for full-featured AST generation.

This session will explore real-life experiences with the parser and generator (Qi/Karma) portions of the Spirit library. As we look at various small and medium-sized parsers/generators employed in various products we will establish some “rules-of-thumb” and guidelines for tackling the parser/generator domain with Qi/Karma. The session will end with the implementation of a usable XML parser and a simplified XPath-like node extractor.

The session will include some lecture and a lot of tutorial. Attendees will walk away with the knowledge and tools to begin parsing and generating with Spirit Qi/Karma.

RAD Spirit

By Joel de Guzman and Hartmut Kaiser

One allure of Boost Spirit Parser, compared to traditional parser generators, is its being embedded in C++. The user of the library specifies a parser’s grammar directly in C++ code using expression templates. There are distinct advantages with this approach, but there are problems as well. The most glaring drawbacks are 1) long compile times and 2) difficult to understand error messages and 3) Difficulty in debugging and testing parsers. For small parsing tasks, these can be tolerated. Yet, after 8 years in active deployment, we’ve come to a point where Spirit is being used in increasingly more complex parsing tasks. 2 and 3 can somehow be alleviated by adding more “smarts” to the expression template engine (using Proto), but that would increase compile times further to the point where the libary will no longer be useful.

It would be interesting to have a tool (both stand-alone or library based) that accepts textual EBNF/PEG expressions and outputs either parsers that are immediately executable or C++ Boost Spirit code. The RAD tool can have all sorts of smarts to make writing parsers as painless as possible (e.g. debugging, error handling, analysis, etc). After the development of the parser is “set in stone”, one can opt to have the tool emit C++ code that can then be included in an application. For more complex parsers, this will be a definite boon.

We would like to present the design and development of such a tool. Obviously, this “Dynamic Spirit” tool will be written using “static” Spirit. This would be a practical real world example using Spirit.

Spirit History and Evolution

By Joel de Guzman and Hartmut Kaiser

This year, we celebrate Spirit’s 10th anniversary from its early beginnings as an offshoot of a much larger GUI library in the 90s and debuted into Boost in May 2001 in the typical “Is there interest in this library?” fashion like all would be Boost libraries. From a humble 7 header file library, Spirit has grown to be one of the most sophisticated Boost libraries and along the way became the incubator of other Boost libraries such as Boost.Fusion, Boost.Phoenix, Boost.Wave and Boost.Proto.

We would like to present Spirit (and the libraries it inspired) in a historical perspective. The presentation will aim to provide a lighter, more intimate perspective into the development of at least 4 libraries with almost a decade’s worth of experience being Boost authors and bonafide crazy template metaprogrammers who abuse operators like Mad Scientists. Of course, we can’t help it if we show off some C++ tricks here and there, but we’ll try to keep it as light as we can.

Before you read on, please be aware that the interfaces described in this post are not set in stone and may change in the future without attempting to be backwards compatible. I’ll write about the current implementation anyway as I think those are important. Generally, understanding attribute handling is crucial to understanding Spirit. Nuff said.

Naturally, Spirit internally implements the means of retrieving the attribute type of any parser expression. So all we need to do is to invoke those and wrap them into a nicely reusable tool.

Let us start by briefly describing some implementation details related to parser construction and attribute composition. As you might already know, any parser expression provided by the user (such as for instance qi::int_ >> qi::double_) is not something you could directly use to parse input. The qi::int_, qi::double_, etc. are just placeholder symbols denoting the type of parser component required to match the input. These parser expressions have to be compiled into real parser instances responsible for the actual matching of the input. Spirit has a built in facility boost::spirit::compile which performs the conversion of the parser expression into the corresponding parser instance. Here is the (simplified) prototype:

namespace boost { namespace spirit
{
    template <typename Domain, typename Expr>
    typename result_of::compile<Domain, Expr>::type
    compile(Expr const& expr);
}}

where Domain is a tag type which in our case is qi::domain, thus identifying Qi (i.e. telling the function we want to create a parser), and Expr is the parser expression to compile. As most (meta-) template facilities in Spirit the compile construct consists out of two parts: the meta-calculation of the resulting type of the parser instance (result_of::compile) and the actual function doing the runtime compilation. For the purpose of this post we are interested in the resulting type of the parser instance only.

Among other things, after compiling the parser expression, the member function parse() is called on the returned parser instance. In the article about Creating Your Own Parser Component for Spirit.Qi we saw that the parser instance additionally exposes its attribute type. Spirit implements another facility we can invoke to extract this attribute type for a given parser instance type:

namespace boost { namespace spirit { namespace traits
{
    template <typename Component, typename Context = unused_type
      , typename Iterator = unused_type>
    struct attribute_of;
}}}

where Component is the type of the parser expression to query for its attribute type, the Context is not relevant in our case and Iterator is the iterator type used to do the actual matching. Most parser components are independent of any iterator type, but components as the raw[] directives are. So most of the time we can leave out supplying an iterator type.

These two Spirit facilities are all what we need! Let us combine the two in order to get the type of the attribute exposed by a parser created from a given parser expression.

template <typename Expr, typename Iterator = spirit::unused_type>
struct attribute_of_parser
{
    typedef typename spirit::result_of::compile<
        spirit::qi::domain, Expr
    >::type parser_expression_type;

    typedef typename spirit::traits::attribute_of<
        parser_expression_type, spirit::unused_type, Iterator
    >::type type;
};

Now as we have a tool to get the attribute type, we can use a simple utility function to print it out to the console (in this example we ignore the iterator type):

template <typename T>
void display_attribute_of_parser(T const&)
{
    typedef typename attribute_of_parser<T>::type attribute_type;
    std::cout << typeid(attribute_type).name() << std::endl;
}

// this will print something like: boost::fusion::vector2<int, double>
display_attribute_of_parser(qi::int_ >> qi::double_);

That’s it! We created a nice, reusable function printing the attribute type of an arbitrary parser expression. I added this as an example to the Boost SVN here and here, allowing you to reuse it for your needs. The last thing to mention is that a similar utility can be easily written for Karma generators. The only thing to change is the passed Domain which needs to be spirit::karma::domain.

Previous versions of Spirit, which are the versions we refer to as Spirit.Classic today, implemented special dynamic parsers allowing to insert control statements into the parsing process (such as if_p, while_p, and for_p). These dynamic parsers are not available anymore in Qi. But it is easy enough to achieve the same behavior using existing Qi components. This ‘Tip of the Day’ describes those techniques, which have been developed and contributed by Carl Barron. Thanks Carl!

Generally, Spirit.Classic allowed to employ the dynamic parser in two ways. The condition to check could be either a function or function object, or a parser expression. Functions or function objects are expected to return values convertible to bool. When the evaluation of the function or function object yields true it will be considered as meeting the condition. When the parser matches the condition is met as well.

General Notation

In the tables below we use the following notation:

• c: A nullary function or function object evaluating to bool
• p, p1, p2, …: Arbitrary parser expressions
• init, step: Arbitrary Phoenix nullary functions or function objects
• _a: a local variable of the rule the expression is assigned to (the type of the local variable should be bool)

All symbols are assumed to be referenced from their respective namespaces (i.e. boost::spirit::classic or boost::spirit::qi).

Conditional Parsing (if_p)

The Qi constructs as shown exactly reproduce the behavior if the if_p construct: if the condition ‘p’ is a parser it will consume input in case it matches. If this is not required you may utilize predicates instead (see the post about What’s the Difference Between Qi’s ‘!’ and ‘~’? for more details).

Parsing With Loops (while_p, do_p, and for_p)

All looping constructs require a local Boolean variable to be defined in the rule the expression is assigned to. It does not necessarily have to be the local variable _a, which in the expressions below is used as a placeholder for any local variable (_a, _b, …_j). At the same time this restricts the function objects ‘init’ and ’step’ to be Phoenix function objects (remember, Phoenix expressions do not mix well with non-Phoenix function objects).

All Qi constructs in the table above employ the same trick as shown in the last installment about local variables (What are Rule Bound Semantic Actions?). By using the eps component we inject arbitrary semantic actions into the parser execution. But please don’t confuse this with the eps(_a) component, which will propagate the current value of _a to its own return code. It will succeed parsing unless the current value of _a is false. In our case this forces the whole loop construct to fail parsing if the body parser ‘p1′ failed.

If you replace the ‘p’ with ‘c’ in any of the Classic expressions above (i.e. you want to use a function object as the condition instead of a parser), the ‘p’ in the equivalent Qi expression needs to be replaced by ‘eps(c)’.

Needless to say, all of the above expressions require you to add the proper #include statements to your code. Please consult the related documentation to see what’s necessary.

The one commonality of the two operators is the same as in Qi: both negate whether the component they are being used with succeeds generating. If the component ‘c’ succeeds, both compound constructs, ‘!c’ and ‘~c’ will fail, and similarly, if ‘c’ fails, the execution of the components ‘!c’ and ‘~c’ will succeed.

Similar to its counterpart in Qi, the unary Karma operator ‘~’ is applicable to character and character class generators only. It negates the set of characters a generator will be allowed to emit. Let me explain. The Karma character set generators, such as char_(“a-z”) or digit will emit their attribute only if the attribute value belongs to the character set described. Consequently, applying the operator ‘~’ will negate the character set the generator it is attached to. Here are some examples:

If a generator can’t emit its attribute it will fail. This is very similar to a failing parser component. The possibility for a generator component to fail is very useful. This can be utilized for alternatives, predicates and other constructs. But this is beyond today’s topic and will be discussed in a different installment of the ‘Tip of the Day’.

The generators created by the operator ‘~’ do not wrap the underlying generator. The operator rather modifies the behavior of the component it is attached to. This means there is no performance difference if compared to the plain character generators.

The unary operator ‘!’ creates a not-predicate generator. Again, similar to its counterpart in Qi, it can be attached to any (arbitrarily complex) generator. The not-predicate generator will succeed if the associated generator fails, and it will fail if its generator succeeds. It invokes the generator it is attached to but the emitted output will not show up in the overall output stream. So effectively, the not-predicate generator will never emit any output. The following example will succeed emitting a floating point number if the first attribute is false (which will make the true_ generator fail), otherwise it will not emit anything and will fail all together:

namespace karma = boost::spirit::karma;
std::string output;
std::back_insert_iterator<std::string> sink(output);
karma::generate(sink, !true_ << double_, false, 1.0); // will emit: 1.0

This example highlights another difference if compared to Qi’s not-predicate. The Karma not-predicate will always consume an attribute. More accurately, it will expose the attribute of the generator it is associated with (while in Qi the not-predicate never exposes any attribute). As mentioned earlier, we utilize the true_ generator’s ability to fail to control what output is generated (or if any output is generated at all as in our case).