The Ro­Register­For­Apartment­Shutdown function is nice, but it doesn’t suit our needs.

One problem is that the object registered for apartment shutdown must be agile. (If you pass a non-agile object, the call fails.) That means that our wrapper must be agile. If we put the non-marshalable non-agile object inside our wrapper, we now have an agile object (the wrapper) that holds a non-agile object (the original object) as a member, which creates the risk that the wrapper will run down in the wrong apartment. The normal way of fixing that is to use Ro­Get­Agile­Reference to create an agile wrapper around the non-agile object, and store that agile wrapper in our container object.

This puts us back where we started: We are trying to implement that agile wrapper!

Even if we got past that problem, we have another problem: At apartment shutdown, COM calls the wrapper object’s UnInitialize method, and there’s a warning on the using the Ro­Unregister­For­Apartment­Shutdown function:

Warning: Don’t call the Ro­Unregister­For­Apartment­Shutdown function from the OnUninitialize callback.

But wait, what if the destructor of the object we are wrapping wants to call Ro­Unregister­For­Apartment­Shutdown? It might happen indirectly because it has its own reference to another object, that in turn tries to call Ro­Unregister­For­Apartment­Shutdown.

For example, it might hold another fake agile wrapper whose UnInitialize method has already been called! (We can work around that in our agile wrapper by nulling out the non-agile pointer so we don’t double-release, but maybe there’s some other object that uses Ro­Unregister­For­Apartment­Shutdown that is less resilient to this scenario.)

Another requirement of Ro­Unregister­For­Apartment­Shutdown is that it must be called from the same apartment that registered. This doesn’t quite put us back where we started, but it makes things more complicated because the whole point of the exercise is to find a way to release the object without having to try to switch to the apartment. I guess what we could do is use IContext­Callback to try to switch back to the apartment to release the original object and unregister the apartment shutdown callback. If the apartment switch fails, then we assume it’s because the apartment has already run down, so we assume that the IApartment­Shutdown::On­Uninitialize method was called, and it released the object for us. (Of course, that still creates the problem that we are cleaning up inside the On­Uninitialize callback, which meas that nobody else can unregister their apartment shutdown callback.)

The last note on this is that the implementation is written on the assumption that apartment shutdown callbacks are rarely unregistered. Registration is O(1), but unregistration is O(n) because it does a linear search through a linked list. By comparison, the Global Interface Table is designed on the assumption that objects are frequently registered, unregistered, and looked up, so it can look up its cookies in O(1) time. The proposed alternate design would unregister a lot of apartment shutdown callbacks, so it’s pushing the apartment shutdown callback list into a usage pattern it wasn’t optimized for.

The post Why didn’t we use <CODE>Ro­Register­For­Apartment­Shutdown</CODE> to clean up the unmarshalable delegate? appeared first on The Old New Thing.

To drain the slide is, as Mark Myers explains it, reading the contents of a slide to the audience when giving a presentation. At least at Microsoft, the term is frequently used by a speaker in the negative, such as “I don’t want to drain the slide here…”

My theory is that the phenomenon of draining the slide has multiple sources.

One is simply the result of inexperience on the part of the presenter. They don’t know what to say when giving a presentation, so they end up just reading out the text that they wrote on the slide.

A more insidious source is the result of tension between the multiple reason for creating a slide deck.

One reason to create a slide deck is to accompany a presentation. The focus of the presentation is the speaker, and the slide deck is there merely as a visual aid.

Another reason to create a slide deck is to provide a lasting record of the presentation and to serve as a reference for the future. If the topic is “How to perform process X”, then you want to make sure that all of the necessary steps are spelled out on the slides, so that you can share a link to the slide deck in the invitation for others to refer to both during the presentation but mostly for reference afterwards. People will refer back to the slides the next time they need to do process X, and they probably don’t want to have to scrub through a one-hour recording just to extract the five steps they need to follow.

Yet another reason for creating a slide deck is when presenting to a senior executive, such as for use in a placemat review. There is probably a standard template for the meeting, such as an “Accomplishments since the last meeting”, perhaps broken into “What’s going well”, “What’s not going well”, and followed by an “Items of future concern”. The meeting may run into a time crunch (because they always run over), and you may not get enough time to get through all your slides, so you want to be able to say, “You can read the rest of the slides after the meeting.” (An extreme case of this is where other teams in the placemat meeting run long and leave you with no time at all!)

A related reason for creating a slide deck is as a “proof of work”. Going into the placement review, you don’t have much to report, but you have to fill the slide. If you don’t, it looks like you haven’t done anything.

I was reminded of this last reason when I worked with a colleague to prepare our team’s weekly placemat report. My colleague would fill the slide with information, and I would recommend replacing most of the bullet points with a much more succicient summary, or just removing the items outright.

Partner Team X is considering the possibility of deploying a version that uses widgets before all of the widget features are fully implemented and rolled out to customers. They understand the consequences, such as impaired reverse widget flow. Some of these missing features will light up when the implementation is ready, but others will be stuck in a broken state unless explicit remediations are performed after the implementation rolls out, and a small number of features will be irreparably broken, with the only recourse being to destroy all the affected widgets and recreate them. Partner Team X has been advised of the various failure modes and how they can recover from the ones that are recoverable, and they are assessing whether they want to proceed with early deployment.

I simplified it to

Partner Team X understands the consequences of and limited recovery from using widgets in production before our work is fully implemented and deployed.

The people who are receiving this slide deck are not interested in the fine details. They want the 25-word summary of how things are going so that they can either decide that our feature is not a point of concern, or decide to ask follow-up questions to understand the issue better. In that case, we can present a so-called back-up slide with the details.

A back-up slide is a slide in a slide deck that you hope that you won’t need to use, but which you create just in case it is needed. You mark it as hidden or put it in an appendix so that it doesn’t show up in the normal presentation flow. But if the audience you are presenting to (usually someone higher up the executive chain) asks a follow-up question about one of your slides, you can reach into your deck and pull out your back-up slide to answer it. “I was ready for that question, so I already have a slide with the data you are asking for.”

Bonus PowerPoint jargon: PowerPoint Karaoke and the eye chart.

The post Microspeak: Drain the slide, back-up slides appeared first on The Old New Thing.

Well, not exactly.

Architecturally, the invalid opcode is a fault, which means that the address of the faulting instruction is reported to the exception handler. On the other hand, the int 6 is a trap, which means that the address of the instruction after the int 6 is reported to the handler.

This distinction between faults and traps reflects how they are used. A fault represents an error that the kernel could attempt to repair and retry. For example, a page fault could be repaired by paging the memory in. An access violation could be repaired by changing the page protections. A trap reports an operation that has already occurred and gives you a chance to do some special work in response to it before resuming execution. In the case of a software interrupt, the handler wants to process whatever the interrupt means, and then resume execution with the next instruction.

Now, some faults are unlikely to be repaired, but reporting them as faults means that you have to take extra steps if you want to execute past them. If you just ignore the fault, the instruction restarts and faults again.

If you manually raise the int 6 interrupt to simulate a #UD undefined instruction fault, the handler will think it’s dealing with a CPU-generated undefined instruction fault, so it expects the instruction pointer in the fault frame to point to the undefined instruction. But since int puts the address of the next instruction into the fault frame, the fault handler will look at the instruction and get confused as to why the invalid instruction fault was raised. “It says here that the faulting instruction is a xor eax, eax. That’s a perfectly legitimate instruction. Why did it fault?”

One case for inspecting the faulting instruction is to emulate missing instructions on older processors.

Furthermore, in protected mode, you need to have permission to issue an int 6. The interrupt descriptor table describes what to do for each interrupt, as well as what CPU privilege level is permitted to invoke it explicitly. And to avoid exactly these types of problems where a user-mode program issues an int 6 and confuses the #UD handler, all operating systems deny int 6 privileges to user mode.

So even if you try to fake a #UD via an explicit int 6, it doesn’t work. To get the #UD undefined instruction fault, you have to issue an actual undefined instruction.

The post Why do we need <CODE>ud2</CODE> when <CODE>int 6</CODE> is right there? appeared first on The Old New Thing.

The std::future is the starting point. It is a move-only class that this takes the source and forwards its wait and wait_for methods to the source, and forwards its get method to consume.

Forwarding the get to consume means that you can call get only once.

The idea is that std::future is for the case where there is exactly one consumer of the value. Since it is move-only, you can’t copy it; you can only give it away and transfer the power to retrieve the value to the recipient.

The std::future has a method called shared(). This transfers the source from the future to a shared_future object. The shared_future also forwards its wait and wait_for methods to the source, but instead of forwarding its get to the source::consume, shared_future::get() just calls source::get to get a reference to the value storage.

This is what makes the “shared future” shared: The shared_future is copyable, so you can have multiple people all with their own shared_future that share the same source, and each one can separately wait for the value and retrieve the value as many times as they want.

The downside of shared_future is that you always get a const reference, so if you want to perform non-const operations on the result, you’ll have to make a copy and operate on the copy. And that’s not possible at all if the result is itself a move-only type, since you can’t move out of a const reference.

You can see that at the end of the day, all of the pieces of the futures library are just different ways of manipulating the invisible source. Some components push values into the source, and others pull values out of the source.

It took me a while to realize that the most important part of the whole futures library is something you can’t even see.

Bonus chatter: All the names in the futures library are horrible. I mean, really, “packaged task”?

Here’s my attempt to give the pieces better names:

I admit, these names aren’t all that good either. But I prefer to use the word “promise” to refer to the thing that “promises” to produce a value. And the pattern promise_source / promise makes the relationship between the two much clearer: A promise source is a thing that makes promises. And it avoids the hilarity of a non-async async().

The post C++ futures part 5: <CODE>std::<WBR>future</CODE> and <CODE>std::<WBR>shared_future</CODE> appeared first on The Old New Thing.

But maybe you want to do it because you are interfacing with a component that wants a std::future. How could you do it?

One way would be to implement the immediately synchronous case manually by setting a value into a promise.

template<typename F, typename... Args>
auto
    make_immediate_future(F&& f, Args&&... args)
-> std::future<
        std::invoke_result_t<std::decay_t<F>,
                             std::decay_t<Args>...>>
{
    std::promise<
        std::invoke_result_t<std::decay_t<F>,
                             std::decay_t<Args>...>> p;
    try {
        p.set_value(std::invoke(auto(f), auto(args)...));
    } catch (...) {
        p.set_exception(std::current_exception());
    }
    return p.get_future();
}

Or you could make packaged_task do the work for you:

template<typename F, typename... Args>
auto
    make_immediate_future(F&& f, Args&&... args)
-> std::future<
        std::invoke_result_t<std::decay_t<F>,
                             std::decay_t<Args>...>>
{
    using R = std::invoke_result_t<std::decay_t<F>,
                                   std::decay_t<Args>...>;

    std::packaged_task<R(Args...)> task(f);
    task(args...);
    return task.get_future();
}

Yet another way would be to ask for an on-demand in-thread execution, and then immediately demand it.

template<typename F, typename... Args>
auto
    make_immediate_future(F&& f, Args&&... args)
-> std::future<
        std::invoke_result_t<std::decay_t<F>,
                             std::decay_t<Args>...>>
{
    auto f = std::async(std::launch::deferred,
                        std::forward<F>(f),
                        std::forward<Args>(args)...);
    f.wait();
    return f;
}

The post C++ futures part 4: What if I want an immediate on-thread <CODE>std::async</CODE>? appeared first on The Old New Thing.

The more complex case is the std::async. You give std::async a callable and optional parameters to pass to that callable, and you also specify how and when you want the callable to be invoked. You get two axes: Do you want it to be calculated immediately or on demand? Do you want the calculation to happen on the thread that requests the value or on its own thread?

Now, you’d think you could fill out the table like this:

However, that’s not what the C++ standard says.

The C++ standard disallows asking for an immediate value on the same thread. There’s really no value to doing it that way, since there is no actual “future”; it all happens in the present! “This future could have been a function call.”

auto f = std::async({}, calculator);

auto result = f.get();

This would, in theory, mean “Immediately call the calculator and wait for it to produce a value. Store it in the source and return a future that fetches the value from that source.” But that’s the same as just

auto result = calculator();

I’m guessing that the C++ standard disallows it because there’s no point.

The C++ standard also says that if you combine both std::launch::deferred and std::launch::async, it doesn’t mean that you get a calculation that lazy-starts on its own thread. Instead, the standard says that the implementation can choose which flag to honor and which to ignore.

So really, the table is

In all cases, the result of the callable (either a value or an exception) is stored in the source.

Now, I noted that there’s no point to asking std::async for an immediate on-thread calculation. But maybe you want to do it anyway because you are interfacing with some other component that expects to receive a future. How could you get the same result?

We’ll look at that next time.

The post C++ futures part 3: The mental model for <CODE>std::async</CODE> appeared first on The Old New Thing.

The lifetime of the source is managed by a std::shared_ptr<source<T>>. Everybody who has a reference to the source uses this shared pointer.

The simplest case is std::promise. The operations on std::promise pretty much map directly to the operations on the source we laid out last time.

The next more complicated case is the std::packaged_task. This takes a callable and saves it for later. You can then call get_future to get a future and hand it to somebody as an IOU for “A value will appear here someday.”

When you want to produce a value, you invoke the packaged_task with whatever parameters you want to pass to the original callable. The result of the invocation (either a value or an exception) is then stored in the source object, which can then be retrieved by the future.

You can imagine a packaged task as just wrapping a promise:

template<typename R, typename... Args>
class packaged_task
{
    std::promise<R> p;
    std::function<R(Args...)> func;

public:
    template<typename F>
    packaged_task(F&& f) : func(std::forward<F>(f)) {}

    auto get_future() { return p.get_future(); }

    void operator()(Args... args) {
        try {
            p.set_value(func(args...));
        } catch (...) {
            p.set_exception(std::current_exception());
        }
    }
};

When you invoke it, it calls the stored callable with the arguments and puts the result in the promise.

Okay, those were the easy cases of pushing values into the source. Next time, we’ll look at the messy case: std::async.

The post C++ futures part 2: The mental model for <CODE>std::promise</CODE>, <CODE>std::packaged_task</CODE> appeared first on The Old New Thing.

The idea is that all of these classes are built of a single unnamed class which the C++ standard library calls the “state”, but for the purpose of discussion, I’m going to call it the source, since it is the thing that produces the value, and it behaves analogously to a C# task completion source.

The source conceptually supports these operations:

• Set a value.
• Set an exception.
• Wait for a value or exception to be set.
• Retrieve the value or rethrow the exception.

Here’s a simplified version that doesn’t support references.

template<typename T>
struct source
{
private:
    enum what {
        nothing,
        value,
        exception,
        consumed,
    } d; // discriminant

    union {
        T value;
        std::exception_ptr ex;
    } r; // result

    std::mutex m;
    std::condition_variable cv;

The discriminant tells us what is in the union: Nothing (because nobody has set anything), a value, an exception, or nothing (because the value has been consumed).

The mutex ensures thread-safe access to the other members, and the condition variable lets us wait for somebody to set a value or exception.

    void must_be_empty() {
        if (d != what::empty) {
            throw std::future_error(future_errc::promise_already_satisfied);
        }
    }

You can call the set methods only when the source is empty, so we’ll factor out the emptiness check.

public:
    void set_value(T& v) {
        std::lock_guard g(m);
        must_be_empty();
        new(&r.value) T(v); // "r.value = v"
        d = what::value;
        cv.notify_all();
    }

You can set a value or exception only once, so if there is already a value or exception, we yell at you. Otherwise, we save the value and notify anybody who was waiting for a value to be set.

I won’t bother with the overload that takes T&&; it’s the same thing, but with a std::move().

There is also a set_value_at_thread_exit which will be another topic for another day.

Instead of setting a value, you can set an exception:

    void set_exception(std::exception_ptr ex) {
        std::lock_guard g(m);
        must_be_empty();
        new(&r.ex) std::exception_ptr(ex); // "r.ex = ex"
        d = what::exception;
        cv.notify_all();
    }

It’s basically the same as set_value except that we store an exception.

That’s how values an exceptions go into the source. The next part is waiting for the source to become nonempty.

    void wait() {
        std::lock_guard g(m);
        while (d == what::empty) {
            cv.wait(g);
        }
    }

    void wait_for(duration dur) {
        std::lock_guard g(m);
        cv.wait_for(g, dur, [&] { return d != what::empty; });
    }

You can either wait indefinitely or wait for a particular duration. The wait_for method is why we have a condition variable at all. If it weren’t for that guy, we could have used std::atomic‘s wait and notify_all methods.

The last part is getting the values out.

    void can_get() {
        if (d == what::consumed) {
            throw std::future_error(future_errc::future_already_retrieved);
        }
        if (d == what::exception) {
            std::rethrow_exception(r.ex);
        }
    }

Regardless of how we intend to retrieve the value, it’s not legal to retrieve a value that has already been consumed. Also, if the result is an exception, we rethrow the exception.

    T consume() {
        wait();
        std::lock_guard g(m);
        can_get();
        d = what::consumed;
        return std::move(r.v);
    }

    const T& get() {
        wait();
        std::lock_guard g(m);
        can_get();
        return r.v;
    }
};

You can get the value, which returns a const reference to the underlying storage. Or you can consume the value, which moves the value out of the underlying storage and marks the value as consumed so nobody can consume or get it (because it’s not there any more).

We can now build our other objects and functions out of this guy. We’ll do that next time.

Bonus chatter: Writing the destructor is left as an exercise for the reader. It’s not hard, but it’s annoying because of the union.

Bonus bonus chatter: I’m writing the series because I always forget how it works, but writing it down helps me remember.

The post C++ futures part 1: The mental model appeared first on The Old New Thing.

This realization lets you reinterpret the C++ access specifiers for C++/WinRT implementation types.

In code:

struct Widget : winrt::implements<Widget, IStringable>
{
public:
    // This is a member of IStringable,
    // so it is accessible to anyone.
    winrt::hstring ToString();

    // This does not implement any member from IStringable,
    // so it is accessible to other code in the same module.
    void Recolor(winrt::Windows::UI::Color newColor);

protected:
    // Accessible to implementation classes that derive from Widget.
    void Reverse();

private:
    // Accessible only to Widget itself.
    void Reset();
};

Note that if you are implementing a runtime class, then C++/WinRT autogenerates a template for you that auto-declares all of the declared interfaces of that class.

// Widget.idl

namespace Contoso
{
    runtimeclass Widget : Windows.Foundation.IStringable
    {
        String Name{ get; };
        void Toggle();
    }
}

The IDL compiler autogenerates an interface IWidget with a property Name and a method Toggle().

// implementation
namespace winrt::Contoso
{
    struct Widget : WidgetT<Widget>
    {
        ⟦ ... ⟧
    };
}

The WidgetT<Widget> expands to implements<Widget, IWidget, Windows::Foundation::IStringable> because those are the interfaces declared by the IDL file.

The post One way of thinking about access specifiers in C++/WinRT implementation types appeared first on The Old New Thing.

Installing two anti-malware programs on the same system is like hiring two different security companies to patrol your building. If they don’t know about each other, each is going to think the other one is an intruder. If you’re lucky, their instructions are merely to report on suspicious activity.

But if you give them weapons and the authority to use them, you may end up with your two security companies pointing their weapons at each other.

In real life, you would introduce the two security companies to each other, or at least make sure they don’t patrol the same floor.

In software, this is harder to do. It’s not like you can invite two programs to lunch and have them get to know each other.

Anti-malware software typically does things that aren’t officially supported, like detouring system functions. And these detours mean that calling a system function no longer does the system thing; instead, it calls into the anti-malware software, and it might decide to do something unrelated to the system function you thought were calling, and that unrelated thing may itself start causing problems, say, by hanging. Not only is there no easy way to identify that this has happened, short of debugging an actual malfunctioning system, but even after you figure out the conflict, there’s usually no way to tell each anti-malware software to “stay on its floor.”

The post If one anti-malware software is good, does that make two better? appeared first on The Old New Thing.