Re: [c++-pthreads] Re: FW: RE: Re: I'm Lost

[David Abrahams <dave@xxxxxxxxxxxxxxxxxxxx>]

Dave Butenhof <david.butenhof@xxxxxx> writes:

>>> This has been THE most contentious issue in every C++/threads
>>> discussion I've encountered since the beginning of (pthread) time.
>>>
>>> My preference has always been that cancellation is an
>>> exception. Period. In our initial CMA architecture, and in our
>>> exception mapping of cancellation/thread-exit onto C language
>>> exceptions in Tru64 UNIX and OpenVMS, it's possible and reasonable
>>> to finalize propagation of a cancel/exit exception. That was
>>> critical for DCE, for example, so that it could trap cancellation
>>> of an RPC server thread, bring the thread back into the server's
>>> work pool, and propagate the exception across the wire to the
>>> client.
>>>     
>> That sounds highly analogous to my case with Python.
>>   
> Sure; and there are other examples. I've just found that the "inverted 
> call stack" of this sort of server setup seems to make sense to a lot of 
> people.

Okay.  So there's a moderate-sized class of applications that need to
do this.

>>> To finalize a cancel/exit under almost any normal circumstance is
>>> simply an application error.
>>
>> The key word being "almost."  In some situations, like those we've
>> both cited, it's absolutely necessary, to even get cancellation to
>> work.
>>   
> Absolutely. There was a strong sub-faction in POSIX that can be
> loosely characterized as "academics" who were determined to try to
> prevent constructs that might be misused. 

You mean, like, computers?

> It's why the realtime people didn't get pthread_abort() to force
> termination without cleanup, why you can't suspend a thread, or
> "force unlock" a mutex that might have been abandoned, and so
> forth. 

Oh, too bad.

> If cancellation can't be finalized, nobody can accidentally
> finalize it; and that's great if you don't trust anyone to know when
> it SHOULD be finalized. I started out as something of an "academic"
> in this sense and evolved into a pramatist... if someone thinks they
> need it, and they're right, don't keep them from doing it.  And if
> they're WRONG, it's their problem, not yours. ;-) Portable Java GC
> would have been a lot easier had POSIX included suspend/resume; and
> does it really matter that nearly anyone else who used it would be
> breaking their application? Well, it all depends on your point of
> view...

I don't know about that suspend/resume, but I don't think legitimate
cases where finalization is needed are nearly so rare as you describe
cases where suspend/resume is needed to be.

>> A ctor that does catch(...) without rethrow is almost always badly
>> designed at best.  There was unfortunate advice going around for many
>> years that you shouldn't throw from ctors, but that's exactly wrong:
>> ctors that throw allow the establishment of strong invariants, and
>> programming without them is much harder.
>>
>> On the other hand, stopping exceptions in dtors is absolutely the
>> right thing to do.
>>   
> Yes, I may have said that backwards. As I've said before, although I use 
> C++, it's not a "native language" for me, and a lot of this is based on 
> opinions others have strongly stated 

It wouldn't surprise me a bit if others had strongly stated ctors
shouldn't throw.  It used to be the advice in Stroustrup's books.

> rather than my own knowledge or experience. Another reason,
> incidentally, for not trying to come down too hard on one side or
> the other where language concerns might trump "threading" concerns.
>
>>> and they have some legitimate concerns about common C++ language
>>> patterns that might pretty much prevent a cancel from ever doing
>>> what a cancel should do.
>>>     
>> Really, legitimate concerns? I can't think of any recommended
>> patterns that would act that way.
>>   
> Hmm. OK. That's interesting. Well, you're the C++ expert here. ;-)

None other than, "prevent exceptions from leaking across language
boundaries."  But if you don't do that, your program is broken anyway.

>> You don't find the idea that exception-safe code implies cancel-safe
>> code technically compelling?  I don't think that's an emotional issue.
>>   
> Well, yes, I do, 

find it technically compelling, or think it's an emotional issue?

> because cancel was always intended to be "just" an exception that
> happened to be thrown from another thread. But then, nothing is ever
> that simple; the asynchronous nature required controls like
> cancelability type and state. 

"Asynchronous nature?"

I haven't even been considering asynchronous cancellation as it's
completely untenable to write anything but the most restricted code
that could work in the face of async cancellation.

> C++ exceptions are synchronous and non-interrupting. (The latter a
> consequence of the former, really.)  One of the main advantages of
> cancellation is that it can break through an extended blocking
> operation; 

If you make all blocking operations cancellation points you can do
that anyway.  No?

> but that's unavoidably an extra condition over "exception-safety".
> Cancel-safe has to mean something more unless we drop
> interruptibility. If we drop it, then cancel-safety is just
> exception-safety but loses much of its value in controlling
> application responsiveness.

You lost me.  I think async cancel safety should be thought of as a
separate level of design.

> In any case, though, I wasn't suggesting that you need to convince
> me.  I'm saying there are diverse and strongly held positions that
> somehow need to be unified in order to get consensus on any
> proposal. I think that I'm the least of your worries. ;-)

Not that you have any obligation to do so, but it might be easier if
you would recognize the weight your opinion carries.  That might mean
learning enough about C++ to form a definite opinion.  That's, at
least, what I've tried to do with threading.

>>> Someone needs to propose and champion "the great exception
>>> compromise"; but if that's to be me I don't yet have the faintest
>>> germ of a notion what it might be. So I sure hope it's going to be
>>> someone else. ;-)
>>>     
>> If "finalized cancellation exceptions result in a new throw at the
>> next cancellation point" isn't enough of a compromise, it isn't going
>> to be me either, because I'm out of new ideas.
>>
>> Okay, how about this one: we count the number of times the
>> cancellation is discarded.  The cancelling thread can specify the
>> number of discards to tolerate, where the default is infinite.  After
>> that, at the next cancellation point all pthread cancellation handlers
>> (but not dtors or catch blocks) are run and the thread is terminated.
>> Heck, at that point I don't care what happens; you're gambling anyway.
>> Run all the dtors and catch blocks for all I care.
>>   
> I do NOT favor any model where "dtor/catch" and "cancellation handler" 
> don't mean the same thing.

Like I said, I don't care at that point.  A forced cancellation is a
big gamble.  If "you" want to roll the dice, it's your funeral.

> I don't think the count is tenable either because although it always 
> feels tempting to add a control dial, it doesn't solve any actual 
> problem if there's nobody who can know to what value the dial should be 
> set. 

Which is why I backed off to the simpler model below.

> If "canceled" state persists when the exception is discarded, then 
> cancel is something different from just "an exception"; 

It already was something different.  The state needed to be stored
somewhere until the next cancellation point.  This just says that the
state persists until otherwise specified.

> which is too bad, but perhaps inevitable. You can't just catch it
> and continue -- you need to somehow also reset that state to recover
> your workgroup thread that's serially running RPC requests (or
> Python code, whatever). 

We could do something awful, like have catch-cancellation-by-value
cause the state to be reset, while catch(...) and
catch-cancellation-by-reference don't.  That would preserve the
convenience, at least.

> A lot of people have suggested various ways of making cancel-pending
> persist after the exception is launched; that's not necessarily
> "wrong", but it isn't "simple" either 
> and somehow it doesn't feel
> right to me.

It's simpler, by most measures I can think of, than resetting the
state upon throwing.

>> A simpler approach might be to have two kinds of exception: "forced"
>> and finalizable.  At least then we can say that exception-safe code
>> implies finalizable cancellation safety.  Then "forced" synchronous
>> cancellation can do whatever people desire.  I personally think it
>> will become a useless appendage sort of like C++ exception
>> specifications, but at least evolution will take care of it.  And if
>> I'm wrong, evolution will wilt my finalizable cancellations.
>>   
> Is this the "unwind" vs "exception" idea? (Where "unwind" is like a new 
> sort of 'throw' that triggers dtors but can't be caught/finalized.) Or 
> something different...?

No, I wasn't suggesting anything that couldn't be caught.  I was just
suggesting an exception that couldn't be stopped.  It could throw
itself in its dtor (not that I'm advocating it, but it might satisfy
the "other side"), for example.

In fact, a general mechanism like:

   cancel( thread_id, exception_object );

is possible, where "cancel" really means throw a copy of the given
exception object when the specified thread reaches the next
cancellation point.

We could call it 

   throw_synchronously( thread_id, exception_object );
   
instead, if "cancel" really means forced execution to too many people.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com