Re: [c++-pthreads] Re: FW: RE: Re: I'm Lost
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [c++-pthreads] Re: FW: RE: Re: I'm Lost

David Abrahams wrote:

Dave Butenhof <david.butenhof@xxxxxx> writes:

David Abrahams wrote:
Dave Butenhof <david.butenhof@xxxxxx> writes: 
However when one writes a robust general-purpose facility (library)
that will be used in an environment supporting cancellation, that
library ought to be written to support cancellation (whether or not
it actually uses cancellation on its own behalf). Such libraries
are not generally tasks taken on by "casual users"; and even so
while hardly ideal it's perfectly adequate to simply say "this
facility isn't cancel-safe; tough luck".

"Industrial strength" libraries in the environment, for example the
"language runtime" itself, whether libc or STL, ought to be
cancel-safe certainly. Even at that, however, because the task can
be monumental, POSIX provided "cheats" -- the list of "optional"
cancellation points allow a libc developer to omit all but the most
critical. A C++ standard for STL *could* provide similar "cheats"
to avoid requiring full implementation of cancel-safety. And again,
if the user of the library (whether the main application or another
library) doesn't choose to use cancellation the point is moot.

Picking this thread up from long ago, lete me say that I'm sort-of in
agreement with the above.  I say "sort of" because Dave B's statement
fails to address the following point (hereafter known as "the
statement"), and I can't tell what side of it he'd come down on:

   Any code that is already exception-safe could be automatically
   cancel-safe depending on our definition of "cancel-safe" and the
   semantics we assign to cancellation exceptions.

In the definition of "cancel-safe" that allows the statement to be
true, cancellation is a request, and doesn't absolutely force
_anything_ to happen.  IIUC, that is the status quo anyway (nobody is
even forced to invoke a cancellation point).

The cancellation exception semantics that allow the statement to be
true are that they act like any other exception, and are not
automatically rethrown at the end of catch blocks.  This is the
question primarily in dispute, IIUC.
This has been THE most contentious issue in every C++/threads discussion I've encountered since the beginning of (pthread) time.
My preference has always been that cancellation is an exception. Period. In our initial CMA architecture, and in our exception mapping of cancellation/thread-exit onto C language exceptions in Tru64 UNIX and OpenVMS, it's possible and reasonable to finalize propagation of a cancel/exit exception. That was critical for DCE, for example, so that it could trap cancellation of an RPC server thread, bring the thread back into the server's work pool, and propagate the exception across the wire to the client. 
That sounds highly analogous to my case with Python.
Sure; and there are other examples. I've just found that the "inverted call stack" of this sort of server setup seems to make sense to a lot of people. 
To finalize a cancel/exit under almost any normal circumstance is
simply an application error. 
The key word being "almost."  In some situations, like those we've
both cited, it's absolutely necessary, to even get cancellation to
work.
Absolutely. There was a strong sub-faction in POSIX that can be loosely characterized as "academics" who were determined to try to prevent constructs that might be misused. It's why the realtime people didn't get pthread_abort() to force termination without cleanup, why you can't suspend a thread, or "force unlock" a mutex that might have been abandoned, and so forth. If cancellation can't be finalized, nobody can accidentally finalize it; and that's great if you don't trust anyone to know when it SHOULD be finalized. I started out as something of an "academic" in this sense and evolved into a pramatist... if someone thinks they need it, and they're right, don't keep them from doing it. And if they're WRONG, it's their problem, not yours. ;-) Portable Java GC would have been a lot easier had POSIX included suspend/resume; and does it really matter that nearly anyone else who used it would be breaking their application? Well, it all depends on your point of view... 
There are many worse application errors, like infinite loops, that
we can't legislate around anyway.  Worrying too much that someone
might finalize the exception unintentionally just seemed like wasted
effort. However it's also important to keep in mind that my
preferences were formed with POSIX cancellation and C language (or
cross-language OS) exceptions. C++ adds a lot of exception semantics
and patterns on top of that.
There have been plenty of people who argue that cancel "can't" be caught; and some of these arguments trace back to the ubiquity of catch(...), especially in constructors; 
A ctor that does catch(...) without rethrow is almost always badly
designed at best.  There was unfortunate advice going around for many
years that you shouldn't throw from ctors, but that's exactly wrong:
ctors that throw allow the establishment of strong invariants, and
programming without them is much harder.

On the other hand, stopping exceptions in dtors is absolutely the
right thing to do.
Yes, I may have said that backwards. As I've said before, although I use C++, it's not a "native language" for me, and a lot of this is based on opinions others have strongly stated rather than my own knowledge or experience. Another reason, incidentally, for not trying to come down too hard on one side or the other where language concerns might trump "threading" concerns. 
and they have some legitimate concerns about common C++ language
patterns that might pretty much prevent a cancel from ever doing
what a cancel should do.

Really, legitimate concerns? I can't think of any recommended patterns that would act that way.

Hmm. OK. That's interesting. Well, you're the C++ expert here. ;-)

The catch lies in whether (and how far) you'll trust application
developers to do the re-throw properly. If we don't clean up all
frames and eventually re-throw the cancel/exit to the runtime's base
frame to terminate the thread, then we don't have cancellation.

I'm of the school that says it's futile and even dangerous to try to
operate correctly in an environment where you have to assume other
code is incorrect.  My library will probably be equally broken if the
user decides to throw out my bad_alloc exception without a proper
response.
True enough. "Over-engineering" is a persistent danger. Sometimes we just need to accept that we can't solve all problems, and that "idiot-proofing" is a losing concept, and move on. ;-) 
On the other hand, if we prevent a catch or force a re-throw, we
lose a lot of C++ (particularly in constructors).

I don't think that should be your concern.  The correct and
well-written C++ we lose is generally sitting at module and language
boundaries.  And then there are dtors.

Part of the reason that you "can't tell what side of it [I'd] come
down on" is that I've long recognized this as an essentially
religious rather than technical argument.  You'll come down on the
side of the semantics toward which you feel the strongest emotional
attachment.

You don't find the idea that exception-safe code implies cancel-safe
code technically compelling?  I don't think that's an emotional issue.
Well, yes, I do, because cancel was always intended to be "just" an exception that happened to be thrown from another thread. But then, nothing is ever that simple; the asynchronous nature required controls like cancelability type and state. C++ exceptions are synchronous and non-interrupting. (The latter a consequence of the former, really.) One of the main advantages of cancellation is that it can break through an extended blocking operation; but that's unavoidably an extra condition over "exception-safety". Cancel-safe has to mean something more unless we drop interruptibility. If we drop it, then cancel-safety is just exception-safety but loses much of its value in controlling application responsiveness.
In any case, though, I wasn't suggesting that you need to convince me. I'm saying there are diverse and strongly held positions that somehow need to be unified in order to get consensus on any proposal. I think that I'm the least of your worries. ;-) 
While I'm happy to express my experience and even preferences, I
also recognize that "the other side" has some equally strong
arguments and expectations, and they (well, most of them!)  are not
"wrong".

I don't think either side is "wrong," either.

Someone needs to propose and champion "the great exception
compromise"; but if that's to be me I don't yet have the faintest
germ of a notion what it might be. So I sure hope it's going to be
someone else. ;-)

If "finalized cancellation exceptions result in a new throw at the
next cancellation point" isn't enough of a compromise, it isn't going
to be me either, because I'm out of new ideas.

Okay, how about this one: we count the number of times the
cancellation is discarded.  The cancelling thread can specify the
number of discards to tolerate, where the default is infinite.  After
that, at the next cancellation point all pthread cancellation handlers
(but not dtors or catch blocks) are run and the thread is terminated.
Heck, at that point I don't care what happens; you're gambling anyway.
Run all the dtors and catch blocks for all I care.
I do NOT favor any model where "dtor/catch" and "cancellation handler" don't mean the same thing.
I don't think the count is tenable either because although it always feels tempting to add a control dial, it doesn't solve any actual problem if there's nobody who can know to what value the dial should be set. In this case, I can't see how either the canceler OR any modular call stack could possibly provide any useful data much less a single numeric value.
If "canceled" state persists when the exception is discarded, then cancel is something different from just "an exception"; which is too bad, but perhaps inevitable. You can't just catch it and continue -- you need to somehow also reset that state to recover your workgroup thread that's serially running RPC requests (or Python code, whatever). A lot of people have suggested various ways of making cancel-pending persist after the exception is launched; that's not necessarily "wrong", but it isn't "simple" either and somehow it doesn't feel right to me. 
A simpler approach might be to have two kinds of exception: "forced"
and finalizable.  At least then we can say that exception-safe code
implies finalizable cancellation safety.  Then "forced" synchronous
cancellation can do whatever people desire.  I personally think it
will become a useless appendage sort of like C++ exception
specifications, but at least evolution will take care of it.  And if
I'm wrong, evolution will wilt my finalizable cancellations.
Is this the "unwind" vs "exception" idea? (Where "unwind" is like a new sort of 'throw' that triggers dtors but can't be caught/finalized.) Or something different...? 
I guess I'm not totally out of ideas ;-)



begin:vcard
fn:Dave Butenhof
n:Butenhof;Dave
org:Hewlett-Packard Company;Manageability Systems Lab
adr;dom:110 Spit Brook Rd;;ZKO2-3/Q18;Nashua;NH;03062
email;internet:david.butenhof@xxxxxx
title:HP Utility Pricing software agent Technical Lead
tel;work:603.884.7460
note;quoted-printable:POSIX thread standards consultant=0D=0A=
	Author of "Programming With POSIX Threads" (Addison-Wesley)=0D=0A=
	Father to Amy (12) and Alyssa (8)
x-mozilla-html:TRUE
version:2.1
end:vcard