[New API Proposal] TSContScheduleToCurrentThread and TSContScheduleToCurrentThreadEvery

+1, we currently have it implemented but did not come around to make a pull
request and a proposal.

Some remarks, instead of current thread we should be able to schedule on an
arbitrary thread (*OnThread)

This can also improve performance by preventing thread jumping and provide
simpler concurrency because there is no concurrency :)

Not having access to source code, but I recall the change not being too
big, it is mostly adding the thread to some calls.

If people are interested I'll try to provide a pr in the next two days

Post by zzz
Hi, team,
The background here is we'd like to clean up a thread local cache after
some timeout. Unfortunately, we're lacking an API to schedule a
continuation to the current thread.
There are two ways I can think of to achieve this.
One is a pair of new APIs *TSContScheduleToCurrentThread(TSCont contp,
ink_hrtime timeout) *and* TSContScheduleToCurrentThreadEvery(TSCont contp,
ink_hrtime every).*
Or we can append the TSThreadPool enum with a new entry called
*TS_THREAD_CURRENT
*and add the logic into *TSContSchedule() *and *TSContScheduleEvery(). *
The issue with the latter solution is *TS_THREAD_CURRENT* doesn't quite fit
in TSThreadPool. If we change TSThreadPool to some more proper name, the
change will not be backward compatible.
Please feel free to share your insights. Any ideas would be much
appreciated!
Thanks,
Zizhong

Otto van der Schaaf

2018-08-15 06:57:18 UTC

+1 for APIs that allow scheduling on an arbitrary thread

Post by Kees Spoelstra
+1, we currently have it implemented but did not come around to make a pull
request and a proposal.
Some remarks, instead of current thread we should be able to schedule on an
arbitrary thread (*OnThread)
This can also improve performance by preventing thread jumping and provide
simpler concurrency because there is no concurrency :)
Not having access to source code, but I recall the change not being too
big, it is mostly adding the thread to some calls.
If people are interested I'll try to provide a pr in the next two days

contp,

Post by zzz
ink_hrtime every).*
Or we can append the TSThreadPool enum with a new entry called
*TS_THREAD_CURRENT
*and add the logic into *TSContSchedule() *and *TSContScheduleEvery(). *
The issue with the latter solution is *TS_THREAD_CURRENT* doesn't quite

fit

Post by zzz
in TSThreadPool. If we change TSThreadPool to some more proper name, the
change will not be backward compatible.
Please feel free to share your insights. Any ideas would be much
appreciated!
Thanks,
Zizhong

Alan Carroll

2018-08-15 14:29:03 UTC

I am fine with extending "TSThreadPool" to include a "this thread" value..
I don't think it breaks the name, it's just specifying a pool of size 1 :-).

For Otto's suggestion, I'd go with adding a new API type, "TSEventThread".
This, in contrast to TSThread, would represent a thread that has an event
loop and which therefore can handle event scheduling. A TSThread does not
have this capability in general. Additional API could be built around this,
such as "TSEventThread TSThisEventThread()" which returns the current
thread. Then you can easily implement what Otto requested,
"TSScheduleOnThread(TSCont, ink_hrtime, TSEventThread)". The original
request would then be "TSScheduleOnThread(contp, 1, TSThisEventThread());".
We might want to add "TSMutexThreadHoldingGet" which gets the thread
holding a mutex. For the kind of scheduling you guys are discussing, this
has proven quite handy in the core. I can work on a write up if you think
this is a reasonable direction to go.

Post by Otto van der Schaaf
+1 for APIs that allow scheduling on an arbitrary thread

Post by Kees Spoelstra
+1, we currently have it implemented but did not come around to make a

pull

Post by Kees Spoelstra
request and a proposal.
Some remarks, instead of current thread we should be able to schedule on

Post by Kees Spoelstra
arbitrary thread (*OnThread)
This can also improve performance by preventing thread jumping and

provide

Post by Kees Spoelstra
simpler concurrency because there is no concurrency :)
Not having access to source code, but I recall the change not being too
big, it is mostly adding the thread to some calls.
If people are interested I'll try to provide a pr in the next two days

contp,

Post by zzz
ink_hrtime every).*
Or we can append the TSThreadPool enum with a new entry called
*TS_THREAD_CURRENT
*and add the logic into *TSContSchedule() *and *TSContScheduleEvery().

Post by zzz
The issue with the latter solution is *TS_THREAD_CURRENT* doesn't quite

fit

Post by zzz
in TSThreadPool. If we change TSThreadPool to some more proper name,

the

Post by zzz
change will not be backward compatible.
Please feel free to share your insights. Any ideas would be much
appreciated!
Thanks,
Zizhong

--
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

CrazyCow

2018-08-15 21:45:03 UTC

Thanks, Kees, Otto, and Alan.
TSScheduleOnThread() seems perfect and much more powerful. Any use cases
you guys can figure for scheduling a cont on a specific different thread?
For scheduling on the current thread case, we don't even need a mutex. If
we'd schedule the cont with mutex on the mutex->thread_holding, we only
need the mutex from the user and don't need to have the new API
TSMutexThreadHoldingGet().

Post by Alan Carroll
I am fine with extending "TSThreadPool" to include a "this thread" value..
I don't think it breaks the name, it's just specifying a pool of size 1 :-).
For Otto's suggestion, I'd go with adding a new API type, "TSEventThread".
This, in contrast to TSThread, would represent a thread that has an event
loop and which therefore can handle event scheduling. A TSThread does not
have this capability in general. Additional API could be built around this,
such as "TSEventThread TSThisEventThread()" which returns the current
thread. Then you can easily implement what Otto requested,
"TSScheduleOnThread(TSCont, ink_hrtime, TSEventThread)". The original
request would then be "TSScheduleOnThread(contp, 1, TSThisEventThread());".
We might want to add "TSMutexThreadHoldingGet" which gets the thread
holding a mutex. For the kind of scheduling you guys are discussing, this
has proven quite handy in the core. I can work on a write up if you think
this is a reasonable direction to go.

Post by Otto van der Schaaf
+1 for APIs that allow scheduling on an arbitrary thread

Post by Kees Spoelstra
+1, we currently have it implemented but did not come around to make a

pull

Post by Kees Spoelstra
request and a proposal.
Some remarks, instead of current thread we should be able to schedule

Post by Otto van der Schaaf
an

Post by Kees Spoelstra
arbitrary thread (*OnThread)
This can also improve performance by preventing thread jumping and

provide

Post by Kees Spoelstra
simpler concurrency because there is no concurrency :)
Not having access to source code, but I recall the change not being

too

Post by Kees Spoelstra
big, it is mostly adding the thread to some calls.
If people are interested I'll try to provide a pr in the next two days

Post by zzz
Hi, team,
The background here is we'd like to clean up a thread local cache

after

Post by zzz
some timeout. Unfortunately, we're lacking an API to schedule a
continuation to the current thread.
There are two ways I can think of to achieve this.
One is a pair of new APIs *TSContScheduleToCurrentThread(TSCont

contp,

Post by zzz
ink_hrtime timeout) *and* TSContScheduleToCurrentThreadEvery(TSCont

contp,

Post by zzz
ink_hrtime every).*
Or we can append the TSThreadPool enum with a new entry called
*TS_THREAD_CURRENT
*and add the logic into *TSContSchedule() *and

*TSContScheduleEvery().

Post by Otto van der Schaaf
*

Post by zzz
The issue with the latter solution is *TS_THREAD_CURRENT* doesn't

quite

Post by Kees Spoelstra
fit

Post by zzz
in TSThreadPool. If we change TSThreadPool to some more proper name,

the

Post by zzz
change will not be backward compatible.
Please feel free to share your insights. Any ideas would be much
appreciated!
Thanks,
Zizhong

Alan Carroll

2018-08-15 22:18:59 UTC

In the core, when a continuation is scheduled on a specific thread that is
not the current thread, it's because there is a locked object with the lock
held on another thread, and the goal is to have the continuation run on
that thread. This can help gather interacting objects on the same thread
where they won't have lock contention (because if I try to get a mutex
that's locked on *this* thread, it is successful).

Otto van der Schaaf

2018-08-16 06:43:19 UTC

A TSEventThread API makes sense to me; maybe Kees has something similar in
the
implementation for XXXOnThread() he mentioned.
Regardless, it sounds like this would allow fine-grained control of where
code associated to a
continuation is executed, which would be great as far as I am concerned.

Otto

On Thu, Aug 16, 2018 at 12:19 AM Alan Carroll

Post by Alan Carroll
In the core, when a continuation is scheduled on a specific thread that is
not the current thread, it's because there is a locked object with the lock
held on another thread, and the goal is to have the continuation run on
that thread. This can help gather interacting objects on the same thread
where they won't have lock contention (because if I try to get a mutex
that's locked on *this* thread, it is successful).

Kees Spoelstra

2018-08-16 07:12:35 UTC

Adding to that, we might need have to look into this:
- getting the TSEventThread from a vconn, txn, ssn instead of implicitly
trusting that the current thread is relevant?
- moving a client vconn to a certain thread (we've seen that the thread
jumping in plugins can really kill performance with pooled client-server
vconns), we ended up pooling vconns per thread for the fastcgi plugin,
which could be a waste and not even possible for some fastcgi servers (more
threads running in TS than connections allowed to the fastcgi server)
- check the callback continuations in the current API? do we want to have
guaranteed callbacks on the same thread or do we want the plugin to handle
this?
- getting the TSEventThread of a TSThread
- some documentation for plugins regarding thread alignment

Naming: TSEventThreadSelf() in line with the current API?

Another advantage of scheduling a unit of work on the same thread is that
we can escape some locks which are on the stack due to the recursive nature
of some handling in ATS, only requesting relevant locks. This reduces the
chances of deadlock or retrying. I know people doing coordination between
txns will love this.
A problem with this approach is that it is not always clear to developers
what the relevant locks are, resulting in very interesting situations (TM)
:)

I remember talking with Alan and Jason about the thread aligned scheduling,
and there were some concerns about possible starvation, but if core already
goes out of its way to align per thread it makes sense to follow that in
plugin land. What are the thoughts about this now?

@Otto: We used the TSThread, but using a TSEventThread type will enhance
readability and protect the developer, the implementation will be the same.

Alan Carroll

2018-08-16 15:08:57 UTC

"getting the TSEventThread from a vconn, txn, ssn instead of implicitly
trusting that the current thread is relevant?"

I'm not sure what this means. Those objects don't have threads explicitly
associated, it's more implicit. If the code is in a callback on a hook for
the object, then the current thread is relevant, effectively by definition.
But as soon as I read this I realized you were going to bring up the
continuation thread affinity issue.

"moving a client vconn to a certain thread" - that would be challenging.
I'd rather move the plugins to the NetVC thread. This is one aspect of
getting the mutex locking thread - that's the thread to which to move.

"check the callback continuations in the current API?" - check them for
what? I don't understand this question.

"getting the TSEventThread of a TSThread" - that's fine, with the
understanding that not every TSThread is a TSEventThread so this may yield
a null result.

Unwinding the call stack to deal with conflicting locks is something that's
done in the core quite a bit, so yes it would be nice if plugins could do
that as well. It's effectively a "yield" with regard to the ATS event
dispatching logic. I will try to remember what we talked about with thread
aligned scheduling, there have been multiple schemes I've discussed with
Jason.

Kees Spoelstra

2018-08-16 17:56:30 UTC

Post by Alan Carroll
"getting the TSEventThread from a vconn, txn, ssn instead of implicitly
trusting that the current thread is relevant?"
I'm not sure what this means. Those objects don't have threads explicitly
associated, it's more implicit. If the code is in a callback on a hook for
the object, then the current thread is relevant, effectively by definition.

When coordinating between txns and reenabling or accessing object a from
object b's hook you might not be so lucky.A hook for A might be called on
an unexpected thread and fun stuff will happen. Often you will be lucky
because core has a lot of explicit alignment checks and reenabling A will
be done on it its original thread.

But as soon as I read this I realized you were going to bring up the

Post by Alan Carroll
continuation thread affinity issue.
"moving a client vconn to a certain thread" - that would be challenging.
I'd rather move the plugins to the NetVC thread. This is one aspect of
getting the mutex locking thread - that's the thread to which to move.

I meant outgoing netvc connections, my bad, but we have the technology to
make it better, :), in the global http server connectionpool we do this
with the netvcs to align with the txn/sm threads. I don't know what has an
higher impact thread jumping or moving the netvc.
Moving netvcs to other threads might have some impact on data in flight, I
think Susan knows about the limitations of this.

Wouldn't it be nice if we had connection pool constructs which we could
reuse :) instead of reinventing.
Somehow when pulling threads the whole world starts to unravel.

Post by Alan Carroll
"check the callback continuations in the current API?" - check them for
what? I don't understand this question.

The callback cont for some async tsapi calls might not be called on the
same thread. It would be nice to know the guaranteed behavior of calls with
callback continuations without checking core code (for example DNS)

Alan Carroll

2018-08-16 17:59:35 UTC

As a quick response, it sounds like most of your issues would be addressed
by thread affinity for continuations. I'll see if I can get some write up
done for the summit. There are other issues beyond yours that I think would
also benefit from that.

.invalid>

for

Post by Alan Carroll
the object, then the current thread is relevant, effectively by

definition.
When coordinating between txns and reenabling or accessing object a from
object b's hook you might not be so lucky.A hook for A might be called on
an unexpected thread and fun stuff will happen. Often you will be lucky
because core has a lot of explicit alignment checks and reenabling A will
be done on it its original thread.
But as soon as I read this I realized you were going to bring up the

I meant outgoing netvc connections, my bad, but we have the technology to
make it better, :), in the global http server connectionpool we do this
with the netvcs to align with the txn/sm threads. I don't know what has an
higher impact thread jumping or moving the netvc.
Moving netvcs to other threads might have some impact on data in flight, I
think Susan knows about the limitations of this.
Wouldn't it be nice if we had connection pool constructs which we could
reuse :) instead of reinventing.
Somehow when pulling threads the whole world starts to unravel.

Post by Alan Carroll
"check the callback continuations in the current API?" - check them for
what? I don't understand this question.

Fei Deng

2018-08-21 14:46:43 UTC

I'm looking into this right now. Already have the TSContScheduleOn
and TSContScheduleOnEvery APIs done. Still working on getting the thread
affinity.

On Thu, Aug 16, 2018 at 12:59 PM Alan Carroll

Post by Alan Carroll
As a quick response, it sounds like most of your issues would be addressed
by thread affinity for continuations. I'll see if I can get some write up
done for the summit. There are other issues beyond yours that I think would
also benefit from that.

.invalid>

Post by Alan Carroll
"getting the TSEventThread from a vconn, txn, ssn instead of

implicitly

Post by Alan Carroll
trusting that the current thread is relevant?"
I'm not sure what this means. Those objects don't have threads

explicitly

Post by Alan Carroll
associated, it's more implicit. If the code is in a callback on a hook

for

Post by Alan Carroll
the object, then the current thread is relevant, effectively by

Post by Alan Carroll
continuation thread affinity issue.
"moving a client vconn to a certain thread" - that would be

challenging.

Post by Alan Carroll
I'd rather move the plugins to the NetVC thread. This is one aspect of
getting the mutex locking thread - that's the thread to which to move.

higher impact thread jumping or moving the netvc.
Moving netvcs to other threads might have some impact on data in flight,

think Susan knows about the limitations of this.
Wouldn't it be nice if we had connection pool constructs which we could
reuse :) instead of reinventing.
Somehow when pulling threads the whole world starts to unravel.

Post by Alan Carroll
"check the callback continuations in the current API?" - check them for
what? I don't understand this question.

The callback cont for some async tsapi calls might not be called on the
same thread. It would be nice to know the guaranteed behavior of calls

with

callback continuations without checking core code (for example DNS)

--
*Fei Deng*

E: ***@oath.com

1908 S First Street
Champaign, IL, 61820

Alan Carroll

2018-08-21 14:50:58 UTC