question about mutexes in TS

[Walt Karas]

In TS, is it important to favor use of continuation mutexes to avoid
thread blocking. For example, should code like this:

before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();

be replaced with code like:

int contf_after(TSCont, TSEvent, void *)
{
after();

return 0;
}

int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();

static TSCont cont_after = TSContCreate(contf_after, nullptr);

TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);

return 0;
}

// ...

before();

static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);

TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);

// ...

(This is plugin code but I assume the same principle would apply to core code.)

[Alan Carroll]

It's a bit more complex than that. One key thing is that if you schedule an
event for a continuation, when the event handler is called the continuation
mutex will be locked. Therefore it's rarely the case a plugin needs to lock
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.

In the core, there is a class, MutexLock, which does the RAII style locking.

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to core code.)

[Leif Hedstrom]

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule an
event for a continuation, when the event handler is called the continuation
mutex will be locked. Therefore it's rarely the case a plugin needs to lock
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style locking.

Right. I think the example (below) makes sense if the critical_section() is blocking for example (for longish times). And then you can schedule that work on the “task” threads I think (and make sure you have enough of those thread).

Cheers,

— leif

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to core code.)

--
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

[Walt Karas]

To what "explicit continuation locking" do you refer?

How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule an
event for a continuation, when the event handler is called the continuation
mutex will be locked. Therefore it's rarely the case a plugin needs to lock
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style locking.

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to core code.)

--
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

[Alan Carroll]

By explicit continuation locking, I mean code like your example -

before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();

The call to TSMutexLock() is an explicit lock of the mutex. It address the
issue by noting that it rarely, if ever, comes up. However what you suggest
is what I would probably do, although it depends quite heavily on why
exactly the lock in question needs to be explicitly locked, rather than
being an event scheduled on a continuation that already has it for a mutex.

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule

an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs to

lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread to

the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

[Pushkar Pradhan]

I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for later.

void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the handler
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule

an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs to

lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread to

the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

[Walt Karas]

It seems like ATS does not have a coherent strategy for threading and mutexes.

One pure strategy is event driven with one thread per core (processing
an event queue). Share data structures are owned by a particular core
(also helps with NUMA), and events that want to access the data
structure are queued on the thread for that core. Few if any mutexes
are needed.

The other strategy (non event driven) is to figure that threads are
low-overhead compared to processes, so don't worry about creating gobs
of them, or the mutex overhead. Most programmers find this approach
to be more straightforward, and it jives better with the API of Unix
and most other OSes.

ATS has static threads, but many more than one per core. The
purpose(s) of the large number and various types of threads is
unclear. Our discussion of mutexes quickly turned into blah blah blah
mumble mumble.
On Wed, Oct 10, 2018 at 11:38 AM Pushkar Pradhan

Post by Pushkar Pradhan
I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for later.
void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the handler
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule

an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs to

lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread to

the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

--
pushkar

[Leif Hedstrom]

Post by Walt Karas
It seems like ATS does not have a coherent strategy for threading and mutexes.
One pure strategy is event driven with one thread per core (processing
an event queue). Share data structures are owned by a particular core
(also helps with NUMA), and events that want to access the data
structure are queued on the thread for that core. Few if any mutexes
are needed.

This should be the preferred strategy when at all possible IMO. This is why we had that discussions, where I mentioned that the intent at least is for the HostDB/DNS lookups to reschedule back to the originating ET_NET thread.

Post by Walt Karas
The other strategy (non event driven) is to figure that threads are
low-overhead compared to processes, so don't worry about creating gobs
of them, or the mutex overhead. Most programmers find this approach
to be more straightforward, and it jives better with the API of Unix
and most other OSes.

That’s just not true. Look at Varnish, it does this approach, and it suffers with 10’s of thousands of threads, and a cache that’s essentially unusable for serious use. Yes, it’s low overhead, but far from zero. Assuming that the OS can handle things was a good idea on (research) papers, but falls apart in reality.

Post by Walt Karas
ATS has static threads, but many more than one per core. The
purpose(s) of the large number and various types of threads is
unclear. Our discussion of mutexes quickly turned into blah blah blah
mumble mumble.

This is “legacy”, If we can figure out where the bottlenecks are, striving towards one ET_NET thread per core should be a goal.

Cheers,

— Leif

Post by Walt Karas
On Wed, Oct 10, 2018 at 11:38 AM Pushkar Pradhan

Post by Pushkar Pradhan
I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for later.
void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the handler
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule

an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs to

lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread to

the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

--
pushkar

[Walt Karas]

Post by Leif Hedstrom

Post by Walt Karas
It seems like ATS does not have a coherent strategy for threading and mutexes.
One pure strategy is event driven with one thread per core (processing
an event queue). Share data structures are owned by a particular core
(also helps with NUMA), and events that want to access the data
structure are queued on the thread for that core. Few if any mutexes
are needed.

This should be the preferred strategy when at all possible IMO. This is why we had that discussions, where I mentioned that the intent at least is for the HostDB/DNS lookups to reschedule back to the originating ET_NET thread.

Why do we need multiple thread types and is the reason explained
anywhere in TS docs?

Post by Leif Hedstrom

Post by Walt Karas
The other strategy (non event driven) is to figure that threads are
low-overhead compared to processes, so don't worry about creating gobs
of them, or the mutex overhead. Most programmers find this approach
to be more straightforward, and it jives better with the API of Unix
and most other OSes.

That’s just not true. Look at Varnish, it does this approach, and it suffers with 10’s of thousands of threads, and a cache that’s essentially unusable for serious use. Yes, it’s low overhead, but far from zero. Assuming that the OS can handle things was a good idea on (research) papers, but falls apart in reality.

Probably depends on the application. It's a trade off between ease of
implementation and performance. With lots of dynamically-created
threads, there can be a lot of implicit context information contained
in the call stack. With the event-driven approach you have to figure
out other ways to store all that context information, which will be
more or less challenging depending on the application. Also,
threading overhead I think is probably much less for RTOSes,
especially ones that run with the MMU turned off.

Post by Leif Hedstrom

Post by Walt Karas
ATS has static threads, but many more than one per core. The
purpose(s) of the large number and various types of threads is
unclear. Our discussion of mutexes quickly turned into blah blah blah
mumble mumble.

This is “legacy”, If we can figure out where the bottlenecks are, striving towards one ET_NET thread per core should be a goal.
Cheers,
— Leif

Post by Walt Karas
On Wed, Oct 10, 2018 at 11:38 AM Pushkar Pradhan

Post by Pushkar Pradhan
I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for later.
void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the handler
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you schedule

an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs to

lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling handles
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to avoid
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread to

the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

--
pushkar

[Alan Carroll]

Yes, that's not well developed. However, it is the reason TS mutexes are
shareable. If there is a resource shared between continuations the TS
approach is to have those continuations share the same mutex leading to
natural rescheduling for lock contention. In addition, that TS mutexes are
recusive makes it so that if you can arrange for all of the continuations
sharing a resource to be on the same ET_NET thread then locking is
effectively cost free, yet remains robust in the cases where there is
actual lock contention.

Because of the proxy (pass-through) nature of I/O in TS, it's quite
difficult to keep shared resources on the same thread if outbound sessions
are shared across threads. It is also problematic in the case of scheduling
to and from different thread pools (although Fei's work should address to a
large extent).

[Walt Karas]

Why do we need thread pools?
On Wed, Oct 10, 2018 at 2:17 PM Alan Carroll

Post by Alan Carroll
Yes, that's not well developed. However, it is the reason TS mutexes are
shareable. If there is a resource shared between continuations the TS
approach is to have those continuations share the same mutex leading to
natural rescheduling for lock contention. In addition, that TS mutexes are
recusive makes it so that if you can arrange for all of the continuations
sharing a resource to be on the same ET_NET thread then locking is
effectively cost free, yet remains robust in the cases where there is
actual lock contention.
Because of the proxy (pass-through) nature of I/O in TS, it's quite
difficult to keep shared resources on the same thread if outbound sessions
are shared across threads. It is also problematic in the case of scheduling
to and from different thread pools (although Fei's work should address to a
large extent).

[Alan Carroll]

To accomodate different behavior requirements on the continuations
scheduled on a thread in that pool. E.g. non-blocking vs. blocking for one.

Post by Walt Karas
Why do we need thread pools?
On Wed, Oct 10, 2018 at 2:17 PM Alan Carroll

Post by Alan Carroll
Yes, that's not well developed. However, it is the reason TS mutexes are
shareable. If there is a resource shared between continuations the TS
approach is to have those continuations share the same mutex leading to
natural rescheduling for lock contention. In addition, that TS mutexes

are

Post by Alan Carroll
recusive makes it so that if you can arrange for all of the continuations
sharing a resource to be on the same ET_NET thread then locking is
effectively cost free, yet remains robust in the cases where there is
actual lock contention.
Because of the proxy (pass-through) nature of I/O in TS, it's quite
difficult to keep shared resources on the same thread if outbound

sessions

Post by Alan Carroll
are shared across threads. It is also problematic in the case of

scheduling

Post by Alan Carroll
to and from different thread pools (although Fei's work should address

to a

Post by Alan Carroll
large extent).

[Walt Karas]

Unfortunately that's like many of our explanations, too open-ended to
be very helpful when someone sits down to write some code.
On Wed, Oct 10, 2018 at 2:26 PM Alan Carroll

Post by Alan Carroll
To accomodate different behavior requirements on the continuations
scheduled on a thread in that pool. E.g. non-blocking vs. blocking for one.

Post by Walt Karas
Why do we need thread pools?
On Wed, Oct 10, 2018 at 2:17 PM Alan Carroll

Post by Alan Carroll
Yes, that's not well developed. However, it is the reason TS mutexes are
shareable. If there is a resource shared between continuations the TS
approach is to have those continuations share the same mutex leading to
natural rescheduling for lock contention. In addition, that TS mutexes

are

Post by Alan Carroll
recusive makes it so that if you can arrange for all of the continuations
sharing a resource to be on the same ET_NET thread then locking is
effectively cost free, yet remains robust in the cases where there is
actual lock contention.
Because of the proxy (pass-through) nature of I/O in TS, it's quite
difficult to keep shared resources on the same thread if outbound

sessions

Post by Alan Carroll
are shared across threads. It is also problematic in the case of

scheduling

Post by Alan Carroll
to and from different thread pools (although Fei's work should address

to a

Post by Alan Carroll
large extent).

--
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

[Pushkar Pradhan]

I thought the purpose of the different threads in ATS was pretty clear.
There are many threads to handle requests (1.5x the number of HT cores).
8 threads to handle disk I/O (cache).
2-3 threads to do miscellaneous tasks.
2-3 threads to handle logging.
One thread for each port which you want to listen on.

Are you running into some perf issue with locking/contention etc?

Post by Walt Karas
It seems like ATS does not have a coherent strategy for threading and mutexes.
One pure strategy is event driven with one thread per core (processing
an event queue). Share data structures are owned by a particular core
(also helps with NUMA), and events that want to access the data
structure are queued on the thread for that core. Few if any mutexes
are needed.
The other strategy (non event driven) is to figure that threads are
low-overhead compared to processes, so don't worry about creating gobs
of them, or the mutex overhead. Most programmers find this approach
to be more straightforward, and it jives better with the API of Unix
and most other OSes.
ATS has static threads, but many more than one per core. The
purpose(s) of the large number and various types of threads is
unclear. Our discussion of mutexes quickly turned into blah blah blah
mumble mumble.
On Wed, Oct 10, 2018 at 11:38 AM Pushkar Pradhan

Post by Pushkar Pradhan
I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for

later.

Post by Pushkar Pradhan
void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the

handler

Post by Pushkar Pradhan
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you

schedule

Post by Pushkar Pradhan

Post by Walt Karas
an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs

to

Post by Pushkar Pradhan

Post by Walt Karas
lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling

handles

Post by Pushkar Pradhan

Post by Walt Karas

Post by Alan Carroll
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to

avoid

Post by Pushkar Pradhan

Post by Walt Karas

Post by Alan Carroll

Post by Walt Karas
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread

to

Post by Pushkar Pradhan

Post by Walt Karas
the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

--
pushkar

[Walt Karas]

But these are not inherently purposes. I could decide there needed to
be a special thread to execute all functions with the letter r in the
name. But that would arbitrary, not an actual purpose. Generally
speaking, in an event driven system, you only need different threads
for multiple cores and/or multiple priorities (if you need
preemption). You don't need separate threads for modularity /
encapsulation purposes. Alan mentions there is some sort of intent to
make sure that events that take the same mutexes are in the same
thread, so the locking is generally vacuous and thus fast. But, when
I write new code, how do I keep the continuations locking the same
mutex on the same thread?
On Wed, Oct 10, 2018 at 5:49 PM Pushkar Pradhan

Post by Pushkar Pradhan
I thought the purpose of the different threads in ATS was pretty clear.
There are many threads to handle requests (1.5x the number of HT cores).
8 threads to handle disk I/O (cache).
2-3 threads to do miscellaneous tasks.
2-3 threads to handle logging.
One thread for each port which you want to listen on.
Are you running into some perf issue with locking/contention etc?

Post by Walt Karas
It seems like ATS does not have a coherent strategy for threading and mutexes.
One pure strategy is event driven with one thread per core (processing
an event queue). Share data structures are owned by a particular core
(also helps with NUMA), and events that want to access the data
structure are queued on the thread for that core. Few if any mutexes
are needed.
The other strategy (non event driven) is to figure that threads are
low-overhead compared to processes, so don't worry about creating gobs
of them, or the mutex overhead. Most programmers find this approach
to be more straightforward, and it jives better with the API of Unix
and most other OSes.
ATS has static threads, but many more than one per core. The
purpose(s) of the large number and various types of threads is
unclear. Our discussion of mutexes quickly turned into blah blah blah
mumble mumble.
On Wed, Oct 10, 2018 at 11:38 AM Pushkar Pradhan

Post by Pushkar Pradhan
I think Alan is referring to the below code.
It's a try lock, so if it doesn't succeed it's just rescheduled for

later.

Post by Pushkar Pradhan
void
EThread::process_event(Event *e, int calling_code)
{
ink_assert((!e->in_the_prot_queue && !e->in_the_priority_queue));
MUTEX_TRY_LOCK_FOR(lock, e->mutex, this, e->continuation);
if (!lock.is_locked()) {
e->timeout_at = cur_time + DELAY_FOR_RETRY;
EventQueueExternal.enqueue_local(e);
} else {
if (e->cancelled) {
free_event(e);
return;
}
Continuation *c_temp = e->continuation;
// Make sure that the contination is locked before calling the

handler

Post by Pushkar Pradhan
//set_cont_flags(e->continuation->control_flags);
e->continuation->handleEvent(calling_code, e);

Post by Walt Karas
To what "explicit continuation locking" do you refer?
How does this address the issue that using TSMutexLock() or MutexLock
in a currently running continuation function (unnecessarily) blocks
all other events waiting in a thread event queue? Whereas the
inability to lock a continuation mutex cause the continuation to be
requeued at the end of the thread event queue, thus allowing
succeeding events in the thread's queue to be handled.
On Tue, Oct 9, 2018 at 3:44 PM Alan Carroll

Post by Alan Carroll
It's a bit more complex than that. One key thing is that if you

schedule

Post by Pushkar Pradhan

Post by Walt Karas
an

Post by Alan Carroll
event for a continuation, when the event handler is called the

continuation

Post by Alan Carroll
mutex will be locked. Therefore it's rarely the case a plugin needs

to

Post by Pushkar Pradhan

Post by Walt Karas
lock

Post by Alan Carroll
its continuations explicitly. For that reason, simply scheduling

handles

Post by Pushkar Pradhan

Post by Walt Karas

Post by Alan Carroll
lock contention without thread blocking.
In the core, there is a class, MutexLock, which does the RAII style

locking.

Post by Alan Carroll

Post by Walt Karas
In TS, is it important to favor use of continuation mutexes to

avoid

Post by Pushkar Pradhan

Post by Walt Karas

Post by Alan Carroll

Post by Walt Karas
before();
TSMutexLock(mutex);
critical_section();
TSMutexUnlock(mutex);
after();
int contf_after(TSCont, TSEvent, void *)
{
after();
return 0;
}
int contf_critical_section(TSCont, TSEvent, void *)
{
critical_section();
static TSCont cont_after = TSContCreate(contf_after, nullptr);
TSContSchedule(cont_after, 0, TS_THREAD_POOL_DEFAULT);
return 0;
}
// ...
before();
static TSCont cont_critical_section =
TSContCreate(contf_critical_section, mutex);
TSContSchedule(cont_critical_section, 0, TS_THREAD_POOL_DEFAULT);
// ...
(This is plugin code but I assume the same principle would apply to

core

Post by Alan Carroll

Post by Walt Karas
code.)

--
*Beware the fisherman who's casting out his line in to a dried up

riverbed.*

Post by Alan Carroll
*Oh don't try to tell him 'cause he won't believe. Throw some bread

to

Post by Pushkar Pradhan

Post by Walt Karas
the

Post by Alan Carroll
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28

--
pushkar

--
pushkar