[PROPOSAL] Rewrite url after running all remap plugins

[Takuya Kitano]

Hi,

I’d like to propose a change about the timing that ATS runs remap plugins.

Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.

```
map http://before-remap.com/ http://after-remap.com/ @plugin=<first plugin>.so @pparam=... @plugin=<second plugin>.so @pparam=...
```

In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.

The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.

My proposal is that ATS should execute all remap plugins and then rewrite url.

More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531


What do you think about this?

Thank you.

[Bryan Call]

We had a discussion on the IRC about this and we agree that treating the first plugin special is not a good idea and is confusing. I looked at the code and approved the PR. I would like to get someone else to look at it too before merging it, since it is an important change to the code.

Thank you for the PR!

-Bryan

Post by Takuya Kitano
Hi,
I’d like to propose a change about the timing that ATS runs remap plugins.
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.
```
```
In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.
My proposal is that ATS should execute all remap plugins and then rewrite url.
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

[Yongming Zhao]

I¡¯d like to take a little step back ask about what is the root causing about this change:

in the origin design
1, the remap plugin works in ordered, which means you can stop(consider it done) remapping at any plugin if you want, and the following plugins will not run after.
2, the remap plugin works in stacks, which means the seconds plugin will continue work on the URL which is rewritten in the first plugin.

when you talking about the origin URL, there is always a pristine URL you can copy, if you really want, so the second plugin can see the origin un-mapped URL. this pristine URL is design for logging, but I think you can take it for your purpose if you like


so, here rise my question:
can the pristine URL full file your second plugin needs? if so, can we keep the origin design?


- Yongming Zhao ÕÔÓÀÃ÷

Hi,
I¡¯d like to propose a change about the timing that ATS runs remap plugins.
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.
```
```
In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.
My proposal is that ATS should execute all remap plugins and then rewrite url.
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

[Bret Palsson]

I don’t normally comment, but feel like I need to chime in here.

I agree in keeping with the stack behavior. This change has the potential to break a lot of plugin logic and will cause a lot of work across many dev teams that maintain many plugins. The pristine URL should be used in subsequent plugins as needed.

Having said that, I am also interested to know and understand the cause for wanting this behavior.

-Bret

Post by Yongming Zhao
in the origin design
1, the remap plugin works in ordered, which means you can stop(consider it done) remapping at any plugin if you want, and the following plugins will not run after.
2, the remap plugin works in stacks, which means the seconds plugin will continue work on the URL which is rewritten in the first plugin.
when you talking about the origin URL, there is always a pristine URL you can copy, if you really want, so the second plugin can see the origin un-mapped URL. this pristine URL is design for logging, but I think you can take it for your purpose if you like
can the pristine URL full file your second plugin needs? if so, can we keep the origin design?
- Yongming Zhao 赵永明

Hi,
I’d like to propose a change about the timing that ATS runs remap plugins.
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.
```
```
In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.
My proposal is that ATS should execute all remap plugins and then rewrite url.
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

[宋辰伟]

Actually there is a problem that URL is only overwritten in first plugin in current logic. That means the request_url is overwritten in first plugin(which return NO_REMAP ) and never changed even if the remain plugin set map_to or map_form, except you set request_url directly. If my understanding is correctly.

scw00 - Song Chenwei

Post by Bret Palsson
I don’t normally comment, but feel like I need to chime in here.
I agree in keeping with the stack behavior. This change has the potential to break a lot of plugin logic and will cause a lot of work across many dev teams that maintain many plugins. The pristine URL should be used in subsequent plugins as needed.
Having said that, I am also interested to know and understand the cause for wanting this behavior.
-Bret

Post by Yongming Zhao
in the origin design
1, the remap plugin works in ordered, which means you can stop(consider it done) remapping at any plugin if you want, and the following plugins will not run after.
2, the remap plugin works in stacks, which means the seconds plugin will continue work on the URL which is rewritten in the first plugin.
when you talking about the origin URL, there is always a pristine URL you can copy, if you really want, so the second plugin can see the origin un-mapped URL. this pristine URL is design for logging, but I think you can take it for your purpose if you like
can the pristine URL full file your second plugin needs? if so, can we keep the origin design?
- Yongming Zhao 赵永明

Hi,
I’d like to propose a change about the timing that ATS runs remap plugins.
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.
```
```
In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.
My proposal is that ATS should execute all remap plugins and then rewrite url.
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

.

[Takuya Kitano]

Thank you. This is exactly what I want to fix.



2018/12/02 21:59 に、"宋辰伟" <***@qq.com> を書き込みました:

Actually there is a problem that URL is only overwritten in first plugin in current logic. That means the request_url is overwritten in first plugin(which return NO_REMAP ) and never changed even if the remain plugin set map_to or map_form, except you set request_url directly. If my understanding is correctly.

scw00 - Song Chenwei

Post by Bret Palsson
I don’t normally comment, but feel like I need to chime in here.
I agree in keeping with the stack behavior. This change has the potential to break a lot of plugin logic and will cause a lot of work across many dev teams that maintain many plugins. The pristine URL should be used in subsequent plugins as needed.
Having said that, I am also interested to know and understand the cause for wanting this behavior.
-Bret

Post by Yongming Zhao
in the origin design
1, the remap plugin works in ordered, which means you can stop(consider it done) remapping at any plugin if you want, and the following plugins will not run after.
2, the remap plugin works in stacks, which means the seconds plugin will continue work on the URL which is rewritten in the first plugin.
when you talking about the origin URL, there is always a pristine URL you can copy, if you really want, so the second plugin can see the origin un-mapped URL. this pristine URL is design for logging, but I think you can take it for your purpose if you like
can the pristine URL full file your second plugin needs? if so, can we keep the origin design?
- Yongming Zhao 赵永明

Hi,
I’d like to propose a change about the timing that ATS runs remap plugins.
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different from that as the second plugin.
```
```
In detail, the first plugin get pre-remapped url information from `rri->requestUrl`,
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes `url_rewrite_remap_request` function after running the first remap plugin.
My proposal is that ATS should execute all remap plugins and then rewrite url.
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

[Alan Carroll]

This came up again on IRC. The effect of the PR is

Before:
If the first remap plugin does not do a remap, then the URL is remapped to
the "from" URL in the remap rule, then the rest of the remap plugins run.

After:
All remap plugins are run, and if *none* of them remap, then the "from" URL
from the remap rule is used.

The primary point of this is to not have the first remap plugin be special.
In particular this means a plugin that's change from first to second
doesn't see different behavior.

Post by Takuya Kitano
Thank you. This is exactly what I want to fix.
Actually there is a problem that URL is only overwritten in first
plugin in current logic. That means the request_url is overwritten in first
plugin(which return NO_REMAP ) and never changed even if the remain plugin
set map_to or map_form, except you set request_url directly. If my
understanding is correctly.
scw00 - Song Chenwei

I don’t normally comment, but feel like I need to chime in here.
I agree in keeping with the stack behavior. This change has the

potential to break a lot of plugin logic and will cause a lot of work
across many dev teams that maintain many plugins. The pristine URL should
be used in subsequent plugins as needed.

Having said that, I am also interested to know and understand the

cause for wanting this behavior.

-Bret

I’d like to take a little step back ask about what is the root
in the origin design
1, the remap plugin works in ordered, which means you can

stop(consider it done) remapping at any plugin if you want, and the
following plugins will not run after.

2, the remap plugin works in stacks, which means the seconds plugin

will continue work on the URL which is rewritten in the first plugin.

when you talking about the origin URL, there is always a pristine

URL you can copy, if you really want, so the second plugin can see the
origin un-mapped URL. this pristine URL is design for logging, but I think
you can take it for your purpose if you like

can the pristine URL full file your second plugin needs? if so, can

we keep the origin design?

- Yongming Zhao 赵氞明

Post by Takuya Kitano
Hi,
I’d like to propose a change about the timing that ATS runs remap

plugins.

Post by Takuya Kitano
Currently, we configure remap.config like the following snippet,
and each plugins use `rri->requestUrl`,
the behavior of a remap plugin as the first plugin is different

from that as the second plugin.

Post by Takuya Kitano
```
map http://before-remap.com/ http://after-remap.com/

@plugin=<first plugin>.so @pparam=... @plugin=<second plugin>.so @pparam=...

Post by Takuya Kitano
```
In detail, the first plugin get pre-remapped url information from

`rri->requestUrl`,

Post by Takuya Kitano
but the second plugin get post-remapped one.
The cause of this behavior is ATS executes

`url_rewrite_remap_request` function after running the first remap plugin.

Post by Takuya Kitano
My proposal is that ATS should execute all remap plugins and then

rewrite url.

Post by Takuya Kitano
More ditails are in
- Issue : https://github.com/apache/trafficserver/issues/2877
- Pull Request : https://github.com/apache/trafficserver/pull/4531
What do you think about this?
Thank you.

.