ngx_mruby v2.7.12
Behavior changes: read before upgrading
Section titled “Behavior changes: read before upgrading”1. A HEAD, 204 or 304 response through an mruby output filter keeps its connection available (#853)
Section titled “1. A HEAD, 204 or 304 response through an mruby output filter keeps its connection available (#853)”- Before: a response that ends at its header, that is the response to a HEAD request, a 304 that nginx answers to a GET with
If-Modified-SinceorIf-None-Match, or a 204 such as that ofreturn 204;, received its header withConnection: keep-alivewhen it passed an mruby output filter, and nginx then closed the connection. A request that the client sent next on that connection got no response. This happened in a location withmruby_output_header_filterormruby_output_header_filter_codeand no mruby body filter, for HEAD, 204 and 304. It happened in a location withmruby_output_body_filterormruby_output_body_filter_codefor a response withoutContent-Length, such as a listing ofautoindexor the 304 of a static file, whose length nginx removes, and for a 204 or 304 with aContent-Length, such as the 204 ofreturn 204;, which nginx gives the length 0. A HEAD response with a known length through a body filter kept its connection since 2.7.5 (#752). - Now: after nginx has sent the header of a response that ends at it (HEAD, 204 or 304), ngx_mruby returns the result of sending it and passes no body on, as nginx’s own handlers do when the header ends the response. nginx keeps an otherwise reusable connection for the next request. The status and the header that the client receives are the same, and no body bytes follow the header. GET responses, a result of
NGX_ERRORfrom sending the header, and an error of the code of a filter keep their handling. - Why it changed:
- The defect: nginx’s header filter sets
r->header_onlywhen it sends the header of a HEAD, 204 or 304 response. The three places where ngx_mruby passes the header on (after the code of a header filter in a location without a body filter; in the body filter, for a response without a length; and after the code of a body filter, inngx_mrb_finalize_body_filter) returnedNGX_ERRORwhen that flag was set, although the header had been sent; the last place did so for every method but HEAD since 2.7.5. nginx’s handlers, such as the static file handler,returnandautoindex, return that result, so nginx ended the request as failed and closed the connection. The same defect was fixed on the development line of 3.0.0 by #848 and #849. - Why the old behavior could not be kept: a HEAD, 204 or 304 response that nginx sent successfully is not an output error, and closing the connection lost the next request that keep-alive announced. A header filter that adds a header to the responses of a location closed the connection after every conditional GET of a static file there that the client could answer from its cache.
- What the documentation promised: the directives of 2.7.11 said nothing about HEAD, 204 or 304 responses in a location with an output filter. Nginx::
Filter said that a HEAD response with a known content length through the body filter keeps its connection, and that other header-only responses kept their behavior.#body=
- The defect: nginx’s header filter sets
- Affected: HEAD, 204 and 304 responses that pass an mruby output header filter or body filter, when the connection can otherwise be kept alive.
- What to do: nothing. To close a connection deliberately, configure nginx’s keep-alive policy (
keepalive_timeout,keepalive_requests) instead of relying on this error. - Tests: port 18112 (
test/conf/conf.d/20-filter-connection.conf,test/t/cases/head_header_filter.rbandtest/t/cases/header_only_keepalive.rb).
2. A request header that Nginx::Headers_in#[]= replaces is sent to the upstream with the script’s value, also when it was the last header of the request (#858)
Section titled “2. A request header that Nginx::Headers_in#[]= replaces is sent to the upstream with the script’s value, also when it was the last header of the request (#858)”- Before: when a script replaced a request header with
Nginx::Headers_in#[]=and the header that it replaced was the last header of the request (a request withHostalone, for example), then, depending on the configuration,proxy_passandgrpc_passdid not send the value of the script, and sent only the value ofproxy_set_header(orgrpc_set_header) or nginx’s own value for that name. When the replaced header was followed by another header, or the script added a new name, the value of the script was sent as well, next to that ofproxy_set_headeror nginx, unless the worker process ended, which, depending on the configuration, could happen (see “Fixes”). - Now: the value of the script is sent as well in every one of these cases, wherever the replaced header was in the request.
- Why it changed:
- The defect: a header that
[]=adds had nolowcase_keyand nonextset, and nginx read them when it sent the request withproxy_passorgrpc_pass(see “Fixes”). The value that nginx read decided whether the module left the script’s header out. - Why the old behavior could not be kept: the fix sets both fields, and the value that it sets makes the result the same in every case: the 2.x line does not match a header that a script adds against the names of the headers that those modules set, which is what happened for every header but the replaced last one.
- What the documentation promised: Nginx::
Headers_in#[]= of 2.7.11 said that it sets the value of the request header, and nothing aboutproxy_passorgrpc_pass.
- The defect: a header that
- Affected: a location with
proxy_passorgrpc_passwhere a script replaces, withNginx::Headers_in#[]=, a request header thatproxy_set_header(orgrpc_set_header) or the module itself also sets, such asHost, and the header is the last one of some requests. WithHost, the upstream now receives twoHostlines for such requests, as it already did for the other requests; nginx as the upstream answers 400 to them. - What to do: to send the value of the script alone, do not set the header with
Nginx::Headers_in#[]=: put the value in a variable, for example withmruby_set_code $backend_host '...';or withNginx::Varon a variable thatsetdeclares, and give it toproxy_set_header Host $backend_host;(orgrpc_set_header). For a name that the module does not set itself, removing itsproxy_set_headeralso works. - Tests: ports 18099 and 18100 (
test/conf/conf.d/74-headers-in-proxy.conf,test/t/cases/headers_in_proxy.rb): the replaced header followed by another header, as the last header and as the only header;Hostreplaced; the variable given toproxy_set_header.
New features
Section titled “New features”None.
- When a response of length 0, such as the 204 of
return 204;, passes an mruby output body filter, the body filter no longer passes NULL to the copy of its empty body. No byte was copied, and the response is the same; a build with UndefinedBehaviorSanitizer reportednull pointer passed as argument 1for the copy (#853) - A header that
Nginx::Headers_in#[]=orNginx::Headers_out#[]=adds has itslowcase_keyandnextset. They were not set: whenproxy_passorgrpc_passsent the request to the upstream, nginx’s proxy or grpc module read thatlowcase_key, which, depending on the configuration, could end the worker process. The added header is sent to the upstream, also whenproxy_set_headerorgrpc_set_headersets the same name; see also the second entry under “Behavior changes: read before upgrading” (#858)
Build and test changes
Section titled “Build and test changes”test/t/cases/head_header_filter.rbandtest/t/cases/header_only_keepalive.rbsend a request whose response ends at the header, or a control, and then GET on the same TCP connection, to new locations of the server on port 18112 (test/conf/conf.d/20-filter-connection.conf): HEAD through a header filter alone on a response ofreturnand on a static file, HEAD through a body filter on anautoindexlisting, also sent at 50 bytes a second (limit_rate 50,send_timeout 3s) so that its header needs more than one write, 304 (If-Modified-Sinceto a static file withif_modified_since before) and 204 (return 204;) through a header filter alone and through a body filter, and 304 to GET and HEAD of a static file that a body filter location reads through an output buffer smaller than the file (output_buffers 1 8,send_timeout 3s).test/t/cases/_head_filter_client.rbtakes request header lines as an optional third argument, and ends the first response at its header for 204 and 304 as for HEAD (#853).test/t/cases/headers_in_proxy.rbadds and replaces request headers withNginx::Headers_in#[]=beforeproxy_pass, on the servers on ports 18099 and 18100 (test/conf/conf.d/74-headers-in-proxy.conf), and checks the headers that the upstream receives: every added header, also one thatproxy_set_headersets and one that the proxy module sets itself, a replaced header wherever it was in the request, and a value given toproxy_set_headerthrough a variable; under valgrind, the build before #858 reports the read of the unsetlowcase_keythere (#858).
Upgrading
Section titled “Upgrading”- No directive was added, renamed or removed. Rebuild the module with this release the way you built 2.7.11, and replace the binary or the dynamic module as usual.
Nginx.module_versionreturns2.7.12. - HEAD, 204 and 304 responses through an mruby output header or body filter: nothing; to close a connection deliberately, configure nginx’s keep-alive policy (#853).
- A request header that
Nginx::Headers_in#[]=replaces in a location withproxy_passorgrpc_pass, when the module also sets that name: to send the value of the script alone, give it toproxy_set_header(orgrpc_set_header) through a variable instead (#858).
Read “Behavior changes: read before upgrading” above before upgrading.