Skip to content

ngx_mruby v2.7.12

1. A HEAD, 204 or 304 response through an mruby output filter keeps its connection available (#853)

Section titled “1. A HEAD, 204 or 304 response through an mruby output filter keeps its connection available (#853)”
  • Before: a response that ends at its header, that is the response to a HEAD request, a 304 that nginx answers to a GET with If-Modified-Since or If-None-Match, or a 204 such as that of return 204;, received its header with Connection: keep-alive when it passed an mruby output filter, and nginx then closed the connection. A request that the client sent next on that connection got no response. This happened in a location with mruby_output_header_filter or mruby_output_header_filter_code and no mruby body filter, for HEAD, 204 and 304. It happened in a location with mruby_output_body_filter or mruby_output_body_filter_code for a response without Content-Length, such as a listing of autoindex or the 304 of a static file, whose length nginx removes, and for a 204 or 304 with a Content-Length, such as the 204 of return 204;, which nginx gives the length 0. A HEAD response with a known length through a body filter kept its connection since 2.7.5 (#752).
  • Now: after nginx has sent the header of a response that ends at it (HEAD, 204 or 304), ngx_mruby returns the result of sending it and passes no body on, as nginx’s own handlers do when the header ends the response. nginx keeps an otherwise reusable connection for the next request. The status and the header that the client receives are the same, and no body bytes follow the header. GET responses, a result of NGX_ERROR from sending the header, and an error of the code of a filter keep their handling.
  • Why it changed:
    • The defect: nginx’s header filter sets r->header_only when it sends the header of a HEAD, 204 or 304 response. The three places where ngx_mruby passes the header on (after the code of a header filter in a location without a body filter; in the body filter, for a response without a length; and after the code of a body filter, in ngx_mrb_finalize_body_filter) returned NGX_ERROR when that flag was set, although the header had been sent; the last place did so for every method but HEAD since 2.7.5. nginx’s handlers, such as the static file handler, return and autoindex, return that result, so nginx ended the request as failed and closed the connection. The same defect was fixed on the development line of 3.0.0 by #848 and #849.
    • Why the old behavior could not be kept: a HEAD, 204 or 304 response that nginx sent successfully is not an output error, and closing the connection lost the next request that keep-alive announced. A header filter that adds a header to the responses of a location closed the connection after every conditional GET of a static file there that the client could answer from its cache.
    • What the documentation promised: the directives of 2.7.11 said nothing about HEAD, 204 or 304 responses in a location with an output filter. Nginx::Filter#body= said that a HEAD response with a known content length through the body filter keeps its connection, and that other header-only responses kept their behavior.
  • Affected: HEAD, 204 and 304 responses that pass an mruby output header filter or body filter, when the connection can otherwise be kept alive.
  • What to do: nothing. To close a connection deliberately, configure nginx’s keep-alive policy (keepalive_timeout, keepalive_requests) instead of relying on this error.
  • Tests: port 18112 (test/conf/conf.d/20-filter-connection.conf, test/t/cases/head_header_filter.rb and test/t/cases/header_only_keepalive.rb).

2. A request header that Nginx::Headers_in#[]= replaces is sent to the upstream with the script’s value, also when it was the last header of the request (#858)

Section titled “2. A request header that Nginx::Headers_in#[]= replaces is sent to the upstream with the script’s value, also when it was the last header of the request (#858)”
  • Before: when a script replaced a request header with Nginx::Headers_in#[]= and the header that it replaced was the last header of the request (a request with Host alone, for example), then, depending on the configuration, proxy_pass and grpc_pass did not send the value of the script, and sent only the value of proxy_set_header (or grpc_set_header) or nginx’s own value for that name. When the replaced header was followed by another header, or the script added a new name, the value of the script was sent as well, next to that of proxy_set_header or nginx, unless the worker process ended, which, depending on the configuration, could happen (see “Fixes”).
  • Now: the value of the script is sent as well in every one of these cases, wherever the replaced header was in the request.
  • Why it changed:
    • The defect: a header that []= adds had no lowcase_key and no next set, and nginx read them when it sent the request with proxy_pass or grpc_pass (see “Fixes”). The value that nginx read decided whether the module left the script’s header out.
    • Why the old behavior could not be kept: the fix sets both fields, and the value that it sets makes the result the same in every case: the 2.x line does not match a header that a script adds against the names of the headers that those modules set, which is what happened for every header but the replaced last one.
    • What the documentation promised: Nginx::Headers_in#[]= of 2.7.11 said that it sets the value of the request header, and nothing about proxy_pass or grpc_pass.
  • Affected: a location with proxy_pass or grpc_pass where a script replaces, with Nginx::Headers_in#[]=, a request header that proxy_set_header (or grpc_set_header) or the module itself also sets, such as Host, and the header is the last one of some requests. With Host, the upstream now receives two Host lines for such requests, as it already did for the other requests; nginx as the upstream answers 400 to them.
  • What to do: to send the value of the script alone, do not set the header with Nginx::Headers_in#[]=: put the value in a variable, for example with mruby_set_code $backend_host '...'; or with Nginx::Var on a variable that set declares, and give it to proxy_set_header Host $backend_host; (or grpc_set_header). For a name that the module does not set itself, removing its proxy_set_header also works.
  • Tests: ports 18099 and 18100 (test/conf/conf.d/74-headers-in-proxy.conf, test/t/cases/headers_in_proxy.rb): the replaced header followed by another header, as the last header and as the only header; Host replaced; the variable given to proxy_set_header.

None.

  • When a response of length 0, such as the 204 of return 204;, passes an mruby output body filter, the body filter no longer passes NULL to the copy of its empty body. No byte was copied, and the response is the same; a build with UndefinedBehaviorSanitizer reported null pointer passed as argument 1 for the copy (#853)
  • A header that Nginx::Headers_in#[]= or Nginx::Headers_out#[]= adds has its lowcase_key and next set. They were not set: when proxy_pass or grpc_pass sent the request to the upstream, nginx’s proxy or grpc module read that lowcase_key, which, depending on the configuration, could end the worker process. The added header is sent to the upstream, also when proxy_set_header or grpc_set_header sets the same name; see also the second entry under “Behavior changes: read before upgrading” (#858)
  • test/t/cases/head_header_filter.rb and test/t/cases/header_only_keepalive.rb send a request whose response ends at the header, or a control, and then GET on the same TCP connection, to new locations of the server on port 18112 (test/conf/conf.d/20-filter-connection.conf): HEAD through a header filter alone on a response of return and on a static file, HEAD through a body filter on an autoindex listing, also sent at 50 bytes a second (limit_rate 50, send_timeout 3s) so that its header needs more than one write, 304 (If-Modified-Since to a static file with if_modified_since before) and 204 (return 204;) through a header filter alone and through a body filter, and 304 to GET and HEAD of a static file that a body filter location reads through an output buffer smaller than the file (output_buffers 1 8, send_timeout 3s). test/t/cases/_head_filter_client.rb takes request header lines as an optional third argument, and ends the first response at its header for 204 and 304 as for HEAD (#853).
  • test/t/cases/headers_in_proxy.rb adds and replaces request headers with Nginx::Headers_in#[]= before proxy_pass, on the servers on ports 18099 and 18100 (test/conf/conf.d/74-headers-in-proxy.conf), and checks the headers that the upstream receives: every added header, also one that proxy_set_header sets and one that the proxy module sets itself, a replaced header wherever it was in the request, and a value given to proxy_set_header through a variable; under valgrind, the build before #858 reports the read of the unset lowcase_key there (#858).
  • No directive was added, renamed or removed. Rebuild the module with this release the way you built 2.7.11, and replace the binary or the dynamic module as usual. Nginx.module_version returns 2.7.12.
  • HEAD, 204 and 304 responses through an mruby output header or body filter: nothing; to close a connection deliberately, configure nginx’s keep-alive policy (#853).
  • A request header that Nginx::Headers_in#[]= replaces in a location with proxy_pass or grpc_pass, when the module also sets that name: to send the value of the script alone, give it to proxy_set_header (or grpc_set_header) through a variable instead (#858).

Read “Behavior changes: read before upgrading” above before upgrading.