ngx_mruby v2.7.6
Behavior changes: read before upgrading
Section titled “Behavior changes: read before upgrading”Output from an HTTP log handler
Section titled “Output from an HTTP log handler”Security advisory: GHSA-r6wq-r8rh-6rc3.
- Before: a log handler could append the text of
Nginx.rputsorNginx.echoafter the response body. - Now: the calls convert their argument as usual and write no bytes. The script continues after a successful call.
- Why it changed:
- The defect: the shared handler finalizer could flush an output chain while nginx was freeing the request.
- Why the old behavior could not be kept: extra bytes could be read as another response on the connection.
- What the documentation promised: the log directives run at the end of request processing for logging.
- Affected: HTTP file and inline log handlers, including calls from their helper methods.
- What to do: log messages with
Nginx.log. Write response bodies in a content handler. - Not changed by this release: argument conversion errors and output from ordinary content handlers.
Output of an mruby content handler or body filter in an SSI subrequest
Section titled “Output of an mruby content handler or body filter in an SSI subrequest”Security advisory: GHSA-79vp-37cx-547v.
- Before: the output of a subrequest’s mruby content handler (
Nginx.rputs,Nginx.echo,Nginx.send_header) or mruby body filter could end the parent response at the subrequest’s output. Over HTTP/1.1 the chunked filter emitted the response-terminator chunk mid-stream, so a length-delimited reader saw two responses where there was one; over HTTP/1.0 or when the parent usedContent-Length, the parent response was cut short at the subrequest’s bytes and the connection closed; and a static file read through the mruby body filter of a subrequest held the main request untilsend_timeout. - Now: a subrequest marks its last buffer with
last_in_chain(andsyncfor an empty buffer), as nginx’s own handlers do. The parent response goes on until the main request’s own last buffer. The static file case no longer stops the main request. - Why it changed:
- The defect: ngx_mruby’s response finalization set
last_bufon every chain, which is correct only for the main request. Nginx’s chunked filter, Content-Length framing and body-end detection act onlast_buf. - Why the old behavior could not be kept: a subrequest’s
last_bufended the parent response early (a split chunked response, a truncated Content-Length or HTTP/1.0 response, read as another response on a shared connection), and the body-filter path held the main request untilsend_timeoutclosed the connection. - What the documentation promised: Nginx.rputs and Nginx::
Filter in 2.7.5 said nothing about how a subrequest’s output interacts with the parent response.
- The defect: ngx_mruby’s response finalization set
- Affected: locations whose mruby content handler calls
Nginx.rputs,Nginx.echoorNginx.send_header, or whose mruby body filter produces a body, when another location includes them with SSI<!--# include virtual=... -->or by another form of subrequest. - What to do: nothing. The subrequest output is framed correctly as part of the parent response.
A value of mruby_set or Nginx::Var whose to_s is missing, raises or does not return a String no longer ends the worker (#768)
Section titled “A value of mruby_set or Nginx::Var whose to_s is missing, raises or does not return a String no longer ends the worker (#768)”- Before: ngx_mruby converted a value that is not a String with its
to_sand used the result as a String without checking its type. For the value that the code ofmruby_setormruby_set_codereturned, it calledto_swhere mruby does not raise an exception ofto_sbut returns it. Ato_sthat returns something other than a String ended the worker process with a segmentation fault, before and afterNginx::Async.sleep. A value withoutto_s(aBasicObject) or whoseto_sraises was read as a String although it was the exception, and the exception stayed in the mruby state of the worker: the request got 500 or no response, and the next mruby handler that the worker ran, in the same request or a later one, could fail with that exception and answer 500. The setter ofNginx::Var(Nginx::Var#<name>=) andNginx::Var#setended the worker process when they got a value, orNginx::Var#seta name, whoseto_sreturns something other than a String. Ato_sthat is missing or raises made these two methods raise its exception, as it does now. - Now: a value of
mruby_setandmruby_set_codethat cannot be converted counts as an exception of the code. Before the code waits inNginx::Async.sleep, ngx_mruby logs the exception, the variable is set to an empty string, and the request goes on. After the sleep, ngx_mruby logs the exception and the request gets 500. The exception isTypeError(... cannot be converted to String) for ato_sthat returns something other than a String,NoMethodErrorfor a value withoutto_s, and the exception ofto_sotherwise. The setter ofNginx::VarandNginx::Var#setraiseTypeErrorwhento_sreturns something other than a String, and the variable keeps its value. No exception stays for a later handler. - Why it changed:
- The defect: ngx_mruby read the result of
to_sas a String without checking its type, and at the end of anmruby_sethandler it calledto_soutside Ruby code, where mruby returns the exception of ato_sthat is missing or raises instead of raising it. The review of #735, which fixed the same conversion onnext, found it. - Why the old behavior could not be kept: the old behavior ended the worker process, with the other requests of that worker, or made a later handler fail with an exception that was not its own. The variable needs a String, and such a value gives none, so the fix handles the value as the exception that ngx_mruby already handles for this code.
- What the documentation promised: mruby_set and mruby_
set_ and Nginx::code Var in 2.7.5 said nothing about values that are not Strings. The documentation ofmruby_setdescribed how the variable and the request go on after an exception of the code, which this release applies to these values.
- The defect: ngx_mruby read the result of
- Affected:
mruby_setandmruby_set_codewhose code returns a value whoseto_sis missing, raises or does not return a String; calls of the setter ofNginx::Varand ofNginx::Var#setwith a value, or a name, whoseto_sdoes not return a String. - What to do: return a String,
nilor a value whoseto_sreturns a String from the code ofmruby_set, and pass such values toNginx::Var. Code that returned or passed the other values did not work on 2.7.5 either: it ended the worker or made a handler fail. - Not changed by this release: a String,
nil, an Integer and any other value whoseto_sreturns a String give the variable the same value as before, and ato_sthat is missing or raises in a call ofNginx::Varraises the same exception as before.
Nginx.rputs, Nginx.echo, Nginx.redirect and Nginx::Filter#body= raise TypeError when to_s of the argument does not return a String (#775)
Section titled “Nginx.rputs, Nginx.echo, Nginx.redirect and Nginx::Filter#body= raise TypeError when to_s of the argument does not return a String (#775)”- Before:
Nginx.rputs,Nginx.echo,Nginx.redirect,Nginx::Filter#body=andNginx::Filter#outputconverted an argument that is not a String with itsto_sand used the result as a String without checking its type. Ato_sthat returns an Integer ended the worker process with a segmentation fault, with the other requests of that worker. Ato_sthat returns another object that is not a String made the method read the memory of that object as a String. Ato_sthat is missing or raises made these methods raise its exception, as it does now. - Now: these methods raise
TypeError(... cannot be converted to String) whento_sof the argument returns something other than a String, as the setter ofNginx::Vardoes since #768. The exception ends the handler like any other exception of the script: arescuein the script catches it, and otherwise ngx_mruby logs it and the request gets 500. Nothing is written, no redirect is made, and the body of the filter stays as it was. - Why it changed:
- The defect: these methods read the result of
to_sas a String without checking its type. #768 fixed the same reading formruby_setandNginx::Var, and the tests of that change found it in these methods. - Why the old behavior could not be kept: the old behavior ended the worker process, with the other requests of that worker, or read memory that was not a String. These methods need a String, and such a value gives none, so they raise the exception that the setter of
Nginx::Varraises for the same value. - What the documentation promised: Nginx.rputs in 2.7.5 said that an argument that is not a String is converted with
to_s, and said nothing about ato_sthat does not return a String. The documentation ofNginx.echo,Nginx.redirectandNginx::Filter#body=said nothing about arguments that are not Strings.
- The defect: these methods read the result of
- Affected: calls of
Nginx.rputs,Nginx.echo,Nginx.redirect,Nginx::Filter#body=andNginx::Filter#outputwith an argument whoseto_sdoes not return a String. - What to do: pass a String,
nilor a value whoseto_sreturns a String to these methods. Code that passed other values did not work on 2.7.5 either: it ended the worker or read memory that was not a String. - Not changed by this release: a String,
niland any other value whoseto_sreturns a String give the same output as before, and ato_sthat is missing or raises makes these methods raise the same exception as before.
An mruby body filter, with or without an mruby header filter, runs once on a proxied response that needs more than one write after an earlier Nginx.rputs (#786)
Section titled “An mruby body filter, with or without an mruby header filter, runs once on a proxied response that needs more than one write after an earlier Nginx.rputs (#786)”- Before: in a location with
mruby_output_body_filter(ormruby_output_body_filter_code), with or withoutmruby_output_header_filter(ormruby_output_header_filter_code), when an earlier handler of the request had written withNginx.rputsand let the request go on (for example a rewrite handler that returnsNginx::DECLINED), the response came fromproxy_pass, and nginx could not send it in one write (for example withsendfile_max_chunkorlimit_rate), the code of the body filter ran twice: once on the body of the upstream response, and once more on an empty body when the upstream response ended. The client got the body of the first run with itsContent-Length, but$sent_http_content_lengthgave0inlog_format. - Now: the code of the body filter runs once for such a response, and
$sent_http_content_lengthgives the length of the body that the client got. A response that nginx sends in one write is not changed by this release: there the code of the body filter still runs a second time on an empty body. - Why it changed:
- The defect: when an earlier handler’s
Nginx.rputsoutput is still in the state of the request after the body filter has run, ngx_mruby drops that state, so that a later handler does not send the output after the response. The dropped state also held the flag that makes the body filter pass the rest of the same response on to the next filter, and the body filter then waited for a new body. A response from a static file stopped after its first write untilsend_timeoutclosed the connection (see “Fixes”). A proxied response went on only because the end of the upstream response made the body filter run once more, on an empty body. The review of #785 found it. - Why the old behavior could not be kept: the stop and the second run have the same cause, and the fix gives the request a new state with that flag whenever nginx still holds a part of the response. Keeping the second run for proxied responses would mean leaving the flag out for them alone, and the code of their body filter would still run on an empty body after it has made the response. The new behavior is the one that the same location already has without the earlier
Nginx.rputs: there the code of the body filter runs once and$sent_http_content_lengthgives the length of the body, whether nginx sends the response in one write or in more. - What the documentation promised: mruby_
output_ and Nginx::body_ filter Filter in 2.7.5 said nothing about how many times the code of a body filter runs for one response.
- The defect: when an earlier handler’s
- Affected: the code of
mruby_output_body_filterandmruby_output_body_filter_codethat counts its runs or acts on an empty body, andlog_formatwith$sent_http_content_length, for responses fromproxy_pass(or another upstream) after an earlierNginx.rputsin a location with an mruby body filter, with or without an mruby header filter, which nginx sends in more than one write. - What to do: nothing for a body filter that only transforms the body, because the body that the client gets is the same. A body filter whose code relied on its second run on an empty body, for example to count responses or to act at the end of one, has to do that in the first run, which now is the only one for such a response.
New features
Section titled “New features”None.
- When a rewrite or access handler calls
Nginx::Async::HTTP.sub_requestafter a rewrite or access handler of the same request waited inNginx::Async.sleep, the subrequest now starts at once. This holds when the handler that slept calls it and when a handler of a later phase calls it, for example an access handler after a rewrite handler that returnedNginx::DECLINEDafter its sleep. Before, the subrequest started only when the next event of the client connection arrived, and until then the client got no response (#772) - When a content or rewrite handler writes its response with
Nginx.rputsin a location withmruby_output_body_filterormruby_output_body_filter_code, and nginx cannot send the response in one write (for example withlimit_rateorsendfile_max_chunk, or to a client that reads slowly), nginx now sends the rest of the response. Before, the client received the header with the fullContent-Lengthand only the part of the body that the first write sent, and nginx closed the connection whensend_timeoutpassed (#784) - When an mruby body filter runs, with or without an mruby header filter in the location, after an earlier handler wrote with
Nginx.rputsand let the request go on (for example a rewrite handler that returnsNginx::DECLINEDbefore a static file is served), and nginx cannot send the response in one write (for example withsendfile_max_chunkorlimit_rate, or to a client that reads slowly), nginx now sends the rest of the response. Before, the client received the header with the fullContent-Lengthand only the part of the body that the first write sent, and nginx closed the connection whensend_timeoutpassed (#786)
Build and test changes
Section titled “Build and test changes”-
directives.rbchecks the complete response stream of two requests on one persistent connection, including file and inline log handlers and their subsequent variable updates. -
New test case:
test/t/cases/var_value_to_s.rb, with the server oftest/conf/conf.d/83-var-value-to-s.confon port 18159 (#768) -
New test case:
test/t/cases/async_sub_request_after_sleep.rb, with the server oftest/conf/conf.d/84-async-sub-request-after-sleep.confon port 18190 (#772) -
New test case:
test/t/cases/output_value_to_s.rb, with the server oftest/conf/conf.d/88-output-value-to-s.confon port 18191 (#775) -
New test case:
test/t/cases/body_filter_split_write.rb, with the server oftest/conf/conf.d/89-body-filter-split-write.confon port 18141 (#784) -
New test case:
test/t/cases/body_filter_leftover_chain.rb, with the server oftest/conf/conf.d/90-body-filter-leftover-chain.confon port 18145 (#786) -
New test case:
test/t/cases/ssi_body_filter.rb, with the server oftest/conf/conf.d/91-ssi-body-filter.confon port 18146.
Upgrading
Section titled “Upgrading”- No directive was added, renamed or removed. Rebuild the module with this release the way you built 2.7.5, and replace the binary or the dynamic module as usual.
Check these before upgrading. Each line is the “What to do” of its entry above.
- The code of
mruby_setandmruby_set_code, and calls ofNginx::Var, with a value whoseto_sis missing, raises or does not return a String: return or pass a String,nilor a value whoseto_sreturns a String. - Calls of
Nginx.rputs,Nginx.echo,Nginx.redirect,Nginx::Filter#body=andNginx::Filter#outputwith an argument whoseto_sdoes not return a String: pass a String,nilor a value whoseto_sreturns a String. - Code of
mruby_output_body_filterandmruby_output_body_filter_codethat relied on running a second time on an empty body for a proxied response sent in more than one write after an earlierNginx.rputs, with or without an mruby header filter in the location: do that in the first run, which now is the only one for such a response.