ngx_mruby v2.7.1
ngx_mruby 2.7.1 is a maintenance and security release of the 2.x line. It fixes the issues described in the security advisory GHSA-h7fw-wf49-8rv5 and several bugs found while extending the test suite and reviewing the code.
Some of these fixes change what an existing configuration or Ruby script can observe, for example a status code, a response body or its length, an exception, whether nginx goes on with the next phase, which certificate a handshake gets, or whether nginx starts. If a script was written for the old behavior, it may need a change. Read the first section before you upgrade. Each entry says what you could observe in 2.7.0 (“Before”), what you observe in 2.7.1 (“Now”), why the behavior had to change (“Why it changed”: “The defect” or “The reason”, “Why the old behavior could not be kept” and “What the documentation promised”), which configurations are affected, what to do, and, where the test suite covers the change, the port of its tests. “Not changed by this release” names a related behavior that stays as it was. “Known limitations” lists behavior that this release leaves as it was.
Behavior changes: read before upgrading
Section titled “Behavior changes: read before upgrading”| If your build, configuration or scripts use | Read |
|---|---|
Redis and the default build_config.rb |
1 |
mruby_output_body_filter*, mruby_output_header_filter* or Nginx::Filter |
2 |
raise after Nginx.rputs or Nginx.echo |
3 |
Nginx::Async or Fiber.yield in a log handler or an output filter |
4 |
Nginx::Async.sleep or Nginx::Async::HTTP.sub_request in a handler that raises or calls Nginx.redirect after it resumes, or sub_request to a location that sleeps |
5 |
mruby_enable_read_request_body on |
6 |
Nginx::SSL or mruby_ssl_handshake_handler* |
7 |
request methods or Nginx.log in mruby_init*, mruby_init_worker*, mruby_exit_worker*, mruby_server_context_handler_code or the handshake handler; Nginx::Server#add_listener or Nginx::Stream.add_listener; the stream init, init worker, exit worker or server context code |
8 |
mruby_set* and Nginx::Async.sleep in a later handler of the same request |
9 |
a configuration without an http {} block |
10 |
mruby_output_header_filter* and mruby_output_body_filter* in one location, with Nginx.rputs or Nginx.echo in the body filter |
11 |
a handler that answers a request with a body (POST, PUT) with Nginx.rputs or Nginx.echo without reading the body |
12 |
mruby_content_handler* that can end without output, or log monitoring that matches response body is empty |
13 |
mruby_enable_read_request_body on with a post-read or access handler, and mruby_log_handler* or an mruby output filter for the same request |
14 |
1. Redis is no longer in the default build
Section titled “1. Redis is no longer in the default build”- Before:
build_config.rbincludedmatsumotory/mruby-redis, so a binary built with the default configuration had theRedisclass. - Now: that line is commented out (#528), because the gem clones its
hiredis dependency from an unpinned branch, which broke the build. A binary
built with the default
build_config.rbhas noRedisclass, and code that refers to it raisesNameError(uninitialized constant Redis). - Why it changed:
- The reason: except on macOS, the build of mruby-redis clones hiredis from
the head of its default branch, so a change there could break every build
made with the default
build_config.rb; the comment that #528 added tobuild_config.rbrecords such a failure of CI under-Werror. - Why the old behavior could not be kept: keeping the gem in the default
list would have left every default build open to such a failure. The
default build has to succeed, and
build_config.rbcannot pin hiredis, because mruby-redis clones it in its own build script, so the gem cannot stay in the default list until mruby-redis pins hiredis. - What the documentation promised: the 2.7.0 documentation listed
mruby-redis among “the default mrbgems”
(docs/install of 2.7.0) and
Redisamong the classes that can be used “with default build configuration” (built-in mrbgems of 2.7.0), so it described the old behavior. Choosing the mrbgems inbuild_config.rbis the documented way to build ngx_mruby with extra gems (docs/install), and it is how you keep Redis.
- The reason: except on macOS, the build of mruby-redis clones hiredis from
the head of its default branch, so a change there could break every build
made with the default
- Affected: scripts that use
Redis, such as the examples in the README and indocs/use_case, and users of the bundledauto-sslmrbgem, whose client takes a Redis connection. A build with your ownbuild_config.rbis not affected. - What to do: if you use Redis, uncomment
conf.gem github: 'matsumotory/mruby-redis'inbuild_config.rbbefore you build. The build of that gem can fail while hiredis has the problem that #528 describes.
2. Output filters: response length and Nginx::Filter
Section titled “2. Output filters: response length and Nginx::Filter”This entry has four parts.
The response length follows the body filter.
- Before: when an mruby handler, such as
mruby_content_handler*, wrote the response body withNginx.rputsorNginx.echoandmruby_output_body_filter*replaced it withNginx::Filter#body=or#output, the response kept the length of theNginx.rputsoutput. A longer replacement body was cut to that length, and a shorter one was padded to it with bytes that were not part of the response. When the body filter itself calledNginx.rputsorNginx.echo, which the documentation has never allowed there,Content-Lengthwas the length of the response before the filter plus the length of that output, and the output itself was not sent. These responses were corrupted, and nothing should depend on them; the repository’s own test of this case expected the cut response, and its expectation changes with this release. This part is about locations without an mruby header filter; for locations with one, see the fourth part. - Now: the body is exactly the output of the body filter, and
Content-Lengthis its length. What the body filter writes withNginx.rputsorNginx.echois neither sent nor counted.
location /filter/output_rputs_origin { mruby_content_handler_code 'Nginx.rputs "origin"'; mruby_output_body_filter_code ' f = Nginx::Filter.new f.output "[#{f.body}] and more" ';}| Version | Content-Length |
Body |
|---|---|---|
| 2.7.0 | 6 |
[origi |
| 2.7.1 | 17 |
[origin] and more |
- Why it changed:
- The defect: ngx_mruby sends as many bytes of the body filter’s buffer as
the length that it keeps for the body, and sets
Content-Lengthto the same number. In 2.7.0 that length came from theContent-LengththatNginx.rputshad counted, and the buffer from the body filter, so the two disagreed whenever the filter changed the length: a longer body was cut, and after a shorter one ngx_mruby sent bytes from beyond the end of the new body, memory that was not part of the response. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: the length and the buffer must agree, and the buffer is the filter’s output, so the length has to follow the output; keeping the old length would mean keeping the cut or the read past the buffer.
- What the documentation promised: the response is now what
Nginx::Filter#body=is documented to do (“Sets the string to the response body”), as it already was for every other origin. The same section of the 2.7.0 documentation (Nginx::Filter Class ) says “You can NOT useNginx.rputsandNginx.echoin the output body handlers”.
- The defect: ngx_mruby sends as many bytes of the body filter’s buffer as
the length that it keeps for the body, and sets
- Affected: locations whose body filter changes the length of a body
written with
Nginx.rputsorNginx.echo, and body filters that callNginx.rputsorNginx.echothemselves, whatever the origin. A body filter that keeps the length and does not call them sees no change. Neither does a body filter on a response from a file,return,proxy_passor another content handler: there the length already followed the filter’s output. - What to do: nothing, unless a client or a test expected the cut response.
Remove
Nginx.rputsandNginx.echofrom body filters: what they write there is not sent. - Tests: port 18112 (
test/t/cases/filter_connection.rb).
Nginx::Filter#body before the body is read.
- Before: in
mruby_output_header_filter*, where the body has not been read yet,Nginx::Filter#bodyreturned a string as long asContent-Lengthwhose bytes were not the response body. When the length was unknown, or 10485760 bytes (10 MiB) or more, it raisedArgumentErrorinstead. - Now: it returns an empty string there, as it already did in a content handler.
- Why it changed:
- The defect: in a header filter the body has not been read yet, and 2.7.0
returned a string as long as
Content-Lengthwhose bytes were not the response body, or raisedArgumentError, as “Before” says. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: there is no body to return at
that point, so the only correct result is an empty string; the length is
available from
$sent_http_content_length. - What the documentation promised: the documentation has always described
Nginx::Filteras a class for the output body filters (“designed to be used in the output body handlers”, Nginx::Filter Class of 2.7.0 ).
- The defect: in a header filter the body has not been read yet, and 2.7.0
returned a string as long as
- Affected: header filters that read
f.body, for example its size to learn the response length. - What to do: read the length with
Nginx::Var.new.sent_http_content_length(a String, ornilwhen the length is unknown), and read the body inmruby_output_body_filter*. - Tests: port 18080 (
test/t/ngx_mruby.rb).
Nginx::Filter#body= and #output only in a body filter.
- Before: they could be called anywhere. When a handler other than a body filter called them in a location that also had a body filter, the worker crashed as soon as that body filter read a response that was not empty. Without a body filter in the location, the call had no effect on the response.
- Now: outside
mruby_output_body_filter*they raiseRuntimeError(Nginx::Filter#body= can only be used in an output body filter), which ends the handler like any other exception. - Why it changed:
- The defect: a body set with
Nginx::Filter#body=or#outputoutside a body filter made the worker crash when a body filter of the location read a response that was not empty, and had no effect without a body filter, as “Before” says. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: the body filter reads the
response into a buffer that it allocates only when no body has been set
yet, and sends that buffer afterwards.
Nginx::Filter#body=set that buffer without the position at which the body filter writes into it, so a body filter that ran later crashed the worker as soon as it read a response that was not empty. 2.x keeps one buffer for both uses, and a body set outside a body filter was never supported, so 2.7.1 refuses the call instead of adding a second buffer. - What the documentation promised: the documentation has always said that
Nginx::Filterdoes not work in other handlers (“It doesn’t work in other handlers likemruby_content_handler_code”, Nginx::Filter Class of 2.7.0 ).
- The defect: a body set with
- Affected: header filters and other handlers that call
body=oroutput. - What to do: set the body in
mruby_output_body_filter*. - Tests: port 18080 (
test/t/ngx_mruby.rb).
With an mruby header filter, the header waits for the body filter.
- Before: in a location with
mruby_output_header_filter*andmruby_output_body_filter*, the header went out as soon as the header filter’s code had run, with theContent-Lengthof the response before the body filter. A body filter that made the body longer then sent more bytes thanContent-Lengthannounced, and one that made it shorter sent fewer. With anNginx.rputshandler as the origin, such a location did not answer at all (see “Fixes”). - Now: the header waits until the body filter has run, as it already did in a location with only a body filter, and goes out with the length of the body filter’s output and with the changes that the header filter made. The header filter’s code still runs before the body filter’s code. A response without a known length is passed on without the body filter, as before.
- Why it changed:
- The defect: sending the header first committed the response to a
Content-Lengththat the body filter then changed, and a client that reads a response by itsContent-Lengthcannot read a body of another length correctly. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: the body filter decides the
length of the body, so the header can carry the right
Content-Lengthonly after the body filter has run. A location with only a body filter already held the header back for this reason. - What the documentation promised: the documentation did not say when the
header is sent in a location that has both filters. It says that
Nginx::Filter#body=“Sets the string to the response body” (Nginx::Filter Class of 2.7.0 ).
- The defect: sending the header first committed the response to a
- Affected: locations with both an mruby header filter and an mruby body filter whose body filter changes the length of the body.
- What to do: nothing.
- Tests: port 18150 (
test/t/cases/header_and_body_filter.rb, with a file and with anNginx.rputshandler as the origin).
3. An exception after Nginx.rputs answers 500
Section titled “3. An exception after Nginx.rputs answers 500”- Before: a post-read, server rewrite, rewrite, access or content handler
that wrote output with
Nginx.rputsorNginx.echoand then raised an exception (without waiting inNginx::Asyncfirst) answered 200 with the output written so far. In an access or post-read handler, nginx then went on with the later phases, so the location’s content handler ran as well and its output followed. - Now: the request gets nginx’s 500 page, the output is dropped, and the
later phases do not run. This is what a handler that raises without output
already got, and what a handler that raises after
Nginx::Asyncgets (entry 5).mruby_set*: in 2.7.0 the output was sent as the response, and the variable got a value that the handler had not set. Now the output is dropped, the variable is an empty string, as for a handler that returnsnil(also when an earlier directive of the location gave it a value), the request goes on, and theContent-Lengthof the response does not count the dropped output. This is for an exception before the handler waits inNginx::Async.sleeporNginx::Async::HTTP.sub_request. AfterNginx::Async.sleepthe request does not go on: it gets nginx’s 500 page without the output (2.7.0 sent the output as the response, with 200). AfterNginx::Async::HTTP.sub_requestnginx goes on with the content phase without waiting for the variable, so the 500 that follows the exception cannot be sent and the client gets no response (entry 5).- Output header filters: an exception there already gave the response status
500 with the body of the origin. A header filter that called
Nginx.rputsorNginx.echocrashed the worker in 2.7.0 (see “Fixes”); when it raises afterwards, its output is now dropped, andContent-Lengthis the length of the origin’s body. Only the header filter’s own output is dropped: when the origin is an mruby content handler that writes withNginx.rputsorNginx.echo, the response has that handler’s output and itsContent-Length, also when the header filter raises without output, which is what 2.7.0 sent in that case. - Log handlers and output body filters: the output is no longer sent after an exception.
- Why it changed:
- The defect: after the exception ngx_mruby logged
mrb_run failed: return 500 HTTP status code to clientand set the status to 500, and then the output path reset the status and sent the partial output with 200, so a failed handler was reported to the client as a success. In an access or post-read handler that path also returned the value that nginx reads as “go on”, so an access check that failed with an exception let the request through. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: an exception has to end the
handler with an error status, or the access decision is wrong, and the
output path cannot send 200 and stop the phase at the same time, so the
output is dropped. The client now gets the 500 that error.log has always
announced, the same as after an exception without output or after
Nginx::Async. Anmruby_set*handler does not answer the request, so the request goes on after its exception: the dropped output must then not count in the length of the response that follows, and the variable needs a defined value, where 2.7.0 let nginx store whatever value its script engine held at that point. - What the documentation promised: the documentation said nothing about a
handler that raises an exception. It shows a content handler that writes
output, calls
Nginx.return Nginx::HTTP_OKand leaves withreturn(Nginx.return of 2.7.0), and that answers 200 in 2.7.0 and in 2.7.1, as “What to do” shows.
- The defect: after the exception ngx_mruby logged
- Affected: scripts that use
raiseto stop a handler after writing output, access checks that write a message and then raise, andmruby_set*handlers that write output and then raise. - What to do: keep
raisefor errors. To end a content handler early with its output, end it without an exception:
# 2.7.0: 200 "under maintenance" 2.7.1: 500Nginx.rputs "under maintenance"raise "stop"# 2.7.0 and 2.7.1: 200 "under maintenance"Nginx.rputs "under maintenance"Nginx.return Nginx::HTTP_OKreturn # leaves the handler; omit it when nothing followsNginx.return with a status other than 200 makes nginx send its own page for
that status without the Nginx.rputs output, as in 2.7.0. Do not write output
in an access or post-read handler: it is sent as the response when the handler
ends, and with Nginx::HTTP_OK or without a status nginx then goes on with the
later phases, whose output follows the response beyond its Content-Length, in
2.7.0 as in 2.7.1 (see “Known limitations”). To stop the request there, return
an error status such as Nginx.return Nginx::HTTP_FORBIDDEN.
- Not changed by this release: a handler that called
Nginx.return,Nginx.send_headerorNginx.status_code=afterNginx.rputsand then raised already got 500. A handler that calledNginx.redirectto a local URI before it raised still ends with the response of the redirect target. - Tests: port 18143 (
test/t/cases/sync_exception.rb).
4. Nginx::Async raises in log handlers and output filters
Section titled “4. Nginx::Async raises in log handlers and output filters”- Before:
Nginx::Async.sleepinmruby_log_handler*left a timer on a request that nginx was already freeing, and the worker could crash when it expired.Nginx::Async::HTTP.sub_requestthere started a subrequest. Inmruby_output_header_filter*andmruby_output_body_filter*,Nginx::Async.sleepstopped the filter at that point: the response went out unfiltered with the status of the origin (200 in the tested configurations), and the code after the sleep did not run, because the request ended before the timer expired.sub_requestin a header filter started a subrequest. A directFiber.yieldin these handlers stopped the handler without a message. - Now: both methods raise
RuntimeErrorthere, for exampleNginx::Async.sleep is not available in a log handler(also... in a header filterand... in a body filter), before they arm a timer or start a subrequest. The exception is handled like any other: error.log gets an[error]line, and in a filter the response gets status 500 (with the unfiltered body in the tested configurations). In a log handler the response has already been sent. A directFiber.yieldthere logs... yielded its fiber, but nginx cannot resume it thereand, in a filter, sets status 500. - Why it changed:
- The defect: nginx runs the log phase while it frees the request, after
the request cleanups and right before the request’s memory is released,
so a timer armed there fired on freed memory and resumed the handler with
a request that no longer existed. Output filters run inside
ngx_http_send_headerandngx_http_output_filter, which return to their caller at once, so a handler that yields there just stops and its filtering is skipped; if a timer or a subrequest resumes it later, the response has already gone out, and a directFiber.yieldis never resumed (#551). - Why the old behavior could not be kept: nginx has no point in these places at which resuming a handler could still change the response, so the call has to fail before it arms a timer or starts a subrequest.
- What the documentation promised: the documentation has always listed
where
Nginx::Asyncworks (the set, rewrite and access handlers); log handlers and filters were never among them (Nginx::Async Class of 2.7.0 ).
- The defect: nginx runs the log phase while it frees the request, after
the request cleanups and right before the request’s memory is released,
so a timer armed there fired on freed memory and resumed the handler with
a request that no longer existed. Output filters run inside
- Affected: log handlers and output filters that call
Nginx::Async.sleep,Nginx::Async::HTTP.sub_requestorFiber.yield. - What to do: do the waiting in a rewrite or access handler (or
mruby_set_code), and hand the result to the filter or the log handler in an nginx variable. - Not changed by this release:
mruby_set*and the post-read, server rewrite, rewrite, access and content handlers can still use both methods (for the content handler, see “Known limitations”). - Tests: ports 18129 (
test/t/cases/async_sleep_in_log.rb) and 18131 (test/t/cases/async_sleep_in_output_filter.rb).
5. An exception after Nginx::Async answers 500
Section titled “5. An exception after Nginx::Async answers 500”This entry concerns a handler that Nginx::Async.sleep or
Nginx::Async::HTTP.sub_request has resumed: when it raises, and when it calls
Nginx.redirect and then raises or calls Nginx.return; and sub_request to a
location that sleeps (#550).
- Before:
- Without output, nginx closed the connection without a response.
- After
Nginx.rputs, the client got 200 with the output written so far. - After a status had been set, for example with
Nginx.return 503, the client got that status. When a handler of the request had calledNginx.return Nginx::DECLINEDbefore (for example a server-level handler), nginx went on with the next phase, for example served the file with 200. - The next request on the same worker that ran any mruby handler failed with 500, and error.log logged the earlier exception again.
- Now: the request gets nginx’s 500 page and the output is dropped. The
next request is not affected.
- When the response had already started, the 500 cannot be sent: nginx logs
header already sentand ends the request, and the client keeps what was sent before. WhenNginx.redirectto a local URI followedNginx::Async.sleep, that is the redirect target’s response. In two cases measured for #550 the client now gets no response at all, where 2.7.0 sent one:Nginx.redirectafterNginx::Async::HTTP.sub_request(2.7.0 sent the redirect target’s response and then kept the connection open until the client closed it), and amruby_set_codethat raises aftersub_request(nginx does not wait for the variable before the content phase; 2.7.0 sent 200 with the content handler’s output). - After
Nginx.redirectfollowed by an exception or byNginx.returnin a resumed handler, nginx closes the connection right after the redirect target’s response. On an HTTP/1.1 keep-alive connection, a request pipelined behind it gets no response, and the client has to send it again on a new connection. 2.7.0 kept the connection open, but the pipelined request did not get its own response either: after an exception it got a 500 page caused by the exception that the worker kept (see the last point of “Before” above), and afterNginx.returnit got nothing until the client or a timeout closed the connection. Nginx::Async::HTTP.sub_requestto a location whose handler waits inNginx::Async.sleepnow completes. 2.7.0 sent nothing until the client orsend_timeoutended the request. When that location raises,Nginx::Async::HTTP.last_response.statusis 0.
- When the response had already started, the 500 cannot be sent: nginx logs
- Why it changed:
- The defect: after an exception in a resumed handler, ngx_mruby logged
return 500 HTTP status code to clientand then replaced the 500 with the status the handler had before the exception: 0 (no response), 200 with the partial output, or a value such asNginx::DECLINEDthat let nginx go on with the next phase. The exception also stayed in the mruby state that all requests of the worker share, so the next request failed with it. In 2.7.0, afterNginx.redirectand thenNginx.returnwith a status, the request and its connection stayed open until the client or a timeout ended them (#550). - Why the old behavior could not be kept: that state must be cleared after an exception, or the next request on the worker fails, and the status must stay an error, or nginx treats the failed handler as a success. Running the requests that nginx posted while it finished the request is how nginx completes a request that is finished from a timer. In 2.7.1 the 500 (or the status) reaches nginx after the redirect target’s response has been sent, so nginx can only end the request by closing the connection, and running the posted requests does that at once; that is why a keep-alive connection is now closed after the redirect target’s response. The client now gets the 500 that error.log announces, as after an exception in a handler that did not wait.
- What the documentation promised: the README of 2.7.0 says that the server
process frees “the exception flag” of the mruby state that the scripts
share, to “extend safety by preventing interference between each script”
(Abstract); the next request that failed with the
exception of an earlier one is such interference. The documentation said
nothing about an exception in a handler that
Nginx::Asynchad resumed, or aboutNginx.redirectin such a handler. It lists the set, rewrite and access handlers as the places whereNginx::Async.sleepandNginx::Async::HTTP.sub_requestwork (Nginx::Async Class of 2.7.0 ).
- The defect: after an exception in a resumed handler, ngx_mruby logged
- Affected: handlers that use
Nginx::Asyncand can raise afterwards, clients that relied on the empty reply or on the status set before the exception,mruby_set_codehandlers that usesub_request, and clients that pipeline requests on a keep-alive connection to a location that callsNginx.redirectin a resumed handler and then raises or callsNginx.return. - What to do: if a handler should answer something other than 500, rescue the exception inside the handler and set the status there.
- Tests: ports 18130 (
test/t/cases/async_exception_state.rb) and 18124 (test/t/cases/async_subrequest_raise.rb).
6. mruby_enable_read_request_body: which handlers wait for the whole body
Section titled “6. mruby_enable_read_request_body: which handlers wait for the whole body”With mruby_enable_read_request_body on, ngx_mruby reads the request body
before it runs the code of a handler. This entry has two parts: the handlers
that can wait for a body that is not complete yet, and those that cannot.
Post-read, server rewrite, rewrite, access and content handlers wait.
- Before: a
mruby_server_rewrite_handler*,mruby_rewrite_handler*ormruby_content_handler*whose request body had not fully arrived when the handler started never ran. The client got no response, and nginx did not end the request when the client closed the connection orclient_body_timeoutexpired. A body that arrives in more than one read, from a large upload or a slow client, was enough. Post-read and access handlers waited for the body and ran with it, but after such a wait a later phase handler that waits for a subrequest, such asauth_request, was never resumed: again the client got no response, and the connection stayed open after the client had closed it. - Now: the handler runs once nginx has read the whole body, and the
request then goes on with the later phases, including a phase handler that
waits for a subrequest, such as
auth_request. On a keep-alive connection the next request is answered as usual, and an exception in the handler answers 500. If the client closes the connection first, nginx logsclient prematurely closed connectionand ends the request; whenclient_body_timeoutexpires, nginx closes the connection without a response, and the access log records 408. The handler does not run in either case (for the log handler and the filters of such a request, see entry 14). - Why it changed:
- The defect: while the body was incomplete, ngx_mruby returned
NGX_AGAIN, which nginx’s rewrite and content phases read as “finished”: nginx stopped reading the body, removedclient_body_timeoutand never ran the handler again, so the client got no answer, and the request and its connection stayed open after the client had left, until the worker exited. After a post-read or access handler had waited for the body, ngx_mruby ran the phases again with the write event handler that nginx sets while it reads a body, one that does nothing, and nginx resumes a phase handler that waits for a subrequest, such asauth_request, through that event handler, so that handler was never resumed. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: these phases leave a request
waiting only when the handler returns
NGX_DONE, and no return value both ends the request and keeps reading the body, so the handler has to wait; ngx_mruby runs the phases again when the body is complete. A phase handler that waits for a subrequest is resumed only through the write event handler, so ngx_mruby now sets it back before it runs the phases again, as nginx’s mirror module does. - What the documentation promised: the handler now gets the body, which is
what the directive is documented to do (“reads a HTTP request body before
calling mruby code”). The 2.7.0 documentation of the directive
(mruby_
enable_ ) also says that “a request body is not always read when invoking a handler like mruby_content_handler” and that “The directive force to read the request body”.read_ request_ body
- The defect: while the body was incomplete, ngx_mruby returned
- Affected: configurations that use the directive with server rewrite,
rewrite or content handlers, and with post-read or access handlers that are
followed by
auth_requestor another phase handler that waits for a subrequest. - What to do: nothing.
- Tests: ports 18144 (the five kinds of handler, keep-alive,
client_body_timeoutand an exception after the body) and 18147 (auth_requestafter a rewrite and an access handler), intest/t/cases/request_body_later.rb.
mruby_set*, log handlers and output filters do not wait.
- Before: these handlers started to read the body before their code ran,
like the others. When nginx could read the whole body at once, for example
because all of it had already arrived, the code ran with it. Otherwise the
code did not run at that point (the variable of
mruby_set*got a value that the handler had not set), and the body read that ngx_mruby had started stayed behind and, when it completed, ran the phases of the request again from wherever the request was then. With nginx 1.31.6 this happened to every HTTP/2 request whose body no earlier handler had read, because nginx reads such a body through events. In the tested configurations, withmruby_set_codeand a body that arrived in two parts:- an mruby content handler in the same location ran twice and answered with the first part of the body only;
- with
proxy_pass, the upstream server got only the part that had arrived. When the upstream server waited for the rest, the client got no response; when it did not, the client got the response, but nginx did not close the HTTP/1.0 connection; - a connection stayed open after its client had closed it.
- Now: these handlers read the body only when the read completes at once:
when an earlier handler or a module has read the body, when the request has
no body, or, for HTTP/1.x with
Content-Length, when the whole body is in the buffer that nginx read the request headers into (client_header_buffer_size, 1k by default, or the buffer oflarge_client_header_buffersthat the headers needed). Otherwise the code runs at once without the body:Nginx::Request#bodyreturnsnil, and error.log gets one line at theinfolevel that ends withthe request body is not complete and an mruby_set handler cannot wait for it, so it runs without the body(ora log handler,a header filter,a body filter). A chunked body, an HTTP/2 or HTTP/3 body, and a body of which only a part is in that buffer count as not complete, even when the rest has already arrived. A later handler that can wait, andproxy_pass, still read the whole body. A body that a post-read, server rewrite, rewrite, access or content handler is still waiting for is not complete either; when nginx has ended that read,Nginx::Request#bodyreturns the part of the body that nginx had read (entry 14). - Why it changed:
- The defect: a read that did not complete at once kept a reference to the request after the handler had returned, and its completion ran the phases again from wherever the request was then, with the results above.
- Why the old behavior could not be kept: these handlers cannot wait:
mruby_set*runs inside the script of nginx’s rewrite module, a log handler while nginx frees the request, and the filters while nginx sends the response, and nginx goes on as soon as they return. So such a handler can only run with a body that is already complete, or without one. ngx_mruby decides this before the read starts, fromContent-Lengthand the bytes in nginx’s header buffer: a chunked body has noContent-Length, and an HTTP/2 or HTTP/3 body is read through events, so they count as not complete. - What the documentation promised: the 2.7.0 documentation of the directive
said that ngx_mruby “reads a HTTP request body before calling mruby code”
(mruby_
enable_ ) and named no handler that cannot wait. Neither 2.7.0 nor 2.7.1 makes these handlers wait for the rest of a body: 2.7.0 did not run their code when the read that it started did not complete at once, and 2.7.1 runs it at once, without the body or, after nginx ended the read of a waiting handler, with the part that nginx had read (entry 14). The documentation of the directive now names these handlers and says when they run without the body.read_ request_ body
- Affected: configurations with
mruby_enable_read_request_body onwhosemruby_set*handlers, log handlers or output filters read the request body. These handlers get no body in 2.7.1 when it is chunked, comes over HTTP/2 or HTTP/3, does not fit in nginx’s header buffer together with the headers, or arrives after the headers, unless an earlier handler or module has read it. In 2.7.0 some of these bodies, such as a chunked body that arrived with the headers or a larger body that had already arrived, were read at once. - What to do: to give these handlers the whole body, read it first in a
handler that can wait and runs before them, for example a server-level
mruby_post_read_handler_codewithmruby_enable_read_request_body on: once it has run, the body is complete for the handlers that run after it. - Tests: ports 18148 and 18149 (
test/t/cases/request_body_later.rb).
7. Nginx::SSL works only in the handshake handler
Section titled “7. Nginx::SSL works only in the handshake handler”- Before:
Nginx::SSLmethods (Nginx::SSL.new,#servername, the certificate setters,#local_port,#tls_version,Nginx::SSL.log) called outsidemruby_ssl_handshake_handler*, which the documentation did not support, could crash the worker or return meaningless values (#339). A handshake whose code did not callNginx::SSL.newcould get the certificate that an earlier handshake on the same worker had set, or fail when the memory of that earlier handshake’s connection had been reused. - Now: outside the handshake handler they raise
RuntimeError(Nginx::SSL can only be used in mruby_ssl_handshake_handler). A handshake whose code sets no certificate, also one that does not callNginx::SSL.new, uses the server’sssl_certificateandssl_certificate_key. - Why it changed:
- The defect: outside the handshake handler,
Nginx::SSLcould crash the worker or return meaningless values (#339), and a handshake whose code did not callNginx::SSL.newcould get the certificate of an earlier handshake, or fail, as “Before” says. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept:
Nginx::SSLkept the state of the handshake (the connection, the server name and the certificate strings) in the server configuration after the handshake had finished, so a call outside the handshake read a null pointer when no handshake had run yet (the crash of #339) or memory that was no longer valid, and a later handshake whose code did not callNginx::SSL.newreused the strings of an earlier connection. Outside the handshake there is no connection and no server name, so the methods have nothing valid to return, and the state has to be cleared when the handshake ends. - What the documentation promised: the documentation has said since 2.1.4
that
Nginx::SSLis for the handshake handlers and does not work in other handlers (Nginx::SSL Class of 2.7.0 ), andssl_certificateis nginx’s certificate for a handshake that does not choose one.
- The defect: outside the handshake handler,
- Affected: handlers that call
Nginx::SSLoutside the handshake handler, for example a content handler that readsservername, and handshake handlers that skipNginx::SSL.newon some handshakes, for example code that returns early on a condition, or that creates the object once and keeps it. Handshake handlers that callNginx::SSL.newon every handshake are not affected by the change of the certificate. - What to do: call
Nginx::SSLonly in the handshake handler (to use a value later, store it there, for example withUserdataas in the documentation). If your handler sets a certificate only on some handshakes without callingNginx::SSL.newon the others, those others now getssl_certificate. - Tests: ports 18089 and 18090 (
test/t/ngx_mruby.rb).
8. Request methods outside a request raise RuntimeError
Section titled “8. Request methods outside a request raise RuntimeError”The code of mruby_init*, mruby_init_worker*, mruby_exit_worker*,
mruby_ssl_handshake_handler* and mruby_server_context_handler_code runs
outside a request: mruby_init* and mruby_server_context_handler_code while
nginx reads the configuration, the others in a worker. In the stream module,
the init, init worker, exit worker and server context code runs outside a
session. This entry has two parts.
Request and session methods, and Nginx.log, outside a request.
- Before: request-scoped methods found the current request through a
pointer that kept the last request the worker had handled.
- Where no request had been handled yet, they crashed: in
mruby_init*andmruby_server_context_handler_codenginx itself, while it read the configuration, and inmruby_init_worker*the worker. - In
mruby_exit_worker*andmruby_ssl_handshake_handler*they read the last request that the worker had handled, whose memory could already be freed (meaningless data or a crash), and crashed as inmruby_init_worker*when the worker had not handled a request yet, for example in its first handshake. Nginx.logandNginx.errloggerraisedRuntimeErrorwhere no request had been handled yet, which inmruby_init*andmruby_server_context_handler_codemade the configuration invalid, and elsewhere they wrote through the last request that the worker had handled.- The stream module had the same problem in its init, init worker and exit worker code.
- Where no request had been handled yet, they crashed: in
- Now: outside a request, request-scoped methods raise
RuntimeError(this method can only be used while processing a request). They are the methods ofNginx::Request,Nginx::Var,Nginx::Headers_in,Nginx::Headers_out,Nginx::Connection,Nginx::Server(exceptadd_listener),Nginx::Filter,Nginx::UpstreamandNginx::Async, andNginx.rputs,Nginx.echo,Nginx.return,Nginx.send_header,Nginx.status_code=andNginx.redirect. In code that runs while nginx reads the configuration, this exception makes the configuration invalid, as any exception there does, unless the code rescues it.Nginx.logandNginx.errloggerwrite to nginx’s error log outside a request. In the stream init, init worker, exit worker and server context code,Nginx::Stream::Connection.new, its methods that read the session, andNginx::Stream::Async.sleepraise (this method can only be used while processing a stream session),Nginx::Stream::Connection.stream_statusandstream_status=do not, andNginx::Stream.logwrites to nginx’s error log. - Why it changed:
- The defect: outside a request, request-scoped methods crashed nginx or the
worker, or used the last request that the worker had handled, and
Nginx.logandNginx.errloggerraised or wrote through that request, as “Before” says. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: request methods found their
request through a process-wide pointer that was set when mruby code
started for a request and never cleared. Code outside any request saw a
null pointer, which crashed the process, or the last request the worker
had handled, which could already be freed, so it read invalid memory;
Nginx.logwrote through that request’s connection. Outside a request there is no request to return, so the methods have to report an error; the old results were a crash or another request’s data.Nginx.logdoes not need a request, so it writes to nginx’s error log there. - What the documentation promised: the documentation shows the request
methods only in request handlers, and the code listed above runs outside
any request. The 2.7.0 documentation of
Nginx.errlogger, of whichNginx.logis an alias, says that it “Sends an error to the log file” and names no handler that it is limited to (Nginx.errlogger of 2.7.0).
- The defect: outside a request, request-scoped methods crashed nginx or the
worker, or used the last request that the worker had handled, and
- Affected: init, init worker, exit worker, server context and handshake
code that uses request data (it could not work reliably before), and code
there that relied on
Nginx.lograising. - What to do: use request data in request handlers only.
Nginx.logcan now be used in the code listed above. - Tests: ports 18080 (
mruby_init_worker_code) and 18089 (handshake handler), intest/t/ngx_mruby.rb.
add_listener only in the server context code.
- Before:
Nginx::Server#add_listenercalled after nginx had read the configuration, for example in a request handler, used configuration data that was no longer valid.Nginx::Stream.add_listenercalled in a stream session (mruby_stream_code) or in the stream init, init worker or exit worker code took the internal context of the stream module, which holds the current session, for a server configuration, and the process crashed. Whilemruby_stream_server_context_coderan,Nginx::Stream.logand the session methods took the server configuration for that internal context. - Now:
Nginx::Server#add_listenerraisesRuntimeError(add_listener can only be used in mruby_server_context_handler_code) outsidemruby_server_context_handler_code, andNginx::Stream.add_listenerraisesRuntimeError(add_listener can only be used in mruby_stream_server_context_code) outsidemruby_stream_server_context_code. Both work as before in that code, while nginx reads the configuration. Inmruby_stream_server_context_code,Nginx::Stream.logwrites to nginx’s error log and the session methods raise, as in the stream init and worker code. - Why it changed:
- The defect:
add_listenercalled outside the server context code used configuration data that was no longer valid or crashed the process, and inmruby_stream_server_context_codeNginx::Stream.logand the session methods took the server configuration for the internal context of the stream module, as “Before” says. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept:
add_listenerneeds the state of nginx’s configuration parser, which exists only while nginx reads the configuration;Nginx::Server#add_listenerof the HTTP module used that state after parsing had ended. The stream module put the server configuration in the place of its internal context whilemruby_stream_server_context_coderan, andNginx::Stream.add_listenerread that place whenever it was called; at any other time the place holds the internal context, which is not a server configuration. The server configuration is now kept apart, and only while that code runs, soadd_listenercan tell where it is called, and the other methods always find the internal context.mruby_stream_server_context_coderuns while nginx reads the configuration, outside any session, so the session methods raise there andNginx::Stream.logwrites to nginx’s error log, as in the stream init and worker code. - What the documentation promised: the 2.7.0 documentation shows
Nginx::Server#add_listenerin a configuration only inmruby_server_context_handler_code(Nginx::Server ), refers to that example for#add_ listener Nginx::Stream.add_listener(Nginx::Stream ), and does not say where else they can be called. It does not describe.add_ listener mruby_stream_server_context_code.
- The defect:
- Affected: code that calls
add_listeneroutside the server context code (it could not work before), andmruby_stream_server_context_codethat callsNginx::Stream.logor session methods. - What to do: call
add_listeneronly in the server context code. There is no session inmruby_stream_server_context_code, so do not call session methods there. - Tests: port 18080 (
Nginx::Server#add_listenerin a content handler,test/t/ngx_mruby.rb) and port 12357 (Nginx::Stream.add_listenerin a session, in a second nginx thattest/t/cases/_second_instance.rbstarts andtest/t/cases/stream.rbcalls throughsecond_instanceoftest/t/cases/_prelude.rb).
9. mruby_set variables are no longer overwritten by later handlers
Section titled “9. mruby_set variables are no longer overwritten by later handlers”- Before: after
mruby_setormruby_set_code(also a server-level one) had run in a request, a later handler of the same request that resumed afterNginx::Async.sleephad its last expression assigned to that variable. - Now: only the
mruby_sethandler’s own result is assigned to its variable (#548). Anmruby_set_codethat sleeps itself still sets its variable when it resumes. - Why it changed:
- The defect:
mruby_setstored the name of its variable in the request’s context and nothing cleared it, so when a later handler of the request resumed afterNginx::Async.sleep, ngx_mruby wrote that handler’s last expression into the variable, for example the$backendof a laterproxy_pass(#548). - Why the old behavior could not be kept: the configuration never asked for that write, and keeping it would mean that a sleep in an unrelated handler changes the variable.
- What the documentation promised: the variable of a
mruby_setis set by thatmruby_set, as the directive is documented (“called at one of the rewrite phases to set a nginx variable”, mruby_set and mruby_set_ ).code of 2.7.0
- The defect:
- Affected: requests that combine
mruby_set*with a later handler that usesNginx::Async.sleep. - What to do: nothing. If a later handler should change the variable, assign
it explicitly, for example
Nginx::Var.new.backend = "...". - Tests: port 18123 (
test/t/cases/async_set_var_target.rb).
10. nginx starts without an http block
Section titled “10. nginx starts without an http block”- Before: with ngx_mruby built in, a configuration without an
http {}block, such as a stream-only one, made every worker crash at start, and nginx kept starting new ones. - Now: nginx starts (#536).
- Why it changed:
- The defect: the HTTP module’s worker start and exit hooks read its main
configuration without checking that it exists; without an
http {}block it does not, and the worker crashed (#536). - Why the old behavior could not be kept: nginx accepts configurations
without
http {}, and the stream module of ngx_mruby already handled this case. The old behavior was a crash, so there is nothing to keep. - What the documentation promised: the documentation said nothing about a
configuration without an
http {}block.
- The defect: the HTTP module’s worker start and exit hooks read its main
configuration without checking that it exists; without an
- Affected: configurations without an
http {}block, such as stream-only ones, in an nginx that has the HTTP module of ngx_mruby built in or loaded. - What to do: nothing.
- Tests: port 12399 (
test/t/ngx_mruby.rb).
11. A body filter that calls Nginx.rputs after an mruby header filter no longer logs “header already sent”
Section titled “11. A body filter that calls Nginx.rputs after an mruby header filter no longer logs “header already sent””- Before: in a location with an mruby header filter and an mruby body
filter whose code calls
Nginx.rputsorNginx.echo(which the documentation says is not supported there), nginx logged[alert] header already sentfor every request, and the body filter set the status to 200 after the header was sent. The client received the status that nginx sent, but the log phase saw 200: a log handler and$statusinlog_formatread 200 for a 404 response. - Now: nginx does not log that alert, and the log phase sees the status that nginx sent. The response is the same as before (#563), unless the body filter changes the length of the body, which entry 2 (fourth part) describes.
- Why it changed:
- The defect: at the end of its code the body filter sent the output of
Nginx.rputsthrough the path for a handler that answers, which sent the header a second time and set the status to 200 (#563). - Why the old behavior could not be kept: keeping that path would keep the
[alert] header already sentline for every request and, in the log phase, a status other than the one that nginx sent, as “Before” says. The body filter no longer takes that path once an mruby header filter is in the location, and the output ofNginx.rputsin a body filter is dropped, as before. - What the documentation promised: the documentation says that
Nginx.rputsandNginx.echocannot be used in the output body filters (“You can NOT useNginx.rputsandNginx.echoin the output body handlers”, Nginx::Filter Class of 2.7.0 ); it said nothing about the status that the log phase sees.
- The defect: at the end of its code the body filter sent the output of
- Affected: locations with
mruby_output_header_filter(or_code) andmruby_output_body_filter(or_code) whose body filter callsNginx.rputsorNginx.echo; access logs, log handlers and alerts that read$statusor theheader already sentline for those requests. - What to do: nothing, unless a log statistic or an alert counts on the
status 200 or on the
header already sentline of those requests. - Tests: port 18138
(
test/t/cases/output_filters_with_content_handler.rb).
12. An unread request body is discarded before the answer
Section titled “12. An unread request body is discarded before the answer”- Before: when a handler answered a request with
Nginx.rputsorNginx.echowithout reading its body, for example a content or rewrite handler that answers aPOSTwithoutmruby_enable_read_request_body, the body stayed on the connection. On a connection that nginx kept open for the next request (HTTP/1.1 keep-alive), nginx then read that body as the beginning of the next request. - Now: before ngx_mruby sends the answer of such a handler, it discards a
request body that no handler has read, as nginx does for its own responses
(
return, its error pages, static files), and the next request on the connection is the one that the client sent after the body. If the body cannot be discarded, nginx closes the connection after the answer. The same holds for the answer that a rewrite handler sends afterNginx::Async.sleep. A body that a handler has read is not affected. - Why it changed:
- The defect: nginx expects a handler that answers a request to read or
discard the request body first, because the next request on a kept
connection starts after the body; nginx’s own handlers do this
(
ngx_http_send_response, the error pages, the static module), and the answer of an mruby handler did not. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect. - Why the old behavior could not be kept: a body left on the connection is read as the beginning of the next request, as “Before” says. Discarding the body keeps the connection usable without making the answer wait for a body that the handler did not want.
- What the documentation promised: the documentation said nothing about a request body that no handler reads.
- The defect: nginx expects a handler that answers a request to read or
discard the request body first, because the next request on a kept
connection starts after the body; nginx’s own handlers do this
(
- Affected: handlers that answer requests with a body (
POST,PUT) withNginx.rputsorNginx.echowithout reading the body, on connections that nginx keeps open for the next request. - What to do: nothing. A handler that needs the body reads it as before,
with
mruby_enable_read_request_body on. - Tests: port 18151 (
test/t/cases/unread_request_body.rb: a content handler, a rewrite handler, and a rewrite handler that answers afterNginx::Async.sleep).
13. A content handler that ends without output answers 500
Section titled “13. A content handler that ends without output answers 500”- Before: an
mruby_content_handler*on the main request that ended without output sent no response when it set no status (it wrote nothing, wrote only""ornilwithNginx.rputs, or only set response headers) or set a status below 300 other than 200, 201 and 204, for exampleNginx::HTTP_ACCEPTED, also when it ended so afterNginx::Async::HTTP.sub_requesthad resumed it. nginx closed an HTTP/1.0 connection without sending a byte and kept an HTTP/1.1 keep-alive connection open without a response untilkeepalive_timeout(75 s by default), so the client waited for its own timeout. - Now: such a request gets 500 with nginx’s error page (or the
error_pageconfigured for 500) andConnection: close, the answer thatNginx.return Nginx::HTTP_OKwithout output already got in 2.7.0 (#566). error.log gets a line at theerrorlevel:status code is not set, and response body is empty. return NGX_HTTP_INTERNAL_SERVER_ERROR, orstatus code is <N>, but response body is empty. return NGX_HTTP_INTERNAL_SERVER_ERROR. - Why it changed:
- The defect: nginx sends a response of its own for 201, 204 and the statuses from 300 up, but for any other status below 300, and when no status is set, it ends the request without sending anything; a handler without output handed exactly such a value to nginx. A request without a response is a defect (#225): the client hangs until its own timeout, and a keep-alive connection is held for nothing.
- Why the old behavior could not be kept: keeping it would leave these
requests without a response. The 2.x line answers these requests the way
it already answered
Nginx::HTTP_OKwithout output, so no request that got a response in 2.7.0 gets a different one. - What the documentation promised: the documentation said nothing about a
content handler that ends without output. It says that
Nginx.return“Returns nginx status code to send back HTTP status to the client” (Nginx.return of 2.7.0); a status such as 202 without output gets 500 in 2.7.1, as 200 without output already did in 2.7.0.
- Affected:
mruby_content_handler*handlers that can end without output, for example a handler that writes only when a condition holds, or one that passes a value that can be empty orniltoNginx.rputs, and log monitoring of these lines: a pattern that matchesstatus code is 200, but response body is emptydoes not match the new lines, and one that matchesresponse body is emptycounts them too. - What to do: a handler that should answer with an empty body returns
Nginx.return Nginx::HTTP_NO_CONTENT(nginx answers 204 itself) or writes a body; a handler that means to deny returns the status it means (Nginx::HTTP_FORBIDDEN,Nginx::HTTP_UNAUTHORIZED). Log monitoring that matchesstatus code is 200, but response body is emptyshould matchresponse body is empty. return NGX_HTTP_INTERNAL_SERVER_ERRORto see the new lines too. - Not changed by this release:
Nginx::HTTP_OKwithout output (500, as before), 201, 204 and the statuses from 300 up (nginx answers them itself), handlers with output,Nginx.redirect, the other kinds of handler, and subrequests:Nginx::Async::HTTP.sub_request, SSI includes andauth_requestsee the same subrequest status as before. A content handler that writes output and then sets a status below 300 other than 200, 201 and 204, for exampleNginx.rputs "x"followed byNginx.return Nginx::HTTP_ACCEPTED, still sends no response (see “Known limitations”). - Tests: port 18136 (
test/conf/conf.d/54-content-empty-body.conf,test/t/cases/content_empty_body.rb).
14. The waiting handler no longer runs after nginx ended its body read
Section titled “14. The waiting handler no longer runs after nginx ended its body read”With mruby_enable_read_request_body on, a post-read, server rewrite, rewrite,
access or content handler waits for the rest of the request body (entry 6).
nginx can end that read without the rest of the body: when
client_body_timeout expires, when the client closes the connection, or when
nginx answers 400 or 413 to the body, for example to a malformed chunked body.
This entry concerns the mruby log handler and the mruby output filters that run
for such a request afterwards.
- Before: when the handler that waited was a post-read or access handler,
such a log handler or filter started a body read of its own, and that read
ran the code of the waiting handler, which had not run yet, after nginx had
already decided the answer. After
client_body_timeout, or after the client had closed the connection, the code of an access handler ran during the log phase. After a malformed chunked body it ran while nginx sent its 400 page, and the response had that page twice under aContent-Lengththat counted it once (157 bytes counted, 314 sent). When the handler that waited was a server rewrite, rewrite or content handler, 2.7.0 did not end the request at all (entry 6), so these handlers did not run. - Now: the log handler or filter does not start a read: it runs at once,
Nginx::Request#bodyreturns the part of the body that nginx had read before it ended the read, ornilwhen that part did not fit in nginx’s body buffer (client_body_buffer_size) and nginx had written it to a temporary file, and error.log gets theinfoline of entry 6 (the request body is not complete and a log handler cannot wait for it, so it runs without the body, ora header filter,a body filter). The code of the waiting handler does not run. Afterclient_body_timeoutnginx closes the connection without a response and the access log records 408; after the client has closed the connection the access log records 400; after a malformed chunked body the client gets nginx’s 400 page once, and nginx closes the connection. - Why it changed:
- The defect: when a body read of the request has already started, nginx calls the completion handler of a new read at once, and the completion handler of ngx_mruby runs the phases of the request again for the handler that waits. A log handler or a filter that started a read after nginx had ended the first one therefore ran those phases again, from the log phase or from inside the response. The security advisory GHSA-h7fw-wf49-8rv5 describes this defect.
- Why the old behavior could not be kept: a read that nginx has ended does not complete, so the waiting handler has no body to run with, and nginx has already answered the request or closed it: running that handler’s code then did its work, such as an access check or a log line, for a request that was over, and sent nginx’s error page a second time. A log handler or a filter cannot wait for a body (entry 6), so it runs with what nginx had read, without starting a read of its own.
- What the documentation promised: the 2.7.0 documentation of the directive
said that ngx_mruby “reads a HTTP request body before calling mruby code”
(mruby_
enable_ ); it said nothing about a read that nginx ends without the whole body.read_ request_ body
- Affected: configurations with
mruby_enable_read_request_body onwhose post-read or access handler reads the body and whose requests also run an mruby log handler or an mruby output filter, for the requests whose body nginx stops reading early (client_body_timeout, a closed connection, 400 or 413 for the body). - What to do: nothing. The code of a handler that waits for the body runs once, with the whole body; when nginx ends the read before the body is complete, it does not run.
- Tests: port 18152 (
test/t/cases/request_body_later.rb: a rewrite handler that waits, with a log handler afterclient_body_timeout, and with a header filter and with a body filter after a malformed chunked body).
New features
Section titled “New features”None.
Fixes described in the security advisory GHSA-h7fw-wf49-8rv5:
- The body filter sends exactly its output, also when the body was written with
Nginx.rputsorNginx.echo, and when the location also has an mruby header filter (entry 2). Nginx::Filter#bodyin a header filter no longer returns bytes that are not the response body, andNginx::Filter#body=and#outputoutside a body filter no longer crash the worker when a body filter follows (entry 2).- An access or post-read handler that raises after
Nginx.rputsstops the request with 500 instead of letting nginx go on with the content handler, and after an exception in anmruby_set*handler or a header filter,Content-Lengthno longer counts the dropped output (entry 3). - Under
mruby_enable_read_request_body, a server rewrite, rewrite or content handler whose request body arrives after it started runs with the body, a later phase handler such asauth_requestis resumed after a body that arrived late, and nginx ends the request when the client closes the connection orclient_body_timeoutexpires, instead of keeping it open (entry 6, first part). Nginx::SSLraisesRuntimeErroroutside the handshake handler instead of crashing the worker or returning meaningless values (entry 7, issue #339).- A later handshake whose code does not call
Nginx::SSL.newno longer gets an earlier handshake’s certificate (entry 7). - Request-scoped methods and stream session methods raise
RuntimeErroroutside a request or a session instead of crashing or using the data of another request,Nginx::Server#add_listenerraises outsidemruby_server_context_handler_code, andNginx.logwrites to nginx’s error log outside a request (entry 8). Nginx::Stream.add_listenerraises outsidemruby_stream_server_context_codeinstead of using the internal context of the stream module as a server configuration (entry 8).- An
Nginx.rputsanswer no longer leaves an unread request body on the connection (entry 12). - A log handler or an output filter that runs after nginx ended a body read without the body no longer runs the code of the handler that waited for the body, and nginx sends its error page for the body once (entry 14).
Other fixes:
Nginx::Asyncraises in log handlers and output filters instead of leaving a timer or a suspended handler behind (#551, entry 4).- An exception after
Nginx::Asyncresumed a handler answers 500, no longer makes the next request on the worker fail, and requests that nginx posted meanwhile run (#550, entry 5). - Under
mruby_enable_read_request_body,mruby_set*, log handlers and filters no longer leave a body read behind (entry 6, second part). - A resumed handler no longer overwrites the variable of an earlier
mruby_set(#548, entry 9). - nginx starts with a configuration that has no
http {}block (#536, entry 10). - A content handler that ends without output answers 500 instead of leaving the client without a response (#566, fixes #225, entry 13).
- A location with an mruby handler that writes the response with
Nginx.rputsorNginx.echo, such as a content or rewrite handler, and an mruby header filter no longer crashes the worker; the header filter runs once (#563, fixes #206). - A header filter that calls
Nginx.rputsorNginx.echono longer crashes the worker (#563), but the call is still not supported. After an origin other than an mruby handler that writes withNginx.rputs, such as nginx’sreturn, the response does not contain the output, although in a location without an mruby body filterContent-Lengthcounts it and a log handler of the location sends it after the response; after an mruby content handler that writes withNginx.rputs, the output is added to that handler’s output and counted inContent-Length. A header filter that raises afterwards drops only its own output, and the output of an mruby content handler stays in the response with itsContent-Length(entry 3). - The
Nginx::SSLcertificate setters copy a value that contains a NUL byte completely. - A failing
Nginx::Async::HTTP.sub_requestno longer keeps its handler’s fiber registered with the garbage collector until the worker exits (#547). - A stream directive that a later directive of the same kind replaces no longer leaks the mruby compiler context of the replaced code at configuration time (#540).
- Setting the
Serverheader from Ruby allocates a zeroed header element (#530).
Known limitations
Section titled “Known limitations”This release does not change the following behavior of 2.7.0.
Nginx::Async.sleepin a content handler ends the request without a response.mruby_content_handler*can callNginx::Async.sleepwithout an exception (entry 4), but nginx does not wait for a content handler that sleeps: the request ends at the sleep, the client gets no response, and the code after the sleep does not run. The documentation lists the set, rewrite and access handlers as the places whereNginx::Asyncworks. Sleep in a rewrite or access handler instead; a rewrite handler can also write the response after the sleep. A later release is planned to let a content handler wait.- An access or post-read handler that writes output lets the request go
on. An access or post-read handler that writes with
Nginx.rputsorNginx.echoand ends without an exception and without an error status sends that output as the response when it ends, and nginx goes on with the later phases: the location’s content handler or nginx’s static module runs as well, and nginx logsheader already sent. What a content handler writes follows the response beyond itsContent-Length. Do not write output in these handlers: to deny a request, return an error status such asNginx.return Nginx::HTTP_FORBIDDEN, and write answers in the content handler. The 3.x line plans to redesign how a handler of each phase ends the response. - Output followed by a status below 300 other than 200, 201 and 204. A
content handler that writes output with
Nginx.rputsorNginx.echoand then sets such a status, for example withNginx.return Nginx::HTTP_ACCEPTED, sends no response: nginx closes an HTTP/1.0 connection without sending a byte, and keeps an HTTP/1.1 keep-alive connection open without a response. Answer withNginx::HTTP_OKwhen the response has a body, and withNginx::HTTP_NO_CONTENTwhen it has none. The 3.x line plans to allow a body with any status.
Build and test changes
Section titled “Build and test changes”- Default nginx:
nginx_version, whichbuild.shandtest.shuse, is 1.31.6 (2.7.0 had 1.27.2). - CI tests nginx 1.31.6, 1.30.5, 1.28.3 and 1.26.3, each as a static and as a
dynamic module, with OpenSSL 1.1.1w installed on the runner as a shared
library and with OpenSSL 1.1.1w compiled into nginx, plus one build of 1.31.6
with AddressSanitizer and UndefinedBehaviorSanitizer. It runs on pull
requests and reports one
ci-okcheck. 2.7.0 tested 1.27.2 and 1.26.2. - CI:
ci-ok, the status check that the rulesets require, now also needs the new jobfilesto succeed. Thefilesjob fails when a pull request adds a file that does not belong in the repository (#578). Its script,test/check_added_files.sh, checks whether a file that the change adds, or that one of its commits adds, matches a pattern oftest/forbidden_paths.txt. The patterns include the contents of.claude/except.claude/settings.json,.claude/commands/and.claude/skills/, build directories at the top such asbuild/andbuild_dynamic/,mruby/build/, files that./configureor the Makefile writes at the top,.logfiles,.envand.env.*, private keys, editor and OS leftovers, and core dumps. An empty file that the change adds also fails, unless it is named.gitkeepor.keep. On a push tomaster,nextorv*.x, the job checks the pushed commits. It needs the whole history of the repository;docs/test/README.mddescribes the check and how to run it before opening a pull request. build_config.rbno longer buildsmruby-redis(#528, entry 1).build.sh: withOPENSSL_SRC_VERSIONset, it passes the OpenSSL source directory that it downloads toconfigureas an absolute path, so OpenSSL’s./configgets the absolute--prefixthat it requires (2.7.0 stopped withDirectory given with --prefix MUST be absolute) (#562, fixes #519). A--with-openssl-srcgiven on the command line ofbuild.shortest.shis still passed toconfigureas it is.test.sh: the HTTP test servers listen on 18080-18090, 18101-18103, 18110-18131, 18136, 18138, 18143-18144 and 18147-18152 instead of 58xxx, below the ephemeral port range; the stream servers use 12345-12359, and one test uses 12399. It also loads the configuration fragments intest/conf/conf.dand runs the case files intest/t/cases(see the test documentation), takes extra compiler and linker flags for nginx fromNGINX_EXTRA_CC_OPTandNGINX_EXTRA_LD_OPT, and compares the nginx version as a whole, so its builds of nginx 1.11.5 and later include the stream access module whatever the patch number. It configures nginx with--with-http_auth_request_module, for the tests of entry 6; a build directory made by an earliertest.shkeeps its old configure options until it is configured again, as the test documentation describes.- The repository has a security policy (
SECURITY.md), contributor and agent instructions (AGENTS.md), and av2.xbranch that mirrorsmasteruntil v3 is promoted tomaster.
Performance
Section titled “Performance”Measured on 2026-10-14 with the callgrind comparison of the next branch
(test/perf/compare.sh at 570cdab05, see “Performance comparison with
callgrind” in its docs/test/README.md): release builds of both trees with
nginx 1.31.6 and the same 21 third-party mrbgems at the same commits, Ubuntu
22.04 in a container on aarch64 (gcc 11.4, valgrind 3.18.1), 20000 requests per
scenario after 2000 warmup requests. Base: master at b3df70f, the 2.x
commit that the fixes of 2.7.1 were merged with (tree 4e88dcc47eca); head:
b3df70f with the fixes of 2.7.1 (tree 3ab6c7888a35, src/ tree
8b6857bd850e). Instructions that nginx executes per request, without mruby’s
garbage collector:
| Scenario | b3df70f |
2.7.1 | Change |
|---|---|---|---|
hello |
11159 | 11297 | +1.23% |
headers |
34154 | 34286 | +0.39% |
var |
19837 | 19976 | +0.70% |
filter |
13229 | 13308 | +0.60% |
sleep |
14719 | 14852 | +0.90% |
sub_request |
47443 | 47617 | +0.37% |
file |
13196 | 13335 | +1.05% |
proxy_plain_2k |
21952 | 21992 | +0.18% |
proxy_plain_64k |
22262 | 22301 | +0.18% |
proxy_stream_plain_50 |
54301 | 54348 | +0.09% |
auth |
46237 | 46299 | +0.13% |
route_json_2k |
106148 | 106224 | +0.07% |
ruby_call_1 |
10736 | 10799 | +0.59% |
ruby_call_10 |
35215 | 35557 | +0.97% |
Every scenario stays below the 3% at which the comparison warns. The largest
relative change is in hello, +1.23% or about 138 instructions per request;
the largest absolute change is in ruby_call_10, 342 instructions per request
for its ten mruby_set_code calls, about 34 per call. The fixes add no
per-request allocation of their own and no blocking wait: a handler that needs
the rest of the request body waits with nginx’s NGX_DONE and runs again when
nginx has read the body, the fixes for an exception after Nginx.rputs record
the chain of Nginx.rputs, its last link and the response length before a
handler runs, and the length again when Nginx.rputs starts a chain, and put
the length back after an exception (for a header filter also the chain), and
the only new call that can allocate is nginx’s own
ngx_http_discard_request_body, for a chunked request body that no handler
read.
Upgrading
Section titled “Upgrading”Supported nginx versions: the nginx mainline and stable lines (see README.md); CI builds the versions listed under “Build and test changes”.
- No directive was added, renamed or removed. Rebuild the module with this
release: build nginx with ngx_mruby 2.7.1 the way you built 2.7.0, and
replace the binary or the dynamic module as usual.
Nginx.module_versionreturns2.7.1. - mruby is unchanged at 3.3.0, and so is ngx_
devel_kit. - The third-party mrbgems in
build_config.rbare still fetched from their repositories without a pinned version on 2.x, so a new build can get newer versions of them than your 2.7.0 build had. Redis needs the change in entry 1.
Check these before upgrading. Each line is the “What to do” of its entry above; the entries that are not listed need no change.
- Redis: uncomment
conf.gem github: 'matsumotory/mruby-redis'inbuild_config.rbbefore you build (entry 1). - Output filters: remove
Nginx.rputsandNginx.echofrom body filters, read the response length in a header filter withNginx::Var.new.sent_http_content_length, and set or read the body only inmruby_output_body_filter*(entry 2). raiseafter output: keepraisefor errors, end a content handler early withNginx.return Nginx::HTTP_OKandreturn, and do not write output in an access or post-read handler (entry 3).Nginx::AsyncorFiber.yieldin a log handler or an output filter: wait in a rewrite or access handler (ormruby_set_code) and hand the result over in an nginx variable (entry 4).- A handler that should answer something other than 500 after an exception
that follows
Nginx::Async: rescue the exception in the handler and set the status there (entry 5). mruby_set*, log handlers and output filters that need the request body: read it first in a handler that can wait, for example a server-levelmruby_post_read_handler_code(entry 6).Nginx::SSL: call it only in the handshake handler. A handler that sets a certificate on some handshakes without callingNginx::SSL.newon the others gives those othersssl_certificatenow (entry 7).- Request methods in code that runs outside a request (the list in entry 8):
use request data in request handlers only, call
add_listeneronly in the server context code, and do not call session methods inmruby_stream_server_context_code(entry 8). - Log statistics or alerts that count on the status 200 or on the
header already sentline of a body filter that callsNginx.rputsafter an mruby header filter: update them; the log phase now sees the status that nginx sent, and nginx no longer logs that line (entry 11). - A content handler that can end without output: return
Nginx::HTTP_NO_CONTENTor write a body, and update log monitoring that matchesstatus code is 200, but response body is empty(entry 13).