diff options
Diffstat (limited to 'CHANGES')
| -rw-r--r-- | CHANGES | 9864 |
1 files changed, 5093 insertions, 4771 deletions
@@ -6,7125 +6,7447 @@ Changelog -Version 7.68.0 (8 Jan 2020) +Version 7.73.0 (14 Oct 2020) -Daniel Stenberg (8 Jan 2020) -- RELEASE-NOTES: 7.68.0 - -- THANKS: updated with names from the 7.68.0 release - -- RELEASE-PROCEDURE: add four future release dates - - and remove four past release dates +Daniel Stenberg (14 Oct 2020) +- RELEASE-NOTES: synced - [skip ci] + for 7.73.0 -Marcel Raad (6 Jan 2020) -- TrackMemory tests: always remove CR before LF - - It was removed for output containing ' =' via `s/ =.*//`. With classic - MinGW, this made lines with `free()` end with CRLF, but lines with e.g. - `malloc()` end with only LF. The tests expect LF only. - - Closes https://github.com/curl/curl/pull/4788 +- THANKS: from 7.73.0 and .mailmap fixes -Daniel Stenberg (6 Jan 2020) -- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - - ... to the private multihhandle.h. It is not for public use and it - wasn't prefixed correctly anyway! - - Closes #4790 +- mailmap: fixups of some contributors -- file: fix copyright year range - - Follow-up to 1b71bc532bd +- projects/build-wolfssl.bat: fix the copyright year range -- curl -w: handle a blank input file correctly +Marc Hoersken (14 Oct 2020) +- [Sergei Nikulov brought this change] + + CI/tests: fix invocation of tests for CMake builds - Previously it would end up with an uninitialized memory buffer that - would lead to a crash or junk getting output. + Update appveyor.yml to set env variable TFLAGS and run tests + Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) + Move testdeps build to build step (per review comments) - Added test 1271 to verify. + Reviewed-by: Marc Hörsken - Reported-by: Brian Carpenter - Closes #4786 + Closes #6066 + Fixes #6052 -- file: on Windows, refuse paths that start with \\ +- tests/server/util.c: fix support for Windows Unicode builds - ... as that might cause an unexpected SMB connection to a given host - name. - - Reported-by: Fernando Muñoz - CVE-2019-15601 - Bug: https://curl.haxx.se/docs/CVE-2019-15601.html + Detected via #6066 + Closes #6070 -Jay Satiro (6 Jan 2020) -- CURLOPT_READFUNCTION.3: fix fopen params in example +Daniel Stenberg (13 Oct 2020) +- [Jay Satiro brought this change] -- CURLOPT_READFUNCTION.3: fix variable name in example + strerror: Revert to local codepage for Windows error string - Reported-by: Paul Joyce + - Change get_winapi_error() to return the error string in the local + codepage instead of UTF-8 encoding. - Fixes https://github.com/curl/curl/issues/4787 - -Daniel Stenberg (5 Jan 2020) -- curl:getparameter return error for --http3 if libcurl doesn't support + Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it + also changed the error string's encoding from local codepage to UTF-8. - Closes #4785 - -- docs: mention CURL_MAX_INPUT_LENGTH restrictions + We return the local codepage version of the error string because if it + is output to the user's terminal it will likely be with functions which + expect the local codepage (eg fprintf, failf, infof). - ... for curl_easy_setopt() and curl_url_set(). + This is essentially a partial revert of bed5f84. The support for xbox + remains but the error string is reverted back to local codepage. - [skip ci] + Ref: https://github.com/curl/curl/pull/6005 - Closes #4783 + Reviewed-by: Marcel Raad + Closes #6065 -- curl: properly free mimepost data +Marc Hoersken (13 Oct 2020) +- CI/tests: use verification curl for test reporting APIs - ... as it could otherwise leak memory when a transfer failed. + Avoid using our own, potentially installed, curl for + the test reporting APIs in case it is broken. - Added test 1293 to verify. + Reviewed-by: Daniel Stenberg - Reported-by: Brian Carpenter - Fixes #4781 - Closes #4782 + Preparation for #6049 + Closes #6063 -- curl: cleanup multi handle on failure +Viktor Szakats (12 Oct 2020) +- windows: fix comparison of mismatched types warning - ... to fix memory leak in error path. + clang 10, mingw-w64: + ``` + vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') + [-Wsign-compare] + if(GetLastError() != CRYPT_E_NOT_FOUND) + ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ + ``` - Fixes #4772 - Closes #4780 - Reported-by: Brian Carpenter + Approved-by: Daniel Stenberg + Closes #6062 -Marcel Raad (3 Jan 2020) -- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` - - Closes https://github.com/curl/curl/pull/4775 +Daniel Stenberg (11 Oct 2020) +- [Viktor Szakats brought this change] -Daniel Stenberg (3 Jan 2020) -- COPYING: it's 2020! + src/Makefile.m32: fix undefined curlx_dyn_* errors + + by linking `lib/dynbuf.c` when building a static curl binary. + Previously this source file was only included when building + a dynamic curl binary. This was likely possibly because no + functions from the `src/Makefile.inc` / `CURLX_CFILES` sources + were actually required for a curl tool build. This has + recently changed with the introduction of `curlx_dyn_*()` + memory functions and their use by the tool sources. - [skip ci] + Closes #6060 -Jay Satiro (3 Jan 2020) -- [Marc Aldorasi brought this change] +- HISTORY: curl verifies SSL certs by default since version 7.10 - tests: Fix bounce requests with truncated writes +Marc Hoersken (8 Oct 2020) +- runtests.pl: use $LIBDIR variable instead of hardcoded path - Prior to this change the swsbounce check in service_connection could - fail because prevtestno and prevpartno were not set, which would cause - the wrong response data to be sent to some tests and cause them to fail. - - Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + Reviewed-by: Daniel Stenberg + Closes #6051 -Marcel Raad (31 Dec 2019) -- tool: make a few char pointers point to const char instead +Daniel Stenberg (7 Oct 2020) +- checksrc: detect // comments on column 0 - These are read-only. + Spotted while working on #6045 - Closes https://github.com/curl/curl/pull/4771 + Closes #6048 + +- [Frederik Wedel-Heinen brought this change] -Jay Satiro (31 Dec 2019) -- tests: Change NTLM tests to require SSL + mbedtls: add missing header when defining MBEDTLS_DEBUG - Prior to this change tests that required NTLM feature did not require - SSL feature. + Closes #6045 + +- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - There are pending changes to cmake builds that will allow enabling NTLM - in non-SSL builds in Windows. In that case the NTLM auth strings created - are different from what is expected by the NTLM tests and they fail: + Previously, it would pass on a define (int) which could make libcurl + read junk as a value - which prevented the CURLOPT_IPRESOLVE option to + "take". This could then make test 2100 do two DoH requests instead of + one! - "The issue with NTLM is that previous non-SSL builds would not enable - NTLM and so the NTLM tests would be skipped." + Fixes #6042 + Closes #6043 + +- RELEASE-NOTES: synced + +- scripts/release-notes.pl: don't "embed" $ in format string for printf() - Assisted-by: marc-groundctl@users.noreply.github.com + ... since they might contain %-codes that mess up the output! + +Jay Satiro (5 Oct 2020) +- [M.R.T brought this change] + + build-wolfssl: fix build with Visual Studio 2019 + + Closes https://github.com/curl/curl/pull/6033 + +Daniel Stenberg (4 Oct 2020) +- runtests: add %repeat[]% for test files - Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + ... and use this new keywords in all the test files larger than 50K to reduce + their sizes and make them a lot easier to read and understand. - Closes https://github.com/curl/curl/pull/4768 + Closes #6040 -- [Michael Forney brought this change] +- [Emil Engler brought this change] - bearssl: Improve I/O handling - - Factor out common I/O loop as bearssl_run_until, which reads/writes TLS - records until the desired engine state is reached. This is now used for - the handshake, read, write, and close. + --help: move two options from the misc category - Match OpenSSL SSL_write behavior, and don't return the number of bytes - written until the corresponding records have been completely flushed - across the socket. This involves keeping track of the length of data - buffered into the TLS engine, and assumes that when CURLE_AGAIN is - returned, the write function will be called again with the same data - and length arguments. This is the same requirement of SSL_write. + The cmdline opts delegation and suppress-connect-headers + fit better into auth and proxy rather than misc. - Handle TLS close notify as EOF when reading by returning 0. + Follow-up to aa8777f63febc + Closes #6038 + +- [Samanta Navarro brought this change] + + docs/opts: fix typos in two manual pages - Closes https://github.com/curl/curl/pull/4748 + Closes #6039 -- travis: Fix error detection +- ldap: reduce the amount of #ifdefs needed - - Stop using inline shell scripts for before_script and script sections. + Closes #6035 + +- runtests: provide curl's version string as %VERSION for tests - Prior to this change Travis could ignore errors from commands in inline - scripts. I don't understand how or why it happens. This is a workaround. + ... so that we can check HTTP requests for User-Agent: curl/%VERSION - Assisted-by: Simon Warta + Update 600+ test cases accordingly. - Ref: https://github.com/travis-ci/travis-ci/issues/1066 + Closes #6037 + +- checksrc: warn on space after exclamation mark - Fixes https://github.com/curl/curl/issues/3730 - Closes https://github.com/curl/curl/pull/3755 + Closes #6034 + +- test1465: verify --libcurl with binary POST data -- tool_operate: fix mem leak when failed config parse +- runtests: allow generating a binary sequence from hex + +- tool_setopt: escape binary data to hex, not octal + +- curl: make --libcurl show binary posts correctly - Found by fuzzing the config file. + Reported-by: Stephan Mühlstrasser + Fixes #6031 + Closes #6032 + +Jay Satiro (1 Oct 2020) +- strerror: fix null deref on winapi out-of-memory - Reported-by: Geeknik Labs + Follow-up to bed5f84 from several days ago. - Fixes https://github.com/curl/curl/issues/4767 + Ref: https://github.com/curl/curl/pull/6005 -- [Xiang Xiao brought this change] +Daniel Stenberg (1 Oct 2020) +- [Kamil Dudka brought this change] - lib: remove erroneous +x file permission on some c files + vtls: deduplicate some DISABLE_PROXY ifdefs - Modified by commit eb9a604 accidentally. + ... in the code of gtls, nss, and openssl - Closes https://github.com/curl/curl/pull/4756 + Closes #5735 -- [Xiang Xiao brought this change] +- RELEASE-NOTES: synced - lib: fix warnings found when porting to NuttX +- [Emil Engler brought this change] + + TODO: Add OpenBSD libtool notice - - Undefine DEBUGASSERT in curl_setup_once.h in case it was already - defined as a system macro. + See #5862 + Closes #6030 + +- tests/unit/README: convert to markdown - - Don't compile write32_le in curl_endian unless - CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. + ... and add to dist! - - Include <arpa/inet.h> in socketpair.c. + Closes #6028 + +- tests/README: convert to markdown - Closes https://github.com/curl/curl/pull/4756 + Closes #6028 -- os400: Add missing CURLE error constants +- include/README: convert to markdown - Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 - Reported-by: Emil Engler + Closes #6028 -- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 +- examples/README: convert to markdown - For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback - is passed two `size_t` parameters which, when multiplied, designate the - number of bytes of data passed in. In practice, CURL always sets the - first parameter (`size`) to 1. + Closes #6028 + +- configure: don't say HTTPS-proxy is enabled when disabled! - This practice is also enshrined in documentation and cannot be changed - in future. The documentation states that the default callback is - `fwrite`, which means `fwrite` must be a suitable function for this - purpose. However, the documentation also states that the callback must - return the number of *bytes* it successfully handled, whereas ISO C - `fwrite` returns the number of items (each of size `size`) which it - wrote. The only way these numbers can be equal is if `size` is 1. + Reported-by: Kamil Dudka + Reviewed-by: Kamil Dudka + Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 + Closes #6029 + +Daniel Gustafsson (30 Sep 2020) +- src: Consistently spell whitespace without whitespace - Since `size` is 1 and can never be changed in future anyway, document - that fact explicitly and let users rely on it. + Whitespace is spelled without a space between white and space, so + make sure to consistently spell it that way across the codebase. - Reported-by: Frank Gevaerts - Commit-message-by: Christopher Head + Closes #6023 + Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Reviewed-by: Emil Engler <me@emilengler.com> + +- MANUAL: update examples to resolve without redirects - Ref: https://github.com/curl/curl/pull/2787 + www.netscape.com is redirecting to a cookie consent form on Aol, and + cool.haxx.se isn't responding to FTP anymore. Replace with examples + that resolves in case users try out the commands when reading the + manual. - Fixes https://github.com/curl/curl/issues/4758 + Closes #6024 + Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Reviewed-by: Emil Engler <me@emilengler.com> -- examples/postinmemory.c: Call curl_global_cleanup always +Daniel Stenberg (30 Sep 2020) +- HISTORY: add some 2020 events + +- sectransp: make it build with --disable-proxy - Prior to this change curl_global_cleanup was not called if - curl_easy_init failed. + Follow-up from #5466 and f3d501dc678d80 + Reported-by: Javier Navarro + Fixes #6025 + Closes #6026 + +- ECH: renamed from ESNI in docs and configure - Reported-by: kouzhudong@users.noreply.github.com + Encrypted Client Hello (ECH) is the current name. - Fixes https://github.com/curl/curl/issues/4751 + Closes #6022 -Daniel Stenberg (21 Dec 2019) -- url2file.c: fix copyright year +- configure: use "no" instead of "disabled" for the end summary - Follow-up to 525787269599b5 - -- [Rickard Hallerbäck brought this change] + ... for consistency but also to make them more distinctly stand out next + to the "enabled" lines. - examples/url2file.c: corrected a comment +- TODO: SSH over HTTPS proxy with more backends - The comment was confusing and suggested that setting CURLOPT_NOPROGRESS - to 0L would both enable and disable debug output at the same time, like - a Schrödinger's cat of CURLOPTs. - - Closes #4745 + ... as right now only the libssh2 backend supports it. -- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 +- libssh2: handle the SSH protocols done over HTTPS proxy + + Reported-by: Robin Douine + Fixes #4295 + Closes #6021 -- RELEASE-NOTES: synced +- [Emil Engler brought this change] -Jay Satiro (20 Dec 2019) -- ngtcp2: Support the latest update key callback type + memdebug: remove 9 year old unused debug function - - Remove our cb_update_key in favor of ngtcp2's new - ngtcp2_crypto_update_key_cb which does the same thing. + There used to be a way to have memdebug fill allocated memory. 9 years + later this has no value there (valgrind and ASAN etc are way better). If + people need to know about it they can have a look at VCS logs. - Several days ago the ngtcp2_update_key callback function prototype was - changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to - fix up our cb_update_key for that change they also added - ngtcp2_crypto_update_key_cb which does the same thing so we'll use that - instead. + Closes #5973 + +- sendf: move Curl_sendf to dict.c and make it static - Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + ... as the only remaining user of that function. Also fix gopher.c to + instead use Curl_write() - Closes https://github.com/curl/curl/pull/4735 + Closes #6020 -Daniel Stenberg (19 Dec 2019) -- sws: search for "Testno:" header uncondtionally if no testno +- ROADMAP: updates and cleanups - Even if the initial request line wasn't found. With the fix to 1455, the - test number is now detected correctly. + Fix the HSTS PR - (Problem found when running tests in random order.) + Remove DoT, thread-safe init and hard-coded localhost. I feel very + little interest for these with users so I downgrade them to plain "TODO" + entries again. + +- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root + + This matches what is returned in other TLS backends in the same + situation. - Closes #4744 + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler + Follow-up to 5a3efb1 + Reported-by: iammrtau on github + Fixes #6003 + Closes #6018 -- tests: set LC_ALL in more tests +- RELEASE-NOTES: synced + +- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL - Follow-up to 23208e330ac0c21 + Added test 348 to verify. Added a 'STOR' command to the test FTP + server to enable test 348. Documented the command in FILEFORMAT.md - Closes #4743 + Reported-by: Duncan Wilcox + Fixes #6016 + Closes #6017 -- test165: set LC_ALL=en_US.UTF-8 too +- pause: only trigger a reread if the unpause sticks - On my current Debian Unstable with libidn2 2.2.0, I get an error if - LC_ALL is set to blank. Then curl errors out with: + As an unpause might itself get paused again and then triggering another + reread doesn't help. - curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 + Follow-up from e040146f22608fd9 (shipped since 7.69.1) - Closes #4738 + Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html + Patch-by: Kunal Chandarana + Fixes #5988 + Closes #6013 -- curl.h: add two defines for the "pre ISO C" case +- test163[12]: require http to be built-in to run - Without this fix, this caused a compilation failure on AIX with IBM xlc - 13.1.3 compiler. + ... as speaking over an HTTPS proxy implies http! - Reported-by: Ram Krushna Mishra - Fixes #4739 - Closes #4740 + Closes #6014 -- create_conn: prefer multiplexing to using new connections - - ... as it would previously prefer new connections rather than - multiplexing in most conditions! The (now removed) code was a leftover - from the Pipelining code that was translated wrongly into a - multiplex-only world. +- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define - Reported-by: Kunal Ekawde - Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html - Closes #4732 + Closes #6012 + +- [Javier Blazquez brought this change] -- test1456: remove the use of a fixed local port + strerror: honor Unicode API choice on Windows - Fixup the test to instead not compare the port number. It sometimes - caused problems like this: + Closes #6005 + +- imap: make imap_send use dynbuf for the send buffer management - "curl: (45) bind failed with errno 98: Address already in use" + Reuses the buffer and thereby reduces number of mallocs over a transfer. - Closes #4733 + Closes #6010 -Jay Satiro (18 Dec 2019) -- CURLOPT_QUOTE.3: fix typos - - Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man - pages would not compile because a variable name was incorrect. +- Curl_send: return error when pre_receive_plain can't malloc - Reported-by: Bylon2@users.noreply.github.com + ... will probably trigger some false DEAD CODE positives on non-windows + code analyzers for the conditional code. - Fixes https://github.com/curl/curl/issues/4736 + Closes #6011 -- [Gisle Vanem brought this change] +- ftp: separate FTPS from FTP over "HTTPS proxy" + + When using HTTPS proxy, SSL is used but not in the view of the FTP + protocol handler itself so separate the connection's use of SSL from the + FTP control connection's sue. + + Reported-by: Mingtao Yang + Fixes #5523 + Closes #6006 - strerror: Fix compiler warning "empty expression" +Dan Fandrich (23 Sep 2020) +- tests/data: Fix some mismatched XML tags in test cases - - Remove the final semi-colon in the SEC2TXT() macro definition. + This allows these test files to pass xmllint. + +Daniel Stenberg (23 Sep 2020) +- pingpong: use a dynbuf for the *_pp_sendf() function - Before: #define SEC2TXT(sec) case sec: txt = #sec; break; + ... reuses the same dynamic buffer instead of doing repeated malloc/free + cycles. - After: #define SEC2TXT(sec) case sec: txt = #sec; break + Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls + after this change in my test setup (132 => 125), curl 7.72.0 needed 140 + calls for this. - Prior to this change SEC2TXT(foo); would generate break;; which caused - the empty expression warning. + Test case 103 makes 9 less allocations now (130). Down from 149 in + 7.72.0. - Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 + Closes #6004 -Daniel Stenberg (18 Dec 2019) -- curl/parseconfig: use curl_free() to free memory allocated by libcurl +- dynbuf: add Curl_dyn_vaddf - Reported-by: bxac on github - Fixes #4730 - Closes #4731 + Closes #6004 -- curl/parseconfig: fix mem-leak +- dynbuf: make *addf() not require extra mallocs - When looping, first trying '.curlrc' and then '_curlrc', the function - would not free the first string. + ... by introducing a printf() function that appends directly into a + dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if + the buffer is already big enough it can just printf directly into it. - Closes #4731 - -- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + Since this less-malloc version requires tthe use of a library internal + printf function, we only provide this version when building libcurl and + not for the dynbuf code that is used when building the curl tool. - [skip ci] + Closes #5998 -- test1270: a basic -w redirect_url test +- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport - Closes #4728 + Closes #5403 -- HISTORY: the SMB(S) support landed in 2014 +- pingpong: remove a malloc per Curl_pp_vsendf call + + This typically makes 7-9 fewer mallocs per FTP transfer. + + Closes #5997 -- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore +- symbian: drop support - It is covered by USE_OPENSSL_ENGINE now. + The OS is deprecated. I see no traces of anyone having actually built + curl for Symbian after 2012. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 + The public headers are unmodified. - Closes #4725 + Closes #5989 -- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style +- RELEASE-NOTES: synced + +- curl_krb5.h: rename from krb5.h - ... even for macros + Follow-up from f4873ebd0be32cf - Reviewed-by: Daniel Gustafsson - Reviewed-by: Jay Satiro - Reported-by: Jay Satiro - Fixes #4683 - Closes #4722 + Turns out some older openssl installations go bananas otherwise. + Reported-by: Tom van der Woerdt + Fixes #5995 + Closes #5996 -- tests: make sure checksrc runs on header files too +- test1297: verify GOT_NOTHING with http proxy tunnel -- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" +- http_proxy: do not count proxy headers in the header bytecount - This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. + ... as that counter is subsequently used to detect if nothing was + returned from the peer. This made curl return CURLE_OK when it should + have returned CURLE_GOT_NOTHING. - Bug: #4683 + Fixes #5992 + Reported-by: Tom van der Woerdt + Closes #5994 -- KNOWN_BUGS: TLS session cache doesn't work with TFO +- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument - [skip ci] - Closes #4301 - -- KNOWN_BUGS: Connection information when using TCP Fast Open + Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the + option is, yeah, not known. Clarified this in the setopt man page too. - Also point to #4296 for more details - Closes #4296 + Closes #5993 -- KNOWN_BUGS: LDAP on Windows doesn't work +- krb5: merged security.c and krb specific FTP functions in here - Closes #4261 - -- docs: TLS SRP doesn't work with TLS 1.3 + These two files were always tightly connected and it was hard to + understand what went into which. This also allows us to make the + ftpsend() function static (moved from ftp.c). - Reported-by: sayrer on github - Closes #4262 - [skip ci] - -Dan Fandrich (16 Dec 2019) -- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. + Removed security.c + Renamed curl_sec.h to krb5.h - A few tests are now passing on FreeBSD, so no longer skip them. - [skip ci] + Closes #5987 -Daniel Stenberg (16 Dec 2019) -- azure: the macos cmake doesn't need to install cmake +- Curl_handler: add 'family' to each protocol - Error: cmake 3.15.5 is already installed - To upgrade to 3.16.1, run `brew upgrade cmake`. + Makes get_protocol_family() faster and it moves the knowledge about the + "families" to each protocol handler, where it belongs. - Closes #4723 + Closes #5986 -Jay Satiro (15 Dec 2019) -- winbuild: Document CURL_STATICLIB requirement for static libcurl +- parsedate: tune the date to epoch conversion - A static libcurl (ie winbuild mode=static) requires that the user define - CURL_STATICLIB when using it in their application. This is already - covered in the FAQ and INSTALL.md, but is a pretty important point so - now it's noted in the BUILD.WINDOWS.txt as well. + By avoiding an unnecessary error check and the temp use of the tm + struct, the time2epoch conversion function gets a little bit faster. + When repeating test 517, the updated version is perhaps 1% faster (on + one particular build on one particular architecture). - Assisted-by: Michael Vittiglio - - Closes https://github.com/curl/curl/pull/4721 - -Daniel Stenberg (15 Dec 2019) -- [Santino Keupp brought this change] + Closes #5985 - libssh2: add support for ECDSA and ed25519 knownhost keys +- cmake: remove scary warning - ... if a new enough libssh2 version is present. + Remove the text saying - Source: https://curl.haxx.se/mail/archive-2019-12/0023.html - Co-Authored-by: Daniel Stenberg - Closes #4714 - -- lib1591: free memory properly on OOM, in the trailers callback + "the curl cmake build system is poorly maintained. Be aware" - Detected by torture tests. + ... not because anything changed just now, but to encourage users to use + it and subsequently improve it. - Closes #4720 + Closes #5984 -- runtests: --repeat=[num] to repeat tests - - Closes #4715 +- docs/MQTT: remove outdated paaragraphs -- RELEASE-NOTES: synced +- docs/MQTT: not experimental anymore + + Follow-up to e37e4468688d8f -- azure: add a torture test on mac +- docs/RESOURCES: remove - Uses --shallow=25 to keep it small enough to get through in time. + This document is not maintained and rather than trying to refresh it, + let's kill it. A more up-to-date document with relevant RFCs is this + page on the curl website: https://curl.haxx.se/rfc/ - Closes #4712 + Closes #5980 -- multi: free sockhash on OOM +- docs/TheArtOfHttpScripting: convert to markdown - This would otherwise leak memory in the error path. + Makes it easier to browse on github etc. Offers (better) links. - Detected by torture test 1540. + It should be noted that this document is already mostly outdated and + "Everything curl" at https://ec.haxx.se/ is a better resource and + tutorial. - Closes #4713 + Closes #5981 -Marcel Raad (13 Dec 2019) -- tests: use DoH feature for DoH tests - - Previously, http/2 was used instead. +- BUGS: convert document to markdown - Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/4692 + Closes #5979 -- hostip: suppress compiler warning +- --help: strdup the category - With `--disable-doh --disable-threaded-resolver`, the `dns` parameter - is not used. + ... since it is converted and the original pointer is freed on Windows + unicode handling. - Closes https://github.com/curl/curl/pull/4692 + Follow-up to aa8777f63febc + Fixes #5977 + Closes #5978 + Reported-by: xwxbug on github -- tests: fix build with `CURL_DISABLE_DOH` - - Closes https://github.com/curl/curl/pull/4692 +- CHECKSRC: document two missing warnings + +- RELEASE-NOTES: synced -Daniel Stenberg (13 Dec 2019) -- azure: add a torture test +- ftp: avoid risk of reading uninitialized integers - Skipping all FTP tests for speed reasons. + If the received PASV response doesn't match the expected pattern, we + could end up reading uninitialized integers for IP address and port + number. - Closes #4697 + Issue pointed out by muse.dev + Closes #5972 -- azure: make the default build use --enable-debug --enable-werror +- [Quentin Balland brought this change] -- ntlm_wb: fix double-free in OOM + easy_reset: clear retry counter - Detected by torture testing test 1310 - - Closes #4710 + Closes #5975 + Fixes #5974 -Dan Fandrich (13 Dec 2019) -- cirrus: Drop the FreeBSD 10.4 build +- ftp: get rid of the PPSENDF macro - Upstream support for 10.4 ended a year ago, and it looks like the image - is now gone, too. - [skip ci] - -Daniel Stenberg (13 Dec 2019) -- unit1620: fix bad free in OOM + The use of such a macro hides some of what's actually going on to the + reader and is generally disapproved of in the project. - Closes #4709 + Closes #5971 -- unit1609: fix mem-leak in OOM +- man pages: switch to https://example.com URLs - Closes #4709 + Since HTTPS is "the new normal", this update changes a lot of man page + examples to use https://example.com instead of the previous "http://..." + + Closes #5969 -- unit1607: fix mem-leak in OOM +- github: remove the duplicate "Security vulnerability" entry + + ... since github adds an entry automatically by itself. - Closes #4709 + Closes #5970 + +- [Emil Engler brought this change] -- lib1559: fix mem-leak in OOM + github: use new issue template feature + + This helps us to avoid getting feature requests as well as security + bugs reported into the issue tracker. - Closes #4709 + Closes #5936 -- lib1557: fix mem-leak in OOM +- [Emil Engler brought this change] + + urlapi: use more Curl_safefree - Closes #4709 + Closes #5968 -- altsvc: make the save function ignore NULL filenames +Marc Hoersken (17 Sep 2020) +- multi: align WinSock mask variables in Curl_multi_wait + + Also skip pre-checking sockets to set timeout_ms to 0 + after the first socket has been detected to be ready. - It might happen in OOM situations. Detected bv torture tests. + Reviewed-by: rcombs on github + Reviewed-by: Daniel Stenberg - Closes #4707 + Follow up to #5886 -- curl: fix memory leak in OOM in etags logic +- multi: reuse WinSock events variable in Curl_multi_wait - Detected by torture tests + Since the struct is quite large (1 long and 10 ints) we + declare it once at the beginning of the function instead + of multiple times inside loops to avoid stack movements. + + Reviewed-by: Viktor Szakats + Reviewed-by: Daniel Stenberg - Closes #4706 + Closes #5886 -- doh: make it behave when built without proxy support +Daniel Stenberg (16 Sep 2020) +- TODO: dynamically decide to use socketpair - Reported-by: Marcel Raad - Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 + Suggested-by: Anders Bakken - Closes #4704 + Closes #4829 -- curl: improved cleanup in upload error path +- TODO: add PR reference for native IDN support on macOS - Memory leak found by torture test 58 + As there was work started on this that never got completed. - Closes #4705 + Closes #5371 -- mailmap: fix Andrew Ishchuk +- tool_help.h: update copyright year range + + Follow-up from aa8777f63febca -- travis: make torture use --shallow=40 +- CI/azure: disable test 571 in the msys2 builds + + It's just too flaky there - As a first step to enable it to run over a more diverse set of tests in - a reasonable time. + Reviewed-by: Marc Hoersken + Closes #5954 -- runtests: introduce --shallow to reduce huge torture tests +- tool_writeout: protect fputs() from NULL - When set, shallow mode limits runtests -t to make no more than NUM fails - per test case. If more are found, it will randomly discard entries until - the number is right. The random seed can also be set. + When the code was changed to do fputs() instead of fprintf() it got + sensitive for NULL pointers; add checks for that. - This is particularly useful when running MANY tests as then most torture - failures will already fail the same functions over and over and make the - total operation painfully tedious. + Follow-up from 0c1e767e83ec66 - Closes #4699 + Closes #5963 -- conncache: CONNECT_ONLY connections assumed always in-use +- test3015: verify stdout "as text" - This makes them never to be considered "the oldest" to be discarded when - reaching the connection cache limit. The reasoning here is that - CONNECT_ONLY is primarily used in combination with using the - connection's socket post connect and since that is used outside of - curl's knowledge we must assume that it is in use until explicitly - closed. + Follow-up from 0c1e767e83e to please win32 tests - Reported-by: Pavel Pavlov - Reported-by: Pavel Löbl - Fixes #4426 - Fixes #4369 - Closes #4696 - -- [Gisle Vanem brought this change] + Closes #5962 - vtls: make BearSSL possible to set with CURL_SSL_BACKEND +- travis: use libressl v3.1.4 instead of master - Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 + ... as their git master seems too fragile to use (and 3.2.1 which is the + latest has a build failure). - Closes #4698 + Closes #5964 -- RELEASE-NOTES: synced +- tests/FILEFORMAT: document type=shell for <command> -- travis: remove "coverage", make it "torture" +- tests/FILEFORMAT: document nonewline support for <file> - The coveralls service and test coverage numbers are just too unreliable. - Removed badge from README.md as well. + The one in <client>, that creates files. - Fixes #4694 - Closes #4695 + Follow-up from b83947c8df7 + +- [anio brought this change] -- azure: add libssh2 and cmake macos builds + tool_writeout: add new writeout variable, %{num_headers} - Removed the macos libssh2 build from travis + This variable gives the number of headers. - Closes #4686 + Closes #5947 -- curl: use errorf() better - - Change series of error outputs to use errorf(). +- tool_urlglob: fix compiler warning "unreachable code" - Only errors that are due to mistakes in command line option usage should - use helpf(), other types of errors in the tool should rather use - errorf(). + (On Windows builds.) - Closes #4691 + Follow-up to 70a3b003d9 -Jay Satiro (9 Dec 2019) -- [Marc Hoersken brought this change] +- [Gergely Nagy brought this change] - tests: make it possible to set executable extensions + vtls: deduplicate client certificates in ssl_config_data - This enables the use of Windows Subsystem for Linux (WSL) to run the - testsuite against Windows binaries while using Linux servers. + Closes #5629 + +- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND + + This is primarily interesting for cases where CURLOPT_NOBODY is set as + previously curl would not return an error for this case. - This commit introduces the following environment variables: - - CURL_TEST_EXE_EXT: set the executable extension for all components - - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only + MDTM getting 550 now also returns this error (it returned + CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for + missing files across protocols and specific FTP commands. - Later testcurl.pl could be adjusted to make use of those variables. - - CURL_TEST_EXE_EXT_SRV: set it for the test servers only + libcurl already returns error on a 550 as a MDTM response (when + CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would + happen subsequently anyway since the RETR command would fail. - (This is one of several commits to support use of WSL for the tests.) + Add test 1913 and 1914 to verify. Updated several tests accordingly due + to the updated SIZE behavior. - Closes https://github.com/curl/curl/pull/3899 + Reported-by: Tomas Berger + Fixes #5953 + Closes #5957 -- [Marc Hoersken brought this change] +- curl: make checkpasswd use dynbuf + + Closes #5952 - tests: fix permissions of ssh keys in WSL +- curl: make glob_match_url use dynbuf - Keys created on Windows Subsystem for Linux (WSL) require it for some - reason. + Closes #5952 + +- curl: make file2memory use dynbuf - (This is one of several commits to support use of WSL for the tests.) + Closes #5952 + +- curl: make file2string use dynbuf - Ref: https://github.com/curl/curl/pull/3899 + Closes #5952 -- [Marc Hoersken brought this change] +- [Antarpreet Singh brought this change] - tests: use \r\n for log messages in WSL + imap: set cselect_bits to CURL_CSELECT_IN initially - Bash in Windows Subsystem for Linux (WSL) requires it for some reason. + ... when continuing a transfer from a FETCH response. - (This is one of several commits to support use of WSL for the tests.) + When the size of the file was small enough that the entirety of the + transfer happens in a single go and schannel buffers holds the entire + data. However, it wasn't completely read in Curl_pp_readresp since a + line break was found before that could happen. So, by the time we are in + imap_state_fetch_resp - there's data in buffers that needs to be read + via Curl_read but nothing to read from the socket. After we setup a + transfer (Curl_setup_transfer), curl just waits on the socket state to + change - which doesn't happen since no new data ever comes. - Ref: https://github.com/curl/curl/pull/3899 + Closes #5961 -- [Andrew Ishchuk brought this change] +- RELEASE-NOTES: synced - winbuild: Define CARES_STATICLIB when WITH_CARES=static - - When libcurl is built with MODE=static, c-ares is forced into static - linkage too. That doesn't happen when MODE=dll so linker would break - over undefined symbols. +- test434: test -K use in a single line without newline - closes https://github.com/curl/curl/pull/4688 + Closes #5946 -Daniel Stenberg (9 Dec 2019) -- conn: always set bits.close with connclose() +- runtests: allow creating files without newlines - Closes #4690 + Closes #5946 -- cirrus: enable clang sanitizers on freebsd 13 - -- conncache: fix multi-thread use of shared connection cache +- curl: use curlx_dynbuf for realloc when loading config files - It could accidentally let the connection get used by more than one - thread, leading to double-free and more. + ... fixes an integer overflow at the same time. - Reported-by: Christopher Reid - Fixes #4544 - Closes #4557 - -- azure: add a vanilla macos build + Reported-by: ihsinme on github + Assisted-by: Jay Satiro - Closes #4685 + Closes #5946 -- curl: make the etag load logic work without fseek - - The fseek()s were unnecessary and caused Coverity warning CID 1456554 +- dynbuf: provide curlx_ names for reuse by the curl tool - Closes #4681 - -- mailmap: Mohammad Hasbini - -- [Mohammad Hasbini brought this change] + Closes #5946 - docs: fix some typos +- dynbuf: make sure Curl_dyn_tail() zero terminates - Closes #4680 + Closes #5959 -- RELEASE-NOTES: synced +- tests: add test1912 to the dist + + Follow-up to 70984ce1be4cab6c -Jay Satiro (5 Dec 2019) -- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN +- docs/LICENSE-MIXING: remove - Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS - and OS400 package spec. + This document is not maintained and I feel that it doesn't provide much + value to users anymore (if it ever did). - Also I added the option to the NameValue list in the tool even though it - isn't exposed as a command-line option (...yet?). (NameValue stringizes - the option name for the curl cmd -> libcurl source generator) + Closes #5955 + +- [Laramie Leavitt brought this change] + + http: consolidate nghttp2_session_mem_recv() call paths - Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. + Previously there were several locations that called + nghttp2_session_mem_recv and handled responses slightly differently. + Those have been converted to call the existing + h2_process_pending_input() function. - Ref: https://github.com/curl/curl/pull/4655 - -- setopt: Fix ALPN / NPN user option when built without HTTP2 + Moved the end-of-session check to h2_process_pending_input() since the + only place the end-of-session state can change is after nghttp2 + processes additional input frames. - - Stop treating lack of HTTP2 as an unknown option error result for - CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. + This will likely fix the fuzzing error. While I don't have a root cause + the out-of-bounds read seems like a use after free, so moving the + nghttp2_session_check_request_allowed() call to a location with a + guaranteed nghttp2 session seems reasonable. - Prior to this change it was impossible to disable ALPN / NPN if libcurl - was built without HTTP2. Setting either option would result in - CURLE_UNKNOWN_OPTION and the respective internal option would not be - set. That was incorrect since ALPN and NPN are used independent of - HTTP2. + Also updated a few nghttp2 callsites to include error messages and added + a few additional error checks. - Reported-by: Shailesh Kapse + Closes #5648 + +- HISTORY: mention alt-svc added in 2019 - Fixes https://github.com/curl/curl/issues/4668 - Closes https://github.com/curl/curl/pull/4672 + ... and make 1996 the first year subtitle -Daniel Stenberg (5 Dec 2019) -- etag: allow both --etag-compare and --etag-save in same cmdline +- base64: also build for pop3 and imap + + Follow-up to the fix in 20417a13fb8f83 - Fixes #4669 - Closes #4678 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5948 -Marcel Raad (5 Dec 2019) -- curl_setup: fix `CURLRES_IPV6` condition +- base64: enable in build with SMTP - Move the definition of `CURLRES_IPV6` to before undefining - `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused - some tests to fail and others to be skipped with c-ares. + The oauth2 support is used with SMTP and it uses base64 functions. - Fixes https://github.com/curl/curl/issues/4673 - Closes https://github.com/curl/curl/pull/4677 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5938 -Daniel Stenberg (5 Dec 2019) -- test342: make it return a 304 as the tag matches +- curl_mime_headers.3: fix the example's use of curl_slist_append + + Reported-by: sofaboss on github + Fixes #5942 + Closes #5943 -Peter Wu (4 Dec 2019) -- CMake: add support for building with the NSS vtls backend +- lib583: fix enum mixup - Options are cross-checked with configure.ac and acinclude.m4. - Tested on Arch Linux, untested on other platforms like Windows or macOS. + grrr the previous follow-up to 17fcdf6a31 was wrong + +- libtest: fix build errors - Closes #4663 - Reviewed-by: Kamil Dudka + Follow-up from 17fcdf6a310d4c8076 -Daniel Stenberg (4 Dec 2019) -- azure: add more builds +- lib: fix -Wassign-enum warnings - ... removed two from travis (that now runs on azure instead) + configure --enable-debug now enables -Wassign-enum with clang, + identifying several enum "abuses" also fixed. - Closes #4671 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 + + Closes #5929 -- CURLOPT_VERBOSE.3: see also ERRORBUFFER +- RELEASE-NOTES: synced -- hostip4.c: bump copyright year range +- [Diven Qi brought this change] -Marcel Raad (3 Dec 2019) -- configure: enable IPv6 support without `getaddrinfo` - - This makes it possible to recognize and connect to literal IPv6 - addresses when `getaddrinfo` is not available, which is already the - case for the CMake build. This affects e.g. classic MinGW because it - still targets Windows 2000 by default, where `getaddrinfo` is not - available, but general IPv6 support is. + url: use blank credentials when using proxy w/o username and password - Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the - CMake build does. + Fixes proxy regression brought in commit ad829b21ae (7.71.0) - Closes https://github.com/curl/curl/pull/4662 + Fixed #5911 + Closes #5914 -- curl_setup: disable IPv6 resolver without `getaddrinfo` +- travis: add a build using libressl (from git master) - Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 - address support. This makes it possible to connect to IPv6 literals by - setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes - the CMake build when using the synchronous resolver without - `getaddrinfo` support. + The v3.2.1 tag (latest release atm) results in a broken build. - Closes https://github.com/curl/curl/pull/4662 + Closes #5932 -Daniel Stenberg (3 Dec 2019) -- github action/azure pipeline: run 'make test-nonflaky' for tests +- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 - To match travis and give more info on failures. - -- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains + Unfortunately, this option is not detecting the same issues as clang's + -Wassign-enum flag, but should still be useful to detect future + mistakes. - Closes #4655 + Closes #5930 -- openssl: set X509_V_FLAG_PARTIAL_CHAIN +- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - Have intermediate certificates in the trust store be treated as - trust-anchors, in the same way as self-signed root CA certificates - are. This allows users to verify servers using the intermediate cert - only, instead of needing the whole chain. + If the error reason from the lib is + SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return + CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. - Other TLS backends already accept partial chains. + This unifies the libcurl return code and makes libressl run test 313 + (CRL testing) fine. - Reported-by: Jeffrey Walton - Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html + Closes #5934 -- curl: show better error message when no homedir is found - - Reported-by: Vlastimil Ovčáčík - Fixes #4644 - Closes #4665 +- FAQ: refreshed some very old language -- OPENSOCKETFUNCTION.3: correct the purpose description +- cmake: make HTTP_ONLY also disable MQTT - Reported-by: Jeff Mears - Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html + ... and alphasort the order of disabling protocols to make it easier to + browse. - Closes #4667 + Closes #5931 -- [Peter Wu brought this change] - - travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty +- libtest: remove lib1541 leftovers - Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. + Caused automake errors. - Reported-by: Jay Satiro - Fixes #4659 - Closes #4661 - Closes #4664 + Follow-up to 8ca54a03ea08a -- azure-pipelines: fix the test script - -- Azure Pipelines: initial CI setup +- tests/libtests: remove test 1900 and 2033 - [skip ci] - -- docs: add "added: 7.68.0" to the --etag-* docs - -- copyright: fix the year ranges for two files + We already remove the test files, now remove the libtest codes as well. - Follow-up to 9c1806ae + Follow-up to e50a877df74 -Jay Satiro (1 Dec 2019) -- build: Disable Visual Studio warning "conditional expression is constant" +Marc Hoersken (7 Sep 2020) +- CI/azure: add test number to title for display in analytics - - Disable warning C4127 "conditional expression is constant" globally - in curl_setup.h for when building with Microsoft's compiler. + To ease identification of tests the test number is added to + the test case title in order to have it on the Azure DevOps + Analytics pages and reports which currently do not show it. - This mainly affects building with the Visual Studio project files found - in the projects dir. + Bump test case revision to make Azure DevOps update titles. - Prior to this change the cmake and winbuild build systems already - disabled 4127 globally for when building with Microsoft's compiler. - Also, 4127 was already disabled for all build systems in the limited - circumstance of the WHILE_FALSE macro which disabled the warning - specifically for while(0). This commit removes the WHILE_FALSE macro and - all other cruft in favor of disabling globally in curl_setup. + Closes #5927 + +Daniel Stenberg (6 Sep 2020) +- altsvc: clone setting in curl_easy_duphandle - Background: + The cache content is not duplicated, like other caches, but the setting + and specified file name are. - We have various macros that cause 0 or 1 to be evaluated, which would - cause warning C4127 in Visual Studio. For example this causes it: + Test 1908 is extended to verify this somewhat. Since the duplicated + handle gets the same file name, the test unfortunately overwrites the + same file twice (with different contents) which makes it hard to check + automatically. - #define Curl_resolver_asynch() 1 + Closes #5923 + +- test1541: remove since it is a known bug - Full behavior is not clearly defined and inconsistent across versions. - However it is documented that since VS 2015 Update 3 Microsoft has - addressed this somewhat but not entirely, not warning on while(true) for - example. + A shared connection cache is not thread-safe is a known issue. Stop + testing this until we believe this issue is addressed. Reduces + occasional test failures we don't care about. - Prior to this change some C4127 warnings occurred when I built with - Visual Studio using the generated projects in the projects dir. + The test code in lib1541.c is left in git to allow us to restore it when + we get to fix this. - Closes https://github.com/curl/curl/pull/4658 + Closes #5922 -- openssl: retrieve reported LibreSSL version at runtime +- tests: remove pipelining tests - - Retrieve LibreSSL runtime version when supported (>= 2.7.1). + Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were + previously disabled. - For earlier versions we continue to use the compile-time version. + The Pipelining code was removed from curl in commit 2f44e94efb3df8e, + April 2019. - Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 - - Closes https://github.com/curl/curl/pull/2425 + Closes #5921 -- strerror: Add Curl_winapi_strerror for Win API specific errors - - - In all code call Curl_winapi_strerror instead of Curl_strerror when - the error code is known to be from Windows GetLastError. +- curl: retry delays in parallel mode no longer sleeps blocking - Curl_strerror prefers CRT error codes (errno) over Windows API error - codes (GetLastError) when the two overlap. When we know the error code - is from GetLastError it is more accurate to prefer the Windows API error - messages. + The previous sleep for retries would block all other concurrent + transfers. Starting now, the retry will instead be properly marked to + not get restarted until after the delay time but other transfers can + still continue in the mean time. - Reported-by: Richard Alcock + Closes #5917 + +- curl:parallel_transfers: make sure retry readds the transfer - Fixes https://github.com/curl/curl/issues/4550 - Closes https://github.com/curl/curl/pull/4581 + Reported-by: htasta on github + Fixes #5905 + Closes #5917 -Daniel Stenberg (2 Dec 2019) -- global_init: undo the "intialized" bump in case of failure +- build: drop support for building with Watcom - ... so that failures in the global init function don't count as a - working init and it can then be called again. + These files are not maintained, they seem to have no users, Watcom + compilers look like not having users nor releases anymore. - Reported-by: Paul Groke - Fixes #4636 - Closes #4653 + Closes #5918 -- parsedate: offer a getdate_capped() alternative +- winbuild/rundebug.cmd: remove - ... and use internally. This function will return TIME_T_MAX instead of - failure if the parsed data is found to be larger than what can be - represented. TIME_T_MAX being the largest value curl can represent. + Seems to have been added by mistake? Not included in dists. - Reviewed-by: Daniel Gustafsson - Reported-by: JanB on github - Fixes #4152 - Closes #4651 + Closes #5919 -- docs: add more references to curl_multi_poll +- curl: in retry output don't call all problems "transient" - Fixes #4643 - Closes #4652 - -- sha256: bump the copyright year range + ... because when --retry-all-errors is used, the error isn't necessarily + transient at all. - Follow-up from 66e21520f + Closes #5916 -Daniel Gustafsson (28 Nov 2019) -- curl_setup_once: consistently use WHILE_FALSE in macros +- easygetopt: pass a valid enum to avoid compiler warning - The WHILE_FALSE construction is used to avoid compiler warnings in - macro constructions. This fixes a few instances where it was not - used in order to keep the code consistent. + "integer constant not in range of enumerated type 'CURLoption'" - Closes #4649 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + + Closes #5915 -Daniel Stenberg (28 Nov 2019) -- [Steve Holme brought this change] +- [Emil Engler brought this change] - http_ntlm: Remove duplicate NSS initialisation + tests: Add tests for new --help - Given that this is performed by the NTLM code there is no need to - perform the initialisation in the HTTP layer. This also keeps the - initialisation the same as the SASL based protocols and also fixes a - possible compilation issue if both NSS and SSPI were to be used as - multiple SSL backends. + This commit is a part of "--help me if you can" - Reviewed-by: Kamil Dudka - Closes #3935 + Closes #5680 + +- [Emil Engler brought this change] -Daniel Gustafsson (28 Nov 2019) -- checksrc: fix regexp for ASSIGNWITHINCONDITION + tool: update --help with categories - The regexp looking for assignments within conditions was too greedy - and matched a too long string in the case of multiple conditionals - on the same line. This is basically only a problem in single line - macros, and the code which exemplified this was essentially: + This commit is a part of "--help me if you can" - do { if((x) != NULL) { x = NULL; } } while(0) + Closes #5680 + +- [Emil Engler brought this change] + + docs: add categories to all cmdline opts - ..where the final parenthesis of while(0) matched the regexp, and - the legal assignment in the block triggered the warning. Fix by - making the regexp less greedy by matching for the tell-tale signs - of the if statement ending. + Adapted gen.pl with 'listcats' - Also remove the one occurrence where the warning was disabled due - to a construction like the above, where the warning didn't apply - when fixed. + This commit is a part of "--help me if you can" - Closes #4647 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Closes #5680 -Daniel Stenberg (28 Nov 2019) - RELEASE-NOTES: synced -- [Maros Priputen brought this change] +- [ihsinme brought this change] - curl: two new command line options for etags - - --etag-compare and --etag-save + connect.c: remove superfluous 'else' in Curl_getconnectinfo - Suggested-by: Paul Hoffman - Fixes #4277 - Closes #4543 + Closes #5912 -Daniel Gustafsson (28 Nov 2019) -- docs: fix typos - -Daniel Stenberg (28 Nov 2019) -- mailmap: Niall O'Reilly's name - -- [Niall O'Reilly brought this change] - - doh: use dedicated probe slots - - ... to easier allow additional DNS transactions. - - Closes #4629 +- [Samuel Marks brought this change] -- travis: build ngtcp2 with --enable-lib-only + CMake: remove explicit `CMAKE_ANSI_CFLAGS` - ... makes it skip the examples and other stuff we don't neeed. + This variable was removed from cmake in commit + https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later + CMake commit removes the variable from the tests, claiming that it was + removed in CMake 2.6 - Closes #4646 + Reviewed-By: Peter Wu + Closes #5439 -- [David Benjamin brought this change] +- [cbe brought this change] - ngtcp2: fix thread-safety bug in error-handling + libssh2: pass on the error from ssh_force_knownhost_key_type - ERR_error_string(NULL) should never be called. It places the error in a - global buffer, which is not thread-safe. Use ERR_error_string_n with a - local buffer instead. - - Closes #4645 - -- travis: export the CC/CXX variables when set - - Suggested-by: Peter Wu - Fixes #4637 - Closes #4640 + Closes #5909 -Marcel Raad (26 Nov 2019) -- dist: add error-codes.pl +- scripts/delta: add diffstat summary - Follow-up to commit 74f441c6d31. - This should fix test 1175 when run via the daily source tarballs. - - Closes https://github.com/curl/curl/pull/4638 + ... and make output more table-like -Daniel Stenberg (26 Nov 2019) -- [John Schroeder brought this change] +- [Martin Bašti brought this change] - curl: fix --upload-file . hangs if delay in STDIN + http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set - Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. + ... in case NO_PROXY takes an effect - When uploading from stdin in non-blocking mode, a delay in reading - the stream (EAGAIN) causes curl to pause sending data - (CURL_READFUNC_PAUSE). Prior to this change, a busy read was - detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. - This change performs the same busy read handling in a - CURLOPT_XFERINFOFUNCTION handler. + Without this patch, the following command crashes: - Fixes #2051 - Closes #4599 - Reported-by: bdry on github - -- [John Schroeder brought this change] - - XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE + $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ + git clone https://github.com/curl/curl.git - (also for PROGRESSFUNCTION) + Minimal libcurl-based reproducer: - By returning this value from the callback, the internal progress - function call is still called afterward. + #include <curl/curl.h> - Closes #4599 - -- [Michael Forney brought this change] - - TLS: add BearSSL vtls implementation + int main() { + CURL *curl = curl_easy_init(); + if(curl) { + CURLcode ret; + curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); + curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); + /* set the proxy type */ + curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); + curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); + curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); + ret = curl_easy_perform(curl); + curl_easy_cleanup(curl); + return ret; + } + return -1; + } - Closes #4597 + Assisted-by: Kamil Dudka + Bug: https://bugzilla.redhat.com/1873327 + Closes #5902 -- curl_multi_wakeup.3: add example and AVAILABILITY +- travis: add a CI job with openssl3 (from git master) - Reviewed-by: Gergely Nagy - Closes #4635 - -- [Gergely Nagy brought this change] + Closes #5908 - multi: add curl_multi_wakeup() +- openssl: avoid error conditions when importing native CA - This commit adds curl_multi_wakeup() which was previously in the TODO - list under the curl_multi_unblock name. + The code section that is OpenSSL 3+ specific now uses the same logic as + is used in the version < 3 section. It caused a compiler error without + it. - On some platforms and with some configurations this feature might not be - available or can fail, in these cases a new error code - (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). - - Fixes #4418 - Closes #4608 - -Jay Satiro (24 Nov 2019) -- [Xiaoyin Liu brought this change] + Closes #5907 - schannel: fix --tls-max for when min is --tlsv1 or default - - Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ - macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS - (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. +- setopt: avoid curl_ on local variable - Closes https://github.com/curl/curl/pull/4633 + Closes #5906 -- checksrc.bat: Add a check for vquic and vssh directories +- mqtt.c: avoid curl_ prefix on local variable - Ref: https://github.com/curl/curl/pull/4607 + Closes #5906 -- projects: Fix Visual Studio projects SSH builds - - - Generate VQUIC and VSSH filenames in Visual Studio project files. - - Prior to this change generated Visual Studio project configurations that - enabled SSH did not build properly. Broken since SSH files were moved to - lib/vssh 3 months ago in 5b2d703. +- wildcard: strip "curl_" prefix from private symbols - Fixes https://github.com/curl/curl/issues/4492 - Fixes https://github.com/curl/curl/issues/4630 - Closes https://github.com/curl/curl/pull/4607 + Closes #5906 -Daniel Stenberg (23 Nov 2019) -- RELEASE-NOTES: synced - -Jay Satiro (22 Nov 2019) -- openssl: Revert to less sensitivity for SYSCALL errors +- vtls: make it 'struct Curl_ssl_session' - - Disable the extra sensitivity except in debug builds (--enable-debug). + Use uppercase C for internal symbols. - - Improve SYSCALL error message logic in ossl_send and ossl_recv so that - "No error" / "Success" socket error text isn't shown on SYSCALL error. + Closes #5906 + +- curl_threads: make it 'struct Curl_actual_call' - Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity - of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were - also considered errors. For example, a server that does not send a known - protocol termination point (eg HTTP content length or chunked encoding) - _and_ does not send a TLS termination point (close_notify alert) would - cause an error if it closed the connection. + Internal names should not be prefixed "curl_" - To be clear that behavior made it into release build 7.67.0 - unintentionally. Several users have reported it as an issue. + Closes #5906 + +- schannel: make it 'struct Curl_schannel*' - Ultimately the idea is a good one, since it can help prevent against a - truncation attack. Other SSL backends may already behave similarly (such - as Windows native OS SSL Schannel). However much more of our user base - is using OpenSSL and there is a mass of legacy users in that space, so I - think that behavior should be partially reverted and then rolled out - slowly. + As internal global names should use captical C. - This commit changes the behavior so that the increased sensitivity is - disabled in all curl builds except curl debug builds (DEBUGBUILD). If - after a period of time there are no major issues then it can be enabled - in dev and release builds with the newest OpenSSL (1.1.1+), since users - using the newest OpenSSL are the least likely to have legacy problems. + Closes #5906 + +- hash: make it 'struct Curl_hash' - Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 - Reported-by: Bjoern Franke + As internal global names should use captical C. - Fixes https://github.com/curl/curl/issues/4624 - Closes https://github.com/curl/curl/pull/4623 + Closes #5906 -- [Daniel Stenberg brought this change] - - openssl: improve error message for SYSCALL during connect +- llist: make it "struct Curl_llist" - Reported-by: Paulo Roberto Tomasi - Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html + As internal global names should use captical C. - Closes https://github.com/curl/curl/pull/4593 + Closes #5906 -Daniel Stenberg (22 Nov 2019) -- test1175: verify symbols-in-versions and libcurl-errors.3 in sync +Marc Hoersken (2 Sep 2020) +- telnet.c: depend on static requirement of WinSock version 2 - Closes #4628 - -- include: make CURLE_HTTP3 use a new error code + Drop dynamic loading of ws2_32.dll and instead rely on the + imported version which is now required to be at least 2.2. - To avoid potential issues with error code reuse. + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats - Reported-by: Christoph M. Becker - Assisted-by: Dan Fandrich - Fixes #4601 - Closes #4627 - -- bump: next release will be 7.68.0 + Closes #5854 -- curl: add --parallel-immediate +- win32: drop support for WinSock version 1, require version 2 - Starting with this change when doing parallel transfers, without this - option set, curl will prefer to create new transfers multiplexed on an - existing connection rather than creating a brand new one. + IPv6, telnet and now also the multi API require WinSock + version 2 which is available starting with Windows 95. - --parallel-immediate can be set to tell curl to prefer to use new - connections rather than to wait and try to multiplex. + Therefore we think it is time to drop support for version 1. - libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default - on parallel transfers. + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats - Suggested-by: Tom van der Woerdt - Closes #4500 + Follow up to #5634 + Closes #5854 -Daniel Gustafsson (20 Nov 2019) -- [Victor Magierski brought this change] - - docs: fix typos +- select: align poll emulation to return all relevant events - Change 'experiemental' to 'experimental'. + The poll emulation via select already consumes POLLRDNORM, + POLLWRNORM and POLLRDBAND as input events. Therefore it + should also return them as output events if signaled. - Closes #4618 - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> - -Jay Satiro (18 Nov 2019) -- projects: Fix Visual Studio wolfSSL configurations + Also fix indentation in input event handling block. - - s/USE_CYASSL/USE_WOLFSSL/ - - - Remove old compatibility macros. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Follow-up to 1c6c59a from several months ago when CyaSSL named symbols - were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL - and we kept using their old name for compatibility reasons, until - earlier this year. + Replaces #5852 + Closes #5883 -Daniel Stenberg (18 Nov 2019) -- RELEASE-NOTES: synced +- CI/azure: MQTT is now enabled by default + + Reviewed-by: Daniel Stenberg + + Follow up to #5858 + Closes #5903 -- [Javier Blazquez brought this change] +Daniel Stenberg (2 Sep 2020) +- copyright.pl: ignore buildconf - ngtcp2: use overflow buffer for extra HTTP/3 data - - Fixes #4525 - Closes #4603 +- test971: show test mismatches "inline" -- altsvc: bump to h3-24 +- lib/Makefile.am: bump VERSIONINFO due to new functions - ... as both ngtcp2 and quiche now support that in their master branches + ... we're generally bad at this, but we are adding new functions for + this release. - Closes #4604 + Closes #5899 -- ngtcp2: free used resources on disconnect +- optiontable: use DEBUGBUILD - Fixes #4614 - Closes #4615 + Follow-up to commit 6e18568ba38 (#5877) -- ngtcp2: handle key updates as ngtcp2 master branch tells us +- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 - Reviewed-by: Tatsuhiro Tsujikawa + If there are more than two items in the list, use commas for all but the + last separator which is set to 'and'. Reads better. - Fixes #4612 - Closes #4613 - -Jay Satiro (17 Nov 2019) -- [Gergely Nagy brought this change] + Closes #5898 - multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds - - Prior to this change: - - The check if an extra wait is necessary was based not on the - number of extra fds but on the pointer. +- curl.1: add see also no-progress-meter on two spots - If a non-null pointer was given in extra_fds, but extra_nfds - was zero, then the wait was skipped even though poll was not - called. + Ref: #5894 - Closes https://github.com/curl/curl/pull/4610 + Closes #5897 -- lib: Move lib/ssh.h -> lib/vssh/ssh.h +- RELEASE-NOTES: synced + +- mqtt: enable by default - Follow-up to 5b2d703 which moved ssh source files to vssh. + No longer considered experimental. - Closes https://github.com/curl/curl/pull/4609 + Closes #5858 -Daniel Stenberg (16 Nov 2019) -- [Andreas Falkenhahn brought this change] +- [Michael Baentsch brought this change] - INSTALL.md: provide Android build instructions + tls: add CURLOPT_SSL_EC_CURVES and --curves - Closes #4606 + Closes #5892 + +- url: remove funny embedded comments in Curl_disonnect calls -- [Niall O'Reilly brought this change] +- [Chris Paulson-Ellis brought this change] - doh: improced both encoding and decoding + conn: check for connection being dead before reuse - Improved estimation of expected_len and updated related comments; - increased strictness of QNAME-encoding, adding error detection for empty - labels and names longer than the overall limit; avoided treating DNAME - as unexpected; + Prevents incorrect reuse of an HTTP connection that has been prematurely + shutdown() by the server. - updated unit test 1655 with more thorough set of proofs and tests + Partial revert of 755083d00deb16 - Closes #4598 + Fixes #5884 + Closes #5893 -- ngtcp2: increase QUIC window size when data is consumed +Marc Hoersken (29 Aug 2020) +- buildconf: exec autoreconf to avoid additional process - Assisted-by: Javier Blazquez - Ref #4525 (partial fix) - Closes #4600 - -- [Melissa Mears brought this change] - - config-win32: cpu-machine-OS for Windows on ARM + Also make buildconf exit with the return code of autoreconf. - Define the OS macro properly for Windows on ARM builds. Also, we might - as well add the GCC-style IA-64 macro. + Reviewed-by: Daniel Stenberg - Closes #4590 + Follow up to #5853 + Closes #5890 -- examples: add multi-poll.c - - Show how curl_multi_poll() makes it even easier to use the multi - interface. +- CI/azure: no longer ignore results of test 1013 - Closes #4596 + Follow up to #5771 + Closes #5889 -- multi_poll: avoid busy-loop when called without easy handles attached +- docs: add description about CI platforms to CONTRIBUTE.md - Fixes #4594 - Closes #4595 - Reported-by: 3dyd on github - -- curl: fix -T globbing + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro - Regression from e59371a4936f8 (7.67.0) + Closes #5882 + +Daniel Stenberg (29 Aug 2020) +- tests/getpart: use MIME::Base64 instead of home-cooked - Added test 490, 491 and 492 to verify the functionality. + Since we already use the base64 package since a while back, we can just + as well switch to that here too. - Reported-by: Kamil Dudka - Reported-by: Anderson Sasaki + It also happens to use the exact same function name, which otherwise + causes a run-time warning. - Fixes #4588 - Closes #4591 - -- HISTORY: added cmake, HTTP/3 and parallel downloads with curl + Reported-by: Marc Hörsken + Fixes #5885 + Closes #5887 -- quiche: reject headers in the wrong order +Marcel Raad (29 Aug 2020) +- ntlm: fix condition for curl_ntlm_core usage - Pseudo header MUST come before regular headers or cause an error. + `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES + backend is fine, but was excluded before. - Reported-by: Cynthia Coan - Fixes #4571 - Closes #4584 + This also fixes test 1013 as the condition for SMB support in + configure.ac didn't match the condition in the source code. Now it + does. + + Fixes https://github.com/curl/curl/issues/1262 + Closes https://github.com/curl/curl/pull/5771 -- openssl: prevent recursive function calls from ctx callbacks +- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode - Follow the pattern of many other callbacks. + The Schannel builds are the most useful to verify as they make the most + use of the Windows API. Classic MinGW doesn't support Unicode at all, + only MinGW-w64 and MSVC do. - Ref: #4546 - Closes #4585 + Closes https://github.com/curl/curl/pull/5843 -- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines +- CMake: add option to enable Unicode on Windows - The disable-scan script used in test 1165 is extended to also verify - that the docs cover all used defines and all defines offered by - configure. + As already existing for winbuild. - Reported-by: SLDiggie on github - Fixes #4545 - Closes #4587 + Closes https://github.com/curl/curl/pull/5843 -- remove_handle: clear expire timers after multi_done() +Marc Hoersken (29 Aug 2020) +- select: simplify return code handling for poll and select - Since 59041f0, a new timer might be set in multi_done() so the clearing - of the timers need to happen afterwards! + poll and select already return -1 on error according to POSIX, + so there is no need to perform a <0 to -1 conversion in code. - Reported-by: Max Kellermann - Fixes #4575 - Closes #4583 - -Marcel Raad (10 Nov 2019) -- test1558: use double slash after file: + Also we can just use one check with <= 0 on the return code. - Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this - test unnecessarily failed when using `file:/` instead of `file:///`. + Assisted-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Closes https://github.com/curl/curl/pull/4554 + Replaces #5852 + Closes #5880 + +Daniel Stenberg (28 Aug 2020) +- RELEASE-NOTES: synced + +- [Jeroen Ooms brought this change] -Daniel Stenberg (10 Nov 2019) -- pause: avoid updating socket if done was already called + tests: add test1912 with typechecks - ... avoids unnecesary recursive risk when the transfer is already done. + Validates that gcc-typecheck macros match the new option type API. - Reported-by: Richard Bowker - Fixes #4563 - Closes #4574 + Closes #5873 -Jay Satiro (9 Nov 2019) -- strerror: Fix an error looking up some Windows error strings +- easyoptions: provide debug function when DEBUGBUILD - - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in - Windows error strings. + ... not CURLDEBUG as they're not always set in conjunction. - Since we are not in control of the error code we don't know what - information may be needed by the error string's format specifiers. + Follow-up to 6ebe63fac23f38df - Prior to this change Windows API error strings which contain specifiers - (think specifiers like similar to printf specifiers) would not be shown. - The FormatMessage Windows API call which turns a Windows error code into - a string could fail and set error ERROR_INVALID_PARAMETER if that error - string contained a format specifier. FormatMessage expects a va_list for - the specifiers, unless inserts are ignored in which case no substitution - is attempted. - - Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 - -- [r-a-sattarov brought this change] + Fixes #5877 + Closes #5878 - system.h: fix for MCST lcc compiler - - Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do - some code cleanup. +Marc Hoersken (28 Aug 2020) +- sockfilt: handle FD_CLOSE winsock event on write socket - e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium - architecture. + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Ref: https://en.wikipedia.org/wiki/Elbrus_2000 - - Closes https://github.com/curl/curl/pull/4576 + Follow up to #5867 + Closes #5879 -Daniel Stenberg (8 Nov 2019) -- TODO: curl_multi_unblock +- multi: handle connection state winsock events - Closes #4418 - -- TODO: Run web-platform-tests url tests + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Closes #4477 - -- TODO: 1.4 alt-svc sharing + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - Closes #4476 + Follow up to #5634 + Closes #5867 -- test1560: require IPv6 for IPv6 aware URL parsing +Daniel Stenberg (28 Aug 2020) +- Curl_pgrsTime - return new time to avoid timeout integer overflow - The URL parser function can't reject a bad IPv6 address properly when - curl was built without IPv6 support. + Setting a timeout to INT_MAX could cause an immediate error to get + returned as timeout because of an overflow when different values of + 'now' were used. - Reported-by: Marcel Raad - Fixes #4556 - Closes #4572 + This is primarily fixed by having Curl_pgrsTime() return the "now" when + TIMER_STARTSINGLE is set so that the parent function will continue using + that time. + + Reported-by: Ionuț-Francisc Oancea + Fixes #5583 + Closes #5847 -- checksrc: repair the copyrightyear check +- TLS: fix SRP detection by using the proper #ifdefs - - Consider a modified file to be committed this year. + USE_TLS_SRP will be true if *any* selected TLS backend can use SRP - - Make the travis CHECKSRC also do COPYRIGHTYEAR scan in examples and - includes + HAVE_OPENSSL_SRP is defined when OpenSSL can use it - - Ignore 0 parents when getting latest commit date of file. + HAVE_GNUTLS_SRP is defined when GnuTLS can use it - since in the CI we're dealing with a truncated repo of last 50 commits, - the file's most recent commit may not be available. when this happens - git log and rev-list show the initial commit (ie first commit not to be - truncated) but that's incorrect so ignore it. + Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is + set if at least one of the supported backends offers SRP. - Ref: https://github.com/curl/curl/pull/4547 + Reported-by: Stefan Strogin + Fixes #5865 + Closes #5870 + +- [Dan Kenigsberg brought this change] + + docs: SSLCERTS: fix English syntax - Closes https://github.com/curl/curl/pull/4549 + Signed-off-by: Dan Kenigsberg <danken@redhat.com> - Co-authored-by: Jay Satiro + Closes #5876 -- copyrights: fix copyright year range +- [Alessandro Ghedini brought this change] + + docs: non-existing macros in man pages - .. because checksrc's copyright year check stopped working. + As reported by man(1) when invoked as: - Ref: https://github.com/curl/curl/pull/4547 + man --warnings -E UTF-8 -l -Tutf8 -Z <file> >/dev/null - Closes https://github.com/curl/curl/pull/4549 + Closes #5846 -- RELEASE-NOTES: synced - -- curlver: bump to 7.67.1 - -- mailmap: fixup Massimiliano Fantuzzi +- [Alessandro Ghedini brought this change] -- scripts/contributors: make committers get included too + curl.1: fix typo invokved -> invoked - in addition to authors + Closes #5846 -Jay Satiro (8 Nov 2019) -- [Massimiliano Fantuzzi brought this change] - - configure: fix typo in help text +- buildconf: invoke 'autoreconf -fi' instead - Closes https://github.com/curl/curl/pull/4570 - -Daniel Stenberg (7 Nov 2019) -- [Christian Schmitz brought this change] - - ntlm: USE_WIN32_CRYPTO check removed to get USE_NTLM2SESSION set + The custom script isn't necessary anymore - but remains for simplicity + and just invokes autoreconf. - Closes #3704 + Closes #5853 -Jay Satiro (6 Nov 2019) -- [Wyatt O'Day brought this change] +- [Emil Engler brought this change] - build: fix for CURL_DISABLE_DOH + lib: make Curl_gethostname accept a const pointer - Fixes https://github.com/curl/curl/issues/4565 - Closes https://github.com/curl/curl/pull/4566 - -- [Leonardo Taccari brought this change] - - configure: avoid unportable `==' test(1) operator + The address of that variable never gets changed, only the data in it so + why not make it a "char * const"? - Closes https://github.com/curl/curl/pull/4567 + Closes #5866 -Version 7.67.0 (5 Nov 2019) - -Daniel Stenberg (5 Nov 2019) -- RELEASE-NOTES: synced +- docs/libcurl: update "Added in" version for curl_easy_option* - The 7.67.0 release + Follow-up to 6ebe63fac23f38 -- THANKS: add new names from 7.67.0 +- scripts: improve the "get latest curl release tag" logic + + ... by insiting on it matching "^curl-". -- configure: only say ipv6 enabled when the variable is set +- configure: added --disable-get-easy-options - Previously it could say "IPv6: enabled" at the end of the configure run - but the define wasn't set because of a missing getaddrinfo(). + To allow disabling of the curl_easy_option APIs in a build. - Reported-by: Marcel Raad - Fixes #4555 - Closes #4560 + Closes #5365 -Marcel Raad (2 Nov 2019) -- certs/Server-localhost-lastSAN-sv: regenerate with sha256 +- options: API for meta-data about easy options + + const struct curl_easyoption *curl_easy_option_by_name(const char *name); + + const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); + + const struct curl_easyoption * + curl_easy_option_next(const struct curl_easyoption *prev); - All other certificates were regenerated in commit ba782baac30, but - this one was missed. - Fixes test3001 on modern systems. + The purpose is to provide detailed enough information to allow for + example libcurl bindings to get option information at run-time about + what easy options that exist and what arguments they expect. - Closes https://github.com/curl/curl/pull/4551 + Assisted-by: Jeroen Ooms + Closes #5365 -Daniel Stenberg (2 Nov 2019) -- [Vilhelm Prytz brought this change] +- [Eric Curtin brought this change] - copyrights: update all copyright notices to 2019 on files changed this year + HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 - Closes #4547 + Closes #5871 -- [Bastien Bouclet brought this change] +- RELEASE-NOTES: synced - mbedtls: add error message for cert validity starting in the future +Jay Satiro (26 Aug 2020) +- openssl: Fix wincrypt symbols conflict with BoringSSL - Closes #4552 - -Jay Satiro (1 Nov 2019) -- schannel_verify: Fix concurrent openings of CA file + OpenSSL undefines the conflicting symbols but BoringSSL does not so we + must do it ourselves. - - Open the CA file using FILE_SHARE_READ mode so that others can read - from it as well. + Reported-by: Samuel Tranchet + Assisted-by: Javier Blazquez - Prior to this change our schannel code opened the CA file without - sharing which meant concurrent openings (eg an attempt from another - thread or process) would fail during the time it was open without - sharing, which in curl's case would cause error: - "schannel: failed to open CA file". + Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 + Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 - Bug: https://curl.haxx.se/mail/lib-2019-10/0104.html - Reported-by: Richard Alcock + Fixes https://github.com/curl/curl/issues/5669 + Closes https://github.com/curl/curl/pull/5857 -Daniel Stenberg (31 Oct 2019) -- gtls: make gnutls_bye() not wait for response on shutdown +Daniel Stenberg (26 Aug 2020) +- socketpair: allow CURL_DISABLE_SOCKETPAIR - ... as it can make it wait there for a long time for no good purpose. + ... to completely disable the use of socketpair - Patched-by: Jay Satiro - Reported-by: Bylon2 on github - Adviced-by: Nikos Mavrogiannopoulos + Closes #5850 + +- curl_get_line: build only if cookies or alt-svc are enabled - Fixes #4487 - Closes #4541 + Closes #5851 -- [Michał Janiszewski brought this change] +- [fullincome brought this change] - appveyor: publish artifacts on appveyor + schannel: fix memory leak when using get_cert_location - This allows obtaining upstream builds of curl directly from appveyor for - all the available configurations + The get_cert_location function allocates memory only on success. + Previously get_cert_location was able to allocate memory and return + error. It wasn't obvious and in this case the memory wasn't + released. - Closes #4509 + Fixes #5855 + Closes #5860 -- url: make Curl_close() NULLify the pointer too +- [Emil Engler brought this change] + + git: ignore libtests in 3XXX area - This is the common pattern used in the code and by a unified approach we - avoid mistakes. + Currently the file tests/libtest/lib3010 is not getting + ignored by git. This fixes it by adding the 3XXX area to + the according .gitignore file. - Closes #4534 + Closes #5859 -- [Trivikram Kamat brought this change] +- [Emil Engler brought this change] - INSTALL: add missing space for configure commands + doh: add error message for DOH_DNS_NAME_TOO_LONG + + When this error code was introduced in b6a53fff6c1d07e8a9, it was + forgotten to be added in the errors array and doh_strerror function. - Closes #4539 + Closes #5863 -- url: Curl_free_request_state() should also free doh handles +- ngtcp2: adapt to the new pkt_info arguments - ... or risk DoH memory leaks. + Guidance-by: Tatsuhiro Tsujikawa - Reported-by: Paul Dreik - Fixes #4463 - Closes #4527 + Closes #5864 -- examples: remove the "this exact code has not been verified" +- winbuild/README.md: make <options> visible - ... as really confuses the reader to not know what to believe! + Follow-up to be753add31c2d8c -- [Trivikram Kamat brought this change] - - HTTP3: fix typo somehere1 > somewhere1 +- winbuild: convert the instruction text to README.md - Closes #4535 + Closes #5861 -Jay Satiro (28 Oct 2019) -- [Javier Blazquez brought this change] +- lib1560: verify "redirect" to double-slash leading URL + + Closes #5849 - HTTP3: fix invalid use of sendto for connected UDP socket +Marc Hoersken (25 Aug 2020) +- multi: expand pre-check for socket readiness - On macOS/BSD, trying to call sendto on a connected UDP socket fails - with a EISCONN error. Because the singleipconnect has already called - connect on the socket when we're trying to use it for QUIC transfers - we need to use plain send instead. + Check readiness of all sockets before waiting on them + to avoid locking in case the one-time event FD_WRITE + was already consumed by a previous wait operation. - Fixes #4529 - Closes https://github.com/curl/curl/pull/4533 + More information about WinSock network events: + https://docs.microsoft.com/en-us/windows/win32/api/ + winsock2/nf-winsock2-wsaeventselect#return-value + + Closes #5634 -Daniel Stenberg (28 Oct 2019) -- RELEASE-NOTES: synced +- [rcombs brought this change] -- [Javier Blazquez brought this change] - - HTTP3: fix Windows build + multi: implement wait using winsock events - The ngtcp2 QUIC backend was using the MSG_DONTWAIT flag for send/recv - in order to perform nonblocking operations. On Windows this flag does - not exist. Instead, the socket must be set to nonblocking mode via - ioctlsocket. + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - This change sets the nonblocking flag on UDP sockets used for QUIC on - all platforms so the use of MSG_DONTWAIT is not needed. + A previous version of this patch failed to account for how in WinSock, + FD_WRITE is set only once when writing becomes possible and not again + until after a send has failed due to the buffer filling. This contrasts + to how FD_READ and FD_OOB continue to be set until the conditions they + refer to no longer apply. This meant that if a user wrote some data to + a socket, but not enough data to completely fill its send buffer, then + waited on that socket to become writable, we'd erroneously stall until + their configured timeout rather than returning immediately. - Fixes #4531 - Closes #4532 - -Marcel Raad (27 Oct 2019) -- appveyor: add --disable-proxy autotools build + This version of the patch addresses that issue by checking each socket + we're waiting on to become writable with select() before the wait, and + zeroing the timeout if it's already writable. - This would have caught issue #3926. - - Also make formatting more consistent. + Assisted-by: Marc Hörsken + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Tested-by: Gergely Nagy + Tested-by: Rasmus Melchior Jacobsen + Tested-by: Tomas Berger - Closes https://github.com/curl/curl/pull/4526 + Replaces #5397 + Reverts #5632 + Closes #5634 -Daniel Stenberg (25 Oct 2019) -- appveyor: make winbuilds with DEBUG=no/yes and VS 2015/2017 +- select: reduce duplication of Curl_poll in Curl_socket_check - ... and invoke "curl -V" once done + Change Curl_socket_check to use select-fallback in Curl_poll + instead of implementing it in Curl_socket_check and Curl_poll. - Co-Authored-By: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Closes #4523 + Replaces #5262 and #5492 + Closes #5707 -- [Francois Rivard brought this change] - - schannel: reverse the order of certinfo insertions +- select: fix poll-based check not detecting connect failure - Fixes #4518 - Closes #4519 - -Marcel Raad (24 Oct 2019) -- test1591: fix spelling of http feature + This commit changes Curl_socket_check to use POLLPRI to + check for connect failure on the write socket, because + POLLPRI maps to fds_err. This is in line with select(2). - The test never got run because the feature name is `http` in lowercase. + The select-based socket check correctly checks for connect + failures by adding the write socket also to fds_err. - Closes https://github.com/curl/curl/pull/4520 - -Daniel Stenberg (23 Oct 2019) -- [Michał Janiszewski brought this change] - - appveyor: Use two parallel compilation on appveyor with CMake + The poll-based implementation (which internally can itself + fallback to select again) did not previously check for + connect failure by using POLLPRI with the write socket. - Appveyor provides 2 CPUs for each builder[1], make sure to use parallel - compilation, when running with CMake. CMake learned this new option in - version 3.12[2] and the version provided by appveyor is fresh enough. + See the follow up commit to this for more information. - Curl doesn't really take that long to build and it is using the slowest - builder available, msbuild, so expect only a moderate improvement in - build times. + This commit makes sure connect failures can be detected + and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. - [1] https://www.appveyor.com/docs/build-environment/ - [2] https://cmake.org/cmake/help/v3.12/release/3.12.html + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Closes #4508 + Replaces #5509 + Prepares #5707 -- conn-reuse: requests wanting NTLM can reuse non-NTLM connections +- select.h: make socket validation macros test for INVALID_SOCKET - Added test case 338 to verify. + With Winsock the valid range is [0..INVALID_SOCKET-1] according to + https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 - Reported-by: Daniel Silverstone - Fixes #4499 - Closes #4514 - -Marcel Raad (23 Oct 2019) -- tests: add missing proxy features - -Daniel Stenberg (22 Oct 2019) -- RELEASE-NOTES: synced - -Marcel Raad (21 Oct 2019) -- tests: use %FILE_PWD for file:// URLs + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - This way, we always have exactly one slash after the host name, making - the tests pass when curl is compiled with the MSYS GCC. + Closes #5760 + +Daniel Stenberg (24 Aug 2020) +- docs: --output-dir is added in 7.73.0, nothing else - Closes https://github.com/curl/curl/pull/4512 + Follow-up to 5620d2cc78c0 -- tests: add `connect to non-listen` keywords +- curl: add --output-dir - These tests try to connect to ports nothing is listening on. + Works with --create-dirs and with -J - Closes https://github.com/curl/curl/pull/4511 + Add test 3008, 3009, 3011, 3012 and 3013 to verify. + + Closes #5637 -- runtests: get textaware info from curl instead of perl +- configure: fix pkg-config detecting wolfssl - The MSYS system on Windows can run the test suite for curl built with - any toolset. When built with the MSYS GCC, curl uses Unix line endings, - while it uses Windows line endings when built with the MinGW GCC, and - `^O` reports 'msys' in both cases. Use the curl executable itself to - determine the line endings instead, which reports 'x86_64-pc-msys' when - built with the MSYS GCC. + When amending the include path with "/wolfssl", this now properly strips + off all whitespace from the path variable! Previously this would lead to + pkg-config builds creating bad command lines. - Closes https://github.com/curl/curl/pull/4506 + Closes #5848 -Daniel Stenberg (20 Oct 2019) -- [Michał Janiszewski brought this change] +- [Michael Musset brought this change] - appveyor: Add MSVC ARM64 build + sftp: add the option CURLKHSTAT_FINE_REPLACE - Closes #4507 - -- http2_recv: a closed stream trumps pause state + Replace the old fingerprint of the host with a new. - ... and thus should return 0, not EAGAIN. + Closes #5685 + +- RELEASE-NOTES: synced - Reported-by: Tom van der Woerdt - Fixes #4496 - Closes #4505 + The next release is now to become 7.73.0 -- http2: expire a timeout at end of stream +- checksrc: verify do-while and spaces between the braces - To make sure that transfer is being dealt with. Streams without - Content-Length need a final read to notice the end-of-stream state. + Updated mprintf.c to comply - Reported-by: Tom van der Woerdt - Fixes #4496 + Closes #5845 -Dan Fandrich (18 Oct 2019) -- travis: Add an ARM64 build +- curl: support XDG_CONFIG_HOME to find .curlrc + + Added test433 to verify. Updated documentation. - Test 323 is failing for some reason, so disable it there for now. + Reviewed-by: Jay Satiro + Suggested-by: Eli Schwartz + Fixes #5829 + Closes #5837 -Marcel Raad (18 Oct 2019) -- examples/sslbackend: fix -Wchar-subscripts warning +- etag: save and use the full received contents - With the `isdigit` implementation that comes with MSYS2, the argument - is used as an array subscript, resulting in a -Wchar-subscripts - warning. `isdigit`'s behavior is undefined if the argument is negative - and not EOF [0]. As done in lib/curl_ctype.h, cast the `char` variable - to `unsigned char` to avoid that. + ... which makes it support weak tags and non-standard etags too! - [0] https://en.cppreference.com/w/c/string/byte/isdigit + Added test case 347 to verify blank incoming ETag: - Closes https://github.com/curl/curl/pull/4503 + Fixes #5610 + Closes #5833 -Daniel Stenberg (18 Oct 2019) -- configure: remove all cyassl references +- setopt: if the buffer exists, refuse the new BUFFERSIZE - In particular, this removes the case where configure would find an old - cyall installation rather than a wolfssl one if present. The library is - named wolfssl in modern days so there's no real need to keep support for - the former. + The buffer only exists during transfer and then we shouldn't change the + size (the setopt is not documented to work then). - Reported-by: Jacob Barthelmeh - Closes #4502 + Reported-by: Harry Sintonen + Closes #5842 -Marcel Raad (17 Oct 2019) -- test1162: disable MSYS2's POSIX path conversion - - This avoids MSYS2 converting the backslasb in the URL to a slash, - causing the test to fail. +- [COFFEETALES brought this change] -Daniel Stenberg (17 Oct 2019) -- RELEASE-NOTES: synced - -Jay Satiro (16 Oct 2019) -- CURLOPT_TIMEOUT.3: Clarify transfer timeout time includes queue time - - Prior to this change some users did not understand that the "request" - starts when the handle is added to the multi handle, or probably they - did not understand that some of those transfers may be queued and that - time is included in timeout. + sftp: add new quote commands 'atime' and 'mtime' - Reported-by: Jeroen Ooms - - Fixes https://github.com/curl/curl/issues/4486 - Closes https://github.com/curl/curl/pull/4489 - -- [Stian Soiland-Reyes brought this change] + Closes #5810 - tool_operate: Fix retry sleep time shown to user when Retry-After +- CURLE_PROXY: new error code - - If server header Retry-After is being used for retry sleep time then - show that value to the user instead of the normal retry sleep time. + Failures clearly returned from a (SOCKS) proxy now causes this return + code. Previously the situation was not very clear as what would be + returned and when. - This is a follow-up to 640b973 (7.66.0) which changed curl tool so that - the value from Retry-After header overrides other retry timing options. + In addition: when this error code is returned, an application can use + CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then + returns a value from the new 'CURLproxycode' enum. - Closes https://github.com/curl/curl/pull/4498 + Closes #5770 -Daniel Stenberg (16 Oct 2019) -- url: normalize CURLINFO_EFFECTIVE_URL +- runtests: make cleardir() erase dot files too - The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as - input in most cases, which made it not get a scheme prefixed like before - if the URL was given without one, and it didn't remove dotdot sequences - etc. + Because test cases might use dot files. - Added test case 1907 to verify that this now works as intended and as - before 7.62.0. - - Regression introduced in 7.62.0 - - Reported-by: Christophe Dervieux - Fixes #4491 - Closes #4493 + Closes #5838 -Marcel Raad (16 Oct 2019) -- tests: line ending fixes for Windows +- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses - Mark some files as text. + Also: the current behavior is now documented in the curl.1 and + CURLOPT_NOPROXY.3 man pages. - Closes https://github.com/curl/curl/pull/4490 + Reported-by: Andrew Barnes + Closes #5745 + Closes #5841 -- tests: use proxy feature +Viktor Szakats (22 Aug 2020) +- Makefile.m32: add ability to override zstd libs [ci skip] - This makes the tests succeed when using --disable-proxy. + Similarly to brotli, where this was already possible. + E.g. it allows to link zstd statically to libcurl.dll. - Closes https://github.com/curl/curl/pull/4488 - -- smbserver: fix Python 3 compatibility + Ref: https://github.com/curl/curl-for-win/issues/12 + Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 - Python 2's `ConfigParser` module is spelled `configparser` in Python 3. + Closes https://github.com/curl/curl/pull/5840 + +Daniel Stenberg (21 Aug 2020) +- runtests: avoid 'fail to start' repeated messages in attempt loops - Closes https://github.com/curl/curl/pull/4484 + Closes #5834 -- security: silence conversion warning +- runtests: clear pid variables when failing to start a server - With MinGW-w64, `curl_socket_t` is is a 32 or 64 bit unsigned integer, - while `read` expects a 32 bit signed integer. - Use `sread` instead of `read` to use the correct parameter type. + ... as otherwise the parent doesn't detect the failure and believe it + actually worked to start. - Closes https://github.com/curl/curl/pull/4483 + Reported-by: Christian Weisgerber + Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html + Closes #5834 -- connect: silence sign-compare warning - - With MinGW-w64 using WinSock, `curl_socklen_t` is signed, while the - result of `sizeof` is unsigned. +- TODO: Virtual external sockets - Closes https://github.com/curl/curl/pull/4483 + Closes #5835 + +- [Don J Olmstead brought this change] -Daniel Stenberg (13 Oct 2019) -- TODO: Handle growing SFTP files + dist: add missing CMake Find modules to the distribution - Closes #4344 + Closes #5836 -- KNOWN_BUGS: remove "CURLFORM_CONTENTLEN in an array" +- RELEASE-NOTES: synced - The curl_formadd() function is deprecated and shouldn't be used so the - real fix for applications is to switch to the curl_mime_* API. + ... and version bumped to 7.72.1 -- KNOWN_BUGS: "LDAP on Windows does authentication wrong" +- tls: provide the CApath verbose log on its own line + + ... not newline separated from the previous line. This makes it output + asterisk prefixed properly like other verbose putput! - Closes #3116 + Reported-by: jmdavitt on github + Fixes #5826 + Closes #5827 -- appveyor: add a winbuild that uses VS2017 +Version 7.72.0 (19 Aug 2020) + +Daniel Stenberg (19 Aug 2020) +- RELEASE-NOTES: synced - Closes #4482 + The curl 7.72.0 release -- [Harry Sintonen brought this change] +- THANKS: add names from curl 7.72.0 release - socketpair: fix include and define for older TCP header systems +Jay Satiro (18 Aug 2020) +- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - fixed build for systems that need netinet/in.h for IPPROTO_TCP and are - missing INADDR_LOOPBACK + Reported-by: plujon@users.noreply.github.com - Closes #4480 + Closes https://github.com/curl/curl/issues/5488 -- socketpair: fix double-close in error case +Daniel Stenberg (17 Aug 2020) +- Curl_easy: remember last connection by id, not by pointer - Follow-up to bc2dbef0afc08 + CVE-2020-8231 + + Bug: https://curl.haxx.se/docs/CVE-2020-8231.html + + Reported-by: Marc Aldorasi + Closes #5824 + +- examples/rtsp.c: correct the copyright year -- gskit: use the generic Curl_socketpair +- RELEASE-PROCEDURE.md: add more future release dates -- asyn-thread: make use of Curl_socketpair() where available +- [H3RSKO brought this change] -- socketpair: an implemention for Windows and more + docs: change "web site" to "website" - Curl_socketpair() is designed to be used and work everywhere if there's - no native version or the native version isn't good enough. + According to wikipedia: - Closes #4466 + While "web site" was the original spelling, this variant has become + rarely used, and "website" has become the standard spelling + + Closes #5822 -- RELEASE-NOTES: synced +- [Bevan Weiss brought this change] -- connect: return CURLE_OPERATION_TIMEDOUT for errno == ETIMEDOUT + CMake: don't complain about missing nroff - Previosly all connect() failures would return CURLE_COULDNT_CONNECT, no - matter what errno said. + The curl_nroff_check() was always being called, and complaining if + *NROFF wasn't found, even when not making the manual. - This makes for example --retry work on these transfer failures. + Only check for nroff (and complain) if actually making the manual - Reported-by: Nathaniel J. Smith - Fixes #4461 - Clsoes #4462 + Closes #5817 -- cirrus: switch off blackhole status on the freebsd CI machines +- [Brian Inglis brought this change] -- tests: use port 2 instead of 60000 for a safer non-listening port + libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin - ... when the tests want "connection refused". - -- KNOWN_BUGS: IDN tests failing on Windows + copy the LDFLAGS approach for adding same option with `libhostname` in + `libtest/Makefile.am`: - Closes #3747 - -Dan Fandrich (9 Oct 2019) -- cirrus: Increase the git clone depth. + - init `libstubgss_la_LDFLAGS_EXTRA` variable, + - add option to variable inside conditional, + - use variable in `libstubgss_la_LDFLAGS` - If more commits are submitted to master between the time of triggering - the first Cirrus build and the time the final build gets started, the - desired commit is no longer at HEAD and the build will error out. - [skip ci] - -Daniel Stenberg (9 Oct 2019) -- docs: make sure the --no-progress-meter docs file is in dist too + Fixes #5819 + Closes #5820 -- docs: document it as --no-progress-meter instead of the reverse +- docs: clarify MAX_SEND/RECV_SPEED functionality - Follow-up to 93373a960c3bb4 + ... in particular what happens if the maximum speed limit is set to a + value that's smaller than the transfer buffer size in use. - Reported-by: infinnovation-dev on github - Fixes #4474 - Closes #4475 + Reported-by: Tomas Berger + Fixes #5788 + Closes #5813 -Dan Fandrich (9 Oct 2019) -- cirrus: Switch the FreeBSD 11.x build to 11.3 and add a 13.0 build. +- test1140: compare stdout - Also, select the images using image_family to get the latest snapshots - automatically. - [skip ci] - -Daniel Stenberg (8 Oct 2019) -- curl: --no-progress-meter + To make problems more immediately obvious when tests fail. - New option that allows a user to ONLY switch off curl's progress meter - and leave everything else in "talkative" mode. + Closes #5814 + +- asyn-ares: correct some bad comments - Reported-by: Piotr Komborski - Fixes #4422 - Closes #4470 + Closes #5812 -- TODO: Consult %APPDATA% also for .netrc +- [Emil Engler brought this change] + + docs: Add video link to docs/CONTRIBUTE.md - Closes #4016 + Closes #5811 -- CURLOPT_TIMEOUT.3: remove the mention of "minutes" +- curl-config: ignore REQUIRE_LIB_DEPS in --libs output - ... just say that limiting operations risk aborting otherwise fine - working transfers. If that means seconds, minutes or hours, we leave to - the user. + Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is + not considered for the --libs output. - Reported-by: Martin Gartner - Closes #4469 + Reported-by: ramsay-jones on github + Assisted-by: Brian Inglis and Ken Brown + Fixes #5793 + Closes #5808 -- [Andrei Valeriu BICA brought this change] +- copyright: update/correct the year range on a few files - docs: added multi-event.c example - - Similar to multi-uv.c but using libevent 2. This is a simpler libevent - integration example then hiperfifo.c. - - Closes #4471 +- scripts/copyright.pl: ignore .muse files -Jay Satiro (5 Oct 2019) -- [Nicolas brought this change] +- [Emil Engler brought this change] - ldap: fix OOM error on missing query string - - - Allow missing queries, don't return NO_MEMORY error in such a case. - - It is acceptable for there to be no specified query string, for example: - - curl ldap://ldap.forumsys.com + multi: Remove 10-year old out-commented code - A regression bug in 1b443a7 caused this issue. + The code hasn't been touched since 2010-08-18 - This is a partial fix for #4261. - - Bug: https://github.com/curl/curl/issues/4261#issuecomment-525543077 - Reported-by: Jojojov@users.noreply.github.com - Analyzed-by: Samuel Surtees - - Closes https://github.com/curl/curl/pull/4467 + Closes #5805 -- [Paul B. Omta brought this change] +- KNOWN_BUGS: A shared connection cache is not thread-safe + + Closes #4915 + Closes #5802 - build: Remove unused HAVE_LIBSSL and HAVE_LIBCRYPTO defines +- CONTRIBUTE: extend git commit message description - Closes https://github.com/curl/curl/pull/4460 + In particular how the first line works. + + Closes #5803 -Daniel Stenberg (5 Oct 2019) - RELEASE-NOTES: synced -- [Stian Soiland-Reyes brought this change] +- [Stefan Yohansson brought this change] - curl: ensure HTTP 429 triggers --retry - - This completes #3794. + transfer: move retrycount from connect struct to easy handle - Also make sure the new tests from #4195 are enabled + This flag was applied to the connection struct that is released on + retry. These changes move the retry counter into Curl_easy struct that + lives across retries and retains the new connection. - Closes #4465 + Reported-by: Cherish98 on github + Fixes #5794 + Closes #5800 -Marcel Raad (4 Oct 2019) -- [apique brought this change] - - winbuild: add ENABLE_UNICODE option +- libssh2: s/ssherr/sftperr/ - Fixes https://github.com/curl/curl/issues/4308 - Closes https://github.com/curl/curl/pull/4309 - -Daniel Stenberg (4 Oct 2019) -- ngtcp2: adapt to API change + The debug output used ssherr instead of sftperr which not only outputs + the wrong error code but also casues a warning on Windows. + + Follow-up to 7370b4e39f1 - Closes #4457 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 + Closes #5799 -- cookies: change argument type for Curl_flush_cookies +- ftp: don't do ssl_shutdown instead of ssl_close - The second argument is really a 'bool' so use that and pass in TRUE/FALSE - to make it clear. + The shutdown function is for downgrading a connection from TLS to plain, + and this is not requested here. - Closes #4455 - -- http2: move state-init from creation to pre-transfer + Have ssl_close reset the TLS connection state. - To make sure that the HTTP/2 state is initialized correctly for - duplicated handles. It would otherwise easily generate "spurious" - PRIORITY frames to get sent over HTTP/2 connections when duplicated easy - handles were used. + This partially reverts commit f002c850d98d - Reported-by: Daniel Silverstone - Fixes #4303 - Closes #4442 + Reported-by: Rasmus Melchior Jacobsen + Reported-by: Denis Goleshchikhin + Fixes #5797 -- urlapi: fix use-after-free bug +Marc Hoersken (9 Aug 2020) +- CI/azure: fix test outcome values and use latest API version - Follow-up from 2c20109a9b5d04 + This makes sure that tests ignored or skipped are not shown + just in the category "Other", but with their correct state. - Added test 663 to verify. + Closes #5796 + +- CI/azure: show runtime stats to investigate slowness - Reported by OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17954 + Also avoid naming conflict of TFLAGS env and tflags variables. - Closes #4453 - -- [Paul Dreik brought this change] + Closes #5776 - cookie: avoid harmless use after free +Daniel Stenberg (8 Aug 2020) +- TLS naming: fix more Winssl and Darwinssl leftovers - This fix removes a use after free which can be triggered by - the internal cookie fuzzer, but otherwise is probably - impossible to trigger from an ordinary application. + The CMake option is now called CMAKE_USE_SCHANNEL - The following program reproduces it: + The winbuild flag is USE_SCHANNEL - curl_global_init(CURL_GLOBAL_DEFAULT); - CURL* handle=curl_easy_init(); - CookieInfo* info=Curl_cookie_init(handle,NULL,NULL,false); - curl_easy_setopt(handle, CURLOPT_COOKIEJAR, "/dev/null"); - Curl_flush_cookies(handle, true); - Curl_cookie_cleanup(info); - curl_easy_cleanup(handle); - curl_global_cleanup(); + The CI jobs and build scripts only use the new names and the new name + options - This was found through fuzzing. + Tests now require 'Schannel' (when necessary) - Closes #4454 - -- [Denis Chaplygin brought this change] + Closes #5795 - docs: add note on failed handles not being counted by curl_multi_perform +- smtp_parse_address: handle blank input string properly - Closes #4446 + Closes #5792 -- CURLMOPT_MAX_CONCURRENT_STREAMS.3: fix SEE ALSO typo - -- [Niall O'Reilly brought this change] - - ESNI: initial build/setup +- runtests: run the DICT server on a random port number + + Removed support for -b (base port number) - Closes #4011 + Closes #5783 - RELEASE-NOTES: synced -- redirect: when following redirects to an absolute URL, URL encode it +- runtests: move the TELNET server to a dynamic port - ... to make it handle for example (RFC violating) embeded spaces. + Rename the port variable to TELNETPORT to better match the existing + pattern. - Reported-by: momala454 on github - Fixes #4445 - Closes #4447 + Closes #5785 -- urlapi: fix URL encoding when setting a full URL - -- tool_operate: rename functions to make more sense - -- curl: create easy handles on-demand and not ahead of time - - This should again enable crazy-large download ranges of the style - [1-10000000] that otherwise easily ran out of memory starting in 7.66.0 - when this new handle allocating scheme was introduced. +- ngtcp2: adapt to error code rename - Reported-by: Peter Sumatra - Fixes #4393 - Closes #4438 + Closes #5786 -- [Kunal Ekawde brought this change] - - CURLMOPT_MAX_CONCURRENT_STREAMS: new setopt +- runtests: move the smbserver to use a dynamic port number - Closes #4410 + Closes #5782 -- chunked-encoding: stop hiding the CURLE_BAD_CONTENT_ENCODING error - - Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the - response is chunked-encoded. +- runtests: run the http2 tests on a random port number - Reported-by: Ilya Kosarev - Fixes #4310 - Closes #4449 + Closes #5779 -Marcel Raad (1 Oct 2019) -- checksrc: fix uninitialized variable warning +- gtls: survive not being able to get name/issuer - The loop doesn't need to be executed without a file argument. - - Closes https://github.com/curl/curl/pull/4444 + Closes #5778 -- urlapi: fix unused variable warning +- runtests: move the gnutls-serv tests to a dynamic port - `dest` is only used with `ENABLE_IPV6`. + Affects test 320, 321, 322 and 324. - Closes https://github.com/curl/curl/pull/4444 + Closes #5778 -- lib: silence conversion warnings +- runtests: support dynamicly base64 encoded sections in tests - Closes https://github.com/curl/curl/pull/4444 - -- AppVeyor: add 32-bit MinGW-w64 build + This allows us to make test cases to use base64 at run-time and still + use and verify information determined at run-time, such as the IMAP test + server's port number in test 842. + + This change makes 12 tests run again that basically never ran since we + moved to dynamic port numbers. - With WinSSL and testing enabled so that it would have detected most of - the warnings fixed in [0] and [1]. + ftpserver.pl is adjusted to load test instructions and test number from + the preprocessed test file. - [0] https://github.com/curl/curl/pull/4398 - [1] https://github.com/curl/curl/pull/4415 + FILEFORMAT.md now documents the new base64 encoding syntax. - Closes https://github.com/curl/curl/pull/4433 + Reported-by: Marcel Raad + Fixes #5761 + Closes #5775 -- AppVeyor: remove MSYS2_ARG_CONV_EXCL for winbuild +- curl.1: add a few missing valid exit codes - It's only used for MSYS2 with MinGW. + 93 - 96 can be returned as well. - Closes - -Daniel Stenberg (30 Sep 2019) -- [Emil Engler brought this change] + Closes #5777 - git: add tests/server/disabled to .gitignore +- TODO: Use multiple parallel transfers for a single download - Closes #4441 + Closes #5774 -- altsvc: accept quoted ma and persist values - - As mandated by the spec. Test 1654 is extended to verify. +- TODO: Set the modification date on an uploaded file - Closes #4443 - -- mailmap: a Lucas fix - -Alessandro Ghedini (29 Sep 2019) -- [Lucas Pardue brought this change] - - quiche: update HTTP/3 config creation to new API + Closes #5768 -Daniel Stenberg (29 Sep 2019) -- BINDINGS: PureBasic, Net::Curl for perl and Nim +- [Thomas M. DuBuisson brought this change] -- BINDINGS: Kapito is an Erlang library, basically a binding - -- BINDINGS: added clj-curl + CI: Add muse CI config - Reported-by: Lucas Severo + Closes #5772 -- [Jay Satiro brought this change] +- [Thomas M. DuBuisson brought this change] - docs: disambiguate CURLUPART_HOST is for host name (ie no port) + travis/script.sh: fix use of `-n' with unquoted envvar - Closes #4424 - -- cookies: using a share with cookies shouldn't enable the cookie engine + Shellcheck tells us "-n doesn't work with unquoted arguments. quote or + use [[ ]]." - The 'share object' only sets the storage area for cookies. The "cookie - engine" still needs to be enabled or activated using the normal cookie - options. + And testing shows: - This caused the curl command line tool to accidentally use cookies - without having been told to, since curl switched to using shared cookies - in 7.66.0. + ``` + docker run --rm -it ubuntu bash + root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" + I ran + root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" + root@fe85ce156856:/# + ``` - Test 1166 verifies + Closes #5773 + +- h2: repair trailer handling - Updated test 506 + The previous h2 trailer fix in 54a2b63 was wrong and caused a + regression: it cannot deal with trailers immediately when read since + they may be read off the connection by the wrong 'data' owner. - Fixes #4429 - Closes #4434 - -- setopt: handle ALTSVC set to NULL - -- RELEASE-NOTES: synced - -- [grdowns brought this change] - - INSTALL: add vcpkg installation instructions + This change reverts the logic back to gathering all trailers into a + single buffer, like before 54a2b63. - Closes #4435 + Reported-by: Tadej Vengust + Fixes #5663 + Closes #5769 -- [Zenju brought this change] - - FTP: add test for FTPFILE_NOCWD: Avoid redundant CWDs +Viktor Szakats (3 Aug 2020) +- windows: disable Unix Sockets for old mingw - Add libtest 661 + Classic mingw and 10y+ old versions of mingw-w64 don't ship with + Windows headers having the typedef necessary for Unix Sockets + support, so try detecting these environments to disable this + feature. - Closes #4417 - -- [Zenju brought this change] - - FTP: url-decode path before evaluation + Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ + + Reviewed-by: Daniel Stenberg - Closes #4428 + Fixes #5674 + Closes #5758 -Marcel Raad (27 Sep 2019) -- tests: fix narrowing conversion warnings +Marcel Raad (3 Aug 2020) +- test1908: treat file as text - `timediff_t` is 64 bits wide also on 32-bit systems since - commit b1616dad8f0. + Fixes the line endings on Windows. - Closes https://github.com/curl/curl/pull/4415 + Closes https://github.com/curl/curl/pull/5767 -Jay Satiro (27 Sep 2019) -- [julian brought this change] - - vtls: Fix comment typo about macosx-version-min compiler flag +- TrackMemory tests: ignore realloc and free in getenv.c - Closes https://github.com/curl/curl/pull/4425 - -Daniel Stenberg (26 Sep 2019) -- [Yechiel Kalmenson brought this change] - - README: minor grammar fix + These are only called for WIN32. - Closes #4431 + Closes https://github.com/curl/curl/pull/5767 -- [Spezifant brought this change] - - HTTP3: fix prefix parameter for ngtcp2 build - - Closes #4430 +Daniel Stenberg (3 Aug 2020) +- tests/FILEFORMAT.md: mention %HTTP2PORT -- quiche: don't close connection at end of stream! +- RELEASE-NOTES: synced -- quiche: set 'drain' when returning without having drained the queues +- tlsv1.3.d. only for TLS-using connections + + ... and rephrase that "not all" TLS backends support it. + + Closes #5764 -- Revert "FTP: url-decode path before evaluation" +- tls-max.d: this option is only for TLS-using connections - This reverts commit 2f036a72d543e96128bd75cb0fedd88815fd42e2. + Ref: #5763 + Closes #5764 -- HTTP3: merged and simplified the two 'running' sections +Marcel Raad (2 Aug 2020) +- [Cameron Cawley brought this change] -- HTTP3: show an --alt-svc using example too + tool_doswin: Simplify Windows version detection + + Closes https://github.com/curl/curl/pull/5754 -- [Zenju brought this change] +- [Cameron Cawley brought this change] - FTP: url-decode path before evaluation + win32: Add Curl_verify_windows_version() to curlx - Closes #4423 + Closes https://github.com/curl/curl/pull/5754 -- openssl: use strerror on SSL_ERROR_SYSCALL +- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL - Instead of showing the somewhat nonsensical errno number, use strerror() - to provide a more relatable error message. + This makes the tests that require the OpenSSL feature also run for + those two compatible libraries. - Closes #4411 + Closes https://github.com/curl/curl/pull/5762 -- HTTP3: update quic.aiortc.org + add link to server list +Daniel Stenberg (1 Aug 2020) +- multi: Condition 'extrawait' is always true + + Reported by Codacy. - Reported-by: Jeremy Lainé + Reviewed-by: Marcel Raad + Closes #5759 -Jay Satiro (26 Sep 2019) -- url: don't set appconnect time for non-ssl/non-ssh connections +Marcel Raad (1 Aug 2020) +- openssl: fix build with LibreSSL < 2.9.1 - Prior to this change non-ssl/non-ssh connections that were reused set - TIMER_APPCONNECT [1]. Arguably that was incorrect since no SSL/SSH - handshake took place. + `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were + introduced in LibreSSL 2.9.1 [0]. - [1]: TIMER_APPCONNECT is publicly known as CURLINFO_APPCONNECT_TIME in - libcurl and %{time_appconnect} in the curl tool. It is documented as - "the time until the SSL/SSH handshake is completed". + [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef - Reported-by: Marcel Hernandez + Closes https://github.com/curl/curl/pull/5757 + +Daniel Stenberg (1 Aug 2020) +- [Marc Aldorasi brought this change] + + multi_remove_handle: close unused connect-only connections - Ref: https://github.com/curl/curl/issues/3760 + Previously any connect-only connections in a multi handle would be kept + alive until the multi handle was closed. Since these connections cannot + be re-used, they can be marked for closure when the associated easy + handle is removed from the multi handle. - Closes https://github.com/curl/curl/pull/3773 + Closes #5749 -Daniel Stenberg (25 Sep 2019) -- ngtcp2: remove fprintf() calls +- checksrc: invoke script with -D to find .checksrc proper - - convert some of them to H3BUF() calls to infof() - - remove some of them completely - - made DEBUG_HTTP3 defined only if CURLDEBUG is set for now + Without the -D command line option, checksrc.pl won't know which + directory to load the ".checksrc" file from when building out of the + source tree. - Closes #4421 + Reported-by: Marcel Raad + Fixes #5715 + Closes #5755 -- [Jay Satiro brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - url: fix the NULL hostname compiler warning case + buildconf: retire ares buildconf invocation - Closes #4403 + no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 -- [Jay Satiro brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - travis: move the go install to linux-only + buildconf: excempt defunct reference to ACLOCAL_FLAGS - ... to repair the build again - Closes #4403 + retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as + the name is generic enough that it might be in use and relied upon from + the environment. -- altsvc: correct the #ifdef for the ngtcp2 backend +- [Carlo Marcelo Arenas Belón brought this change] -- altsvc: save h3 as h3-23 + buildconf: avoid array concatenation in die() + + reported as error SC2145[1] by shellcheck, but not expected to cause + any behavioural differences otherwise. + + [1] https://github.com/koalaman/shellcheck/wiki/SC2145 - Follow-up to d176a2c7e5 + Closes #5701 -- urlapi: question mark within fragment is still fragment +- travis: add ppc64le and s390x builds - The parser would check for a query part before fragment, which caused it - to do wrong when the fragment contains a question mark. + Closes #5752 + +Marc Hoersken (31 Jul 2020) +- connect: remove redundant message about connect failure - Extended test 1560 to verify. + Reviewed-by: Daniel Stenberg - Reported-by: Alex Konev - Fixes #4412 - Closes #4413 + Closes #5708 -- [Alex Samorukov brought this change] +- tests/sshserver.pl: fix compatibility with OpenSSH for Windows + + Follow up to #5721 - HTTP3.md: move -p for mkdir, remove -j for make +- CI/azure: install libssh2 for use with msys2-based builds - - mkdir on OSX/Darwin requires `-p` argument before dir + This enables building and running the SFTP tests. + Unfortunately OpenSSH for Windows does not support SCP (yet). - - portabbly figuring out number of cores is an exercise for somewhere - else + Reviewed-by: Daniel Stenberg - Closes #4407 + Closes #5721 -Patrick Monnerat (24 Sep 2019) -- os400: getpeername() and getsockname() return ebcdic AF_UNIX sockaddr, +- CI/azure: increase Windows job timeout once again - As libcurl now uses these 2 system functions, wrappers are needed on os400 - to convert returned AF_UNIX sockaddrs to ascii. + Avoid aborted jobs due to performance issues on Azure DevOps. - This is a follow-up to commit 7fb54ef. - See also #4037. - Closes #4214 - -Jay Satiro (24 Sep 2019) -- [Lucas Pardue brought this change] + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + + Closes #5738 - strcase: fix raw lowercasing the letter X +Jay Satiro (30 Jul 2020) +- TODO: Schannel: 'Add option to allow abrupt server closure' - Casing mistake in Curl_raw_tolower 'X' wasn't lowercased as 'x' prior to - this change. + We should offer an option to allow abrupt server closures (server closes + SSL transfer without sending a known termination point such as length of + transfer or close_notify alert). Abrupt server closures are usually + because of misconfigured or very old servers. - Follow-up to 0023fce which added the function several days ago. + Closes https://github.com/curl/curl/issues/4427 + +- url: fix CURLU and location following - Ref: https://github.com/curl/curl/pull/4401#discussion_r327396546 + Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was + incorrectly used for the location follow, resulting in infinite requests + to the original location. - Closes https://github.com/curl/curl/pull/4408 - -Daniel Stenberg (23 Sep 2019) -- http2: Expression 'stream->stream_id != - 1' is always true + Reported-by: sspiri@users.noreply.github.com - PVS-Studio warning - Fixes #4402 + Fixes https://github.com/curl/curl/issues/5709 + Closes https://github.com/curl/curl/pull/5713 -- http2: A value is being subtracted from the unsigned variable - - PVS-Studio warning - Fixes #4402 +Daniel Stenberg (30 Jul 2020) +- RELEASE-NOTES: synced -- libssh: part of conditional expression is always true: !result - - PVS-Studio warning - Fixed #4402 +- [divinity76 brought this change] -- libssh: part of conditional expression is always true + docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - PVS-Studio warning - Fixes #4402 - -- libssh: The expression is excessive or contains a misprint + it helps make it obvious that most developers don't have to care about + the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 + years old, November 4 2009) - PVS-Studio warning - Fixes #4402 + Closes #5744 -- quiche: The expression must be surrounded by parentheses +Jay Satiro (29 Jul 2020) +- tool_cb_wrt: fix outfile mode flags for Windows - PVS-Studio warning - Fixes #4402 - -- vauth: The parameter 'status' must be surrounded by parentheses + - Use S_IREAD and S_IWRITE mode permission flags to create the file + on Windows instead of S_IRUSR, S_IWUSR, etc. - PVS-Studio warning - Fixes #4402 - -- [Paul Dreik brought this change] - - doh: allow only http and https in debug mode + Windows only accepts a combination of S_IREAD and S_IWRITE. It does not + acknowledge other combinations, for which it may generate an assertion. - Otherwise curl may be told to use for instance pop3 to - communicate with the doh server, which most likely - is not what you want. + This is a follow-up to 81b4e99 from yesterday, which improved the + existing file check with -J. - Found through fuzzing. + Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks + Ref: https://github.com/curl/curl/pull/5731 - Closes #4406 - -- [Paul Dreik brought this change] + Closes https://github.com/curl/curl/pull/5742 - doh: return early if there is no time left +Daniel Stenberg (28 Jul 2020) +- checksrc: ban gmtime/localtime - Closes #4406 - -- [Barry Pollard brought this change] + They're not thread-safe so they should not be used in libcurl code. + + Explictly enabled when deemed necessary and in examples and tests + + Reviewed-by: Nicolas Sterchele + Closes #5732 - http: lowercase headernames for HTTP/2 and HTTP/3 +- transfer: fix data_pending for builds with both h2 and h3 enabled - Closes #4401 - Fixes #4400 + Closes #5734 -Marcel Raad (23 Sep 2019) -- vtls: fix narrowing conversion warnings +- curl_multi_setopt: fix compiler warning "result is always false" - Curl_timeleft returns `timediff_t`, which is 64 bits wide also on - 32-bit systems since commit b1616dad8f0. + On systems with 32 bit long the expression is always false. Avoid + the warning. - Closes https://github.com/curl/curl/pull/4398 - -Daniel Stenberg (23 Sep 2019) -- [Joel Depooter brought this change] + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 + Closes #5736 - winbuild: Add manifest to curl.exe for proper OS version detection +- curl: improve the existing file check with -J - This is a small fix to commit ebd213270a017a6830928ee2e1f4a9cabc799898 - in pull request #1221. That commit added the CURL_EMBED_MANIFEST flag to - CURL_RC_FLAGS. However, later in the file CURL_RC_FLAGS is - overwritten. The fix is to append values to CURL_RC_FLAGS instead of - overwriting + Previously a file that isn't user-readable but is user-writable would + not be properly avoided and would get overwritten. - Closes #4399 + Reported-by: BrumBrum on hackerone + Assisted-by: Jay Satiro + Bug: https://hackerone.com/reports/926638 + Closes #5731 -- RELEASE-NOTES: synced +- [Jonathan Nieder brought this change] -Marcel Raad (22 Sep 2019) -- openssl: fix compiler warning with LibreSSL - - It was already fixed for BoringSSL in commit a0f8fccb1e0. - LibreSSL has had the second argument to SSL_CTX_set_min_proto_version - as uint16_t ever since the function was added in [0]. + multi: update comment to say easyp list is linear - [0] https://github.com/libressl-portable/openbsd/commit/56f107201baefb5533486d665a58d8f57fd3aeda + Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, + 2013-08-02), the easy handle list is not circular but ends with + ->next pointing to NULL. - Closes https://github.com/curl/curl/pull/4397 + Reported-by: Masaya Suzuki <masayasuzuki@google.com> + Closes #5737 -Daniel Stenberg (22 Sep 2019) -- curl: exit the create_transfers loop on errors +- CURLOPT_NOBODY.3: fix the syntax for referring to options - When looping around the ranges and given URLs to create transfers, all - errors should exit the loop and return. Previously it would keep - looping. + As test 1140 fails otherwise! - Reported-by: SumatraPeter on github - Bug: #4393 - Closes #4396 + Follow-up to e1bac81cc815 -Jay Satiro (21 Sep 2019) -- socks: Fix destination host shown on SOCKS5 error +- ngtcp2: store address in sockaddr_storage - Prior to this change when a server returned a socks5 connect error then - curl would parse the destination address:port from that data and show it - to the user as the destination: - - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.12.206 (locally resolved) - * Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) - curl: (7) Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) - - That's incorrect because the address:port included in the connect error - is actually a bind address:port (typically unused) and not the - destination address:port. This fix changes curl to show the destination - information that curl sent to the server instead: + Reported-by: Tatsuhiro Tsujikawa + Closes #5733 + +- CURLOPT_NOBODY.3: clarify what setting to 0 means - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.7.14:99 (locally resolved) - * Can't complete SOCKS5 connection to 172.217.7.14:99. (1) - curl: (7) Can't complete SOCKS5 connection to 172.217.7.14:99. (1) + ... and mention that HTTP with other methods than HEAD might get a body and + there's no option available to stop that. - curld -v --socks5-hostname 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to google.com:99 (remotely resolved) - * Can't complete SOCKS5 connection to google.com:99. (1) - curl: (7) Can't complete SOCKS5 connection to google.com:99. (1) + Closes #5729 + +- setopt: unset NOBODY switches to GET if still HEAD - Ref: https://tools.ietf.org/html/rfc1928#section-6 + Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented + action but before 7.71.0 that used to switch back to GET and with this + change (assuming the method is still set to HEAD) this behavior is + brought back. - Closes https://github.com/curl/curl/pull/4394 + Reported-by: causal-agent on github + Fixes #5725 + Closes #5728 -Daniel Stenberg (21 Sep 2019) -- travis: enable ngtcp2 h3-23 builds +- [Ehren Bendler brought this change] -- altsvc: both backends run h3-23 now + configure: cleanup wolfssl + pkg-config conflicts when cross compiling. - Closes #4395 - -- http: fix warning on conversion from int to bit + Also choose a different wolfSSL function to test for NTLM support. - Follow-up from 03ebe66d70 + Fixes #5605 + Closes #5682 -- urldata: use 'bool' for the bit type on MSVC compilers +- configure: show zstd "no" in summary when built without it - Closes #4387 - Fixes #4379 + Reported-by: Marc Hörsken + Fixes #5720 + Closes #5730 -- appveyor: upgrade VS2017 to VS2019 +- quiche: handle calling disconnect twice - Closes #4383 + Reported-by: lilongyan-huawei on github + Fixes #5726 + Closes #5727 -- [Zenju brought this change] +- [Nicolas Sterchele brought this change] - FTP: FTPFILE_NOCWD: avoid redundant CWDs + getinfo: reset retry-after value in initinfo + + - Avoid re-using retry_after value from preceding request + - Add libtest 3010 to verify - Closes #4382 + Reported-by: joey-l-us on github + Fixes #5661 + Closes #5672 -- cookie: pass in the correct cookie amount to qsort() +Marcel Raad (27 Jul 2020) +- WIN32: stop forcing narrow-character API - As the loop discards cookies without domain set. This bug would lead to - qsort() trying to sort uninitialized pointers. We have however not found - it a security problem. + Except where the results are only used for character output. + getenv is not touched because it's part of the public API, and having + it return UTF-8 instead of ANSI would be a breaking change. - Reported-by: Paul Dreik - Closes #4386 + Fixes https://github.com/curl/curl/issues/5658 + Fixes https://github.com/curl/curl/issues/5712 + Closes https://github.com/curl/curl/pull/5718 -- [Paul Dreik brought this change] +Jay Satiro (27 Jul 2020) +- [Tobias Stoeckmann brought this change] - urlapi: avoid index underflow for short ipv6 hostnames + mprintf: Fix stack overflows - If the input hostname is "[", hlen will underflow to max of size_t when - it is subtracted with 2. + Stack overflows can occur with precisions for integers and floats. - hostname[hlen] will then cause a warning by ubsanitizer: + Proof of concepts: + - curl_mprintf("%d, %.*1$d", 500, 1); + - curl_mprintf("%d, %+0500.*1$f", 500, 1); - runtime error: addition of unsigned offset to 0x<snip> overflowed to - 0x<snip> + Ideally, compile with -fsanitize=address which makes this undefined + behavior a bit more defined for debug purposes. - I think that in practice, the generated code will work, and the output - of hostname[hlen] will be the first character "[". + The format strings are valid. The overflows occur due to invalid + arguments. If these arguments are variables with contents controlled + by an attacker, the function's stack can be corrupted. - This can be demonstrated by the following program (tested in both clang - and gcc, with -O3) + Also see CVE-2016-9586 which partially fixed the float aspect. - int main() { - char* hostname=strdup("["); - size_t hlen = strlen(hostname); + Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org> - hlen-=2; - hostname++; - printf("character is %d\n",+hostname[hlen]); - free(hostname-1); - } - - I found this through fuzzing, and even if it seems harmless, the proper - thing is to return early with an error. - - Closes #4389 + Closes https://github.com/curl/curl/pull/5722 -- [Tatsuhiro Tsujikawa brought this change] +- [Tobias Stoeckmann brought this change] - ngtcp2: compile with latest ngtcp2 + nghttp3 draft-23 + mprintf: Fix dollar string handling - Closes #4392 - -- THANKS-filter: deal with my typos 'Jat' => 'Jay' - -- travis: use go master + Verify that specified parameters are in range. If parameters are too + large, fail early on and avoid out of boundary accesses. - ... as the boringssl builds needs a very recent version + Also do not read behind boundaries of illegal format strings. - Co-authored-by: Jat Satiro - Closes #4361 - -- tool_operate: removed unused variable 'done' + These are defensive measures since it is expected that format strings + are well-formed. Format strings should not be modifiable by user + input due to possible generic format string attacks. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5722 -- tool_operate: Expression 'config->resume_from' is always true +Daniel Stenberg (26 Jul 2020) +- ntlm: free target_info before (re-)malloc - Fixes warning detected by PVS-Studio - Fixes #4374 - -- tool_getparam: remove duplicate switch case + OSS-Fuzz found a way this could get called again with the pointer still + pointing to a malloc'ed memory, leading to a leak. - Fixes warning detected by PVS-Studio - Fixes #4374 - -- libssh2: part of conditional expression is always true: !result + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #5724 -- urlapi: Expression 'storep' is always true +Marcel Raad (26 Jul 2020) +- CI/macos: set minimum macOS version - Fixes warning detected by PVS-Studio - Fixes #4374 - -- urlapi: 'scheme' is always true + This enables some deprecation warnings. + Previously, autotools defaulted to 10.8. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5723 -- urlapi: part of conditional expression is always true: (relurl[0] == '/') - - Fixes warning detected by PVS-Studio - Fixes #4374 +Daniel Stenberg (26 Jul 2020) +- RELEASE-NOTES: synced -- setopt: store CURLOPT_RTSP_SERVER_CSEQ correctly +Marcel Raad (25 Jul 2020) +- CI/macos: enable warnings as errors for CMake builds - Fixes bug detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5716 -- mime: make Curl_mime_duppart() assert if called without valid dst +- CMake: fix test for warning suppressions - Fixes warning detected by PVS-Studio - Fixes #4374 - -- http_proxy: part of conditional expression is always true: !error + GCC doesn't warn for unknown `-Wno-` options, except if there are other + warnings or errors [0]. This was problematic with `CURL_WERROR` as that + warning-as-error cannot be suppressed. Notably, this always happened + with `-Wno-pedantic-ms-format` when not targeting Windows. So test for + the positive form of the warning instead, which should always result in + a diagnostic if unknown. - Fixes warning detected by PVS-Studio - Fixes #4374 - -- imap: merged two case-branches performing the same action + [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5714 -- multi: value '2L' is assigned to a boolean +Jay Satiro (23 Jul 2020) +- curl.h: update CURLINFO_LASTONE - Fixes warning detected by PVS-Studio - Fixes #4374 - -- easy: part of conditional expression is always true: !result + CURLINFO_LASTONE should have been updated when + CURLINFO_EFFECTIVE_METHOD was added. - Fixes warning detected by PVS-Studio - Fixes #4374 - -- netrc: part of conditional expression is always true: !done + Reported-by: xwxbug@users.noreply.github.com - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes https://github.com/curl/curl/issues/5711 -- version: Expression 'left > 1' is always true +Marc Hoersken (22 Jul 2020) +- CI/azure: unconditionally enable warnings-as-errors with autotools - Fixes warning detected by PVS-Studio - Fixes #4374 - -- url: remove dead code + Reviewed-by: Marcel Raad - Fixes warning detected by PVS-Studio - Fixes #4374 + Follow up to #5694 + Closes #5706 -- url: part of expression is always true: (bundle->multiuse == 0) +Marcel Raad (21 Jul 2020) +- doh: remove redundant cast - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5704 -- ftp: the conditional expression is always true +- CI/macos: unconditionally enable warnings-as-errors with autotools - ... both !result and (ftp->transfer != FTPTRANSFER_BODY)! + Previously, warnings were only visible in the output for most jobs. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5694 -- ftp: Expression 'ftpc->wait_data_conn' is always false +- util: silence conversion warnings - Fixes warning detected by PVS-Studio - Fixes #4374 - -- ftp: Expression 'ftpc->wait_data_conn' is always true + timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might + be a 64-bit integer. This is the case when building for recent macOS + versions, for example. Just treat tv_usec as an int, which should + hopefully always be sufficient on systems with + `HAVE_CLOCK_GETTIME_MONOTONIC`. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/5695 -- ftp: part of conditional expression is always true: !result +- md(4|5): don't use deprecated macOS functions - Fixes warning detected by PVS-Studio - Fixes #4374 - -- http: fix Expression 'http->postdata' is always false + They are marked as deprecated for -mmacosx-version-min >= 10.15, + which might result in warnings-as-errors. - Fixes warning detected by PVS-Studio - Fixes #4374 - Reported-by: Valerii Zapodovnikov + Closes https://github.com/curl/curl/pull/5695 -- [Niall O'Reilly brought this change] - - doh: avoid truncating DNS QTYPE to lower octet +Daniel Stenberg (18 Jul 2020) +- strdup: remove the odd strlen check - Closes #4381 - -- [Jens Finkhaeuser brought this change] - - urlapi: CURLU_NO_AUTHORITY allows empty authority/host part + It confuses code analyzers with its use of -1 for unsigned value. Also, + a check that's not normally used in strdup() code - and not necessary. - CURLU_NO_AUTHORITY is intended for use with unknown schemes (i.e. not - "file:///") to override cURL's default demand that an authority exists. - - Closes #4349 - -- version: next release will be 7.67.0 + Closes #5697 -- RELEASE-NOTES: synced +- [Alessandro Ghedini brought this change] -- url: only reuse TLS connections with matching pinning + travis: update quiche builds for new boringssl layout - If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the - connection should not be reused. + This is required after https://github.com/cloudflare/quiche/pull/593 + moved BoringSSL around slightly. - Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html - Reported-by: Sebastian Haglund + This also means that Go is not needed to build BoringSSL anymore (the + one provided by quiche anyway). - Closes #4347 + Closes #5691 -- README: add OSS-Fuzz badge [skip ci] +Marcel Raad (17 Jul 2020) +- configure: allow disabling warnings - Closes #4380 - -Michael Kaufmann (18 Sep 2019) -- http: merge two "case" statements + When using `--enable-warnings`, it was not possible to disable warnings + via CFLAGS that got explicitly enabled. Now warnings are not enabled + anymore if they are explicitly disabled (or enabled) in CFLAGS. This + works for at least GCC, clang, and TCC as they have corresponding + `-Wno-` options for every warning. + + Closes https://github.com/curl/curl/pull/5689 -Daniel Stenberg (18 Sep 2019) -- [Zenju brought this change] +Daniel Stenberg (16 Jul 2020) +- ngtcp2: adjust to recent sockaddr updates + + Closes #5690 - FTP: remove trailing slash from path for LIST/MLSD +- page-header: provide protocol details in the curl.1 man page + + Add protocol and version specific information about all protocols curl + supports. - Closes #4348 + Fixes #5679 + Reported-by: tbugfinder on github + Closes #5686 -- mime: when disabled, avoid C99 macro +Daniel Gustafsson (16 Jul 2020) +- docs: Update a few leftover mentions of DarwinSSL - Closes #4368 + Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the + more correct/common name Secure Transport, but a few mentions in the docs + remained. + + Closes #5688 + Reviewed-by: Daniel Stenberg <daniel@haxx.se> -- url: cleanup dangling DOH request headers too +Daniel Stenberg (16 Jul 2020) +- file2memory: use a define instead of -1 unsigned value - Follow-up to 9bc44ff64d9081 + ... to use the maximum value for 'size_t' when detecting integer overflow. + Changed the limit to max/4 as already that seems unreasonably large. - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17269 + Codacy didn't like the previous approach. - Closes #4372 + Closes #5683 -- [Christoph M. Becker brought this change] - - http2: relax verification of :authority in push promise requests +- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream - If the :authority pseudo header field doesn't contain an explicit port, - we assume it is valid for the default port, instead of rejecting the - request for all ports. + ... by adding support for a new dedicated return code. - Ref: https://curl.haxx.se/mail/lib-2019-09/0041.html + Suggested-by: Jonathan Cardoso + Assisted-by: Erik Johansson + URL: https://curl.haxx.se/mail/lib-2020-06/0099.html + Closes #5636 + +- [Baruch Siach brought this change] + + nss: fix build with disabled proxy support - Closes #4365 + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. + + Closes #5667 + +- test1139: make it display the difference on test failures -- doh: clean up dangling DOH handles and memory on easy close +- test1119: verify stdout in the test - If you set the same URL for target as for DoH (and it isn't a DoH - server), like "https://example.com" in both, the easy handles used for - the DoH requests could be left "dangling" and end up not getting freed. + So that failures will be displayed in the terminal, as it makes test failures + visually displayed easier and faster. - Reported-by: Paul Dreik - Closes #4366 + Closes #5644 -- unit1655: make it C90 compliant +- curl: add %{method} to the -w variables - Unclear why this was not detected in the CI. + Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. - Follow-up to b7666027296a + Added test 1197 to verify. -- smb: check for full size message before reading message details +- CURLINFO_EFFECTIVE_METHOD: added - To avoid reading of uninitialized data. + Provide the HTTP method that was used on the latest request, which might + be relevant for users when there was one or more redirects involved. - Assisted-by: Max Dymond - Bug: https://crbug.com/oss-fuzz/16907 - Closes #4363 + Closes #5511 -- quiche: persist connection details +Viktor Szakats (14 Jul 2020) +- windows: add unicode to feature list - ... like we do for other protocols at connect time. This makes "curl -I" - and other things work. + Reviewed-by: Marcel Raad + Reviewed-by: Marc Hörsken - Reported-by: George Liu - Fixes #4358 - Closes #4360 + Closes #5491 -- openssl: fix warning with boringssl and SSL_CTX_set_min_proto_version +Daniel Stenberg (14 Jul 2020) +- multi: remove two checks always true - Follow-up to ffe34b7b59 - Closes #4359 + Detected by Codacy + Closes #5676 -- [Paul Dreik brought this change] +Marc Hoersken (13 Jul 2020) +- workflows: limit what branches to run CodeQL on + + Align CodeQL action with existing CI actions: + - Update branch filter to avoid duplicate CI runs. + - Shorten workflow name due to informative job name. + + Reviewed-by: Daniel Stenberg + + Closes #5660 - doh: fix undefined behaviour and open up for gcc and clang optimization +- appveyor: collect libcurl.dll variants with prefix or suffix - The undefined behaviour is annoying when running fuzzing with - sanitizers. The codegen is the same, but the meaning is now not up for - dispute. See https://cppinsights.io/s/516a2ff4 + On some platforms libcurl is build with a platform-specific + prefix and/or a version number suffix. - By incrementing the pointer first, both gcc and clang recognize this as - a bswap and optimizes it to a single instruction. See - https://godbolt.org/z/994Zpx + Assisted-by: Jay Satiro - Closes #4350 + Closes #5659 -- [Paul Dreik brought this change] +Daniel Stenberg (12 Jul 2020) +- [ihsinme brought this change] - doh: fix (harmless) buffer overrun + socks: use size_t for size variable - Added unit test case 1655 to verify. - Close #4352 + Use the unsigned type (size_t) in the arithmetic of pointers. In this + context, the signed type (ssize_t) is used unnecessarily. - the code correctly finds the flaws in the old code, - if one temporarily restores doh.c to the old version. + Authored-by: ihsinme on github + Closes #5654 -Alessandro Ghedini (15 Sep 2019) -- docs: remove trailing ':' from section names in CURLOPT_TRAILER* man - -- docs: fix typo in CURLOPT_HTTP_VERSION man +- RELEASE-NOTES: synced + + ... and bumped to 7.72.0 as the next release version number -GitHub (14 Sep 2019) -- [Daniel Stenberg brought this change] +- [Gilles Vollant brought this change] - CI: inintial github action job + content_encoding: add zstd decoding support - First shot at a CI build on github actions + include zstd curl patch for Makefile.m32 from vszakats + and include Add CMake support for zstd from Peter Wu + + Helped-by: Viktor Szakats + Helped-by: Peter Wu + Closes #5453 -Daniel Stenberg (13 Sep 2019) -- appveyor: add a winbuild +- asyn.h: remove the Curl_resolver_getsock define - Assisted-by: Marcel Raad - Assisted-by: Jay Satiro + - not used + - used the wrong number of arguments + - confused the Codeacy code analyzer - Closes #4324 + Closes #5647 -- FTP: allow "rubbish" prepended to the SIZE response - - This is a protocol violation but apparently there are legacy proprietary - servers doing this. +- [Nicolas Sterchele brought this change] + + configure.ac: Sort features name in summary - Added test 336 and 337 to verify. + - Same as protocols - Reported-by: Philippe Marguinaud - Closes #4339 + Closes #5656 -- [Zenju brought this change] +- [Matthias Naegler brought this change] - FTP: skip CWD to entry dir when target is absolute + cmake: fix windows xp build - Closes #4332 + Reviewed-by: Marcel Raad + Closes #5662 -Kamil Dudka (13 Sep 2019) -- curl: fix memory leaked by parse_metalink() +- ngtcp2: update to modified qlog callback prototype - This commit fixes a regression introduced by curl-7_65_3-5-gb88940850. - Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind - and libmetalink enabled. - - Closes #4326 + Closes #5675 -Daniel Stenberg (13 Sep 2019) -- parsedate: still provide the name arrays when disabled +- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle - If FILE or FTP are enabled, since they also use them! + Added test case 674 to reproduce and verify the bug report. - Reported-by: Roland Hieber - Fixes #4325 - Closes #4343 + Fixes #5665 + Reported-by: NobodyXu on github + Closes #5673 -- [Gilles Vollant brought this change] +- [Baruch Siach brought this change] - curl:file2string: load large files much faster + bearssl: fix build with disabled proxy support - ... by using a more efficient realloc scheme. + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Bug: https://curl.haxx.se/mail/lib-2019-09/0045.html - Closes #4336 + Reviewed-by: Nicolas Sterchele + Closes #5666 -- openssl: close_notify on the FTP data connection doesn't mean closure - - For FTPS transfers, curl gets close_notify on the data connection - without that being a signal to close the control connection! +- RELEASE-NOTES: synced + +Jay Satiro (11 Jul 2020) +- [Carlo Marcelo Arenas Belón brought this change] + + cirrus-ci: upgrade 11-STABLE to 11.4 - Regression since 3f5da4e59a556fc (7.65.0) + Meant to be the last of the 11 series and so make sure that all + other references reflect all 11 versions so they can be retired + together later. - Reported-by: Zenju on github - Reviewed-by: Jay Satiro - Fixes #4329 - Closes #4340 + Closes https://github.com/curl/curl/pull/5668 -- [Jimmy Gaussen brought this change] +- [Filip Salomonsson brought this change] - docs/HTTP3: fix `--with-ssl` ngtcp2 configure flag + CURLINFO_CERTINFO.3: fix typo - Closes #4338 + Closes https://github.com/curl/curl/pull/5655 -- RELEASE-NOTES: synced - -- curlver: bump to 7.66.1 +Daniel Stenberg (4 Jul 2020) +- http2: only do the *done() cleanups for HTTP + + Follow-up to ef86daf4d3 + + Closes #5650 + Fixes #5646 -- [Zenju brought this change] +- [Alex Kiernan brought this change] - setopt: make it easier to add new enum values + gnutls: repair the build with `CURL_DISABLE_PROXY` - ... by using the *_LAST define names better. + `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` + if `CURL_DISABLE_PROXY` is enabled. Repair the build with that + configuration. - Closes #4321 + Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> + Closes #5645 -- asyn-thread: s/AF_LOCAL/AF_UNIX for Solaris +Alex Kiernan (3 Jul 2020) +- gnutls: Fetch backend when using proxy - Reported-by: Dagobert Michelsen - Fixes #4328 - Closes #4333 + Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") + Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> -- [Bernhard Walle brought this change] +Daniel Stenberg (3 Jul 2020) +- [Laramie Leavitt brought this change] - winbuild/MakefileBuild.vc: Add vssh + http2: close the http2 connection when no more requests may be sent - Without that modification, the Windows build using the makefiles doesn't - work. + Well-behaving HTTP2 servers send two GOAWAY messages. The first + message is a warning that indicates that the server is going to + stop accepting streams. The second one actually closes the stream. - Signed-off-by: Bernhard Walle <bernhard.walle@posteo.eu> + nghttp2 reports this state (and the other state of no more stream + identifiers) via the call nghttp2_session_check_request_allowed(). + In this state the client should not create more streams on the + session (tcp connection), and in curl this means that the server + has requested that the connection is closed. - Fixes #4322 - Closes #4323 - -Bernhard Walle (11 Sep 2019) -- winbuild/MakefileBuild.vc: Fix line endings + It would be also be possible to put the connclose() call into the + on_http2_frame_recv() function that triggers on the GOAWAY message. - The file had mixed line endings. + This fixes a bug seen when the client sees the following sequence of + frames: - Signed-off-by: Bernhard Walle <bernhard.walle@posteo.eu> - -Jay Satiro (11 Sep 2019) -- ldap: Stop using wide char version of ldapp_err2string + // advisory GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] + ... some additional frames - Despite ldapp_err2string being documented by MS as returning a - PCHAR (char *), when UNICODE it is mapped to ldap_err2stringW and - returns PWCHAR (wchar_t *). + // final GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] - We have lots of code that expects ldap_err2string to return char *, - most of it failf used like this: + Before this change, curl will attempt to reuse the connection even + after the last stream, will encounter this error: - failf(data, "LDAP local: Some error: %s", ldap_err2string(rc)); + * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] + * Re-using existing connection! (#0) with host localhost + * Connected to localhost (::1) port 10443 (#0) + * Using Stream ID: 9 (easy handle 0x5595f0a72e30) + > GET /index.html?5 HTTP/2 + > Host: localhost:10443 + > user-agent: curl/7.68.0 + > accept: */* + > + * stopped the pause stream! + * Connection #0 to host localhost left intact + curl: (16) Error in the HTTP2 framing layer - Closes https://github.com/curl/curl/pull/4272 - -Version 7.66.0 (10 Sep 2019) + This error may posion the connection cache, causing future requests + which resolve to the same curl connection to go through the same error + path. + + Closes #5643 -Daniel Stenberg (10 Sep 2019) -- RELEASE-NOTES: curl 7.66.0 +- ftpserver: don't verify SMTP MAIL FROM names + + Rely on tests asking the names to get refused instead - test servers + should be as dumb as possible. Edited test 914, 955 and 959 accordingly. + + Closes #5639 -- THANKS: from the 7.66.0 release +- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated + + This came up in #5640. It make sense to clarify this in the docs! + + Reminded-by: Kamil Dudka + Closes #5642 -- curl: make sure the parallel transfers do them all +Kamil Dudka (3 Jul 2020) +- tool_getparam: make --krb option work again - The logic could erroneously break the loop too early before all - transfers had been transferred. + It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. - Reported-by: Tom van der Woerdt - Fixes #4316 - Closes #4317 + Bug: https://bugzilla.redhat.com/1833193 + Closes #5640 -- urlapi: one colon is enough for the strspn() input (typo) +Daniel Stenberg (2 Jul 2020) +- [Jeremy Maitin-Shepard brought this change] -- urlapi: verify the IPv6 numerical address + http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages - It needs to parse correctly. Otherwise it could be tricked into letting - through a-f using host names that libcurl would then resolve. Like - '[ab.be]'. + Confusingly, nghttp2 has two different error code enums: - Reported-by: Thomas Vegas - Closes #4315 - -- [Clément Notin brought this change] + - nghttp2_error, to be used with nghttp2_strerror + - nghttp2_error_code, to be used with nghttp2_http2_strerror + + Closes #5641 - openssl: use SSL_CTX_set_<min|max>_proto_version() when available +Marcel Raad (2 Jul 2020) +- url: silence MSVC warning - OpenSSL 1.1.0 adds SSL_CTX_set_<min|max>_proto_version() that we now use - when available. Existing code is preserved for older versions of - OpenSSL. + Since commit f3d501dc678, if proxy support is disabled, MSVC warns: + url.c : warning C4701: potentially uninitialized local variable + 'hostaddr' used + url.c : error C4703: potentially uninitialized local pointer variable + 'hostaddr' used - Closes #4304 + That could actually only happen if both `conn->bits.proxy` and + `CURL_DISABLE_PROXY` were enabled. + Initialize it to NULL to silence the warning. + + Closes https://github.com/curl/curl/pull/5638 -- [Clément Notin brought this change] +Daniel Stenberg (1 Jul 2020) +- RELEASE-NOTES: synced - openssl: indent, re-organize and add comments +Version 7.71.1 (30 Jun 2020) -- [migueljcrum brought this change] +Daniel Stenberg (30 Jun 2020) +- RELEASE-NOTES: curl 7.71.1 - sspi: fix memory leaks - - Closes #4299 +- THANKS: add contributors to 7.71.1 -- travis: disable ngtcp2 builds (again) +- scripts/copyright.pl: skip .dcignore -- Curl_fillreadbuffer: avoid double-free trailer buf on error +- Revert "multi: implement wait using winsock events" - Reviewed-by: Jay Satiro - Reported-by: Thomas Vegas + This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. - Closes #4307 - -- tool_setopt: handle a libcurl build without netrc support + That commit (from #5397) introduced a regression in 7.71.0. - Reported-by: codesniffer13 on github - Fixes #4302 - Closes #4305 + Reported-by: tmkk on github + Fixes #5631 + Closes #5632 -- security:read_data fix bad realloc() - - ... that could end up a double-free - - CVE-2019-5481 - Bug: https://curl.haxx.se/docs/CVE-2019-5481.html +- TODO: Add flag to specify download directory -- [Thomas Vegas brought this change] +- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection - tftp: Alloc maximum blksize, and use default unless OACK is received +- cirrus-ci: disable FreeBSD 13 (again) - Fixes potential buffer overflow from 'recvfrom()', should the server - return an OACK without blksize. + It has been failing for a good while again. This time we better leave it + disabled until we have more reason to believe it behaves. - Bug: https://curl.haxx.se/docs/CVE-2019-5482.html - CVE-2019-5482 + Closes #5628 -- [Thomas Vegas brought this change] +- ngtcp2: sync with current master + + ngtcp2 added two new callbacks + + Reported-by: Lucien Zürcher + Fixes #5624 + Closes #5627 - tftp: return error when packet is too small for options +- examples/multithread.c: call curl_global_cleanup() + + Reported-by: qiandu2006 on github + Fixes #5622 + Closes #5623 -- KNOWN_BUGS/TODO: cleanup and remove outdated issues +- vtls: compare cert blob when finding a connection to reuse + + Reported-by: Gergely Nagy + Fixes #5617 + Closes #5619 - RELEASE-NOTES: synced -- netrc: free 'home' on error - - Follow-up to f9c7ba9096ec2 +- terminology: call them null-terminated strings - Coverity CID 1453474 + Updated terminology in docs, comments and phrases to refer to C strings + as "null-terminated". Done to unify with how most other C oriented docs + refer of them and what users in general seem to prefer (based on a + single highly unscientific poll on twitter). - Closes #4291 + Reported-by: coinhubs on github + Fixes #5598 + Closes #5608 -- urldata: avoid 'generic', use dedicated pointers +- http: fix proxy auth with blank password - For the 'proto' union within the connectdata struct. + Regression in 7.71.0 - Closes #4290 + Added test case 346 to verify. + + Reported-by: Kristoffer Gleditsch + Fixes #5613 + Closes #5616 -- cleanup: move functions out of url.c and make them static +- .dcignore: ignore tests and docs directories - Closes #4289 + This is a config file for deepcode.ai, a static code analyzer. -- smtp: check for and bail out on too short EHLO response +Jay Satiro (26 Jun 2020) +- tool_cb_hdr: Fix etag warning output and return code - Otherwise, a three byte response would make the smtp_state_ehlo_resp() - function misbehave. + - Return 'failure' on failure, to follow the existing style. - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/16918 + - Put Warning: and the warning message on the same line. - Assisted-by: Max Dymond + Ref: https://github.com/curl/curl/issues/5610 - Closes #4287 + Closes https://github.com/curl/curl/pull/5612 -- smb: init *msg to NULL in smb_send_and_recv() - - ... it might otherwise return OK from this function leaving that pointer - uninitialized. +Daniel Stenberg (26 Jun 2020) +- CURLOPT_READFUNCTION.3: provide the upload data size up front - Bug: https://crbug.com/oss-fuzz/16907 - - Closes #4286 + Assisted-by: Jay Satiro + Closes #5607 -- ROADMAP: updated after recent user poll +- test1539: do a HTTP 1.0 POST without a set size (fails) - In rough prio order + Attempt to reproduce #5593. Test case 1514 is very similar but uses + HTTP/1.1 and thus switches to chunked. + + Closes #5595 -- THANKS: remove duplicate +- [Baruch Siach brought this change] -- Curl_addr2string: take an addrlen argument too + mbedtls: fix build with disabled proxy support - This allows the function to figure out if a unix domain socket has a - file name or not associated with it! When a socket is created with - socketpair(), as done in the fuzzer testing, the path struct member is - uninitialized and must not be accessed. + Don't reference fields that do not exist. Fixes build failure: - Bug: https://crbug.com/oss-fuzz/16699 + vtls/mbedtls.c: In function 'mbed_connect_step1': + vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' - Closes #4283 - -- [Rolf Eike Beer brought this change] - - CMake: remove needless newlines at end of gss variables + Closes #5615 -- [Rolf Eike Beer brought this change] - - CI: remove duplicate configure flag for LGTM.com +- codeql-analysis.yml: fix the 'languages' setting + + It needs a 'with:' in front of it. -- [Rolf Eike Beer brought this change] +GitHub (26 Jun 2020) +- [Daniel Stenberg brought this change] - CMake: use platform dependent name for dlopen() library + gtihub: codeql-analysis.yml - Closes #4279 + enables code security scanning with github actions -- quiche: expire when poll returned data +Daniel Stenberg (25 Jun 2020) +- tests: verify newline in username and password for HTTP - ... to make sure we continue draining the queue until empty + test 1296 is a simply command line test - Closes #4281 + test 1910 is a libcurl test including a redirect -- quiche: decrease available buffer size, don't assign it! +- url: allow user + password to contain "control codes" for HTTP(S) - Found-by: Jeremy Lainé - -- RELEASE-NOTES: synced - -- [Kyohei Kadota brought this change] - - curl: fix include conditions - -- [Kyohei Kadota brought this change] + Reported-by: Jon Johnson Jr + Fixes #5582 + Closes #5592 - plan9: fix installation instructions +- escape: make the URL decode able to reject only %00 bytes - Closes #4276 - -- ngtcp2: on h3 stream close, call expire - - ... to trigger a new read to detect the stream close! + ... or all "control codes" or nothing. - Closes #4275 - -- [Tatsuhiro Tsujikawa brought this change] + Assisted-by: Nicolas Sterchele - ngtcp2: build latest ngtcp2 and ngtcp2_crypto_openssl +- http2: set the correct URL in pushed transfers - Closes #4278 - -- ngtcp2: set flow control window to stream buffer size + ...previously CURLINFO_EFFECTIVE_URL would report the URL of the + original "mother transfer", not the actually pushed resource. - Closes #4274 + Reported-by: Jonathan Cardoso Machado + Fixes #5589 + Closes #5591 -- [Christopher Head brought this change] +Jay Satiro (25 Jun 2020) +- [Javier Blazquez brought this change] - CURLOPT_HEADERFUNCTION.3: clarify + openssl: Fix compilation on Windows when ngtcp2 is enabled + + - Include wincrypt before OpenSSL includes so that the latter can + properly handle any conflicts between the two. - Closes #4273 + Closes https://github.com/curl/curl/pull/5606 -- CURLINFO docs: mention that in redirects times are added +Daniel Stenberg (25 Jun 2020) +- test543: extended to verify zero length input - Suggested-by: Brandon Dong - Fixes #4250 - Closes #4269 + As was reported in #5601 -- travis: enable ngtcp2 builds again +- escape: zero length input should return a zero length output - Switched to the openssl-quic-draft-22 openssl branch. + Regression added in 7.71.0. - Closes #4271 - -- HTTP3: switched openssl branch to use - -- [Tatsuhiro Tsujikawa brought this change] + Fixes #5601 + Reported-by: Kristoffer Gleditsch + Closes #5602 - ngtcp2: Build with latest ngtcp2 and ngtcp2_crypto_openssl +- Curl_inet_ntop: always check the return code - Closes #4270 + Reported-by: Siva Sivaraman + Fixes #5412 + Closes #5597 -- http2: when marked for closure and wanted to close == OK +- sendf: improve the message on client write errors - It could otherwise return an error even when closed correctly if GOAWAY - had been received previously. + Replace "Failed writing body (X != Y)" with + "Failure writing output to destination". Possibly slightly less cryptic. - Reported-by: Tom van der Woerdt - Fixes #4267 - Closes #4268 + Reported-by: coinhubs on github + Fixes #5594 + Closes #5596 - RELEASE-NOTES: synced -- build-openssl: fix build with Visual Studio 2019 - - Reviewed-by: Marcel Raad - Contributed-by: osabc on github - Fixes #4188 - Closes #4266 +- curlver: start working on 7.71.1 -Kamil Dudka (26 Aug 2019) -- vauth: return CURLE_AUTH_ERROR on gss_init_sec_context() failure - - This is a follow-up to https://github.com/curl/curl/pull/3864 . - - Closes #4224 +- [Denis Baručić brought this change] -Daniel Stenberg (26 Aug 2019) -- KNOWN_BUGS: USE_UNIX_SOCKETS on Windows + DYNBUF.md: fix a typo: trail => tail - Closes #4040 + Closes #5599 -- quiche: send the HTTP body correctly on callback uploads - - Closes #4265 +Version 7.71.0 (23 Jun 2020) -- travis: disable ngtcp2 builds (temporarily) - - Just too many API changes right now - - Closes #4264 +Daniel Stenberg (23 Jun 2020) +- RELEASE-NOTES: curl 7.71.0 release -- ngtcp2: add support for SSLKEYLOGFILE - - Closes #4260 +- THANKS: curl 7.71.0 additions -- ngtcp2: improve h3 response receiving +- url: make sure pushed streams get an allocated download buffer - Closes #4259 - -- ngtcp2: use nghttp3_version() - -- ngtcp2: sync with upstream API changes + Follow-up to c4e6968127e876b0 - Assisted-by: Tatsuhiro Tsujikawa - -- [Kyle Abramowitz brought this change] - - scp: fix directory name length used in memcpy + When a new transfer is created, as a resuly of an acknowledged push, + that transfer needs a download buffer allocated. - Fix read off end of array due to bad pointer math in getworkingpath for - SCP home directory case. - - Closes #4258 + Closes #5590 -- http: the 'closed' struct field is used by both ngh2 and ngh3 +Jay Satiro (22 Jun 2020) +- openssl: Don't ignore CA paths when using Windows CA store - and remove 'header_recvbuf', not used for anything + This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does + not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default + locations. Instead the CA store can now be used at the same time. - Reported-by: Jeremy Lainé + The change is due to the impending release. The issue is still being + discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and + is now documented as experimental. - Closes #4257 + Ref: bc052cc (parent commit) + Ref: https://github.com/curl/curl/issues/5585 -- ngtcp2: accept upload via callback +- tool_operate: Don't use Windows CA store as a fallback - Closes #4256 - -- defines: avoid underscore-prefixed defines + Background: - Double-underscored or underscore plus uppercase letter at least. + 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate + store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides + CURLOPT_CAINFO if both are set. The curl tool will fall back to + CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set + via CURLOPT_CAINFO. - ... as they're claimed to be reserved. + Problem: - Reported-by: patnyb on github + libcurl may be built with hardcoded paths to a certificate bundle or + directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are + ignored. - Fixes #4254 - Closes #4255 - -- travis: add a build using ngtcp2 + nghttp3 (and a patched OpenSSL) + Solution: - Runs no tests + A solution is still being discussed but since there's an impending + release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. - Closes #4253 + Ref: https://github.com/curl/curl/issues/5585 -- travis: bump to using nghttp2 version 1.39.2 +- openssl: Fix CA fallback logic for OpenSSL 3.0 build + + Prior to this change I assume a build error would occur when + CURL_CA_FALLBACK was used. - Closes #4252 + Closes https://github.com/curl/curl/pull/5587 -- [Gisle Vanem brought this change] +Daniel Stenberg (22 Jun 2020) +- copyright: update mismatched copyright years + +- test1460: verify that -Ji is not ok - docs/examples/curlx: fix errors +- tool_getparam: -i is not OK if -J is used - Initialise 'mimetype' and require the -p12 arg. + Reported-by: sn on hackerone + Bug: https://curl.haxx.se/docs/CVE-2020-8177.html + +- [Peter Wu brought this change] + + CMake: ignore INTERFACE_LIBRARY targets for pkg-config file - Closes #4248 + Reviewed-by: Marcel Raad + Fixes #5512 + Closes #5517 + +- [Valentyn Korniienko brought this change] -- cleanup: remove DOT_CHAR completely + multibyte: Fixed access-> waccess to file for Windows Plarform - Follow-up to f9c7ba9096ec + Reviewed-by: Marcel Raad + Closes #5580 + +- altsvc: bump to h3-29 - The use of DOT_CHAR for ".ssh" was probably a mistake and is removed - now. + Closes #5584 + +- urlglob: treat literal IPv6 addresses with zone IDs as a host name - Pointed-out-by: Gisle Vanem - Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638 + ... and not as a "glob". Now done by passing the supposed host to the + URL parser which supposedly will do a better job at identifying "real" + numerical IPv6 addresses. - Closes #4247 + Reported-by: puckipedia on github + Fixes #5576 + Closes #5579 -- spnego_sspi: add typecast to fix build warning +- test1179: verify error message for non-existing cmdline option + +- tool_getparam: repair the error message for unknown flag - Reported in build "Win32 target on Debian Stretch (64-bit) - - i686-w64-mingw32 - gcc-20170516" + Follow-up to 9e5669f3880674 + Detected by Coverity CID 1464582 ("Logically dead code") - Closes #4245 + Closes #5577 -- openssl: build warning free with boringssl - - Closes #4244 +- FILEFORMAT: describe verify/stderr -- curl: make --libcurl use CURL_HTTP_VERSION_3 +- connect: improve happy eyeballs handling + + For QUIC but also for regular TCP when the second family runs out of IPs + with a failure while the first family is still trying to connect. - Closes #4243 + Separated the timeout handling for IPv4 and IPv6 connections when they + both have a number of addresses to iterate over. -- ngtcp2: make postfields-set posts work +- ngtcp2: never call fprintf() in lib code in release version + +- ngtcp2: fix happy eyeballs quic connect crash - Closes #4242 + Reported-by: Peter Wu + Fixes #5565 + Closes #5568 -- http: remove chunked-encoding and expect header use for HTTP/3 +- select: remove the unused ELAPSED_MS() macro + + Closes #5573 -- [Alessandro Ghedini brought this change] +Marc Hoersken (17 Jun 2020) +- [rcombs brought this change] - configure: use pkg-config to detect quiche + multi: implement wait using winsock events - This removes the need to hard-code the quiche target path in - configure.ac. + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - This depends on https://github.com/cloudflare/quiche/pull/128 + Reviewed-by: Gergely Nagy + Reviewed-by: Marc Hörsken - Closes #4237 + Closes #5397 -- CURLOPT_SSL_VERIFYHOST: treat the value 1 as 2 +Daniel Stenberg (17 Jun 2020) +- manpage: add three missing environment variables - For a long time (since 7.28.1) we've returned error when setting the - value to 1 to make applications notice that we stopped supported the old - behavior for 1. Starting now, we treat 1 and 2 exactly the same. + CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE - Closes #4241 + Closes #5571 -- curl: use .curlrc (with a dot) on Windows as well +- RELEASE-NOTES: synced + +- configure: for wolfSSL, check for the DES func needed for NTLM - Fall-back to _curlrc if the dot-version is missing. + Also adds pkg-config support for the wolfSSL detection. + +- [Ruurd Beerstra brought this change] + + ntlm: enable NTLM support with wolfSSL - Co-Authored-By: Steve Holme + When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* + functions that OpenSSL has. This change take advantage of that. - Closes #4230 + Co-authored-by: Daniel Stenberg + Closes #5556 + Fixes #5548 -- netrc: make the code try ".netrc" on Windows as well +- http: move header storage to Curl_easy from connectdata - ... but fall back and try "_netrc" too if the dot version didn't work. + Since the connection can be used by many independent requests (using + HTTP/2 or HTTP/3), things like user-agent and other transfer-specific + data MUST NOT be kept connection oriented as it could lead to requests + getting the wrong string for their requests. This struct data was + lingering like this due to old HTTP1 legacy thinking where it didn't + mattered.. - Co-Authored-By: Steve Holme + Fixes #5566 + Closes #5567 -- ngtcp2: use ngtcp2_version() to get the run-time version +- CODE_REVIEW.md: how to do code reviews in curl - ... which of course doesn't have to be the same used at build-time. + Assisted-by: Daniel Gustafsson + Assisted-by: Rich Salz + Assisted-by: Hugo van Kemenade + Assisted-by: James Fuller + Assisted-by: Marc Hörsken + Assisted-by: Jay Satiro - Function just recently merged in ngtcp2. + Closes #5555 -- ngtcp2: move the h3 initing to immediately after the rx key +- altsvc: remove the num field from the altsvc struct - To fix a segfault and to better deal with 0-RTT + It was superfluous since we have the list.size alredy - Assisted-by: Tatsuhiro Tsujikawa - -- [Alessandro Ghedini brought this change] + Reported-by: Jay Satiro + Fixes #5553 + Closes #5563 - quiche: register debug callback once and earlier +- version.d: expanded and alpha-sorted - The quiche debug callback is global and can only be initialized once, so - make sure we don't do it multiple times (e.g. if multiple requests are - executed). + Added a few missing features not previously mentioned. Ordered them + alphabetically. - In addition this initializes the callback before the connection is - created, so we get logs for the handshake as well. + Closes #5558 + +- ABI.md: rename to .md and polish the markdown - Closes #4236 + Closes #5562 -- ssh: add a generic Curl_ssh_version function for SSH backends +- HELP-US: add a section for "smaller tasks" + + The point of this section is to meet the CII Best Practices gold level + critera: + + "The project MUST clearly identify small tasks that can be performed by + new or casual contributors" - Closes #4235 + Closes #5560 -- base64: check for SSH, not specific SSH backends +- TODO: retry on the redirected-to URL + + Closes #5462 -- vssh: move ssh init/cleanup functions into backend code +- mailmap: Nicolas Sterchele -- vssh: create directory for SSH backend code +- [Nicolas Sterchele brought this change] -- TODO/ROADMAP: remove "refuse downgrade redirects" and HTTP/3 + TODO: remove 19.3 section title - HTTP3 is now already in full progress + Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on + curl documentation website - Downgrade redirects can be achived almost exactly like that by setting - CURLOPT_REDIR_PROTOCOLS. + Closes #5561 -- RELEASE-NOTES: synced +- [Martin V brought this change] -- travis: add a quiche build + test1560: avoid possibly negative association in wording - Closes #4207 + Closes #5549 -- http: fix use of credentials from URL when using HTTP proxy +- share: don't set the share flag it something fails - When a username and password are provided in the URL, they were wrongly - removed from the stored URL so that subsequent uses of the same URL - wouldn't find the crendentials. This made doing HTTP auth with multiple - connections (like Digest) mishave. + When asking for a specific feature to be shared in the share object, + that bit was previously set unconditionally even if the shared feature + failed or otherwise wouldn't work. - Regression from 46e164069d1a5230 (7.62.0) + Closes #5554 + +- buildconf: remove -print from the find command that removes files - Test case 335 added to verify. + It's just too annoying and unnecessary to get a long list of files shown + +- RELEASE-NOTES: synced + +- wording: avoid blacklist/whitelist stereotypes - Reported-by: Mike Crowe + Instead of discussing if there's value or meaning (implied or not) in + the colors, let's use words without the same possibly negative + associations. - Fixes #4228 - Closes #4229 + Closes #5546 -- [Mike Crowe brought this change] - - tests: Replace outdated test case numbering documentation +Jay Satiro (9 Jun 2020) +- tool_getparam: fix memory leak in parse_args - Tests are no longer grouped by numeric range[1]. Let's stop saying that - and provide some alternative advice for numbering tests. + Prior to this change in Windows Unicode builds most parsed options would + not be freed. - [1] https://curl.haxx.se/mail/lib-2019-08/0043.html + Found using _CrtDumpMemoryLeaks(). - Closes #4227 + Ref: https://github.com/curl/curl/issues/5545 -- travis: reduce number of torture tests in 'coverage' +Daniel Stenberg (8 Jun 2020) +- socks: detect connection close during handshake - ... to make it complete in time. This cut seems not almost not affect - the coverage percentage and yet completes within 35 minutes on travis - where the previous runs recently always timed out after 50. + The SOCKS4/5 state machines weren't properly terminated when the proxy + connection got closed, leading to a busy-loop. - Closes #4223 + Reported-By: zloi-user on github + Fixes #5532 + Closes #5542 -- [Igor Makarov brought this change] +- [James Fuller brought this change] - configure: use -lquiche to link to quiche + multi: add defensive check on data->multi->num_alive - Closes #4226 + Closes #5540 -- ngtcp2: provide the callbacks as a static struct +- Curl_addrinfo: use one malloc instead of three - ... instead of having them in quicsocket - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: add missing nghttp3_conn_add_write_offset call + To reduce the amount of allocations needed for creating a Curl_addrinfo + struct, make a single larger malloc instead of three separate smaller + ones. - Closes #4225 - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: deal with stream close - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: Consume QUIC STREAM data properly + Closes #5533 -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: don't reinitialize SSL on Retry +- [Alessandro Ghedini brought this change] -- multi: getsock improvements for QUIC connecting + quiche: update SSLKEYLOGFILE support + + quiche now requires the application to explicitly set the keylog path + for each connection, rather than reading the environment variable + itself. + + Closes #5541 -- connect: connections are persistent by default for HTTP/3 +- tests: add two simple tests for --login-options + + Test 895 and 896 - as a follow-up to a3e972313b + + Closes #5539 -- quiche: happy eyeballs +- ngtcp2: update with recent API changes + + Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. - Closes #4220 + Assisted-by: Tatsuhiro Tsujikawa + Closes #5538 -- ngtcp2: do QUIC connections happy-eyeballs friendly +- [James Fuller brought this change] -- curl_version: bump string buffer size to 250 + socks: remove unreachable breaks in socks.c and mime.c - With HTTP/3 libs and plenty TLS libs, I manged to hit the limit (which - causes a truncated output). + Closes #5537 -- CURLOPT_ALTSVC.3: use a "" file name to not load from a file +- tool_cfgable: free login_options at exit + + Memory leak + Reported-by: Geeknik Labs + Fixes #5535 + Closes #5536 -Jay Satiro (14 Aug 2019) -- vauth: Use CURLE_AUTH_ERROR for auth function errors +- libssh2: keep sftp errors as 'unsigned long' - - Add new error code CURLE_AUTH_ERROR. + Remove weird work-around for storing the SFTP errors as int instead of + the "unsigned long" that libssh2 actually returns for SFTP errors. - Prior to this change auth function errors were signaled by - CURLE_OUT_OF_MEMORY and CURLE_RECV_ERROR, and neither one was - technically correct. + Closes #5534 + +Marc Hoersken (6 Jun 2020) +- timeouts: move ms timeouts to timediff_t from int and long - Ref: https://github.com/curl/curl/pull/3848 + Now that all functions in select.[ch] take timediff_t instead + of the limited int or long, we can remove type conversions + and related preprocessor checks to silence compiler warnings. - Co-authored-by: Dominik Hölzl + Avoiding conversions from time_t was already done in 842f73de. - Closes https://github.com/curl/curl/pull/3864 + Based upon #5262 + Supersedes #5214, #5220 and #5221 + Follow up to #5343 and #5479 + Closes #5490 -Daniel Stenberg (13 Aug 2019) -- curl_version_info: make the quic_version a const +Daniel Stenberg (6 Jun 2020) +- [François Rigault brought this change] + + openssl: set FLAG_TRUSTED_FIRST unconditionally - Follow-up from 1a2df1518ad8653f + On some systems, openssl 1.0.x is still the default, but it has been + patched to contain all the recent security fixes. As a result of this + patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be + defined, while the previous behavior of openssl to not look at trusted + chains first, remains. - Closes #4222 - -- examples: add http3.c, altsvc.c and http3-present.c + Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to + probe for the behavior of openssl based on the existence ofmacros. - Closes #4221 + Closes #5530 -Peter Wu (13 Aug 2019) -- nss: use TLSv1.3 as default if supported +- server/util: fix logmsg format using curl_off_t argument - SSL_VersionRangeGetDefault returns (TLSv1.0, TLSv1.2) as supported - range in NSS 3.45. It looks like the intention is to raise the minimum - version rather than lowering the maximum, so adjust accordingly. Note - that the caller (nss_setup_connect) initializes the version range to - (TLSv1.0, TLSv1.3), so there is no need to check for >= TLSv1.0 again. + ... this caused segfaults on armv7. - Closes #4187 - Reviewed-by: Daniel Stenberg - Reviewed-by: Kamil Dudka + Regression added in dd0365d560aea5a (7.70.0) + + Reviewed-by: Jay Satiro + Closes #5529 -Daniel Stenberg (13 Aug 2019) -- quic.h: remove unused proto +- RELEASE-NOTES: synced -- curl_version_info.3: mentioned ALTSVC and HTTP3 - - ... and sorted the list alphabetically +- [Cherish98 brought this change] -- lib/quic.c: unused - removed + socks: fix expected length of SOCKS5 reply + + Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the + reply ATYP is X'01'. This resulted in erroneously expecting more bytes + when the request length is greater than the reply length (e.g., when + remotely resolving the hostname). + + Closes #5527 -- CURLOPT_ALTSVC_CTRL.3: remove CURLALTSVC_ALTUSED +Marc Hoersken (5 Jun 2020) +- .gitignore: add directory containing the stats repo - Follow-up to 98c3f148 that removed it from the header file + Since the new curl/stats repository is designed to be + checked out into the curl repository working tree as stats/ + it should be on the ignore list to aid in commit staging. -- [Junho Choi brought this change] +Daniel Stenberg (5 Jun 2020) +- [Adnan Khan brought this change] - docs/HTTP3: simplify quiche build instruction + HTTP3.md: clarify cargo build directory - Use --recursive to get boringssl in one line + Cargo needs to be called from within the 'quiche' directory. - Closes #4219 + Closes #5522 -- altsvc: make it use h3-22 with ngtcp2 as well - -- ngtcp2: initial h3 request work +- user-agent.d: spell out what happens given a blank argument - Closes #4217 + Closes #5525 -- curl_version_info: offer quic (and h3) library info +- trailers: switch h1-trailer logic to use dynbuf - Closes #4216 - -- HTTP3: use ngtcp2's draft-22 branch - -- RELEASE-NOTES: synced + In the continued effort to remove "manual" realloc schemes. + + Closes #5524 -- CURLOPT_READFUNCTION.3: provide inline example +- CURLINFO_ACTIVESOCKET.3: clarify the description - ... instead of mentioning one in another place + Reported-by: Jay Satiro + Fixes #5299 + Closes #5520 -- [Tatsuhiro Tsujikawa brought this change] +- mailmap: Don J Olmstead - ngtcp2: send HTTP/3 request with nghttp3 +- configure: only strip first -L from LDFLAGS - This commit makes sending HTTP/3 request with nghttp3 work. It - minimally receives HTTP response and calls nghttp3 callbacks, but no - processing is made at the moment. + In the logic that works out if a given OpenSSL path works, it stripped + off a possibly leading -L flag using an incorrect sed pattern which + would remove all instances of -L in the string, including if the path + itself contained that two-letter sequence! - Closes #4215 - -- nghttp3: initial h3 template code added + The same pattern was used and is now updated in multiple places. Now it + only removes -L if it starts the strings. + + Reported-by: Mohamed Osama + Fixes #5519 + Closes #5521 -- nghttp3: required when ngtcp2 is used for QUIC +Peter Wu (4 Jun 2020) +- quiche: advertise draft 28 support - - checked for by configure - - updated docs/HTTP3.md - - shown in the version string + Fix the verbose message while at it, quiche currently supports draft + 27 and draft 28 simultaneously. - Closes #4210 + Closes #5518 -- [Eric Wong brought this change] +Daniel Stenberg (4 Jun 2020) +- KNOWN_BUGS: RTSP authentication breaks without redirect support + + Closes #4750 - asyn-thread: issue CURL_POLL_REMOVE before closing socket +Jay Satiro (4 Jun 2020) +- projects: Add crypt32.lib to dependencies for all OpenSSL configs - This avoids EBADF errors from EPOLL_CTL_DEL operations in the - ephiperfifo.c example. EBADF is dangerous in multi-threaded - applications where I rely on epoll_ctl to operate on the same - epoll description from different threads. + Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO + need crypt32. - Follow-up to eb9a604f8d7db8 + Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. - Bug: https://curl.haxx.se/mail/lib-2019-08/0026.html - Closes #4211 - -- [Carlo Marcelo Arenas Belón brought this change] - - configure: avoid undefined check_for_ca_bundle + The changes that are in this commit were made by script. - instead of using a "greater than 0" test, check for variable being - set, as it is always set to 1, and could be left unset if non of - OPENSSL MBEDTLS GNUTLS WOLFSSL is being configured for. + Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 - Closes #4213 - -- [Tatsuhiro Tsujikawa brought this change] + Closes https://github.com/curl/curl/pull/5516 - ngtcp2: Send ALPN h3-22 +Marc Hoersken (3 Jun 2020) +- CI/macos: fix 'is already installed' errors by using bundle - Closes #4212 - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: use ngtcp2_settings_default and specify initial_ts + Avoid failing CI builds due to nghttp2 being already installed. + + Closes #5513 -- curl_global_init_mem.3: mention it was added in 7.12.0 +Daniel Stenberg (3 Jun 2020) +- altsvc: fix 'dsthost' may be used uninitialized in this function -- [Tatsuhiro Tsujikawa brought this change] +- RELEASE-NOTES: synced - ngtcp2: make the QUIC handshake work +- urldata: let the HTTP method be in the set.* struct + + When the method is updated inside libcurl we must still not change the + method as set by the user as then repeated transfers with that same + handle might not execute the same operation anymore! + + This fixes the libcurl part of #5462 - Closes #4209 + Test 1633 added to verify. + + Closes #5499 -- [Alex Mayorga brought this change] +- hostip: fix the memory-leak introduced in 67d2802 + + Fixes #5503 + Closes #5504 - HTTP3.md: Update quiche build instructions +- test970: make it require proxy support - Added cloning for quiche and BoringSSL and modified the build - instructions so they work on a clean folder. + This test verifies the -w %json output and the test case includes a full + generated "blob". If there's no proxy support built into libcurl, it + will return an error for proxy related info variables and they will not + be included in the json, thus causing a mismatch and this test fails. - Closes #4208 + Reported-by: Marc Hörsken + Fixes #5501 + Closes #5502 + +- [Radoslav Georgiev brought this change] -- CURLOPT_H3: removed + examples/http2-down/upload: add error checks - There's no use for this anymore and it was never in a release. + If `index.html` does not exist in the directory from which the example + is invoked, the fopen(upload, "rb") invocation in `setup` would fail, + returning NULL. This value is subsequently passed as the FILE* argument + of the `fread` invocation in the `read_callback` function, which is the + actual cause of the crash (apparently `fread` assumes that argument to + be non-null). - Closes #4206 + In addition, mitigate some possible crashes of similar origin. + + Closes #5463 + +- [kotoriのねこ brought this change] -- http3: make connection reuse work + examples/ephiperfifo: turn off interval when setting timerfd - Closes #4204 + Reported-by: therealhirudo on github + Fixes #5485 + Closes #5497 -- quiche: add SSLKEYLOGFILE support +- [Saleem Abdulrasool brought this change] -- cleanup: s/curl_debug/curl_dbg_debug in comments and docs + vtls: repair the build with `CURL_DISABLE_PROXY` - Leftovers from the function rename back in 76b63489495 + `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` + is enabled. Repair the build with that configuration. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/f3e0f071b14fcb46a453f69bdf4e062bcaacf362#com - mitcomment-34601751 + Follow-up to f3d501dc67 - Closes #4203 - -- RELEASE-NOTES: synced + Closes #5498 -- alt-svc: add protocol version selection masking +- transfer: remove k->str NULL check - So that users can mask in/out specific HTTP versions when Alt-Svc is - used. + "Null-checking k->str suggests that it may be null, but it has already + been dereferenced on all paths leading to the check" - and it can't + legally be NULL at this point. Remove check. - - Removed "h2c" and updated test case accordingly - - Changed how the altsvc struct is laid out - - Added ifdefs to make the unittest run even in a quiche-tree + Detected by Coverity CID 1463884 - Closes #4201 + Closes #5495 -- http3: fix the HTTP/3 in the request, make alt-svc set right versions +Marc Hoersken (1 Jun 2020) +- select: always use Sleep in Curl_wait_ms on Win32 - Closes #4200 + Since Win32 almost always will also have USE_WINSOCK, + we can reduce complexity and always use Sleep there. + + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Follow up to #5343 + Closes #5489 -- alt-svc: send Alt-Used: in redirected requests +Daniel Stenberg (31 May 2020) +- conncache: download buffer needs +1 size for trailing zero - RFC 7838 section 5: + Follow-up to c4e6968127e + Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 + +Marc Hoersken (31 May 2020) +- azure: use matrix strategy to avoid configuration redundancy - When using an alternative service, clients SHOULD include an Alt-Used - header field in all requests. + This also includes the following changes: - Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus - this is deemed ok). + - Use the same timeout for all jobs on Linux (60 minutes) + and Windows (90 minutes) + - Use CLI stable apt-get install -y instead of apt install + which warns about that and run apt-get update first + - Enable MQTT for Windows msys2 builds instead of + legacy msys1 builds + - Add ./configure --prefix parameter to the msys2 builds + - The MSYSTEM environment variable is now preset inside + the container images for the msys2 builds - You can disable sending this header just like you disable any other HTTP - header in libcurl. + Note: on Azure Pipelines the matrix strategy is basically + just a simple list of job copies and not really a matrix. - Closes #4199 + Closes #5468 -- CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly +Daniel Stenberg (30 May 2020) +- build: disable more code/data when built without proxy support - Even though it cannot fall-back to a lower HTTP version automatically. The - safer way to upgrade remains via CURLOPT_ALTSVC. + Added build to travis to verify - CURLOPT_H3 no longer has any bits that do anything and might be removed - before we remove the experimental label. + Closes #5466 + +- url: alloc the download buffer at transfer start - Updated the curl tool accordingly to use "--http3". + ... and free it as soon as the transfer is done. It removes the extra + alloc when a new size is set with setopt() and reduces memory for unused + easy handles. - Closes #4197 + In addition: the closure_handle now doesn't use an allocated buffer at + all but the smallest supported size as a stack based one. + + Closes #5472 -- docs/ALTSVC: remove what works and the experimental explanation +- timeouts: change millisecond timeouts to timediff_t from time_t - Also, put the TODO items at the bottom. + For millisecond timers we like timediff_t better. Also, time_t can be + unsigned so returning a negative value doesn't work then. - Closes #4198 - -- docs/EXPERIMENTAL: explain what it means and what's experimental now + Closes #5479 -- curl: make use of CURLINFO_RETRY_AFTER when retrying +Marc Hoersken (30 May 2020) +- select: add overflow checks for timeval conversions - If a Retry-After: header was used in the response, that value overrides - other retry timing options. + Using time_t and suseconds_t if suseconds_t is available, + long on Windows (maybe others in the future) and int elsewhere. - Fixes #3794 - Closes #4195 - -- curl: use CURLINFO_PROTOCOL to check for HTTP(s) + Also handle case of ULONG_MAX being greater or equal to INFINITE. - ... instead of CURLINFO_EFFECTIVE_URL to avoid string operations. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Part of #5343 -- CURLINFO_RETRY_AFTER: parse the Retry-After header value +- select: use timediff_t instead of time_t and int for timeout_ms - This is only the libcurl part that provides the information. There's no - user of the parsed value. This change includes three new tests for the - parser. + Make all functions in select.[ch] take timeout_ms as timediff_t + which should always be large enough and signed on all platforms + to take all possible timeout values and avoid type conversions. - Ref: #3794 - -- docs/ALTSVC.md: first basic file format description - -- curl: have -w's 'http_version' show '3' for HTTP/3 + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #4196 + Replaces #5107 and partially #5262 + Related to #5240 and #5286 + Closes #5343 -- curl.h: add CURL_HTTP_VERSION_3 to the version enum +- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' - It can't be set for CURLOPT_HTTP_VERSION, but it can be extracted with - CURLINFO_HTTP_VERSION. - -- quiche: make use of the connection timeout API properly - -- quiche: make POSTFIELDS posts work - -- quiche: improved error handling and memory cleanups - -- quiche: flush egress in h3_stream_recv() too - -- RELEASE-NOTES: synced - -Jay Satiro (6 Aug 2019) -- [Patrick Monnerat brought this change] - - os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid(). + GCC 10 warns about this with warning: implicit conversion + from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + Since 'expected_result' is not really of type 'CURLcode' and + it is not exposed in any way, we can just use 'SANITIZEcode'. - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/4186 + Closes #5476 -- tests: Fix the line endings for the SASL alt-auth tests +- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - - Change data and protocol sections to CRLF line endings. + Since curl_setup.h now makes use of curlx_win32_fopen for Win32 + builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, + we need to include the relevant files for tests using fopen, + because the libtest sources are also including curl_setup.h - Prior to this change the tests would fail or hang, which is because - certain sections such as protocol require CRLF line endings. + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Follow-up to grandparent commit which added the tests. + Follow up to #3784 (ffdddb45d9) + Closes #5475 + +- appveyor: add non-debug plain autotools-based build - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + This should enable us to catch linking issues with the + testsuite early, like the one described/fixed in #5475. - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/4186 + Closes #5477 -- [Steve Holme brought this change] +Daniel Stenberg (29 May 2020) +- RELEASE-NOTES: synced - examples: Added SASL PLAIN authorisation identity (authzid) examples +- Revert "buildconf: use find -execdir" - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + This partially reverts commit c712009838f44211958854de431315586995bc61. - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Keep the ares_ files removed but bring back the older way to run find, + to make it work with busybox's find, as apparently that's being used. - Closes https://github.com/curl/curl/pull/4186 + Reported-by: Max Peal + Fixes #5483 + Closes #5484 -- [Steve Holme brought this change] +- server/sws: fix asan warning on use of uninitialized variable - curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool +- libssh2: improved error output for wrong quote syntax - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + Reported-by: Werner Stolz - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. - - Closes https://github.com/curl/curl/pull/4186 + Closes #5474 -- [Steve Holme brought this change] +- mk-lib1521: generate code for testing BLOB options as well + + Follow-up to cac5374298b3 + + Closes #5478 - sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID +- configure: repair the check if argv can be written to - Added the ability for the calling program to specify the authorisation - identity (authzid), the identity to act as, in addition to the - authentication identity (authcid) and password when using SASL PLAIN - authentication. + Due to bad escaping of the test code, the test wouldn't build and thus + result in a negative test result, which would lead to the unconditional + assumption that overwriting the arguments doesn't work and thus curl + would never hide credentials given in the command line, even when it + would otherwise be possible. - Fixes #3653 - Closes #3790 + Regression from commit 2d4c2152c (7.60.0) - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Reported-by: huzunhao on github + Fixes #5470 + Closes #5471 + +Peter Wu (28 May 2020) +- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - Closes https://github.com/curl/curl/pull/4186 + Otherwise the build might fail due to missing source files, as + demonstrated by the recent keylog.c addition on an existing build dir. + + Closes #5469 -Daniel Stenberg (6 Aug 2019) -- docs/HTTP3: refreshed as it is now in master and HTTP/3 can be tested +Daniel Stenberg (28 May 2020) +- urldata: fix comments: Curl_done() is called multi_done() now + + ... since 575e885db -- [Yiming Jing brought this change] +Peter Wu (27 May 2020) +- ngtcp2: use common key log routine for better thread-safety + + Tested with ngtcp2 built against the OpenSSL library. Additionally + tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + + The TLS backend (independent of QUIC) may or may not already have opened + the keylog file before. Therefore Curl_tls_keylog_open is always called + to ensure the file is open. - mesalink: implement client authentication +- wolfssl: add SSLKEYLOGFILE support + + Tested following the same curl and tshark commands as in commit + "vtls: Extract and simplify key log file handling from OpenSSL" using + WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with + `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. - Closes #4184 + Full support for this feature requires certain wolfSSL build options, + see "Availability note" in lib/vtls/wolfssl.c for details. + + Closes #5327 -- curl_multi_poll: a sister to curl_multi_wait() that waits more +- vtls: Extract and simplify key log file handling from OpenSSL - Repeatedly we see problems where using curl_multi_wait() is difficult or - just awkward because if it has no file descriptor to wait for - internally, it returns immediately and leaves it to the caller to wait - for a small amount of time in order to avoid occasional busy-looping. + Create a set of routines for TLS key log file handling to enable reuse + with other TLS backends. Simplify the OpenSSL backend as follows: - This is often missed or misunderstood, leading to underperforming - applications. + - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. + - Do not perform dynamic memory allocation when preparing a log entry. + Unless the TLS specifications change we can suffice with a reasonable + fixed-size buffer. + - Simplify state tracking when SSL_CTX_set_keylog_callback is + unavailable. My original sslkeylog.c code included this tracking in + order to handle multiple calls to SSL_connect and detect new keys + after renegotiation (via SSL_read/SSL_write). For curl however we can + be sure that a single master secret eventually becomes available + after SSL_connect, so a simple flag is sufficient. An alternative to + the flag is examining SSL_state(), but this seems more complex and is + not pursued. Capturing keys after server renegotiation was already + unsupported in curl and remains unsupported. - This change introduces curl_multi_poll() as a replacement drop-in - function that accepts the exact same set of arguments. This function - works identically to curl_multi_wait() - EXCEPT - for the case when - there's nothing to wait for internally, as then this function will by - itself wait for a "suitable" short time before it returns. This - effectiely avoids all risks of busy-looping and should also make it less - likely that apps "over-wait". + Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f + (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) + against an OpenSSL 1.1.1f server configured with: - This also changes the curl tool to use this funtion internally when - doing parallel transfers and changes curl_easy_perform() to use it - internally. + # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 + openssl s_server -www -tls1 + # Likewise, but fail the server handshake. + openssl s_server -www -tls1 -Verify 2 + # TLS 1.3 test. No need to test the failing server handshake. + openssl s_server -www -tls1_3 - Closes #4163 - -- quiche:h3_stream_recv return 0 at end of stream + Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly + written using Wireshark. For the first and third case, expect four + matches per connection (decrypted Server Finished, Client Finished, HTTP + Request, HTTP Response). For the second case where the handshake fails, + expect a decrypted Server Finished only. + + tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ + -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ + -dtls.port==4433,http -ohttp.desegment_body:FALSE \ + -Y 'tls.handshake.verify_data or http' - ... and remove some verbose messages we don't need. Made transfers from - facebook.com work better. + A single connection can easily be identified via the `tcp.stream` field. -- altsvc: make quiche use h3-22 now +Daniel Stenberg (27 May 2020) +- FILEFORMAT: add more features that tests can depend on -- quiche: show the actual version number +- [Michael Kaufmann brought this change] -- quiche: first working HTTP/3 request + transfer: close connection after excess data has been read - - enable debug log - - fix use of quiche API - - use download buffer - - separate header/body + For HTTP 1.x, it's a protocol error when the server sends more bytes + than announced. If this happens, don't reuse the connection, because the + start position of the next response is undefined. - Closes #4193 + Closes #5440 + +- [Estanislau Augé-Pujadas brought this change] -- http09: disable HTTP/0.9 by default in both tool and library + Revert "ssh: ignore timeouts during disconnect" - As the plan has been laid out in DEPRECATED. Update docs accordingly and - verify in test 1174. Now requires the option to be set to allow HTTP/0.9 - responses. + This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in + curl 7.54.1. - Closes #4191 - -- quiche: initial h3 request send/receive + Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html + Closes #5465 -- lib/Makefile.am: make checksrc run in vquic too - -- altsvc: fix removal of expired cache entry +- urldata: connect related booleans live in struct ConnectBits - Closes #4192 - -- RELEASE-NOTES: synced + And remove a few unused booleans! + + Closes #5461 -Steve Holme (4 Aug 2019) -- md4: Use our own MD4 implementation when no crypto libraries are available +- hostip: on macOS avoid DoH when given a numerical IP address + + When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that + numerical IP addresses still need to get "resolved" - but not with DoH. - Closes #3780 + Reported-by: Viktor Szakats + Fixes #5454 + Closes #5459 -- md4: No need to include Curl_md4.h for each TLS library +- ngtcp2: cleanup memory when failing to connect + + Reported-by: Peter Wu + Fixes #5447 (the ngtcp2 side of it) + Closes #5451 -- md4: No need for the NTLM code to call Curl_md4it() for each TLS library +- quiche: clean up memory properly when failing to connect - As the NTLM code no longer calls any of TLS libraries' specific MD4 - functions, there is no need to call this function for each #ifdef. + Addresses the quiche side of #5447 + Reported-by: Peter Wu + Closes #5450 -- md4: Move the mbed TLS MD4 implementation out of the NTLM code +- cleanup: use a single space after equals sign in assignments -- md4: Move the WinCrypt implementation out of the NTLM code +- url: accept "any length" credentials for proxy auth + + They're only limited to the maximum string input restrictions, not to + 256 bytes. + + Added test 1178 to verify + + Reported-by: Will Roberts + Fixes #5448 + Closes #5449 -- md4: Move the SecureTransport implementation out of the NTLM code +- [Maksim Stsepanenka brought this change] -- md4: Use the Curl_md4it() function for OpenSSL based NTLM + test1167: fixes in badsymbols.pl + + Closes #5442 -- md4: Move the GNU TLS gcrypt MD4 implementation out of the NTLM code +- altsvc: fix parser for lines ending with CRLF + + Fixed the alt-svc parser to treat a newline as end of line. + + The unit tests in test 1654 were done without CRLF and thus didn't quite + match the real world. Now they use CRLF as well. + + Reported-by: Peter Wu + Assisted-by: Peter Wu + Assisted-by: Jay Satiro + Fixes #5445 + Closes #5446 -- md4: Move the GNU TLS Nettle MD4 implementation out of the NTLM code +Viktor Szakats (25 May 2020) +- all: fix codespell errors + + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes https://github.com/curl/curl/pull/5452 -Jay Satiro (4 Aug 2019) -- OS400: Add CURLOPT_H3 symbols +Peter Wu (25 May 2020) +- ngtcp2: fix build with current ngtcp2 master implementing draft 28 - Follow-up to 3af0e76 which added experimental H3 support. + Based on client.cc changes from ngtcp2. Tested with current git master, + ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. - Closes https://github.com/curl/curl/pull/4185 + Fixes #5444 + Closes #5443 -Daniel Stenberg (3 Aug 2019) -- url: make use of new HTTP version if alt-svc has one +Daniel Stenberg (25 May 2020) +- RELEASE-NOTES: synced + + moved the new setopts up to a "change" -- url: set conn->transport to default TCP at init time +- RELEASE-NOTES: synced -- altsvc: with quiche, use the quiche h3 alpn string +- copyright: updated year ranges out of sync - Closes #4183 + ... and whitelisted a few more files in the the copyright.pl script. -- alt-svc: more liberal ALPN name parsing - - Allow pretty much anything to be part of the ALPN identifier. In - particular minus, which is used for "h3-20" (in-progress HTTP/3 - versions) etc. +- [Gilles Vollant brought this change] + + setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - Updated test 356. - Closes #4182 + Closes #5431 -- quiche: use the proper HTTP/3 ALPN +- curl: remove -J "informational" written on stdout + + curl would previously show "curl: Saved to filename 'name from header'" + if -J was used and a name was picked from the Content-Disposition + header. That output could interfer with other stdout output, such as -w. + + This commit removes that output line. + Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html + Reported-by: Коваленко Анатолий Викторович + Closes #5435 -- quiche: add failf() calls for two error cases +Peter Wu (22 May 2020) +- travis: simplify quiche build instructions wrt boringssl - To aid debugging + quiche builds boringssl as static library, reuse that instead of + building another shared library. - Closes #4181 + Closes #5438 -- mailmap: added Kyohei Kadota +- configure: fix pthread check with static boringssl + + A shared boringssl/OpenSSL library requires -lcrypto only for linking. + A static build additionally requires `-ldl -lpthread`. In the latter + case `-lpthread` is added to LIBS which prevented `-pthread` from being + added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. -Kamil Dudka (1 Aug 2019) -- http_negotiate: improve handling of gss_init_sec_context() failures +Daniel Stenberg (22 May 2020) +- Revert "sendf: make failf() use the mvsnprintf() return code" - If HTTPAUTH_GSSNEGOTIATE was used for a POST request and - gss_init_sec_context() failed, the POST request was sent - with empty body. This commit also restores the original - behavior of `curl --fail --negotiate`, which was changed - by commit 6c6035532383e300c712e4c1cd9fdd749ed5cf59. + This reverts commit 74623551f306990e70c7c5515b88972005604a74. - Add regression tests 2077 and 2078 to cover this. + Instead mark the function call with (void). Getting the return code and + using it instead triggered Coverity warning CID 1463596 because + snprintf() can return a negative value... - Fixes #3992 - Closes #4171 + Closes #5441 -Daniel Stenberg (1 Aug 2019) -- mailmap: added 4 more names +- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' - Evgeny Grin, Peter Pih, Anton Malov and Marquis de Muesli + Reported-by: Billyzou0741326 on github + Fixes #5432 + Closes #5436 -- mailmap: add Giorgos Oikonomou +- tests/server/util.h: add extern to silence compiler warning + + Follow-up from a3b0699d5c1 -- src/makefile: fix uncompressed hugehelp.c generation +- typecheck-gcc.h: fix the OFF_T check - Regression from 5cf5d57ab9 (7.64.1) + The option number also needs to be less than CURLOPTTYPE_BLOB. - Fixed-by: Lance Ware - Fixes #4176 - Closes #4177 + Follow-up to cac5374298 + Reported-by: Jeroen Ooms + Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 -- appveyor: pass on -k to make +- TODO: --dry-run + + Closes #5426 -- timediff: make it 64 bit (if possible) even with 32 bit time_t +- TODO: Ratelimit or wait between serial requests - ... to make it hold microseconds too. + Closes #5406 + +- tool_paramhlp: fixup C89 mistake - Fixes #4165 - Closes #4168 + Follow-up to c5f0a9db22. -- ROADMAP: parallel transfers are merged now +- [Siva Sivaraman brought this change] -- getenv: support up to 4K environment variable contents on windows + tool_paramhlp: fixed potentially uninitialized strtol() variable - Reported-by: Michal Čaplygin - Fixes #4174 - Closes #4175 + Seems highly unlikely to actually be possible, but better safe than + sorry. + + Closes #5417 -- [Kyohei Kadota brought this change] +- [Siva Sivaraman brought this change] - plan9: add support for running on Plan 9 + tool_operate: fixed potentially uninitialized variables - Closes #3701 - -- [Kyohei Kadota brought this change] + ... in curl_easy_getinfo() calls. They're harmless but clearing the + variables makes the code safer and comforts the reader. + + Closes #5416 - ntlm: explicit type casting +- sha256: move assign to the declaration line + + Follow-up to fae30656. Should've been squashed with that commit... -- [Justin brought this change] +- [Siva Sivaraman brought this change] - curl.h: fix outdated comment + sha256: fixed potentially uninitialized variable - Closes #4167 + Closes #5414 -- curl: remove outdated comment +- sendf: make failf() use the mvsnprintf() return code - Turned bad with commit b8894085000 + ... and avoid a strlen() call. Fixes a MonocleAI warning. - Reported-by: niallor on github - Fixes #4172 - Closes #4173 + Reported-by: MonocleAI + Fixes #5413 + Closes #5420 -- cleanup: remove the 'numsocks' argument used in many places +- hostip: make Curl_printable_address not return anything - It was used (intended) to pass in the size of the 'socks' array that is - also passed to these functions, but was rarely actually checked/used and - the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries - that should be used instead. + It was not used much anyway and instead we let it store a blank buffer + in case of failure. - Closes #4169 + Reported-by: MonocleAI + Fixes #5411 + Closes #5418 -- readwrite_data: repair setting the TIMER_STARTTRANSFER stamp - - Regression, broken in commit 65eb65fde64bd5f (curl 7.64.1) - - Reported-by: Jonathan Cardoso Machado - Assisted-by: Jay Satiro +- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) - Fixes #4136 - Closes #4162 + They're done on purpose, make that visible in the code. + Reported-by: MonocleAI + Fixes #5412 + Closes #549 -- mailmap: Amit Katyal - -- asyn-thread: removed unused variable +- TODO: forbid TLS post-handshake auth and do TLS record padding - Follow-up to eb9a604f. Mistake caused by me when I edited the commit - before push... + Closes #5396 + Closes #5398 - RELEASE-NOTES: synced -- [Amit Katyal brought this change] - - asyn-thread: create a socketpair to wait on +- dynbuf: return NULL when there's no buffer length + + ... as returning a "" is not a good idea as the string is supposed to be + allocated and returning a const string will cause issues. - Closes #4157 + Reported-by: Brian Carpenter + Follow-up to ed35d6590e72c + Closes #5405 -- curl: cap the maximum allowed values for retry time arguments +Peter Wu (16 May 2020) +- travis: upgrade to bionic, clang-9, improve readability - ... to avoid integer overflows later when multiplying with 1000 to - convert seconds to milliseconds. + Changes, partially to reduce build failures from external dependencies: + - Upgrade Ubuntu and drop unnecessary third-party repos. + - Properly clone apt config to ensure retries. + - Upgrade to clang-9 from the standard repos. + - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey + fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use + focal everywhere yet since Travis CI has not documented this option. + In focal, python-impacket (Py2.7) has been removed, leaving only + python3-impacket. Since it is only needed for SMB tests and not SSH, + skip it for the libssh job since it might need more work. + - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. - Added test 1269 to verify. + Non-functional cleanups: + - Simplify test matrix, drop redundant os and compiler keys. + - Deprecation fixes: remove sudo, rename matrix -> jobs. + - Every job has an 'env' key, put this key first in a list item. - Reported-by: Jason Lee - Closes #4166 + Closes #5370 -- progress: reset download/uploaded counter +- travis: whitespace-only changes for consistency - ... to make CURLOPT_MAX_RECV_SPEED_LARGE and - CURLOPT_MAX_SEND_SPEED_LARGE work correctly on subsequent transfers that - reuse the same handle. + Automatically apply a consistent indentation with: - Fixed-by: Ironbars13 on github - Fixes #4084 - Closes #4161 - -- http2_recv: trigger another read when the last data is returned + python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' - ... so that end-of-stream is detected properly. + followed by manually re-indenting three comments. - Reported-by: Tom van der Woerdt - Fixes #4043 - Closes #4160 + Closes #5370 -- curl: avoid uncessary libcurl timeouts (in parallel mode) - - When curl_multi_wait() returns OK without file descriptors to wait for, - it might already have done a long timeout. +- CMake: add libssh build support - Closes #4159 - -- [Balazs Kovacsics brought this change] + Closes #5372 - HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown +Daniel Stenberg (15 May 2020) +- KNOWN_BUGS: wolfssh: publickey auth doesn't work - If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set, - automatically add a Transfer-Encoding: chunked header, same as it is - already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME. Update - test 1514 according to the new behaviour. + Closes #4820 + +- KNOWN_BUGS: OS400 port requires deprecated IBM library - Closes #4138 + Closes #5176 -Jay Satiro (29 Jul 2019) -- [Daniel Stenberg brought this change] +- [Vyron Tsingaras brought this change] - winbuild: add vquic to list of build directories + http2: keep trying to send pending frames after req.upload_done - This fixes the winbuild build method which broke several days ago - when experimental quic support was added in 3af0e76. - - Reported-by: Michael Lee - - Fixes https://github.com/curl/curl/issues/4158 + Fixes #1410 + Closes #5401 -- easy: resize receive buffer on easy handle reset +- [Gilles Vollant brought this change] + + setopt: support certificate options in memory with struct curl_blob - - In curl_easy_reset attempt to resize the receive buffer to its default - size. If realloc fails then continue using the previous size. + This change introduces a generic way to provide binary data in setopt + options, called BLOBs. - Prior to this change curl_easy_reset did not properly handle resetting - the receive buffer (data->state.buffer). It reset the variable holding - its size (data->set.buffer_size) to the default size (READBUFFER_SIZE) - but then did not actually resize the buffer. If a user resized the - buffer by using CURLOPT_BUFFERSIZE to set the size smaller than the - default, later called curl_easy_reset and attempted to reuse the handle - then a heap overflow would very likely occur during that handle's next - transfer. + This change introduces these new setopts: - Reported-by: Felix Hädicke + CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, + CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. - Fixes https://github.com/curl/curl/issues/4143 - Closes https://github.com/curl/curl/pull/4145 - -- [Brad Spencer brought this change] + Reviewed-by: Daniel Stenberg + Closes #5357 - examples: Avoid reserved names in hiperfifo examples - - - Trade in __attribute__((unused)) for the classic (void)x to silence - unused symbols. +- source cleanup: remove all custom typedef structs - Because the classic way is not gcc specific. Also because the prior - method mapped to symbol _Unused, which starts with _ and a capital - letter which is reserved. + - Stick to a single unified way to use structs + - Make checksrc complain on 'typedef struct {' + - Allow them in tests, public headers and examples - Assisted-by: The Infinnovation team + - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually + typedef different types/structs depending on build conditions. - Bug: https://github.com/curl/curl/issues/4120#issuecomment-512542108 + Closes #5338 + +- travis: remove the .checksrc fiddling + +- ftp: make domore_getsock() return the secondary socket properly - Closes https://github.com/curl/curl/pull/4153 + Previously, after PASV and immediately after the data connection has + connected, the function would only return the control socket to wait for + which then made the data connection simply timeout and not get polled + correctly. This become obvious when running test 1631 and 1632 event- + based. -Daniel Stenberg (25 Jul 2019) -- RELEASE-NOTES: synced +- test1632: verify FTP through HTTPS-proxy with connection re-use -- [Felix Hädicke brought this change] +- test1631: verify FTP download through HTTPS-proxy - ssh-libssh: do not specify O_APPEND when not in append mode +- sws: as last resort, get test number from server cmd file - Specifying O_APPEND in conjunction with O_TRUNC and O_CREAT does not - make much sense. And this combination of flags is not accepted by all - SFTP servers (at least not Apache SSHD). + If it can't be found in the request. Also support --cmdfile to set it to + a custom file name. - Fixes #4147 - Closes #4148 + runtests.pl always writes this file with the test number in it since a + while back. -- [Gergely Nagy brought this change] +- ftp: shut down the secondary connection properly when SSL is used + + Reported-by: Neal Poole + Fixes #5340 + Closes #5385 - multi: call detach_connection before Curl_disconnect +Marcel Raad (14 May 2020) +- KNOWN_BUGS: adapt 5.5 to recent changes - Curl_disconnect bails out if conn->easyq is not empty, detach_connection - needs to be called first to remove the current easy from the queue. + It only applies to non-Unicode builds now. + Also merge 5.10 into it as it's effectively a duplicate. - Fixes #4144 - Closes #4151 + Closes https://github.com/curl/curl/pull/3784 -Jay Satiro (23 Jul 2019) -- tool_operate: fix implicit call to easysrc_cleanup - - easysrc_cleanup is only defined when CURL_DISABLE_LIBCURL_OPTION is not - defined, and prior to this change would be called regardless. +- curl_setup: support Unicode functions to open files on Windows - Bug: https://github.com/curl/curl/pull/3804#issuecomment-513922637 - Reported-by: Marcel Raad + Use them only if `_UNICODE` is defined, in which case command-line + arguments have been converted to UTF-8. - Closes https://github.com/curl/curl/pull/4142 + Closes https://github.com/curl/curl/pull/3784 -Daniel Stenberg (22 Jul 2019) -- curl:create_transfers check return code from curl_easy_setopt +- tool: support UTF-16 command line on Windows - From commit b8894085 + - use `wmain` instead of `main` when `_UNICODE` is defined [0] + - define `argv_item_t` as `wchar_t *` in this case + - use the curl_multibyte gear to convert the command-line arguments to + UTF-8 - Pointed out by Coverity CID 1451703 + This makes it possible to pass parameters with characters outside of + the current locale on Windows, which is required for some tests, e.g. + the IDN tests. Out of the box, this currently only works with the + Visual Studio project files, which default to Unicode, and winbuild + with the `ENABLE_UNICODE` option. - Closes #4134 + [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + + Ref: https://github.com/curl/curl/issues/3747 + Closes https://github.com/curl/curl/pull/3784 -- HTTP3: initial (experimental) support +- curl_multibyte: add to curlx - USe configure --with-ngtcp2 or --with-quiche + This will also be needed in the tool and tests. - Using either option will enable a HTTP3 build. - Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me> + Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 + Closes https://github.com/curl/curl/pull/3784 + +Daniel Stenberg (14 May 2020) +- url: make the updated credentials URL-encoded in the URL - Closes #3500 + Found-by: Gregory Jefferis + Reported-by: Jeroen Ooms + Added test 1168 to verify. Bug spotted when doing a redirect. + Bug: https://github.com/jeroen/curl/issues/224 + Closes #5400 -- curl: remove dead code +- tests: add https-proxy support to the test suite - The loop never loops (since b889408500), pointed out by Coverity (CID - 1451702) + Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like + HTTP proxy but with a full TLS connection to the proxy. - Closes #4133 + Closes #5399 -- docs/PARALLEL-TRANSFERS: correct the version number +- mailmap: James Fuller -- docs/PARALLEL-TRANSFERS: added +- [Major_Tom brought this change] -- curl: support parallel transfers + vauth/cleartext: fix theoretical integer overflow + + Fix theoretical integer overflow in Curl_auth_create_plain_message. - This is done by making sure each individual transfer is first added to a - linked list as then they can be performed serially, or at will, in - parallel. + The security impact of the overflow was discussed on hackerone. We + agreed this is more of a theoretical vulnerability, as the integer + overflow would only be triggerable on systems using 32-bits size_t with + over 4GB of available memory space for the process. - Closes #3804 + Closes #5391 -- docs/MANUAL.md: converted to markdown from plain text +Jay Satiro (13 May 2020) +- curl.1: Quote globbed URLs + + - Quote the globbing example URLs that contain characters [] {} since + otherwise they may be interpreted as shell metacharacters. - ... will make it render as a nicer web page. + Bug: https://github.com/curl/curl/issues/5388 + Reported-by: John Simpson - Closes #4131 + Closes https://github.com/curl/curl/pull/5394 -- curl_version_info: provide nghttp2 details +Daniel Stenberg (14 May 2020) +- checksrc: enhance the ASTERISKSPACE and update code accordingly - Introducing CURLVERSION_SIXTH with nghttp2 info. + Fine: "struct hello *world" - Closes #4121 + Not fine: "struct hello* world" (and variations) + + Closes #5386 -- bump: start working on 7.66.0 +- docs/options-in-versions: which version added each cmdline option + + Added test 971 to verify that the list is in sync with the files in + cmdline-opts. The check also verifies that .d-files that uses Added: + specify the same version number as the options-in-versions file does. + + Closes #5381 -- source: remove names from source comments +- docs: unify protocol lists - Several reasons: + We boast support for 25 transfer protocols. Make sure the lists are + consistent - - we can't add everyone who's helping out so its unfair to just a few - selected ones. - - we already list all helpers in THANKS and in RELEASE-NOTES for each - release - - we don't want to give the impression that some parts of the code is - "owned" or "controlled" by specific persons + Closes #5384 + +- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN - Assisted-by: Daniel Gustafsson - Closes #4129 + ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. + + Reported-by: Michael Kaufmann + Fixes #5374 + Closes #5376 -Version 7.65.3 (19 Jul 2019) +- tls13-ciphers.d: shorten the Arg -Daniel Stenberg (19 Jul 2019) -- RELEASE-NOTES: 7.65.3 +- sasl-authzid.d: add Arg: and shorten the desc -- THANKS: 7.65.3 status +- cert-type.d: mention the available types in the desc -- progress: make the progress meter appear again +- tool: shorten 3 --help descriptions + + --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort - Fix regression caused by 21080e1 + gen.pl already warned about these lines but we didn't listen - Reported-by: Chih-Hsuan Yen - Fixes #4122 - Closes #4124 + Closes #5379 -- version: bump to 7.65.3 +- configure: the wolfssh backend does not provide SCP + + Closes #5387 -- RELEASE-NOTES: Contributors or now 1990 +- RELEASE-NOTES: synced -Version 7.65.2 (17 Jul 2019) +- url: reject too long input when parsing credentials + + Since input passed to libcurl with CURLOPT_USERPWD and + CURLOPT_PROXYUSERPWD circumvents the regular string length check we have + in Curl_setstropt(), the input length limit is enforced in + Curl_parse_login_details too, separately. + + Reported-by: Thomas Bouzerar + Closes #5383 -Daniel Stenberg (17 Jul 2019) -- RELEASE-NOTES: 7.65.2 +- list-only.d: this option existed already in 4.0 -- THANKS: add contributors from 7.65.2 +Jay Satiro (12 May 2020) +- retry-all-errors.d: Shorten the summary line + + Follow-up to b995bb5 from a few moments ago. + + Reported-by: Daniel Stenberg + + Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 -Jay Satiro (17 Jul 2019) -- [aasivov brought this change] +- [denzor brought this change] - cmake: Fix finding Brotli on case-sensitive file systems + easy: fix dangling pointer on easy_perform fail - - Find package "Brotli" instead of "BROTLI" since the former is the - casing used for CMake/FindBrotli.cmake, and otherwise find_package - may fail on a case-sensitive file system. + Closes https://github.com/curl/curl/pull/5363 + +- tool: Add option --retry-all-errors to retry on any error - Fixes https://github.com/curl/curl/issues/4117 + The "sledgehammer" of retrying. + + Closes https://github.com/curl/curl/pull/5185 + +Daniel Stenberg (12 May 2020) +- [James Le Cuirot brought this change] -- CURLOPT_RANGE.3: Caution against using it for HTTP PUT + libcurl.pc: Merge Libs.private into Libs for static-only builds - AFAICT CURLOPT_RANGE does not support ranged HTTP PUT uploads so I've - cautioned against using it for that purpose and included a workaround. + A project being built entirely statically will call pkg-config with + --static, which utilises the Libs.private field. Conversely it will + not use --static when not being built entirely statically, even if + there is only a static build of libcurl available. This will most + likely cause the build to fail due to underlinking unless we merge the + Libs fields. - Bug: https://curl.haxx.se/mail/lib-2019-04/0075.html - Reported-by: Christopher Head + Consider that this is what the Meson build system does when it + generates pkg-config files. - Closes https://github.com/curl/curl/issues/3814 + I have also reflected this in the --libs argument of curl-config even + though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + + Closes #5373 -- [Stefano Simonelli brought this change] +- [Peter Wu brought this change] - CURLOPT_SEEKDATA.3: fix variable name - - Closes https://github.com/curl/curl/pull/4118 + CMake: fix runtests.pl with CMake, add new test targets + + * runtests.pl: + - Fix out-of-tree build under CMake when srcdir is not set. Default + srcdir to the location of runtests.pl. + - Add a hack to allow CMake to use the TFLAGS option as documented + in tests/README and used in scripts/travis/script.sh. + * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie + support (no one should care, it is already EOL.). + * Remove CTest since it defines its own 'test' target with no tests + since all unittests are already broken and not built by default. + * Add new test targets based on the options from Makefile.am. Since + new test targets are rarely added, I opted for duplicating the + runtests.pl options as opposed to creating a new Makefile.inc file. + Use top-level target names (test-x) instead of x-test since that is + used by CI and others. + + Closes #5358 -- [Giorgos Oikonomou brought this change] +- [Peter Wu brought this change] - CIPHERS.md: Explain Schannel error SEC_E_ALGORITHM_MISMATCH + CMake: do not build test programs by default - If the SSL backend is Schannel and the user specifies an Schannel CALG_ - that is not supported by the protocol or the server then curl returns - CURLE_SSL_CONNECT_ERROR (35) SEC_E_ALGORITHM_MISMATCH. + The default target should only build libcurl and curl. Add a dedicated + 'testdeps' target which will be used later when running tests. Note that + unittests are currently broken in CMake and already excluded. - Fixes https://github.com/curl/curl/issues/3389 - Closes https://github.com/curl/curl/pull/4106 + Closes #5368 -- [Daniel Gustafsson brought this change] +- FILEFORMAT: moved up the variables section and further polished - nss: inspect returnvalue of token check +- runtests: remove ftp2 support, not used - PK11_IsPresent() checks for the token for the given slot is available, - and sets needlogin flags for the PK11_Authenticate() call. Should it - return false, we should however treat it as an error and bail out. + We once supported two separate ftp instances in the test suite. Has not + been used the last decade. - Closes https://github.com/curl/curl/pull/4110 + Closes #5375 -- docs: Explain behavior change in --tlsv1. options since 7.54 +- url: sort the protocol schemes in rough popularity order + + When looking for a protocol match among supported schemes, check the + most "popular" schemes first. It has zero functionality difference and + for all practical purposes a speed difference will not be measureable + but it still think it makes sense to put the least likely matches last. - Since 7.54 --tlsv1. options use the specified version or later, however - older versions of curl documented it as using just the specified version - which may or may not have happened depending on the TLS library. - Document this discrepancy to allay confusion for users familiar with the - old documentation that expect just the specified version. + "Popularity" based on the 2019 user survey. - Fixes https://github.com/curl/curl/issues/4097 - Closes https://github.com/curl/curl/pull/4119 + Closes #5377 -- libcurl: Restrict redirect schemes (follow-up) +Marc Hoersken (11 May 2020) +- test1238: avoid tftpd being busy for tests shortly following - - Allow FTPS on redirect. + The tftpd server may still be busy if the total timeout of + 25 seconds has not been reached or no sread error was received + during or after the execution of the timeout test 1238. - - Update default allowed redirect protocols in documentation. + Once the next TFTP test comes around (eg. 1242 or 1243), + those will fail because the tftpd server is still waiting + on data from curl due to the UDP protocol being stateless + and having no connection close. On Linux this error may not + happen, because ICMP errors generated due to a swrite error + can also be returned async on the next sread call instead. - Follow-up to 6080ea0. + Therefore we will now just kill the tftpd server after test + 1238 to make sure that the following tests are not affected. - Ref: https://github.com/curl/curl/pull/4094 + This enables us to no longer ignore tests 1242, 1243, 2002 + and 2003 on the CI platforms CirrusCI and AppVeyor. - Closes https://github.com/curl/curl/pull/4115 + Assisted-by: Peter Wu + Closes #5364 -Daniel Stenberg (16 Jul 2019) -- test1173: make it also check all libcurl option man pages - - ... and adjust those that cause errors +Daniel Stenberg (11 May 2020) +- write-out.d: added "response_code" + +- KNOWN_BUGS: Build with staticly built dependency - Closes #4116 + I rewrote the item 5.4 to be more generic about static dependencies. -- curl: only accept COLUMNS less than 10000 +- ROADMAP: remove old entries - ... as larger values would rather indicate something silly (and could - potentially cause buffer problems). + MQTT - the start has already landed - Reported-by: pendrek at hackerone - Closes #4114 - -- dist: add manpage-syntax.pl + tiny-curl - also mostly landed and is a continuous work - follow-up to 7fb66c403 + make menuconfig - basically no interest from users, not pushing there -- test1173: detect some basic man page format mistakes - - Triggered by PR #4111 +- [Peter Wu brought this change] + + travis: Add ngtcp2 and quiche tests for CMake - Closes #4113 + To avoid an explosion of jobs, extend the existing CMake tests with + ngtcp2 and quiche support. macOS was previously moved to GitHub actions, + so the non-Linux case can be dropped. -Jay Satiro (15 Jul 2019) -- [Bjarni Ingi Gislason brought this change] +- [Peter Wu brought this change] - docs: Fix missing lines caused by undefined macros + CMake: add ENABLE_ALT_SVC option - - Escape apostrophes at line start. + Tested alt-svc with quiche. While at it, add missing MultiSSL reporting + (not tested). + +- [Peter Wu brought this change] + + CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) - Some lines begin with a "'" (apostrophe, single quote), which is then - interpreted as a control character in *roff. + Add three new CMake Find modules (using the curl license, but I grant + others the right to apply the CMake BSD license instead). - Such lines are interpreted as being a call to a macro, and if - undefined, the lines are removed from the output. + This CMake config is simpler than the autotools one because it assumes + ngtcp2 and nghttp3 to be used together. Another difference is that this + CMake config checks whether QUIC is actually supported by the TLS + library (patched OpenSSL or boringssl) since this can be a common + configuration mistake that could result in build errors later. - Bug: https://bugs.debian.org/926352 - Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> + Unlike autotools, CMake does not warn you that the features are + experimental. The user is supposed to already know that and read the + documentation. It requires a very special build environment anyway. - Submitted-by: Alessandro Ghedini + Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from + current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure + out which features (libldap-2.4, libssh2) to disable due to conflicts + with boringssl. - Closes https://github.com/curl/curl/pull/4111 + Closes #5359 -Daniel Stenberg (14 Jul 2019) -- libcurl-security.3: update to new CURLOPT_REDIR_PROTOCOLS defaults +Marc Hoersken (10 May 2020) +- tests/server/tftpd.c: fix include and enhance debug logging + + setjmp.h should only be included if HAVE_SETJMP_H is defined. - follow-up to 6080ea098 + Add additional log statements to see wether reads and writes + are blocking or finishing before an alarm signal is received. + + Assisted-by: Peter Wu + Part of #5364 -- [Linos Giannopoulos brought this change] +Daniel Stenberg (10 May 2020) +- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present + + Reported-by: Marcel Raad + Follow-up to 148534db5 + Fixes #5367 + Closes #5369 - libcurl: Add testcase for gopher redirects +Marc Hoersken (9 May 2020) +- appveyor: update comments to be clear about toolchain - The testcase ensures that redirects to CURLPROTO_GOPHER won't be - allowed, by default, in the future. Also, curl is being used - for convenience while keeping the testcases DRY. + - CMake-based MSYS builds use mingw-w64 to cross-compile. + - autotools-based builds are compiled using msys2-devel. - The expected error code is CURLE_UNSUPPORTED_PROTOCOL when the client is - redirected to CURLPROTO_GOPHER + The difference is that the later ones are not cross-compiled + to Windows and instead require the msys2 runtime to be present. - Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr> + At the moment only the Azure Pipelines CI builds actually + run autotools-based cross-compilation builds for Windows. -- [Linos Giannopoulos brought this change] - - libcurl: Restrict redirect schemes +- TODO: update regarding missing Schannel features - All protocols except for CURLPROTO_FILE/CURLPROTO_SMB and their TLS - counterpart were allowed for redirect. This vastly broadens the - exploitation surface in case of a vulnerability such as SSRF [1], where - libcurl-based clients are forced to make requests to arbitrary hosts. + Some aspects have already been implemented over the years. - For instance, CURLPROTO_GOPHER can be used to smuggle any TCP-based - protocol by URL-encoding a payload in the URI. Gopher will open a TCP - connection and send the payload. + 15.1 Client certificates are now supported: - Only HTTP/HTTPS and FTP are allowed. All other protocols have to be - explicitly enabled for redirects through CURLOPT_REDIR_PROTOCOLS. + - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 + - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 - [1]: https://www.acunetix.com/blog/articles/server-side-request-forgery-vulnerability/ + 15.2 Ciphers can now be specified through: - Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr> + - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 - Closes #4094 + Reviewed-by: Daniel Stenberg and Marcel Raad + Closes #5358 -- [Zenju brought this change] +Daniel Stenberg (8 May 2020) +- checksrc: close the .checksrc file handle when done reading - openssl: define HAVE_SSL_GET_SHUTDOWN based on version number +- RELEASE-NOTES: synced - Closes #4100 + And bumped next version to 7.71.0 -- [Peter Simonyi brought this change] +- [Gilles Vollant brought this change] - http: allow overriding timecond with custom header - - With CURLOPT_TIMECONDITION set, a header is automatically added (e.g. - If-Modified-Since). Allow this to be replaced or suppressed with - CURLOPT_HTTPHEADER. + CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) - Fixes #4103 - Closes #4109 + Closes #4346 -Jay Satiro (11 Jul 2019) -- [Juergen Hoetzel brought this change] +- TODO: native IDN support on macOS - smb: Use the correct error code for access denied on file open - - - Return CURLE_REMOTE_ACCESS_DENIED for SMB access denied on file open. +- urlapi: accept :: as a valid IPv6 address - Prior to this change CURLE_REMOTE_FILE_NOT_FOUND was returned instead. + Text 1560 is extended to verify. - Closes https://github.com/curl/curl/pull/4095 + Reported-by: Pavel Volgarev + Fixes #5344 + Closes #5351 -- [Daniel Gustafsson brought this change] +- THANKS-filter: Peter Wang - DEPRECATE: fixup versions and spelling +- [Peter Wang brought this change] + + *_sspi: fix bad uses of CURLE_NOT_BUILT_IN - Correctly set the July 17 version to 7.65.2, and update spelling to - be consistent. Also fix a typo. + Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other + instances of QuerySecurityPackageInfo failing, as in + commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. - Closes https://github.com/curl/curl/pull/4107 + Closes #5355 -- [Gisle Vanem brought this change] +- docs/HTTP3: add qlog to the quiche build instruction - system_win32: fix clang warning +- ngtcp2: introduce qlog support - - Declare variable in header as extern. + If the QLOGDIR environment variable is set, enable qlogging. - Bug: https://github.com/curl/curl/commit/48b9ea4#commitcomment-34084597 - -Daniel Gustafsson (10 Jul 2019) -- headers: Remove no longer exported functions + ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for + QUIC functions that are backend independent. - There were a leftover few prototypes of Curl_ functions that we used to - export but no longer do, this removes those prototypes and cleans up any - comments still referring to them. - - Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free() - Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn() - were made static in 05b100aee247bb9bec8e9a1b0166496aa4248d1c. - Curl_http_perhapsrewind() made static in 574aecee208f79d391f10d57520b3. + Closes #5353 + +- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - For the remainder, I didn't trawl the Git logs hard enough to capture - their exact time of deletion, but they were all gone: Curl_splayprint(), - Curl_http2_send_request(), Curl_global_host_cache_dtor(), - Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(), - Curl_http_auth_stage() and Curl_close_connections(). + That return code is reserved for build-time conditional code not being + present while this was a regular run-time error from a Windows API. - Closes #4096 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Reported-by: wangp on github + Fixes #5349 + Closes #5350 -- CMake: fix typos and spelling +- runtests: show elapsed test time with higher precision (ms) -- [Kyle Edwards brought this change] +- RELEASE-NOTES: synced - CMake: Convert errant elseif() to else() +- http2: simplify and clean up trailer handling - CMake interprets an elseif() with no arguments as elseif(FALSE), - resulting in the elseif() block not being executed. That is not what - was intended here. Change the empty elseif() to an else() as it was - intended. + Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in + ed35d6590e72. This should make the trailer handling more straight forward and + hopefully less error-prone. - Closes #4101 - Reported-by: Artalus <artalus-mail@yandex.ru> - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> - -- buildconf: fix header filename - - The header file inclusion had a typo, it should be .h and not .hd. - Fix by renaming. + Deliver the trailer header to the callback already at receive-time. No + longer caches the trailers to get delivered at end of stream. - Fixes #4102 - Reported-by: AceCrow on Github + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 + Closes #5348 -- [Jan Chren brought this change] +Marc Hoersken (7 May 2020) +- appveyor: disable test 1139 instead of ignoring it + + Spending time on manpage checking makes no sense + for these builds due to lacking manpage support. - configure: fix --disable-code-coverage +- appveyor: disable flaky test 1501 and ignore broken 1056 - This fixes the case when --disable-code-coverage supplied to ./configure - would result in coverage="yes" being set. + Test 1501 is flaky on Windows CI due to being time sensitive + and the testsuite relying on taskkill.exe to check for the + existance of processes which can take to much time itself. - Closes #4099 - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> - -- cleanup: fix typo in comment + Test 1056 is broken in autotools-based Windows builds due + to scope ID support missing in these builds at the moment. -- RELEASE-NOTES: synced - -Jay Satiro (6 Jul 2019) -- [Daniel Gustafsson brought this change] - - nss: support using libnss on macOS +- test613.pl: make tests 613 and 614 work with OpenSSH for Windows - The file suffix for dynamically loadable objects on macOS is .dylib, - which need to be added for the module definitions in order to get the - NSS TLS backend to work properly on macOS. + OpenSSH for Windows shows group and other/world permissions as *, + because those concepts do not exist on Windows. It also does not + show the current or parent directory, so we just ignore those. - Closes https://github.com/curl/curl/pull/4046 + Reviewed-by: Daniel Stenberg + Closes #5328 -- [Daniel Gustafsson brought this change] +Daniel Stenberg (6 May 2020) +- runtests: set +x mode again - nss: don't set unused parameter +- libssh2: convert over to use dynbuf - The value of the maxPTDs parameter to PR_Init() has since at least - NSPR 2.1, which was released sometime in 1998, been marked ignored - as is accordingly not used in the initialization code. Setting it - to a value when calling PR_Init() is thus benign, but indicates an - intent which may be misleading. Reset the value to zero to improve - clarity. + In my very basic test that lists sftp://127.0.0.1/tmp/, this patched + code makes 161 allocations compared to 194 in git master. A 17% + reduction. - Closes https://github.com/curl/curl/pull/4054 + Closes #5336 -- [Daniel Gustafsson brought this change] +- travis: add "qlog" as feature in the quiche build - nss: only cache valid CRL entries +- quiche: enable qlog output - Change the logic around such that we only keep CRLs that NSS actually - ended up caching around for later deletion. If CERT_CacheCRL() fails - then there is little point in delaying the freeing of the CRL as it - is not used. + quiche has the potential to log qlog files. To enable this, you must + build quiche with the qlog feature enabled `cargo build --features + qlog`. curl then passes a file descriptor to quiche, which takes + ownership of the file. The FD transfer only works on UNIX. - Closes https://github.com/curl/curl/pull/4053 - -- [Gergely Nagy brought this change] - - lib: Use UTF-8 encoding in comments + The convention is to enable logging when the QLOGDIR environment is + set. This should be a path to a folder where files are written with the + naming template <SCID>.qlog. - Some editors and IDEs assume that source files use UTF-8 file encodings. - It also fixes the build with MSVC when /utf-8 command line option is - used (this option is mandatory for some other open-source projects, this - is useful when using the same options is desired for building all - libraries of a project). + Co-authored-by: Lucas Pardue + Replaces #5337 + Closes #5341 + +- urldata.h: remove #define HEADERSIZE, not used anymore - Closes https://github.com/curl/curl/pull/4087 + Follow-up to ed35d6590e72c -- [Caleb Raitto brought this change] +- ngtcp2: convert to dynbuf + + Closes #5335 - CURLOPT_HEADEROPT.3: Fix example +- connect: make happy eyeballs work for QUIC (again) - Fix an issue where example builds a curl_slist, but fails to actually - use it, or free it. + Follow-up from dbd16c3e256c6c (regression in 7.70.0) - Closes https://github.com/curl/curl/pull/4090 + Closes #5334 -- [Shankar Jadhavar brought this change] +- connect: add two asserts to clue code analyzers in a little + +- http_proxy: ported to use dynbuf instead of a static size buffer + + Removes a 16K static buffer from the easy handle. Simplifies the code. - winbuild: Change Makefile to honor ENABLE_OPENSSL_AUTO_LOAD_CONFIG +- dynbuf: introduce internal generic dynamic buffer functions - - Made changes so that ENABLE_OPENSSL_AUTO_LOAD_CONFIG will be honored. + A common set of functions instead of many separate implementations for + creating buffers that can grow when appending data to them. Existing + functionality has been ported over. - - Also removed some ^M chars from file. + In my early basic testing, the total number of allocations seem at + roughly the same amount as before, possibly a few less. - Prior to this change while building on Windows platform even if we pass - the ENABLE_OPENSSL_AUTO_LOAD_CONFIG option with value as "no" it does - not set the CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG flag. + See docs/DYNBUF.md for a description of the API. - Closes https://github.com/curl/curl/pull/4086 + Closes #5300 -Daniel Stenberg (4 Jul 2019) -- doh-url.d: added in 7.62.0 +- runtests: remove sleep calls + + Remove many one second sleeps that were done *after* each newly started + test server already has been verified. They should not have any purpose + there. + + Closes #5323 -Jay Satiro (30 Jun 2019) -- docs: Fix links to OpenSSL docs +- asyn-*: remove support for never-used NULL entry pointers - OpenSSL changed their manual locations and does not redirect to the new - locations. + ... and instead convert those to asserts to make sure they are truly + never NULL. - Bug: https://curl.haxx.se/mail/lib-2019-06/0056.html - Reported-by: Daniel Stenberg + Closes #5324 -Daniel Stenberg (26 Jun 2019) -- [Gaël PORTAY brought this change] +- [Emil Engler brought this change] - curl_multi_wait.3: escape backslash in example - - The backslash in the character Line Feed must be escaped. + doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax - The current man-page outputs the code as following: + Closes #5325 + +Jay Satiro (2 May 2020) +- asyn-thread: fix cppcheck warning - fprintf(stderr, "curl_multi failed, code %d.0, mc); + - Check for NULL entry parameter before attempting to deref entry in + Curl_resolver_is_resolved, like is already done in asyn-ares. - The commit fixes it as follow: + This is to silence cppcheck which does not seem to understand that + asyn-ares and asyn-thread have separate Curl_resolver_is_resolved + and those units are mutually exclusive. Prior to this change it warned + of a scenario where asyn-thread's Curl_resolver_is_resolved is called + with a NULL entry from asyn-ares, but that couldn't happen. - fprintf(stderr, "curl_multi failed, code %d\n", mc); + Reported-by: rl1987@users.noreply.github.com - Closes #4079 + Fixes https://github.com/curl/curl/issues/5326 -- openssl: disable engine if OPENSSL_NO_UI_CONSOLE is defined +- select: fix overflow protection in Curl_socket_check + + Follow-up to a96c752 which changed the timeout_ms type from time_t to + timediff_t. - ... since that needs UI_OpenSSL() which isn't provided when OpenSSL is - built with OPENSSL_NO_UI_CONSOLE which happens when OpenSSL is built for - UWP (with "VC-WIN32-UWP"). + Ref: https://github.com/curl/curl/pull/5240 - Reported-by: Vasily Lobaskin - Fixes #4073 - Closes #4077 + Closes https://github.com/curl/curl/pull/5286 -- test1521: adapt to SLISTPOINT +Marc Hoersken (2 May 2020) +- sockfilt: make select_ws stop waiting on exit signal event - The header now has the slist-using options marked as SLISTPOINT so this - makes sure test 1521 understands that. + This makes sure that select_ws behaves similar to real select + which stops waiting on a signal handler being triggered. - Follow-up to ae99b4de1c443ae989 + This makes it possible to gracefully stop sockfilt.exe on + Windows with taskkill /IM sockfilt.exe (without /F force flag). - Closes #4074 + Reviewed-by: Jay Satiro + Part of #5260 -- win32: make DLL loading a no-op for UWP +- tests/server/util.[ch]: add exit event to stop waiting on Windows - Reported-by: Michael Brehm - Fixes #4060 - Closes #4072 - -- [1ocalhost brought this change] - - configure: fix typo '--disable-http-uath' + This commit adds a global exit event to the test servers that + Windows-specific wait routines can use to get triggered if the + program was signaled to be terminated, eg. select_ws in sockfilt.c + + The exit event will be managed by the signal handling code and is + set to not reset automatically to support multiple wait routines. - Closes #4076 + Reviewed-by: Jay Satiro + Closes #5260 -- [Niklas Hambüchen brought this change] +- tests/server/util.c: fix thread handle not being closed + + Reviewed-by: Jay Satiro + Part of #5260 - docs: fix string suggesting HTTP/2 is not the default +- tests/server/util.c: use raise instead of calling signal handler - Commit 25fd1057c9c86e3 made HTTP2 the default, and further down in the - man page that new default is mentioned, but the section at the top - contradicted it until now. + Use raise to trigger signal handler instead of calling it + directly and causing potential unexpected control flow. - Also remove claim that setting the HTTP version is not sensible. + Reviewed-by: Jay Satiro + Part of #5260 + +- tests: add support for SSH server variant specific transfer paths - Closes #4075 + OpenSSH for Windows requires paths in the format of /C:/ + instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ + + Reviewed-by: Daniel Stenberg + Closes #5298 +Daniel Stenberg (2 May 2020) - RELEASE-NOTES: synced -- [Stephan Szabo brought this change] - - tests: update fixed IP for hostip/clientip split +- libssh2: set the expected total size in SCP upload init - These tests give differences for me on linux when using a hostip - pointing to the external ip address for the local machine. + ... as otherwise the progress callback gets called without that + information, making the progress meter have less info. - Closes #4070 + Reported-by: Murugan Balraj + Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html + Closes #5317 -Daniel Gustafsson (24 Jun 2019) -- http: clarify header buffer size calculation +- runtests: make the logmsg from the ssh server only show in verbose + +- tests: make test 1248 + 1249 use %NOLISTENPORT - The header buffer size calculation can from static analysis seem to - overlow as it performs an addition between two size_t variables and - stores the result in a size_t variable. Overflow is however guarded - against elsewhere since the input to the addition is regulated by - the maximum read buffer size. Clarify this with a comment since the - question was asked. + ... instead of a port of a non-running server so that it works + stand-alone. - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Closes #5318 -Daniel Stenberg (24 Jun 2019) -- KNOWN_BUGS: Don't clear digest for single realm +- examples: remove asiohiper.cpp - Closes #3267 - -- KNOWN_BUGS: Schannel disable CURLOPT_SSL_VERIFYPEER and verify hostname + This example has repeatedly been reported to contain bugs, and as users + copy and paste code from this into production, I now deem it better to + not provide the example at all. - Closes #3284 + Closes #5090 + Closes #5322 + +- [Emil Engler brought this change] -- http2: call done_sending on end of upload + doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 - To make sure a HTTP/2 stream registers the end of stream. + Closes #5320 + +- [Emil Engler brought this change] + + KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" - Bug #4043 made me find this problem but this fix doesn't correct the - reported issue. + It was fixed in 9a2cbf3 - Closes #4068 - -- [James Brown brought this change] + Closes #5319 - c-ares: honor port numbers in CURLOPT_DNS_SERVERS +- cirrus: disable SFTP and SCP tests - By using ares_set_servers_ports_csv on new enough c-ares. + ... as we can't seem to start the sshd server on it. Those problems + existed before d1239b50bececd (running the SSH server on a random port), + but they're more noticable now since there are more failed attempts in + the logs. - Fixes #4066 - Closes #4067 + Closes #5315 -Daniel Gustafsson (24 Jun 2019) -- CURLMOPT_SOCKETFUNCTION.3: fix typo +- [Emil Engler brought this change] -Daniel Stenberg (24 Jun 2019) -- [Koen Dergent brought this change] + runtests: fix typo in the existence of disabled tests checker + + Closes #5316 - curl: skip CURLOPT_PROXY_CAPATH for disabled-proxy builds +Dan Fandrich (30 Apr 2020) +- test75: Remove precheck test - Closes #4061 + This has not been needed since commit 9fa42bed and often prevents it + from running at all with dynamic test ports. -- test153: fix content-length to avoid occasional hang +- tests: Stop referring to server ports when they're not used - Closes #4065 + Several tests referred to specific server ports even when the test + didn't actually use that server or specify that it's needed. In such + cases, the test harness substitutes the text "[not running]" as the port + number which causes many such tests to fail due to the inability to + parse the URL. These tests are changed to use %NOLISTENPORT which will + always be substituted correctly. -- RELEASE-NOTES: synced +Daniel Stenberg (30 Apr 2020) +- [Emil Engler brought this change] -- multi: enable multiplexing by default (again) + GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT - It was originally made default in d7c4213bd0c (7.62.0) but mistakenly - reverted in commit 2f44e94efb3d (7.65.0). Now enabled again. - - Closes #4051 + Closes #5287 -- typecheck: add 3 missing strings and a callback data pointer +- conncache: various concept cleanups - Closes #4050 - -- tests: add disable-scan.pl to dist + More connection cache accesses are protected by locks. + + CONNCACHE_* is a beter prefix for the connection cache lock macros. - follow-up from 29177f422a5 + Curl_attach_connnection: now called as soon as there's a connection + struct available and before the connection is added to the connection + cache. - Closes #4059 + Curl_disconnect: now assumes that the connection is already removed from + the connection cache. + + Ref: #4915 + Closes #5009 -- http2: don't call stream-close on already closed streams +- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports + + As stunnel is an external tool and it has no specific option to export + the actually used port number when asked to listen to 0, runtests + instead iterates over ten randomly picked high number ports and sticks + to the first one stunnel can listen to. - Closes #4055 + Closes #5267 -Marcel Raad (20 Jun 2019) -- travis: enable alt-svc for coverage build +- tests: pick a random port number for SSH - Closes + Since sshd doesn't have such an option by itself, we iterate over a + series of random ports until one works. + + Closes #5273 + +- [Rikard Falkeborn brought this change] -- travis: enable libssh2 for coverage build + libtest/cmake: Remove commented code - It was enabled by default before commit c92d2e14cfb. + These were commented out in e9dd0998706a when Makefile.inc was included + instead. 11 years have passed since then and the commented code is of + course very outdated. Remove it to avoid confusion. + + Closes #5311 + +- schannel: source code reindent - Disable torture tests 600 and 601 because of - https://github.com/curl/curl/issues/1678. + White space edits only. Conform better to standard curl source code + indenting style. - Closes + Closes #5305 -- travis: disable threaded resolver for coverage build +Kamil Dudka (29 Apr 2020) +- test1177: look for curl.h in source directory - This enables more tests. + If we use a separate build directory, there is no copy of the header. - Closes + Closes #5310 -- travis: enable brotli for all xenial jobs +- tests: look for preprocessed tests in build directory - There's no need for a separate job, and no need to build it from source - with Xenial. + ... which is not always the same directory as source directory - Closes + Closes #5310 -- travis: enable warnings-as-errors for coverage build +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: synced - Closes + ... and bumped curlver.h to 7.70.1 -GitHub (20 Jun 2019) -- [Gisle Vanem brought this change] +Version 7.70.0 (29 Apr 2020) + +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: 7.70.0 - system_win32: fix typo +- THANKS: synced with the 7.70.0 release -Daniel Stenberg (20 Jun 2019) -- typecheck: CURLOPT_CONNECT_TO takes an slist too +- headers: copyright range fix + +- [Rikard Falkeborn brought this change] + + doh: Constify some input pointers + + Closes #5306 + +- nss: check for PK11_CreateDigestContext() returning NULL - Additionally, add an alias in curl.h for slist-using options so that - we can grep/parse those out at will. + ... to avoid crashes! - Closes #4042 + Reported-by: Hao Wu + Fixes #5302 + Closes #5303 -- [Stephan Szabo brought this change] +- travis: bump the wolfssl CI build to use 4.4.0 + + Closes #5301 + +- copyright updates: adjust year ranges - tests: support non-localhost HOSTIP for dict/smb servers +Marc Hoersken (26 Apr 2020) +- CI: do not include */ci branches in PR builds - smbserver.py/dictserver.py were explicitly using localhost/127.0.0.1 for - binding the server which when we were running the tests with a separate - HOSTIP and CLIENTIP had failures verifying the server from the device we - were testing. + Align Azure Pipelines with GitHub Actions. + +Daniel Stenberg (25 Apr 2020) +- runtests: check for the disabled tests relative srcdir - This changes them to take the address from runtests.py and default to - localhost/127.0.0.1 if none is given. + To make it work correctly for out-of-tree builds. - Closes #4048 + Follow-up to 75e8feb6fb08b + + Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 + Reported-by: Marcel Raad + Closes #5297 -- test1523: basic test of CURLOPT_LOW_SPEED_LIMIT +- runtests: revert commenting out a line I did for debugging + + Follow-up to 11091cd4d. It was not meant to be pushed! -- configure: --disable-progress-meter +- smtp: set auth correctly - Builds libcurl without support for the built-in progress meter. + Regression since 7.69.0 and 68fb25fa3fcff. - Closes #4023 - -- curl: improved skip-setopt-options when built with disabled features + The code wrongly assigned 'from' instead of 'auth' which probably was a + copy and paste mistake from other code, leading to that auth could + remain NULL and later cause an error to be returned. - Reduces #ifdefs in src/tool_operate.c + Assisted-by: Eric Sauvageau + Fixes #5294 + Closes #5295 + +Marcel Raad (25 Apr 2020) +- lib: clean up whitespace - Follow-up from 4e86f2fc4e6 - Closes #3936 + This fixes CodeFactor warnings. -Steve Holme (18 Jun 2019) -- netrc: Return the correct error code when out of memory +Daniel Stenberg (25 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: avoid options override by configuration files - Introduced in 763c5178. + Previously, options set explicitly through command line options could be + overridden by the configuration files parsed automatically when + ssh_connect() was called. - Closes #4036 - -Daniel Stenberg (18 Jun 2019) -- config-os400: add getpeername and getsockname defines + By calling ssh_options_parse_config() explicitly, the configuration + files are parsed before setting the options, avoiding the options + override. Once the configuration files are parsed, the automatic + configuration parsing is not executed. - Reported-by: jonrumsey on github - Fixes #4037 - Closes #4039 + Fixes #4972 + Closes #5283 + Signed-off-by: Anderson Toshiyuki Sasaki <ansasaki@redhat.com> -- runtests: keep logfiles around by default +- runtests: when <killserver> mentions http, kill http/2 too - Make '-k' a no-op. The singletest function now clears the log directory - BEFORE each individual test and not after, which makes it possible to - always keep the logfiles around after a test has been run. No need to - specify -k anymore. Keeping the option parsing around to work with users - of old habits. + Since the http2 test server is a mere proxy that needs to know about the + dynamic port the HTTP server is using, it too needs to get restarted + when the http server is killed. - Some tests also didn't work properly when -k was used (since the old - logs would be kep when a new test starts) which this change also fixes. + A regression caused by 80d6515. - Closes #4035 + Fixes #5289 + Closes #5291 -- [Gergely Nagy brought this change] +- [Yuri Slobodyanyuk brought this change] - openssl: fix pubkey/signature algorithm detection in certinfo + docs: fix two typos - Certinfo gives the same result for all OpenSSL versions. - Also made printing RSA pubkeys consistent with older versions. + Closes #5292 + +- [Emil Engler brought this change] + + tests/git: ignore mqttd and port files - Reported-by: Michael Wallner - Fixes #3706 - Closes #4030 + Closes #5290 -- conn_maxage: move the check to prune_dead_connections() +- tests: make runtests check that disabled tests exists - ... and avoid the locking issue. + ... and error out if so. Removed '536' from DISABLED as there is no such + test file. - Reported-by: Kunal Ekawde - Fixes #4029 - Closes #4032 + Closes #5288 + +- test1154: set a proper name -- tests: have runtests figure out disabled features +- select: make Curl_socket_check take timediff_t timeout - ... so that runtests can skip individual test cases that test features - that are explicitly disabled in this build. This new logic is intended - for disabled features that aren't otherwise easily visible through the - curl_version_info() or other API calls. + Coverity found CID 1461718: - tests/server/disabled is a newly built executable that will output a - list of disabled features. Outputs nothing for a default build. + Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > + 9223372036854775807L" is always false regardless of the values of its + operands. This occurs as the logical second operand of "||". - Closes #3950 + Closes #5240 -- test188/189: fix Content-Length - - This cures the flaky test results +- [i-ky brought this change] + + libcurl-multi.3: added missing full stop - Closes #4034 + Closes #5285 -- [Thomas Gamper brought this change] +Jay Satiro (22 Apr 2020) +- transfer: Switch PUT to GET/HEAD on 303 redirect + + Prior to this change if there was a 303 reply to a PUT request then + the subsequent request to respond to that redirect would also be a PUT. + It was determined that was most likely incorrect based on the language + of the RFCs. Basically 303 means "see other" resource, which implies it + is most likely not the same resource, therefore we should not try to PUT + to that different resource. + + Refer to the discussions in #5237 and #5248 for more information. + + Fixes https://github.com/curl/curl/issues/5237 + Closes https://github.com/curl/curl/pull/5248 - winbuild: use WITH_PREFIX if given +Daniel Stenberg (22 Apr 2020) +- lib/mk-ca-bundle: skip empty certs - Closes #4031 + Reviewed-by: Emil Engler + Reported-by: Ashwin Metpalli + Fixes #5278 + Closes #5280 -Daniel Gustafsson (17 Jun 2019) -- openssl: remove outdated comment +- version: skip idn2_check_version() check and add precaution - OpenSSL used to call exit(1) on syntax errors in OPENSSL_config(), - which is why we switched to CONF_modules_load_file() and introduced - a comment stating why. This behavior was however changed in OpenSSL - commit abdd677125f3a9e3082f8c5692203590fdb9b860, so remove the now - outdated and incorrect comment. The mentioned commit also declares - OPENSSL_config() deprecated so keep the current coding. + A gcc-10's -fanalyze complaint made me spot and do these improvements. - Closes #4033 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Closes #5281 -Daniel Stenberg (16 Jun 2019) - RELEASE-NOTES: synced -Patrick Monnerat (16 Jun 2019) -- os400: make vsetopt() non-static as Curl_vsetopt() for os400 support. +- [Brian Bergeron brought this change] + + curl.h: update comment typo - Use it in curl_easy_setopt_ccsid(). + "routines with be invoked" -> "routines will be invoked" - Reported-by: jonrumsey on github - Fixes #3833 - Closes #4028 + Closes #5279 -Daniel Stenberg (15 Jun 2019) -- runtests: report single test time + total duration - - ... after each successful test. +- [Emil Engler brought this change] + + GnuTLS: Don't skip really long certificate fields - Closes #4027 + Closes #5271 -- multi: fix the transfer hash function +- gnutls: bump lowest supported version to 3.1.10 - Follow-up from 8b987cc7eb + GnuTLS 3.1.10 added new functions we want to use. That version was + released on Mar 22, 2013. Removing support for older versions also + greatly simplifies the code. - Reported-by: Tom van der Woerdt - Fixes #4018 - Closes #4024 + Ref: #5271 + Closes #5276 + +- mqtt: make NOSTATE get within the debug name array -- unit1654: cleanup on memory failure +- tests: run the RTSP test server on a dynamic port number - ... to make it handle torture tests properly. + To avoid port collisions. - Reported-by: Marcel Raad - Fixes #4021 - Closes #4022 + Closes #5272 -Marcel Raad (13 Jun 2019) -- krb5: fix compiler warning +- tests: add %NOLISTENPORT and use it - Even though the variable was used in a DEBUGASSERT, GCC 8 warned in - debug mode: - krb5.c:324:17: error: unused variable 'maj' [-Werror=unused-variable] + The purpose with this variable is to provide a port number that is + reasonably likely to not have a listener on the local host so that tests + can try connect failures against it. It uses port 47 - "reserved" + according to IANA. - Just suppress the warning and declare the variable unconditionally - instead of only for DEBUGBUILD (which also missed the check for - HAVE_ASSERT_H). + Updated six tests to use it instead of the previous different ports. - Closes https://github.com/curl/curl/pull/4020 + Assisted-by: Emil Engler + Closes #5270 -Daniel Stenberg (13 Jun 2019) -- quote.d: asterisk prefix works for SFTP as well +- mqtt: remove code with no purpose - Reported-by: Ben Voris - Fixes #4017 - Closes #4019 - -- multi: fix the transfer hashes in the socket hash entries + Detected by Coverity. CID 1462319. - - The transfer hashes weren't using the correct keys so removing entries - failed. + "The same code is executed when the condition result is true or false, + because the code in the if-then branch and after the if statement is + identical." - - Simplified the iteration logic over transfers sharing the same socket and - they now simply are set to expire and thus get handled in the "regular" - timer loop instead. + Closes #5275 + +- mqtt: fix Curl_read() error handling while reading remaining length - Reported-by: Tom van der Woerdt - Fixes #4012 - Closes #4014 + Detected by Coverity. CID 1462320. + + Closes #5274 -Jay Satiro (12 Jun 2019) -- [Cliff Crosland brought this change] +- server/tftpd: fix compiler warning + + Follow-up from 369ce38ac1d + Reported-by: Marc Hörsken - url: Fix CURLOPT_MAXAGE_CONN time comparison +- http: free memory when Alt-Used header creation fails due to OOM - Old connections are meant to expire from the connection cache after - CURLOPT_MAXAGE_CONN seconds. However, they actually expire after 1000x - that value. This occurs because a time value measured in milliseconds is - accidentally divided by 1M instead of by 1,000. + Reported-by: James Fuller + Fixes #5268 + Closes #5269 + +Daniel Gustafsson (20 Apr 2020) +- lib: fix typos in comments and errormessages - Closes https://github.com/curl/curl/pull/4013 + This fixes a few randomly spotted typos in recently merged code, most + notably one in a userfacing errormessage the schannel code. -Daniel Stenberg (11 Jun 2019) -- test1165: verify that CURL_DISABLE_ symbols are in sync +Daniel Stenberg (20 Apr 2020) +- tests: run the SOCKS test server on a dynamic port number - between configure.ac and source code. They should be possible to switch - on/off in configure AND be used in source code. + Closes #5266 -- configure: remove CURL_DISABLE_TLS_SRP +- [Johannes Schindelin brought this change] + + multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - It isn't used by code so stop providing the define. + When cURL is compiled with support for multiple SSL backends, it is + possible to configure an SSL backend via `curl_global_sslset()`, but + only *before* `curl_global_init()` was called. - Closes #4010 - -- Revert "cmake: add SMB to list of disabled protocols if HTTP_ONLY is specified" + If another SSL backend should be used after that, a user might be + tempted to call `curl_global_cleanup()` to start over. However, we did + not foresee that use case and forgot to reset the SSL backend in that + cleanup. - This reverts commit 36738caeb78603ce24e3ea089a167b8c216fb938. + Let's allow that use case. - Apparently several of the appveyor windows builds broke. + Fixes #5255 + Closes #5257 + Reported-by: davidedec on github + Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> -- [sergey-raevskiy brought this change] +- tests: run the TFTP test server on a dynamic port number + + Picking a dynamic unused port is better than a fixed to avoid the + collision risk. + + Closes #5265 - cmake: add SMB to list of disabled protocols if HTTP_ONLY is specified +- mqtt: improve the state machine + + To handle PUBLISH before SUBACK and more. - Reviewed-by: Jakub Zakrzewski - Closes #3770 + Updated the existing tests and added three new ones. + + Reported-by: Christoph Krey + Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html + Closes #5246 + +- runtests: always put test number in servercmd file - RELEASE-NOTES: synced -- http2: remove CURL_DISABLE_TYPECHECK define +- release-notes.pl: fix parsing typo + +James Fuller (20 Apr 2020) +- ensure all references to ports are replaced by vars + +- add more alt-svc test coverage + +Daniel Stenberg (20 Apr 2020) +- test1247: use http server to get the port number set - ... in http2-less builds as it served no use. + Follow-up to 0f5db7b263f -- configure: more --disable switches to toggle off individual features +- runtests: use a unix domain socket path with the pid in the name - ... actual support in the code for disabling these has already landed. + To make it impossible for test cases to access the file name without + using the proper variable for the purpose. - Closes #4009 + Closes #5264 -- wolfssl: fix key pinning build error - - follow-up from deb9462ff2de8 +Daniel Gustafsson (19 Apr 2020) +- [Mipsters on github brought this change] -- CURLMOPT_SOCKETFUNCTION.3: clarified + src: Remove C99 constructs to ensure C89 compliance - Moved away the callback explanation from curl_multi_socket_action.3 and - expanded it somewhat. + This fixes the error: 'for' loop initial declaration used outside C99 + mode by declaring the loop increment variable in the beginning of the + block instead of inside the for loop. - Closes #4006 + Fixes #5254 + Reviewed-by: Daniel Gustafsson <daniel@yesql.se> -- wolfssl: fixup for SNI use - - follow-up from deb9462ff2de8 +Daniel Stenberg (19 Apr 2020) +- runtests: dummy init the ports variables to avoid warnings - Closes #4007 + ... and generate something that can help debug test cases. -- CURLOPT_CAINFO.3: polished wording +- [Patrick Monnerat brought this change] + + mime: properly check Content-Type even if it has parameters - Clarify the functionality when built to use Schannel and Secure - Transport and stop calling it the "recommended" or "preferred" way and - instead rather call it the default. + New test 669 checks this fix is effective. - Removed the reference to the ssl comparison table as it isn't necessary. + Fixes #5256 + Closes #5258 + Reported-by: thanhchungbtc on github + +- tests/FILEFORMAT: converted to markdown and extended - Reported-by: Richard Alcock - Bug: https://curl.haxx.se/mail/lib-2019-06/0019.html - Closes #4005 + Closes #5261 -GitHub (10 Jun 2019) -- [Daniel Stenberg brought this change] +- test1245: make it work with dynamic FTP server port - SECURITY.md: created - - Brief security policy description for use/display on github. +- test1055: make it work with dynamic FTP port + +- test1028: make it run on dynamic FTP server port -Daniel Gustafsson (10 Jun 2019) -- tool_cb_prg: Fix integer overflow in progress bar +- tests: move pingpong server to dynamic listening port - Commit 61faa0b420c236480bc9ef6fd52b4ecc1e0f8d17 fixed the progress bar - width calculation to avoid integer overflow, but failed to account for - the fact that initial_size is initialized to -1 when the file size is - retrieved from the remote on an upload, causing another signed integer - overflow. Fix by separately checking for this case before the width - calculation. + FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic + ports - Closes #3984 - Reported-by: Brian Carpenter (Geeknik Labs) - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Test 842-845 are unfortunately a bit hard to move over to this concept + right now and require "default port" still... + +- test1056: work with dynamic HTTP ipv6 port -Daniel Stenberg (10 Jun 2019) -- wolfssl: refer to it as wolfSSL only +- test1448: work with dynamic HTTP server port + +- tests: introduce preprocessed test cases - Remove support for, references to and use of "cyaSSL" from the source - and docs. wolfSSL is the current name and there's no point in keeping - references to ancient history. + The runtests script now always performs variable replacement on the + entire test source file before the test gets executed, and saves the + updated version in a temporary file (log/test[num]) so that all test + case readers/servers can use that version (if present) and thus enjoy + the powers of test case variable substitution. - Assisted-by: Daniel Gustafsson + This is necessary to allow complete port number freedom. - Closes #3903 + Test 309 is updated to work with a non-fixed port number thanks to this. -- RELEASE-NOTES: synced +- tests: make 2006-2010 handle different port number lengths -- bindlocal: detect and avoid IP version mismatches in bind() +- tests: run the sws server on "any port" - Reported-by: Alex Grebenschikov - Fixes #3993 - Closes #4002 - -- multi: make sure 'data' can present in several sockhash entries + Makes the test servers for HTTP and Gopher pop up on a currently unused + port and runtests adapts to that! - Since more than one socket can be used by each transfer at a given time, - each sockhash entry how has its own hash table with transfers using that - socket. + Closes #5247 + +Marc Hoersken (18 Apr 2020) +- sockfilt: tidy variable naming and data structure in select_ws - In addition, the sockhash entry can now be marked 'blocked = TRUE'" - which then makes the delete function just set 'removed = TRUE' instead - of removing it "for real", as a way to not rip out the carpet under the - feet of a parent function that iterates over the transfers of that same - sockhash entry. + This commit does not introduce any logical changes to the code. - Reported-by: Tom van der Woerdt - Fixes #3961 - Fixes #3986 - Fixes #3995 - Fixes #4004 - Closes #3997 + Reviewed-by: Jay Satiro and Marcel Raad + Closes #5238 -- [Sorcus brought this change] +Daniel Stenberg (17 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] - libcurl-tutorial.3: Fix small typo (mutipart -> multipart) + libssh: Use new ECDSA key types to check known hosts + + From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA + keys depending on the curve. - Fixed-by: MrSorcus on github - Closes #4000 + Signed-off-by: Anderson Toshiyuki Sasaki <ansasaki@redhat.com> + Fixes #5252 + Closes #5253 -- unpause: trigger a timeout for event-based transfers +Marcel Raad (17 Apr 2020) +- appveyor: add Unicode winbuild jobs - ... so that timeouts or other state machine actions get going again - after a changing pause state. For example, if the last delivery was - paused there's no pending socket activity. + These are cheap as they don't build tests. - Reported-by: sstruchtrup on github - Fixes #3994 - Closes #4001 + Closes https://github.com/curl/curl/pull/5063 -Marcel Raad (9 Jun 2019) -- travis: use xenial LLVM package for scan-build +Daniel Stenberg (16 Apr 2020) +- mqttd: s/errno/SOCKERRNO - I missed that in commit 99a49d6. + To behave proper on Windows + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 + Closes #5241 -- travis: update scan-build job to xenial +- buildconf: use find -execdir instead, remove -print and the ares files - Closes https://github.com/curl/curl/pull/3999 + Follow-up to 1e41bec96a6e + + Suggested-by: Marc Hörsken -Daniel Stenberg (8 Jun 2019) -- bump: start working on 7.65.2 +- [Alexander V. Tikhonov brought this change] -Marcel Raad (5 Jun 2019) -- examples/htmltitle: use C++ casts between pointer types + buildconf: avoid using tempfile when removing files - Compilers and static analyzers warn about using C-style casts here. + Closes #5213 + +- copyright: bump the copyright year range + +- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords + +- [JP Mens brought this change] + + docs/MQTT: replace confusing 80 by 75 + + I was a bit surprised by the `80`: first thought: what's HTTP doing + here? ;) - Closes https://github.com/curl/curl/pull/3975 + Closes #5236 -- examples/fopen: fix comparison +- [Brad King brought this change] + + cmake: Avoid MSVC C4273 warnings in send/recv checks + + We use `check_c_source_compiles` to check possible send/recv signatures + by reproducing the forward declarations from system headers. On Windows + the `winsock2.h` header adds dll linkage settings to its forward + declaration. If ours does not match the compiler warns: - As want is size_t, (file->buffer_pos - want) is unsigned, so checking - if it's less than zero makes no sense. - Check if file->buffer_pos is less than want instead to avoid the - unsigned integer wraparound. + warning C4273: 'recv': inconsistent dll linkage - Closes https://github.com/curl/curl/pull/3975 + Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so + that our linkage is consistent with that from `winsock2.h`. + + Fixes #4764 + Closes #5232 -- build: fix Codacy warnings +Jay Satiro (14 Apr 2020) +- KNOWN_BUGS: Add entry 'Blocking socket operations' + + - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of + known blocking operations. + + - New known bugs entry 'Blocking socket operations in non-blocking API' + that directs to the TODO's list of known blocking operations. - Reduce variable scopes and remove redundant variable stores. + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 - Closes https://github.com/curl/curl/pull/3975 + Reported-by: Marc Hoersken + + Closes https://github.com/curl/curl/pull/5216 -- sws: remove unused variables +Marc Hoersken (14 Apr 2020) +- test2043: use revoked.badssl.com instead of revoked.grc.com + + The certificate of revoked.grc.com has expired on 2020-04-13. - Unused since commit 2f44e94. + Reviewed-by: Jay Satiro + + Closes #5233 + +- sockfilt: fix broken pipe on Windows to be ready in select_ws - Closes https://github.com/curl/curl/pull/3975 + Closes #5228 + +Daniel Stenberg (14 Apr 2020) +- RELEASE-NOTES: synced -Version 7.65.1 (4 Jun 2019) +- scripts/release-notes: fix duplicate output header -Daniel Stenberg (4 Jun 2019) -- RELEASE-NOTES: 7.65.1 +- github/workflow: enable MQTT in the macOS debug build -- THANKS: new contributors from 7.65.1 +- azure: add mqtt support to one of the Windows builds -Steve Holme (4 Jun 2019) -- [Frank Gevaerts brought this change] +- travis: add mqtt job on Linux - ssl: Update outdated "openssl-only" comments for supported backends +- tests: add four MQTT tests 1190 - 1193 + +- tests: add the mqtt test server mqttd + +- tests: support hex encoded data and mqtt server - These are for features that used to be openssl-only but were expanded - over time to support other SSL backends. + The mqtt server is started using a "random" port. + +- [Björn Stenberg brought this change] + + mqtt: add new experimental protocol - Closes #3985 + Closes #5173 -Daniel Stenberg (4 Jun 2019) -- curl_share_setopt.3: improve wording [ci ship] +- TODO: Consider convenience options for JSON and XML? - Reported-by: Carlos ORyan + Closes #5203 -Steve Holme (4 Jun 2019) -- tool_parsecfg: Use correct return type for GetModuleFileName() +- tool: do not declare functions with Curl_ prefix - GetModuleFileName() returns a DWORD which is a typedef of an unsigned - long and not an int. + To avoid collision risks with private libcurl symbols when linked with + static versions (or just versions not hiding internal symbols). - Closes #3980 + Reported-by: hydra3333 on github + Fixes #5219 + Closes #5234 -Daniel Stenberg (3 Jun 2019) -- TODO: "at least N milliseconds between requests" [ci skip] - - Suggested-by: dkwolfe4 on github - Closes #3920 +- [Nathaniel R. Lewis brought this change] -Steve Holme (2 Jun 2019) -- tests/server/.gitignore: Add socksd to the ignore list - - Missed in 04fd6755. + cmake: add aliases so exported target names are available in tree - Closes #3978 + Reviewed-by: Brad King + Closes #5206 -- tool_parsecfg: Fix control flow issue (DEADCODE) +- version: increase buffer space for ssl version output - Follow-up to 8144ba38. + To avoid it getting truncated, especially when several SSL backends are + built-in. - Detected by Coverity CID 1445663 - Closes #3976 + Reported-by: Gisle Vanem + Fixes #5222 + Closes #5226 -Daniel Stenberg (2 Jun 2019) -- [Sergey Ogryzkov brought this change] +Marc Hoersken (13 Apr 2020) +- cirrus: no longer ignore test 504 which is working again + + The test is working again, because TCP blackholing is disabled. - NTLM: reset proxy "multipass" state when CONNECT request is done +- appveyor: completely disable tests that fail to timeout early - Closes #3972 + The tests changed from ignored to disabled are tests that are + about connecting to non-listening socket. On AppVeyor these + tests are not reliable, because for some unknown reason the + connect is not timing out before the test time limit is reached. -- test334: verify HTTP 204 response with chunked coding header +Daniel Stenberg (13 Apr 2020) +- test1908: avoid using fixed port number in test data - Verifies that a bodyless response don't parse this content-related - header. + Closes #5225 -- [Michael Kaufmann brought this change] +Jay Satiro (12 Apr 2020) +- [Andrew Kurushin brought this change] - http: don't parse body-related headers bodyless responses + schannel: Fix blocking timeout logic - Responses with status codes 1xx, 204 or 304 don't have a response body. For - these, don't parse these headers: + - Fix schannel_send for the case when no timeout was set. - - Content-Encoding - - Content-Length - - Content-Range - - Last-Modified - - Transfer-Encoding + Prior to this change schannel would error if the socket was not ready + to send data and no timeout was set. - This change ensures that HTTP/2 upgrades work even if a - "Content-Length: 0" or a "Transfer-Encoding: chunked" header is present. + This commit is similar to parent commit 89dc6e0 which recently made the + same change for SOCKS, for the same reason. Basically it was not well + understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms + but actually means no timeout. - Co-authored-by: Daniel Stenberg - Closes #3702 - Fixes #3968 - Closes #3977 + Fixes https://github.com/curl/curl/issues/5177 + Closes https://github.com/curl/curl/pull/5221 -- tls13-docs: mention it is only for OpenSSL >= 1.1.1 +- socks: Fix blocking timeout logic - Reported-by: Jay Satiro - Co-authored-by: Jay Satiro - Fixes #3938 - Closes #3946 - -- dump-header.d: spell out that no headers == empty file [ci skip] + - Document in Curl_timeleft's comment block that returning 0 signals no + timeout (ie there's infinite time left). - Reported-by: wesinator at github - Fixes #3964 - Closes #3974 - -- singlesocket: use separate variable for inner loop + - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. - An inner loop within the singlesocket() function wrongly re-used the - variable for the outer loop which then could cause an infinite - loop. Change to using a separate variable! + Prior to this change if the timeout had a value of 0 and that was passed + to SOCKET_READABLE it would return right away instead of blocking. That + was likely because it was not well understood that when Curl_timeleft + returns 0 it is not a timeout of 0 ms but actually means no timeout. - Reported-by: Eric Wu - Fixes #3970 - Closes #3973 - -- RELEASE-NOTES: synced + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + + Closes https://github.com/curl/curl/pull/5220 -- [Josie Huddleston brought this change] +- [Marc Hoersken brought this change] - http2: Stop drain from being permanently set on + gopher: check remaining time left during write busy loop - Various functions called within Curl_http2_done() can have the - side-effect of setting the Easy connection into drain mode (by calling - drain_this()). However, the last time we unset this for a transfer (by - calling drained_transfer()) is at the beginning of Curl_http2_done(). - If the Curl_easy is reused for another transfer, it is then stuck in - drain mode permanently, which in practice makes it unable to write any - data in the new transfer. + Prior to this change gopher's blocking code would block forever, + ignoring any set timeout value. - This fix moves the last call to drained_transfer() to later in - Curl_http2_done(), after the functions that could potentially call for a - drain. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Fixes #3966 - Closes #3967 - Reported-by: Josie-H + Similar to #5220 and #5221 + Closes #5214 -Steve Holme (29 May 2019) -- conncache: Remove the DEBUGASSERT on length check +Daniel Stenberg (13 Apr 2020) +- [Dirkjan Bussink brought this change] + + gnutls: ensure TLS 1.3 when SRP isn't requested - We trust the calling code as this is an internal function. + When SRP is requested in the priority string, GnuTLS will disable + support for TLS 1.3. Before this change, curl would always add +SRP to + the priority list, effectively always disabling TLS 1.3 support. - Closes #3962 - -Jay Satiro (29 May 2019) -- [Gisle Vanem brought this change] + With this change, +SRP is only added to the priority list when SRP + authentication is also requested. This also allows updating the error + handling here to not have to retry without SRP. This is because SRP is + only added when requested and in that case a retry is not needed. + + Closes #5223 - system_win32: fix function prototype +Marc Hoersken (12 Apr 2020) +- tests/server: add hidden window to gracefully handle WM_CLOSE - - Change if_nametoindex parameter type from char * to const char *. + Forward Window events as signals to existing signal event handler. + +- tests/server: add CTRL event handler for Win32 consoles - Follow-up to 09eef8af from this morning. + Forward CTRL events as signals to existing signal event handler. + +- tests/server: move all signal handling routines to util.[ch] - Bug: https://github.com/curl/curl/commit/09eef8af#r33716067 + Avoid code duplication to prepare for portability enhancements. -Marcel Raad (29 May 2019) -- appveyor: add Visual Studio solution build +Daniel Stenberg (12 Apr 2020) +- compressed.d: stress that the headers are not modified - Closes https://github.com/curl/curl/pull/3941 + Suggested-by: Michael Osipov + Assisted-by: Jay Satiro + Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 + Closes #5217 -- appveyor: add support for other build systems +Marc Hoersken (11 Apr 2020) +- tests/server/util.c: use curl_off_t instead of long for pid - Introduce BUILD_SYSTEM variable, which is currently always CMake. + Avoid potential overflow of huge PIDs on Windows. - Closes https://github.com/curl/curl/pull/3941 + Related to #5188 + Assisted-by: Marcel Raad -Steve Holme (29 May 2019) -- url: Load if_nametoindex() dynamically from iphlpapi.dll on Windows +- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - This fixes the static dependency on iphlpapi.lib and allows curl to - build for targets prior to Windows Vista. + Since the Windows versions of both programs would write Windows + PIDs to their pidfiles which we cannot handle, we need to use + our known perl.exe Cygwin/msys PID together with exec() in order + to tie the spawned processes to the existance of our perl.exe - This partially reverts 170bd047. + The perl.exe that is executing secureserver.pl and sshserver.pl + has a Cygwin/msys PID, because it is started inside Cygwin/msys. - Fixes #3960 - Closes #3958 + Related to #5188 -Daniel Stenberg (29 May 2019) -- http: fix "error: equality comparison with extraneous parentheses" +- tests: add Windows compatible pidwait like pidkill and pidterm + + Related to #5188 -- parse_proxy: make sure portptr is initialized +- tests: fix conflict between Cygwin/msys and Windows PIDs - Reported-by: Benbuck Nason + Add 65536 to Windows PIDs to allow Windows specific treatment + by having disjunct ranges for Cygwin/msys and Windows PIDs. - fixes #3959 - -- url: default conn->port to the same as conn->remote_port + See also: + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=b5e1003722cb14235c4f166be72c09acdffc62ea + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe - ... so that it has a sensible value when ConnectionExists() is called which - needs it set to differentiate host "bundles" correctly on port number! + Replaces #5178 + Closes #5188 + +Daniel Stenberg (11 Apr 2020) +- RELEASE-NOTES: synced + +- release-notes.pl: detect the start of the references in cleanup mode + +- Revert "file: on Windows, refuse paths that start with \\" - Also, make conncache:hashkey() use correct port for bundles that are proxy vs - host connections. + This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. - Probably a regression from 7.62.0 + Reminded-by: Chris Roberts + Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html - Reported-by: Tom van der Woerdt - Fixes #3956 - Closes #3957 + Closes #5215 -- conncache: make "bundles" per host name when doing proxy tunnels +Jay Satiro (11 Apr 2020) +- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - Only HTTP proxy use where multiple host names can be used over the same - connection should use the proxy host name for bundles. + - If loss of data may occur converting a timediff_t to time_t and + the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - Reported-by: Tom van der Woerdt - Fixes #3951 - Closes #3955 + This is a follow-up to 8843678 which removed the (time_t) typecast + from the macros so that conversion warnings could be identified. + + Closes https://github.com/curl/curl/pull/5199 -- multi: track users of a socket better +- test1148: tolerate progress updates better (again) + + - Ignore intermediate progress updates. - They need to be removed from the socket hash linked list with more care. + - Support locales that use a character other than period as decimal + separator (eg 100,0%). - When sh_delentry() is called to remove a sockethash entry, remove all - individual transfers from the list first. To enable this, each Curl_easy struct - now stores a pointer to the sockethash entry to know how to remove itself. + test1148 checks that the progress finishes at 100% and has the right + bar width. Prior to this change the test assumed that the only progress + reported for such a quick transfer was 100%, however in rare instances + (like in the CI where transfer time can slow considerably) there may be + intermediate updates. For example, below is stderrlog1148 from a failed + CI run with explicit \r and \n added (it is one line; broken up so that + it's easier to understand). - Reported-by: Tom van der Woerdt and Kunal Ekawde + \r + \r################################## 48.3% + \r######################################################################## 100.0% + \n - Fixes #3952 - Fixes #3904 - Closes #3953 + Closes https://github.com/curl/curl/pull/5194 -Steve Holme (28 May 2019) -- curl-win32.h: Enable Unix Domain Sockets based on the Windows SDK version +Marc Hoersken (10 Apr 2020) +- sshserver.pl: use cached Win32 environment check variable + +- appveyor: partially revert 3413a110 to keep build without proxy - Microsoft added support for Unix Domain Sockets in Windows 10 1803 - (RS4). Rather than expect the user to enable Unix Domain Sockets by - uncommenting the #define that was added in 0fd6221f we use the RS4 - pre-processor variable that is present in newer versions of the - Windows SDK. + Ref: #5211 and #4526 + Reported-by: Marcel Raad + +- appveyor: ignore failing 'connect to non-listening proxy' tests - Closes #3939 + Closes #5211 -Daniel Stenberg (28 May 2019) -- [Jonas Vautherin brought this change] +- CI/macos: convert CRLF to LF and align indentation - cmake: support CMAKE_OSX_ARCHITECTURES when detecting SIZEOF variables +Daniel Stenberg (9 Apr 2020) +- url: allow non-HTTPS altsvc-matching for debug builds - Closes #3945 + This is already partly supported but this part was missing. + Reported-by: James Fuller + + Closes #5205 -Marcel Raad (27 May 2019) -- HAProxy tests: add keywords +- server/resolve: remove AI_CANONNAME to make macos tell the truth - Add the proxy and haproxy keywords in order to be able to exclude or - run these specific tests. + With this bit set, my mac successfully resolves "ip6-localhost" when in + fact there is no such host known to my machine! That in turn made test + 241 wrongly execute and fail. - Closes https://github.com/curl/curl/pull/3949 + Closes #5202 -Daniel Stenberg (27 May 2019) -- [Maksim Stsepanenka brought this change] - - tests: make test 1420 and 1406 work with rtsp-disabled libcurl +- runtests: fix warning about using an undefined variable - Closes #3948 + Follow-up from 4d939ef6ceb2db1 -Kamil Dudka (27 May 2019) -- [Hubert Kario brought this change] +- release-notes: fix the initial reference list output - nss: allow to specify TLS 1.3 ciphers if supported by NSS +- github actions: run when pushed to master or */ci + PRs + + Avoid double-builds when using "local" branches for PRs. For both macos + and fuzz jobs. - Closes #3916 + Closes #5201 -Daniel Stenberg (26 May 2019) -- RELEASE-NOTES: synced +- runtests: provide nicer errormsg when protocol "dump" file is empty -- [Jay Satiro brought this change] +- [Gilles Vollant brought this change] - Revert all SASL authzid (new feature) commits - - - Revert all commits related to the SASL authzid feature since the next - release will be a patch release, 7.65.1. + schannel: support .P12 or .PFX client certificates - Prior to this change CURLOPT_SASL_AUTHZID / --sasl-authzid was destined - for the next release, assuming it would be a feature release 7.66.0. - However instead the next release will be a patch release, 7.65.1 and - will not contain any new features. + Used with curl command line option like this: --cert + <filename>:<password> --cert-type p12 - After the patch release after the reverted commits can be restored by - using cherry-pick: + Closes #5193 + +- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690 + test1294: "split request" being when the entire request isn't sent in + the first go, and the remainder is sent in the PERFORM state. A GET + request is otherwise not sending anything during PERFORM. - Details for all reverted commits: + test1295: same kind of split but with POST - Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()." + Closes #5197 + +- http: don't consider upload done if the request isn't completely sent off - This reverts commit 0edf6907ae37e2020722e6f61229d8ec64095b0a. + Fixes #4919 + Closes #5197 + +- http: allow Curl_add_buffer_send() to do a short first send by force - Revert "tests: Fix the line endings for the SASL alt-auth tests" + In a debug build, settting the environment variable "CURL_SMALLREQSEND" + will make the first HTTP request send not send more bytes than the set + amount, thus ending up verifying that the logic for handling a split + HTTP request send works correctly. + +- connect: store connection info for QUIC connections - This reverts commit c2a8d52a1356a722ff9f4aeb983cd4eaf80ef221. + Restores the --head functionality to the curl utility which extracts + 'protocol' that is stored that way. - Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples" + Reported-by: James Fuller + Fixes #5196 + Closes #5198 + +- tests/README: update the port numbers list - This reverts commit 8c1cc369d0c7163c6dcc91fd38edfea1f509ae75. + Since the pipelining server is long gone. + Reported-by: James Fuller + +- select: remove typecast from SOCKET_WRITABLE/READABLE macros - Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool" + So that they don't hide conversions-by-mistake - This reverts commit a9499ff136d89987af885e2d7dff0a066a3e5817. + Reviewed-by: Jay Satiro + Closes #5190 + +- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also - Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID" + Closes #5192 + +- release-notes: output trailing references sorted numerically + +- cleanup: correct copyright year range on a few files + +- configure: remove use of -vec-report0 from CFLAGS with icc - This reverts commit a14d72ca2fec5d4eb5a043936e4f7ce08015c177. + ... as it apparently isn't (always) supported. + Reported-by: Alain Miniussi + Fixes #5096 + Closes #5191 -- [dbrowndan brought this change] +- warnless: remove code block for icc that didn't work + + Reported-by: Alain Miniussi + Fixes #5096 - FAQ: more minor updates and spelling fixes +Marc Hoersken (6 Apr 2020) +- dist: add missing setup-win32.h - Closes #3937 + Follow up to d820224b8b +Daniel Stenberg (6 Apr 2020) - RELEASE-NOTES: synced -- sectransp: handle errSSLPeerAuthCompleted from SSLRead() +- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance - Reported-by: smuellerDD on github - Fixes #3932 - Closes #3933 + This script helps putting entries in the RELEASE-NOTES using a coherent + style and sorting with a minimal human editing effort - as long as the + first line in the commit message is good enough! There's a short howto + at the top of the file. -GitHub (24 May 2019) -- [Gisle Vanem brought this change] +- [Dennis Felsing brought this change] - Fix typo. - -Daniel Stenberg (23 May 2019) -- tool_setopt: for builds with disabled-proxy, skip all proxy setopts() + configure: don't check for Security.framework when cross-compiling - Reported-by: Marcel Raad - Fixes #3926 - Closes #3929 - -Steve Holme (23 May 2019) -- winbuild: Use two space indentation + Since it checks for the local file, not the cross-compiled one. - Closes #3930 + Closes #5189 -GitHub (23 May 2019) -- [Gisle Vanem brought this change] +- TODO: Option to make -Z merge lined based outputs on stdout + + Closes #5175 - tool_parse_cfg: Avoid 2 fopen() for WIN32 +- lib: never define CURL_CA_BUNDLE with a getenv - Using the memdebug.h mem-leak feature, I noticed 2 calls like: - FILE tool_parsecfg.c:70 fopen("c:\Users\Gisle\AppData\Roaming\_curlrc","rt") - FILE tool_parsecfg.c:114 fopen("c:\Users\Gisle\AppData\Roaming\_curlrc","rt") + - it breaks the build (since 6de756c9b1de34b7a1) + - it's not documented and not consistent across platforms + - the curl tool does that getenv magic - No need for 'fopen(), 'fclose()' and a 'fopen()' yet again. + Bug: https://github.com/curl/curl/commit/6de756c#r38127030 + Reported-by: Gisle Vanem + + Closes #5187 -Daniel Stenberg (23 May 2019) -- md4: include the mbedtls config.h to get the MD4 info +Marc Hoersken (5 Apr 2020) +- lib670: use the same Win32 API check as all other lib tests -- md4: build correctly with openssl without MD4 +- appveyor: use random test server ports based upon APPVEYOR_API_URL + + Avoid conflicts of test server ports with AppVeyor API on localhost. - Reported-by: elsamuko at github - Fixes #3921 - Closes #3922 + Closes #5034 -Patrick Monnerat (23 May 2019) -- os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid(). +- appveyor: sort builds by type and add two new variants + + Related to #5034 and #5063 -Daniel Stenberg (23 May 2019) -- .github/FUNDING: mention our opencollective "home" [ci skip] +- appveyor: show failed tests in log even if test is ignored + + And print API response with newline only if there is one -Marcel Raad (23 May 2019) -- [Zenju brought this change] +- appveyor: turn disabled tests into ignored result tests - config-win32: add support for if_nametoindex and getsockname +Daniel Stenberg (5 Apr 2020) +- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" - Closes https://github.com/curl/curl/pull/3923 + Fixed with #5170 (commit 23a870f2fd041278) -Jay Satiro (23 May 2019) -- tests: Fix the line endings for the SASL alt-auth tests - - - Change data and protocol sections to CRLF line endings. +- test1566: verify --etag-compare that gets a 304 back - Prior to this change the tests would fail or hang, which is because - certain sections such as protocol require CRLF line endings. + Verifies the fix in #5183 - Follow-up to a9499ff from today which added the tests. - - Ref: https://github.com/curl/curl/pull/3790 + Closes #5186 + +- [Kwon-Young Choi brought this change] -Daniel Stenberg (23 May 2019) -- url: fix bad #ifdef + CURLINFO_CONDITION_UNMET: return true for 304 http status code - Regression since e91e48161235272ff485. + In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the + output file if the server did not transfered a file based on time + condition. In the same manner, getting a 304 HTTP response back from the + server, for example after passing a custom If-Match-* header, also + fulfill this condition. - Reported-by: Tom Greenslade - Fixes #3924 - Closes #3925 + Fixes #5181 + Closes #5183 -- Revert "progress: CURL_DISABLE_PROGRESS_METER" +- [Kwon-Young Choi brought this change] + + curl: allow both --etag-compare and --etag-save with same file name + + This change inverse the order of processing for the --etag-compare and + --etag-save option to process first --etag-compare. This in turn allows + to use the same file name to compare and save an etag. - This reverts commit 3b06e68b7734cb10a555f9d7e804dd5d808236a4. + The original behavior of not failing if the etag file does not exists is + conserved. - Clearly this change wasn't good enough as it broke CURLOPT_LOW_SPEED_LIMIT + - CURLOPT_LOW_SPEED_TIME + Fixes #5179 + Closes #5180 + +Viktor Szakats (4 Apr 2020) +- windows: enable UnixSockets with all build toolchains - Reported-by: Dave Reisner + Extend existing unix socket support in Windows builds to be + enabled for all toolchain vendors or versions. (Previously + it was only supported with certain MSVC versions + more recent + Windows 10 SDKs) - Fixes #3927 - Closes #3928 + Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ + Ref: https://github.com/curl/curl/issues/5162 + Closes: https://github.com/curl/curl/pull/5170 -Steve Holme (22 May 2019) -- examples: Added SASL PLAIN authorisation identity (authzid) examples +Daniel Stenberg (4 Apr 2020) +- KNOWN_BUGS: Store TLS context per transfer instead of per connection + + Closes #5102 -- curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool +Marc Hoersken (3 Apr 2020) +- sockfilt: remove redundancy in timeout handling + + And update other logmsg output in select_ws on Windows. -- sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID +- sockfilt: fix handling of ready closed sockets on Windows - Added the ability for the calling program to specify the authorisation - identity (authzid), the identity to act as, in addition to the - authentication identity (authcid) and password when using SASL PLAIN - authentication. + Replace the incomplete workaround regarding FD_CLOSE + only signalling once by instead doing a pre-check with + standard select and storing the result for later use. - Fixed #3653 - Closes #3790 + select keeps triggering on closed sockets on Windows while + WSAEventSelect fires only once with data still available. + By doing the pre-check we do not run in a deadlock + due to waiting forever for another FD_CLOSE event. -Marc Hoersken (22 May 2019) -- tests: add support to test against OpenSSH for Windows +- sockfilt: fix race-condition of waiting threads and event handling - Testing against OpenSSH for Windows requires v7.7.0.0 or newer - due to the use of AllowUsers and DenyUsers. For more info see: - https://github.com/PowerShell/Win32-OpenSSH/wiki/sshd_config + Fix race-condition of waiting threads finishing while events are + already being processed which lead to invalid or skipped events. + + Use mutex to check for one event at a time or do post-processing. + In addition to mutex-based locking use specific event as signal. + + Closes #5156 -Daniel Stenberg (22 May 2019) -- bump: start on the next release +Daniel Stenberg (2 Apr 2020) +- [Leo Neat brought this change] -Marcel Raad (22 May 2019) -- examples: fix "clarify calculation precedence" warnings + CI-fuzz: increase fuzz time to 40 minutes - Closes https://github.com/curl/curl/pull/3919 + Closes #5174 -- hiperfifo: remove unused variable +Marc Hoersken (2 Apr 2020) +- CI: increase Azure Pipelines timeouts due to performance issues - Closes https://github.com/curl/curl/pull/3919 + The current demand on Azure negatively impacts the CI performance. -- examples: remove dead variable stores - - Closes https://github.com/curl/curl/pull/3919 +- runtests.pl: log host OS as detected by Perl environment -- examples: reduce variable scopes - - Closes https://github.com/curl/curl/pull/3919 +- ftpserver.pl: log before and after data connection is closed -- http2-download: fix format specifier - - Closes https://github.com/curl/curl/pull/3919 +Daniel Stenberg (1 Apr 2020) +- RELEASE-NOTES: synced + +- RELEASE-PROCEDURE.md: run the copyright.pl script! -Daniel Stenberg (22 May 2019) -- PolarSSL: deprecate support step 1. Removed from configure. +- vquic/ngtcp2.h: update copyright year range - Also removed mentions from most docs. + Follow-up to 0736ee73d346a52 + +- [Daiki Ueno brought this change] + + CI: add build with ngtcp2 + gnutls on Travis CI + +- [Daiki Ueno brought this change] + + vquic: add support for GnuTLS backend of ngtcp2 - Discussed: https://curl.haxx.se/mail/lib-2019-05/0045.html + Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile + time. Therefore OpenSSL support needs to be explicitly disabled. - Closes #3888 + Signed-off-by: Daiki Ueno <dueno@redhat.com> + Closes #5148 -- configure/cmake: check for if_nametoindex() +- [Gisle Vanem brought this change] + + examples/sessioninfo.c: add include to fix compiler warning - - adds the check to cmake + Fixes #5171 + +- misc: copyright year updates - - fixes the configure check to work for cross-compiled windows builds + Follow-up to 7a71965e9 + +- [Harry Sintonen brought this change] + + build: fixed build for systems with select() in unistd.h - Closes #3917 + Closes #5169 -- parse_proxy: use the IPv6 zone id if given +- memdebug: don't log free(NULL) - If the proxy string is given as an IPv6 numerical address with a zone - id, make sure to use that for the connect to the proxy. + ... it serves no purpose and fills up the log. + +- cleanup: insert newline after if() conditions - Reported-by: Edmond Yu + Our code style mandates we put the conditional block on a separate + line. These mistakes are now detected by the updated checksrc. + +- checksrc: warn on obvious conditional blocks on the same line as if() - Fixes #3482 - Closes #3918 + Closes #5164 -Version 7.65.0 (22 May 2019) +- [Roger Orr brought this change] -Daniel Stenberg (22 May 2019) -- RELEASE-NOTES: 7.65.0 release + cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + + Fixes #5165 + Closes #5167 -- THANKS: from the 7.65.0 release-notes +- [Daiki Ueno brought this change] -- url: convert the zone id from a IPv6 URL to correct scope id + ngtcp2: update to git master for the key installation API change - Reported-by: GitYuanQu on github - Fixes #3902 - Closes #3914 - -- configure: detect getsockname and getpeername on windows too + This updates the ngtcp2 OpenSSL backend to follow the API change in + commit 32e703164 of ngtcp2. - Made detection macros for these two functions in the same style as other - functions possibly in winsock in the hope this will work better to - detect these functions when cross-compiling for Windows. + Notable changes are: + - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace + ngtcp2_crypto_derive_and_install_key + - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key + has been removed - Follow-up to e91e4816123 + Fixes #5166 + Closes #5168 + +- [Cyrus brought this change] + + SECURITY.md: minor rephrase - Fixes #3913 - Closes #3915 + Closes #5158 -Marcel Raad (21 May 2019) -- examples: remove unused variables +- output.d: quote the URL when globbing - Fixes Codacy/CppCheck warnings. + Some shells do globbing of their own unless the URL is quoted, so maybe + encourage this. - Closes + Co-authored-by: Jay Satiro + Closes #5160 -Daniel Gustafsson (21 May 2019) -- udpateconninfo: mark variable unused +- dist: add tests/version-scan.pl to tarball - When compiling without getpeername() or getsockname(), the sockfd - paramter to Curl_udpateconninfo() became unused after commit e91e481612 - added ifdef guards. + ... used in test 1177. - Closes #3910 - Fixes https://curl.haxx.se/dev/log.cgi?id=20190520172441-32196 - Reviewed-by: Marcel Raad, Daniel Stenberg + Follow-up to a97d826f6de3 + +- test1177: verify that all the CURL_VERSION_ bits are documented -- ftp: move ftp_ccc in under featureflag +- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented - Commit e91e48161235272ff485ff32bd048c53af731f43 moved ftp_ccc in under - the FTP featureflag in the UserDefined struct, but vtls callsites were - still using it unprotected. + Considered experimental and therefore we can do this. - Closes #3912 - Fixes: https://curl.haxx.se/dev/log.cgi?id=20190520044705-29865 - Reviewed-by: Daniel Stenberg, Marcel Raad + Closes #5157 -Daniel Stenberg (20 May 2019) -- curl: report error for "--no-" on non-boolean options +- KNOWN_BUGS: DoH doesn't inherit all transfer options - Reported-by: Olen Andoni - Fixes #3906 - Closes #3907 + Closes #4578 + Closes #4579 -- [Guy Poizat brought this change] - - mbedtls: enable use of EC keys +- KNOWN_BUGS: DoH leaks memory after followlocation - Closes #3892 + Closes #4592 -- lib1560: add tests for parsing URL with too long scheme +- KNOWN_BUGS: "FTPS needs session reuse" - Ref: #3905 + Closes #4654 -- [Omar Ramadan brought this change] +- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed - urlapi: increase supported scheme length to 40 bytes - - The longest currently registered URI scheme at IANA is 36 bytes long. +- TODO: Set custom client ip when using haproxy protocol - Closes #3905 - Closes #3900 + Closes #5125 -Marcel Raad (20 May 2019) -- lib: reduce variable scopes +Michael Kaufmann (27 Mar 2020) +- writeout_json: Fix data type issues - Fixes Codacy/CppCheck warnings. + Load long values correctly (e.g. for http_code). - Closes https://github.com/curl/curl/pull/3872 - -- tool_formparse: remove redundant assignment + Use curl_off_t (not long) for: + - size_download (CURLINFO_SIZE_DOWNLOAD_T) + - size_upload (CURLINFO_SIZE_UPLOAD_T) - Just initialize word_begin with the correct value. + The unit for these values is bytes/second, not microseconds: + - speed_download (CURLINFO_SPEED_DOWNLOAD_T) + - speed_upload (CURLINFO_SPEED_UPLOAD_T) - Closes https://github.com/curl/curl/pull/3873 + Fixes #5131 + Closes #5152 -- ssh: move variable declaration to where it's used - - This way, we need only one call to free. +Daniel Stenberg (27 Mar 2020) +- mailmap: fixup a few author names/fields - Closes https://github.com/curl/curl/pull/3873 + Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and + Svyatoslav Mishyn -- ssh-libssh: remove unused variable +- version: add 'cainfo' and 'capath' to version info struct - sock was only used to be assigned to fd_read. + Suggested-by: Timothe Litt + URL: https://curl.haxx.se/mail/lib-2020-03/0090.html + Reviewed-by: Jay Satiro - Closes https://github.com/curl/curl/pull/3873 + Closes #5150 -Daniel Stenberg (20 May 2019) -- test332: verify the blksize fix +- RELEASE-NOTES: synced -- tftp: use the current blksize for recvfrom() +Jay Satiro (26 Mar 2020) +- SSLCERTS.md: Fix example code for setting CA cert file + + Prior to this change the documentation erroneously said use + CURLOPT_CAPATH to set a CA cert file. + + Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html + Reported-by: Timothe Litt - bug: https://curl.haxx.se/docs/CVE-2019-5436.html - Reported-by: l00p3r on hackerone - CVE-2019-5436 + Closes https://github.com/curl/curl/pull/5151 -Daniel Gustafsson (19 May 2019) -- version: make ssl_version buffer match for multi_ssl +Marc Hoersken (26 Mar 2020) +- sockfilt: add logmsg output to select_ws_wait_thread on Windows - When running a multi TLS backend build the version string needs more - buffer space. Make the internal ssl_buffer stack buffer match the one - in Curl_multissl_version() to allow for the longer string. For single - TLS backend builds there is no use in extended to buffer. This is a - fallout from #3863 which fixes up the multi_ssl string generation to - avoid a buffer overflow when the buffer is too small. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #3875 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Closes #5086 -Steve Holme (18 May 2019) -- http_ntlm_wb: Handle auth for only a single request +Daniel Stenberg (26 Mar 2020) +- docs/make: generate curl.1 from listed files only - Currently when the server responds with 401 on NTLM authenticated - connection (re-used) we consider it to have failed. However this is - legitimate and may happen when for example IIS is set configured to - 'authPersistSingleRequest' or when the request goes thru a proxy (with - 'via' header). + Previously it rendered the page from files matching "*.d" in the correct + directory, which worked fine in git builds when the files were added but + made it easy to forget adding the files to the dist. - Implemented by imploying an additional state once a connection is - re-used to indicate that if we receive 401 we need to restart - authentication. + Now, only man page sections listed in DPAGES in Makefile.inc will be + used, thus "forcing" us to update this to get the man page right and get + it included in the dist at the same time. - Missed in fe6049f0. + Ref: #5146 + Closes #5149 -- http_ntlm_wb: Cleanup handshake after clean NTLM failure +- openssl: adapt to functions marked as deprecated since version 3 - Missed in 50b87c4e. - -- http_ntlm_wb: Return the correct error on receiving an empty auth message + OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES + functions we use. - Missed in fe20826b as it wasn't implemented in http.c in b4d6db83. + Fix the MD4 and SSL_CTX_load_verify_locations warnings. - Closes #3894 - -Daniel Stenberg (18 May 2019) -- curl: make code work with protocol-disabled libcurl + In configure, detect OpenSSL v3 and if so, inhibit the deprecation + warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and + until we rewrite the code to use non-deprecated functions we better + ignore these warnings as they don't help us. - Closes #3844 + Closes #5139 -- libcurl: #ifdef away more code for disabled features/protocols - -- progress: CURL_DISABLE_PROGRESS_METER - -- hostip: CURL_DISABLE_SHUFFLE_DNS +- dist: add mail-rcpt-allowfails.d to the tarball + + Reported-by: Maksim Stsepanenka + Reviewed-by: Jat Satiro + + Closes #5146 -- netrc: CURL_DISABLE_NETRC +- travis: update the ngtcp2 build to use the latest OpenSSL patch + + ... which also makes it OpenSSL 1.1.1d based and not v3. -Viktor Szakats (16 May 2019) -- docs: Markdown and misc improvements [ci skip] +Marc Hoersken (24 Mar 2020) +- CI: remove default Ubuntu build from GitHub Actions - Approved-by: Daniel Stenberg - Closes #3896 + We are already running a very similar Ubuntu build on Travis CI. + The macOS variant of this default build is kept on Github Actions. -- docs/RELEASE-PROCEDURE: link to live iCalendar [ci skip] +- CI: bring GitHub Actions fuzzing job in line with macOS jobs - Ref: https://github.com/curl/curl/commit/0af41b40b2c7bd379b2251cbe7cd618e21fa0ea1#commitcomment-33563135 - Approved-by: Daniel Stenberg - Closes #3895 + Update YAML formatting, job naming and triggers. -Daniel Stenberg (16 May 2019) -- travis: add an osx http-only build +- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions + + Reduce workload on Azure Pipelines and Travis CI while + consolidating macOS jobs onto less utilized GitHub Actions. + + Reviewed-by: Daniel Stenberg - Closes #3887 + Closes #5124 -- cleanup: remove FIXME and TODO comments +Daniel Stenberg (24 Mar 2020) +- config: remove all defines of HAVE_DES_H - They serve very little purpose and mostly just add noise. Most of them - have been around for a very long time. I read them all before removing - or rephrasing them. + As there's no code using it. - Ref: #3876 - Closes #3883 + Closes #5144 -- curl: don't set FTP options for FTP-disabled builds +- copyright: fix out-of-date copyright ranges and missing headers - ... since libcurl has started to be totally unaware of options for - disabled protocols they now return error. + Reported by the new script 'scripts/copyright.pl'. The script has a + regex whitelist for the files that don't need copyright headers. - Bug: https://github.com/curl/curl/commit/c9c5304dd4747cbe75d2f24be85920d572fcb5b8#commitcomment-33533937 + Removed three (mostly usesless) README files from docs/ - Reported-by: Marcel Raad - Closes #3886 + Closes #5141 + +- packages: add OS400/chkstrings.c to the dist + + Reported-by: Jon Rumsey + Fixes #5142 + Closes #5143 -Steve Holme (16 May 2019) -- http_ntlm_wb: Move the type-2 message processing into a dedicated function +- [Clément Notin brought this change] + + nghttp2: 1.12.0 required - This brings the code inline with the other HTTP authentication mechanisms. + since nghttp2_session_set_local_window_size is needed - Closes #3890 + Closes #5140 -Daniel Stenberg (15 May 2019) - RELEASE-NOTES: synced -- docs/RELEASE-PROCEDURE: updated coming releases dates [ci skip] +- [Calvin Buckley brought this change] -- CURLOPT_READFUNCTION.3: see also CURLOPT_UPLOAD_BUFFERSIZE [ci skip] + OS400: Update strings for ccsid-ifier - Reported-by: Roy Bellingan - Bug: #3885 - -- parse_proxy: use the URL parser API + Fixes build. - As we treat a given proxy as a URL we should use the unified URL parser - to extract the parts out of it. - - Closes #3878 + Closes #5132 -Steve Holme (15 May 2019) -- http_negotiate: Move the Negotiate state out of the negotiatedata structure +- cirrus: make freebsd ignore the tests instead of skipping - Given that this member variable is not used by the SASL based protocols - there is no need to have it here. + To allow us to see in the CI logs how they actually behave - Closes #3882 + Closes #5091 -- http_ntlm: Move the NTLM state out of the ntlmdata structure - - Given that this member variable is not used by the SASL based protocols - there is no need to have it here. +- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 -- url: Move the negotiate state type into a dedicated enum +- Revert "cirrus-ci: disable the FreeBSD 13 builds" + + This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. -- url: Remove duplicate clean up of the winbind variables in conn_shutdown() +- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - Given that Curl_disconnect() calls Curl_http_auth_cleanup_ntlm() prior - to calling conn_shutdown() and it in turn performs this, there is no - need to perform the same action in conn_shutdown(). + To let debug-builds return fake values, like in test 970. - Closes #3881 + Ref: #5131 + Closes #5136 -Daniel Stenberg (14 May 2019) -- urlapi: require a non-zero host name length when parsing URL +- test970: improve the test - Updated test 1560 to verify. + - send more data to make problems more obvious + - don't start the data with minus, it makes diffs harder to read + - skip the headers in the stdout comparison + - save to a file name to also verify 'filename_effective' - Closes #3880 + Ref: #5131 -- configure: error out if OpenSSL wasn't detected when asked for +- CURLINFO_NUM_CONNECTS: improve accuracy - If --with-ssl is used and configure still couldn't enable SSL this - creates an error instead of just silently ignoring the fact. + The counter was not bumped in all cases correctly. - Suggested-by: Isaiah Norton - Fixes #3824 - Closes #3830 + Reported-by: Marcel Raad + Ref: #5131 + Closes #5135 -Daniel Gustafsson (14 May 2019) -- imap: Fix typo in comment +- TODO: Use "random" ports for the test servers -Steve Holme (14 May 2019) -- url: Remove unnecessary initialisation from allocate_conn() - - No need to set variables to zero as calloc() does this for us. +- lib/curl_setup: adjust the copyright year range - Closes #3879 + Follow-up from d820224b8 -Daniel Stenberg (14 May 2019) -- CURLOPT_CAINFO.3: with Schannel, you want Windows 8 or later [ci skip] +Jay Satiro (21 Mar 2020) +- curl_setup: define _WIN32_WINNT_[OS] symbols - Clues-provided-by: Jay Satiro - Clues-provided-by: Jeroen Ooms - Fixes #3711 - Closes #3874 - -Daniel Gustafsson (13 May 2019) -- vtls: fix potential ssl_buffer stack overflow + .. because not all Windows build systems have those symbols, and even + those that do may be missing newer symbols (eg the Windows 7 SDK does + not define _WIN32_WINNT_WIN10). - In Curl_multissl_version() it was possible to overflow the passed in - buffer if the generated version string exceeded the size of the buffer. - Fix by inverting the logic, and also make sure to not exceed the local - buffer during the string generation. + Those symbols are used in build-time logic to decide which API to use + and prior to this change if the symbols were missing it would have + resulted in deprecated API being used when more recent functions were + available (eg GetVersionEx used instead of VerifyVersionInfo). - Closes #3863 - Reported-by: nevv on HackerOne/curl - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Reported-by: FuccDucc@users.noreply.github.com + + Probably fixes https://github.com/curl/curl/issues/4995 + Closes https://github.com/curl/curl/pull/5057 -Daniel Stenberg (13 May 2019) -- RELEASE-NOTES: synced +- [Ross Burton brought this change] -- appveyor: also build "/ci" branches like travis + curl-functions.m4: remove inappropriate AC_REQUIRE + + AC_REQUIRE means "if this macro hasn't been executed already, execute + it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) + isn't correct at that will execute AC_RUN_IFELSE without any arguments. + + With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, + AC_RUN_IFELSE without a default value when cross-compiling is fatal. + The result is that curl with autoconf 2.70 cannot cross-compile. + + Fixes https://github.com/curl/curl/issues/5126 + Closes https://github.com/curl/curl/pull/5130 + +Marc Hoersken (20 Mar 2020) +- ci/tests: fix Azure Pipelines not running Windows containers + + Workaround posted here: microsoft/azure-pipelines-agent#2864 + + Assisted-by: Simon Chalifoux + Assisted-by: Tommy Petty + + Fixes #5117 + Closes #5129 -- pingpong: disable more when no pingpong enabled +Daniel Stenberg (20 Mar 2020) +- tests: add test 430, 431 and 432 to verify the --config fix + + Verify the fixes in 4e0b4fee4 -- proxy: acknowledge DISABLE_PROXY more +- [Rici Lake brought this change] -- parsedate: CURL_DISABLE_PARSEDATE + cmdline: fix handling of OperationConfig linked list (--next) + + Ensures that -K/--config inserts new items at the end of the list + instead of overwriting the second item, and that after a -K/--config + option has been parsed, the option parser's view of the current config + is update. + + Fixes #5120 + Closes #5123 -- sasl: only enable if there's a protocol enabled using it +Marc Hoersken (20 Mar 2020) +- test2100: fix static port instead of dynamic value being used -- mime: acknowledge CURL_DISABLE_MIME +- test970: fix static ip:port instead of dynamic values being used -- wildcard: disable from build when FTP isn't present +Daniel Stenberg (19 Mar 2020) +- secure transport: remove the BACKEND define kludge + + Closes #5122 -- http: CURL_DISABLE_HTTP_AUTH +- mbedtls: remove the BACKEND define kludge -- base64: build conditionally if there are users +- bearssl: remove the BACKEND define kludge -- doh: CURL_DISABLE_DOH +- wolfssl: remove the BACKEND define kludge -Steve Holme (12 May 2019) -- auth: Rename the various authentication clean up functions - - For consistency and to a avoid confusion. - - Closes #3869 +- nss: remove the BACKEND define kludge -Daniel Stenberg (12 May 2019) -- [Jay Satiro brought this change] +- gnutls: remove the BACKEND define kludge - docs/INSTALL: fix broken link [ci skip] +- openssl: remove the BACKEND define kludge - Reported-by: Joombalaya on github - Fixes #3818 + Use a proper variable instead to make it easier to use a debugger and + read the code. -Marcel Raad (12 May 2019) -- easy: fix another "clarify calculation precedence" warning +Marc Hoersken (19 Mar 2020) +- tests: make Python-based servers compatible with Python 2 and 3 - I missed this one in commit 6b3dde7fe62ea5a557fd1fd323fac2bcd0c2e9be. - -- build: fix "clarify calculation precedence" warnings + Update smbserver.py and negtelnetserver.py to be compatible with + Python 3 while staying backwards-compatible to support Python 2. - Codacy/CppCheck warns about this. Consistently use parentheses as we - already do in some places to silence the warning. + Fix string encoding and handling of echoed and transferred data. - Closes https://github.com/curl/curl/pull/3866 - -- cmake: restore C89 compatibility of CurlTests.c + Tested with both Python 2.7.17 and Python 3.7.7 - I broke it in d1b5cf830bfe169745721b21245d2217d2c2453e and - 97de97daefc2ed084c91eff34af2426f2e55e134. + Reported-by: Daniel Stenberg + Assisted-by: Kamil Dudka + Reviewed-by: Marcel Raad - Reported-by: Viktor Szakats - Ref: https://github.com/curl/curl/commit/97de97daefc2ed084c91eff34af2426f2e55e134#commitcomment-33499044 - Closes https://github.com/curl/curl/pull/3868 + Fixes #5104 + Closes #5110 -Steve Holme (11 May 2019) -- http_ntlm: Corrected the name of the include guard +Daniel Stenberg (18 Mar 2020) +- writeout_json: use curl_off_t printf() option for the time output - Missed in f0bdd72c. + Follow-up to: 04c03416e68fd635a15 - Closes #3867 + Closes #5115 -- http_digest: Don't expose functions when HTTP and Crypto Auth are disabled +- RELEASE-NOTES: synced - Closes #3861 - -- http_negotiate: Don't expose functions when HTTP is disabled + Uh, I missed this in 1a46b218db -Daniel Stenberg (11 May 2019) -- SECURITY-PROCESS: fix links [ci skip] - -Marcel Raad (11 May 2019) -- CMake: suppress unused variable warnings +- RELEASE-NOTES: synced - I missed these in commit d1b5cf830bfe169745721b21245d2217d2c2453e. + ... and bumped curlver.h to 7.70.0 -Daniel Stenberg (11 May 2019) -- doh: disable DOH for the cases it doesn't work +Jay Satiro (18 Mar 2020) +- http2: Fix erroneous debug message that h2 connection closed - Due to limitations in Curl_resolver_wait_resolv(), it doesn't work for - DOH resolves. This fix disables DOH for those. + Prior to this change in libcurl debug builds http2 stream closure was + erroneously referred to as connection closure. - Limitation added to KNOWN_BUGS. + Before: + * nread <= 0, server closed connection, bailing - Fixes #3850 - Closes #3857 - -Jay Satiro (11 May 2019) -- checksrc.bat: Ignore snprintf warnings in docs/examples + After: + * nread == 0, stream closed, bailing - .. because we allow snprintf use in docs/examples. + Closes https://github.com/curl/curl/pull/5118 + +Daniel Stenberg (18 Mar 2020) +- tool_setopt: correct the copyright year range - Closes https://github.com/curl/curl/pull/3862 + Follow-up to 5450428491 + +Jay Satiro (18 Mar 2020) +- [Johannes Schindelin brought this change] -Steve Holme (10 May 2019) -- vauth: Fix incorrect function description for Curl_auth_user_contains_domain() + schannel: add "best effort" revocation check option - ...and misalignment of these comments. From a78c61a4. + - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and + --ssl-revoke-best-effort to allow a "best effort" revocation check. - Closes #3860 - -Jay Satiro (10 May 2019) -- Revert "multi: support verbose conncache closure handle" + A best effort revocation check ignores errors that the revocation check + was unable to take place. The reasoning is described in detail below and + discussed further in the PR. - This reverts commit b0972bc. + --- - - No longer show verbose output for the conncache closure handle. + When running e.g. with Fiddler, the schannel backend fails with an + unhelpful error message: - The offending commit was added so that the conncache closure handle - would inherit verbose mode from the user's easy handle. (Note there is - no way for the user to set options for the closure handle which is why - that was necessary.) Other debug settings such as the debug function - were not also inherited since we determined that could lead to crashes - if the user's per-handle private data was used on an unexpected handle. + Unknown error (0x80092012) - The revocation function was unable + to check revocation for the certificate. - The reporter here says he has a debug function to capture the verbose - output, and does not expect or want any output to stderr; however - because the conncache closure handle does not inherit the debug function - the verbose output for that handle does go to stderr. + Sadly, many enterprise users who are stuck behind MITM proxies suffer + the very same problem. - There are other plausible scenarios as well such as the user redirects - stderr on their handle, which is also not inherited since it could lead - to crashes when used on an unexpected handle. + This has been discussed in plenty of issues: + https://github.com/curl/curl/issues/3727, + https://github.com/curl/curl/issues/264, for example. - Short of allowing the user to set options for the conncache closure - handle I don't think there's much we can safely do except no longer - inherit the verbose setting. + In the latter, a Microsoft Edge developer even made the case that the + common behavior is to ignore issues when a certificate has no recorded + distribution point for revocation lists, or when the server is offline. + This is also known as "best effort" strategy and addresses the Fiddler + issue. - Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html - Reported-by: Kristoffer Gleditsch + Unfortunately, this strategy was not chosen as the default for schannel + (and is therefore a backend-specific behavior: OpenSSL seems to happily + ignore the offline servers and missing distribution points). - Ref: https://github.com/curl/curl/pull/3598 - Ref: https://github.com/curl/curl/pull/3618 + To maintain backward-compatibility, we therefore add a new flag + (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option + (`--ssl-revoke-best-effort`) to select the new behavior. - Closes https://github.com/curl/curl/pull/3856 - -Steve Holme (10 May 2019) -- ntlm: Fix misaligned function comments for Curl_auth_ntlm_cleanup() + Due to the many related issues Git for Windows and GitHub Desktop, the + plan is to make this behavior the default in these software packages. - From 6012fa5a. + The test 2070 was added to verify this behavior, adapted from 310. - Closes #3858 - -Daniel Stenberg (9 May 2019) -- BUG-BOUNTY: minor formatting fixes [ci skip] - -- RELEASE-NOTES: synced + Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com> + Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com> + Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> + + Closes https://github.com/curl/curl/pull/4981 -- BUG-BOUNTY.md: add the Dropbox "bonus" extra payout ability [ci skip] +- multi: Improve parameter check for curl_multi_remove_handle + + - If an easy handle is owned by a multi different from the one specified + then return CURLM_BAD_EASY_HANDLE. + + Prior to this change I assume user error could cause corruption. - Closes #3839 + Closes https://github.com/curl/curl/pull/5116 -Kamil Dudka (9 May 2019) -- http_negotiate: do not treat failure of gss_init_sec_context() as fatal +Viktor Szakats (17 Mar 2020) +- windows: suppress UI in all CryptAcquireContext() calls - Fixes #3726 - Closes #3849 + Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters + Reviewed-by: Marc Hörsken + Closes https://github.com/curl/curl/pull/5088 -- spnego_gssapi: fix return code on gss_init_sec_context() failure +Daniel Stenberg (17 Mar 2020) +- writeout_json: add missing comma to fix the HTTP version - Fixes #3726 - Closes #3849 + Follow-up to 04c03416e68fd635a15 -Steve Holme (9 May 2019) -- gen_resp_file.bat: Removed unnecessary @ from all but the first command +- test 970: verify --write-out '%{json}' - There is need to use @ on every command once echo has been turned off. + Makes curl_easy_getinfo() of "variable" numerical content instead return + the number set in the env variable `CURL_TIME`. - Closes #3854 - -Jay Satiro (8 May 2019) -- http: Ignore HTTP/2 prior knowledge setting for HTTP proxies + Makes curl_version() of "variable" textual content. This guarantees a + stable version string which can be tested against. Environment variable + `CURL_VERSION` defines the content. - - Do not switch to HTTP/2 for an HTTP proxy that is not tunnelling to - the destination host. + Assisted-by: Mathias Gumz + +- [Mathias Gumz brought this change] + + writeout: support to generate JSON output - We already do something similar for HTTPS proxies by not sending h2. [1] + This commit adds support to generate JSON via the writeout feature: - Prior to this change setting CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE would - incorrectly use HTTP/2 to talk to the proxy, which is not something we - support (yet?). Also it's debatable whether or not that setting should - apply to HTTP/2 proxies. + -w "%{json}" - [1]: https://github.com/curl/curl/commit/17c5d05 + It leverages the existing infrastructure as much as possible. Thus, + generating the JSON on STDERR is possible by: - Bug: https://github.com/curl/curl/issues/3570 - Bug: https://github.com/curl/curl/issues/3832 + -w "%{stderr}%{json}" - Closes https://github.com/curl/curl/pull/3853 - -Marcel Raad (8 May 2019) -- travis: update mesalink build to xenial + This implements a variant of + https://github.com/curl/curl/wiki/JSON#--write-out-json. - Closes https://github.com/curl/curl/pull/3842 + Closes #4870 -Daniel Stenberg (8 May 2019) -- [Ricky Leverence brought this change] +- CI: stop ignoring 323, it is disabled - OpenSSL: Report -fips in version if OpenSSL is built with FIPS +- DISABLED: disable test 323 + + The test uses SRP to "a server not supporting it" but modern stunnel + versions will silently accept it and remain happy. The test is therefore + faulty. - Older versions of OpenSSL report FIPS availabilty via an OPENSSL_FIPS - define. It uses this define to determine whether to publish -fips at - the end of the version displayed. Applications that utilize the version - reported by OpenSSL will see a mismatch if they compare it to what curl - reports, as curl is not modifying the version in the same way. This - change simply adds a check to see if OPENSSL_FIPS is defined, and will - alter the reported version to match what OpenSSL itself provides. This - only appears to be applicable in versions of OpenSSL <1.1.1 + I haven't figured out how to make stunnel explicitly reject SRP-using + connects. - Closes #3771 + Reported-by: Marc Hörsken + Fixes #5105 + Closes #5113 -Kamil Dudka (7 May 2019) -- [Frank Gevaerts brought this change] +Marc Hoersken (17 Mar 2020) +- ci/tests: increase timeouts for torture builds on Azure Pipelines + + For some reason the torture builds have slowed down recently. + + Reported-by: Daniel Stenberg - nss: allow fifos and character devices for certificates. +Daniel Stenberg (16 Mar 2020) +- cmake: add support for building with wolfSSL - Currently you can do things like --cert <(cat ./cert.crt) with (at least) the - openssl backend, but that doesn't work for nss because is_file rejects fifos. + My working build cmdline: - I don't actually know if this is sufficient, nss might do things internally - (like seeking back) that make this not work, so actual testing is needed. + $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . - Closes #3807 + Assisted-by: Brad King + Closes #5095 -Daniel Gustafsson (6 May 2019) -- test2100: Fix typos in test description - -Daniel Stenberg (6 May 2019) -- ssh: define USE_SSH if SSH is enabled (any backend) +- tool_operate: fix add_parallel_transfers when more are in queue - Closes #3846 - -Steve Holme (5 May 2019) -- winbuild: Add our standard copyright header to the winbuild batch files - -- makedebug: Fix ERRORLEVEL detection after running where.exe + Trying to return early from the function if no new transfers were added + would break the "morep" argument and cause issues. This could lead to + zero content "transfers" (within quotes since they would never be + started) when parallel-max was reduced. - Closes #3838 + Reported-by: Gavin Wong + Analyzed-by: Jay Satiro + Fixes #4937 + Closes #5112 -Daniel Stenberg (5 May 2019) -- urlapi: add CURLUPART_ZONEID to set and get +- vtls: free ssl_config leftovers on out-of-memory - The zoneid can be used with IPv6 numerical addresses. + Torture testing 2034 and 2037 found this. - Updated test 1560 to verify. + Reported-by: Marc Hörsken + Fixes #5108 + Closes #5109 + +Marc Hoersken (16 Mar 2020) +- ci/tests: fix Azure Pipelines not running for pull requests - Closes #3834 + Closes #5111 -- [Taiyu Len brought this change] +Daniel Stenberg (15 Mar 2020) +- gskit: update the copyright year range + + Follow-up from 083603c63a3 - WRITEFUNCTION: add missing set_in_callback around callback +Marc Hoersken (15 Mar 2020) +- gskit: use our internal select wrapper for portability - Closes #3837 + Follow up to c52b342 + Closes #5106 -- RELEASE-NOTES: synced +- tests: fix verification of stdout in test 1452 due to newline + + Fixes test1452:41:1: error: missing </stdout> tag before </verify> -- CURLMOPT_TIMERFUNCTION.3: warn about the recursive risk [ci skip] +- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI + + Also force the package index/cache to be updated before installing. - Reported-by: Ricardo Gomes + Closes #5103 + +- tests/README: add note about manually installing python-impacket - Bug: #3537 - Closes #3836 + Follow up to 4be2560 -- CURLOPT_CHUNK_BGN_FUNCTION.3: document the struct and time value +Daniel Stenberg (15 Mar 2020) +- transfer: cap retries of "dead connections" to 5 + + When libcurl retries a connection due to it being "seemingly dead" or by + REFUSED_STREAM, it will now only do it up five times before giving up, + to avoid never-ending loops. - The time field in the curl_fileinfo struct will always be zero. No code - was ever implemented to actually convert the date string to a time_t. + Reported-by: Dima Tisnek + Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html + Closes #5074 + +- TODO: TLS-PSK with OpenSSL - Fixes #3829 - Closes #3835 + Closes #5081 -- OS400/ccsidcurl.c: code style fixes +Marc Hoersken (15 Mar 2020) +- select: add 'timeout_ms' wrap-around precaution to Curl_select -- OS400/ccsidcurl: replace use of Curl_vsetopt +- select: fix 'pending_ms' is assigned a value that is never used - (and make the code style comply) + Detected by Codacy + +- select: move duplicate select preparation code into Curl_select - Fixes #3833 + Reviewed by Daniel Stenberg + Reviewed by Marcel Raad + Closes #5078 -- urlapi: strip off scope id from numerical IPv6 addresses +Daniel Stenberg (15 Mar 2020) +- connect: happy eyeballs cleanup - ... to make the host name "usable". Store the scope id and put it back - when extracting a URL out of it. + Make sure each separate index in connn->tempaddr[] is used for a fixed + family (and only that family) during the connection process. - Also makes curl_url_set() syntax check CURLUPART_HOST. + If family one takes a long time and family two fails immediately, the + previous logic could misbehave and retry the same family two address + repeatedly. - Fixes #3817 - Closes #3822 + Reported-by: Paul Vixie + Reported-by: Jay Satiro + Fixes #5083 + Fixes #4954 + Closes #5089 -- RELEASE-NOTES: synced +Marc Hoersken (15 Mar 2020) +- ci/tests: fix and align setting TFLAGS for make test-nonflaky -- multiif.h: remove unused protos +- ci/tests: install test suite dependencies stunnel and impacket + +- tests: remove python_dependencies for smbserver from our tree - ... for functions related to pipelining. Those functions were removed in - 2f44e94efb3df. + Users of the SMB tests will have to install impacket manually. - Closes #3828 - -- [Yiming Jing brought this change] - - travis: mesalink: temporarily disable test 3001 + Reasoning: our in-tree version of impacket was quite outdated + and only compatible with Python 2 which is already end-of-life. + Upgrading to Python 3 and a compatible impacket version would + require to import additional Python-only and CPython-extension + dependencies. This would have hindered portability enormously. - ... due to SHA-1 signatures in test certs - -- [Yiming Jing brought this change] + Closes #5094 - travis: upgrade the MesaLink TLS backend to v1.0.0 +Jay Satiro (14 Mar 2020) +- Makefile.m32: Improve windres parameter compatibility - Closes #3823 - Closes #3776 - -- ConnectionExists: improve non-multiplexing use case + - s/COFF/coff/ - - better log output + Some versions of windres do not recognize uppercase COFF as a valid + way to specify the COFF output format. - - make sure multiplex is enabled for it to be used - -- multi: provide Curl_multiuse_state to update information + Reported-by: Steven Penny - As soon as a TLS backend gets ALPN conformation about the specific HTTP - version it can now set the multiplex situation for the "bundle" and - trigger moving potentially queued up transfers to the CONNECT state. + Fixes https://github.com/curl/curl/issues/5099 + Closes https://github.com/curl/curl/pull/5101 -- process_pending_handles: mark queued transfers as previously pending +- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - With transfers being queued up, we only move one at a a time back to the - CONNECT state but now we mark moved transfers so that when a moved - transfer is confirmed "successful" (it connected) it will trigger the - move of another pending transfer. Previously, it would otherwise wait - until the transfer was done before doing this. This makes queued up - pending transfers get processed (much) faster. - -- http: mark bundle as not for multiuse on < HTTP/2 response + - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in + curl_easy_duphandle. - Fixes #3813 - Closes #3815 - -Daniel Gustafsson (1 May 2019) -- cookie: Guard against possible NULL ptr deref + Prior to this change if c-ares was used as the resolver backend and + either it was too old or libcurl was built without IPv6 support then + some of our resolver functions could return CURLE_NOT_BUILT_IN to + curl_easy_duphandle causing it to fail. - In case the name pointer isn't set (due to memory pressure most likely) - we need to skip the prefix matching and reject with a badcookie to avoid - a possible NULL pointer dereference. + Caused by c8f086b which shipped in 7.69.1. - Closes #3820 #3821 - Reported-by: Jonathan Moerman - Reviewed-by: Daniel Stenberg <daniel@haxx.se> - -Patrick Monnerat (30 Apr 2019) -- os400: Add CURLOPT_MAXAGE_CONN to ILE/RPG bindings - -Kamil Dudka (29 Apr 2019) -- nss: provide more specific error messages on failed init + Reported-by: Karl Chen - Closes #3808 - -Daniel Stenberg (29 Apr 2019) -- [Reed Loden brought this change] + Fixes https://github.com/curl/curl/issues/5097 + Closes https://github.com/curl/curl/pull/5100 - docs: minor polish to the bug bounty / security docs +Daniel Stenberg (13 Mar 2020) +- docs: add warnings about FILE: URLs on Windows - Closes #3811 + - --url man page section + - libcurl-security.3 gets the full text + - CURLOPT_URL.3 + + Reported-by: Tim Sedlmeyer -- CURL_MAX_INPUT_LENGTH: largest acceptable string input size +- server/getpart: make the "XML-parser" stricter - This limits all accepted input strings passed to libcurl to be less than - CURL_MAX_INPUT_LENGTH (8000000) bytes, for these API calls: - curl_easy_setopt() and curl_url_set(). + When extracting a <section> <part> and there's no </part> before + </section>, this now outputs an error and returns a wrong string to + make users spot the mistake. - The 8000000 number is arbitrary picked and is meant to detect mistakes - or abuse, not to limit actual practical use cases. By limiting the - acceptable string lengths we also reduce the risk of integer overflows - all over. + Ref: #5070 + Closes #5071 + +Marc Hoersken (13 Mar 2020) +- impacket: some more Python 3 code compatibility updates - NOTE: This does not apply to `CURLOPT_POSTFIELDS`. + This makes smbserver load on Python 3, but still not work completely. + +- smbserver: pin Python version to 2 since we are not yet 3 compatible - Test 1559 verifies. + Even though the existing code can be fixed to run on Python 3, the + tests will fail due to the Unicode transition the protocol is invalid. - Closes #3805 + Follow up to ee63837 + Closes #5085 -- [Tseng Jun brought this change] +Daniel Stenberg (12 Mar 2020) +- [Viktor Szakats brought this change] - curlver.h: use parenthesis in CURL_VERSION_BITS macro + cleanup: fix some text/comment typos - Closes #3809 + Closes #5087 -Marcel Raad (27 Apr 2019) -- [Simon Warta brought this change] - - cmake: rename CMAKE_USE_DARWINSSL to CMAKE_USE_SECTRANSP +Marc Hoersken (12 Mar 2020) +- smbserver: fix Python version specific ConfigParser import - Closes https://github.com/curl/curl/pull/3769 - -Steve Holme (23 Apr 2019) -- ntlm: Missed pre-processor || (or) during rebase for cd15acd0 + Follow up to ee63837 and 8c7c4a6 + Fixes #5077 -- ntlm: Support the NT response in the type-3 when OpenSSL doesn't include MD4 +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: synced - Just like we do for mbed TLS, use our local implementation of MD4 when - OpenSSL doesn't support it. This allows a type-3 message to include the - NT response. + bumped to 7.69.2 -Daniel Gustafsson (23 Apr 2019) -- INTERNALS: fix misindentation of ToC item +Dan Fandrich (11 Mar 2020) +- tests/data: Fix some XML formatting issues in test cases - Kerberos was incorrectly indented as a subsection under FTP, which is - incorrect as they are both top level sections. A fix for this was first - attempted in commit fef38a0898322f285401c5ff2f5e7c90dbf3be63 but that - was a few paddles short of being complete. + This allows these test files to pass xmllint. -- [Aron Bergman brought this change] +Daniel Stenberg (11 Mar 2020) +- [Muhammad Herdiansyah brought this change] - INTERNALS: Add structs to ToC + Makefile: run the cd commands in a subshell - Add the subsections under "Structs in libcurl" to the table of contents. + In bmake, if the directory is changed (with cd or anything else), bmake + won't return to the "root directory" on the next command (in the same + Makefile rule). This commit runs the cd command in a subshell so it + would work in bmake. - Reviewed-by: Daniel Stenberg <daniel@haxx.se> - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> - -- [Aron Bergman brought this change] + Closes #5073 - INTERNALS: Add code highlighting +- configure: convert -I to -isystem as a last step - Make all struct members under the Curl_handler section - print in monospace font. + As all the -I uses in CFLAGS at that point are for system headers and + third party libraries this helps us remove/ignore warnings on those! - Closes #3801 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> + Closes #5060 -Daniel Stenberg (22 Apr 2019) -- docs/BUG-BOUNTY: bug bounty time [skip ci] +- configure: fix -pedantic-errors for GCC 5 and later - Introducing the curl bug bounty program on hackerone. We now recommend - filing security issues directly in the hackerone ticket system which - only is readable to curl security team members. + If --enable-werror is used. - Assisted-by: Daniel Gustafsson + Follow-up to d5c0351055d5709da which added it too early in the configure + script before $compiler_num was set correctly and thus this option was + never used. - Closes #3488 + Reported-by: Stepan Efremov + Fixes #5067 + Closes #5068 -Steve Holme (22 Apr 2019) -- sasl: Don't send authcid as authzid for the PLAIN mechanism as per RFC 4616 +- configure: document 'compiler_num' for gcc + + The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + + MINOR and ignores the patch version, and since gcc version 7 it only + sets it to MAJOR*100. - RFC 4616 specifies the authzid is optional in the client authentication - message and that the server will derive the authorisation identity - (authzid) from the authentication identity (authcid) when not specified - by the client. + Reported-by: Stepan Efremov + Ref: #5067 + Closes #5069 -Jay Satiro (22 Apr 2019) -- [Gisle Vanem brought this change] +Version 7.69.1 (11 Mar 2020) - memdebug: fix variable name - - Follow-up to 76b6348 which renamed logfile as curl_dbg_logfile. - - Ref: https://github.com/curl/curl/commit/76b6348#r33259088 +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: 7.69.1 -Steve Holme (21 Apr 2019) -- vauth/cleartext: Don't send the authzid if it is empty - - Follow up to 762a292f. +- THANKS: from the 7.69.1 release -Daniel Stenberg (21 Apr 2019) -- test 196,197,198: add 'retry' keyword [skip ci] +- [Marc Hoersken brought this change] -- RELEASE-NOTES: synced + test1129: fix invalid case of closing XML-tag and Content-Length + + Fixes #5070 + Closes #5072 -- CURLOPT_MAXAGE_CONN: set the maximum allowed age for conn reuse +Marc Hoersken (10 Mar 2020) +- tests/data: fix static ip instead of dynamic value being used - ... and disconnect too old ones instead of trying to reuse. + Follow up to 94ced8e + +- tests/data: fix static ip:port instead of dynamic values being used - Default max age is set to 118 seconds. + Closes #5065 + +- tests/server: fix missing use of exe_ext helper function - Ref: #3722 - Closes #3782 + Follow up to 9819984 and 3dce984 + Reviewed-By: Daniel Stenberg + Closes #5064 + +- runtests: log minimal and maximal used port numbers -Daniel Gustafsson (20 Apr 2019) -- [Po-Chuan Hsieh brought this change] +Daniel Stenberg (9 Mar 2020) +- [James Fuller brought this change] - altsvc: Fix building with cookies disables + sftp: fix segfault regression introduced by #4747 - ALTSVC requires Curl_get_line which is defined in lib/cookie.c inside a #if - check of HTTP and COOKIES. That makes Curl_get_line undefined if COOKIES is - disabled. Fix by splitting out the function into a separate file which can - be included where needed. + This fix adds a defensive check for the case where the char *name in + struct libssh2_knownhost is NULL - Closes #3717 - Reviewed-by: Daniel Gustafsson <daniel@yesql.se> - Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com> + Fixes #5041 + Closes #5062 -Daniel Stenberg (20 Apr 2019) -- test1002: correct the name [skip ci] +- RELEASE-NOTES: synced -- test660: verify CONNECT_ONLY with IMAP +- socks4: fix host resolve regression - which basically just makes sure LOGOUT is *not* issued on disconnect - -- Curl_disconnect: treat all CONNECT_ONLY connections as "dead" + 1. The socks4 state machine was broken in the host resolving phase - Since the connection has been used by the "outside" we don't know the - state of it anymore and curl should not use it anymore. + 2. The code now insists on IPv4-only when using SOCKS4 as the protocol + only supports that. - Bug: https://curl.haxx.se/mail/lib-2019-04/0052.html + Regression from #4907 and 4a4b63d, shipped in 7.69.0 - Closes #3795 + Reported-by: amishmm on github + Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 + Closes #5061 -- multi: fix the statenames (follow-up fix from 2f44e94efb3df8e) +- [Patrick Monnerat brought this change] + + silly web server: silent a compilation warning + + Recent gcc warns when byte count of strncpy() equals the destination + buffer size. Since the destination buffer is previously cleared and + the source string is always shorter, reducing the byte count by one + silents the warning without affecting the result. - The list of names must be in sync with the defined states in the header - file! + Closes #5059 -Steve Holme (16 Apr 2019) -- openvms: Remove pre-processors for Windows as VMS cannot support them +- [Patrick Monnerat brought this change] -- openvms: Remove pre-processor for SecureTransport as VMS cannot support it + cookie: get_top_domain() sets zero length for null domains - Fixes #3768 - Closes #3785 + This silents a compilation warning with gcc -O3. -Jay Satiro (16 Apr 2019) -- TODO: Add issue link to an existing entry - -Daniel Stenberg (16 Apr 2019) -- RELEASE-NOTES: synced +- [Patrick Monnerat brought this change] -Jay Satiro (16 Apr 2019) -- tool_help: Warn if curl and libcurl versions do not match - - .. because functionality may be affected if the versions differ. - - This commit implements TODO 18.7 "warning if curl version is not in sync - with libcurl version". + test 1560: avoid valgrind false positives - Ref: https://github.com/curl/curl/blob/curl-7_64_1/docs/TODO#L1028-L1033 + When using maximum code optimization level (-O3), valgrind wrongly + detects uses of uninitialized values in strcmp(). - Closes https://github.com/curl/curl/pull/3774 + Preset buffers with all zeroes to avoid that. -Steve Holme (16 Apr 2019) -- md5: Update the function signature following d84da52d +Steve Holme (8 Mar 2020) +- sha256: Added WinCrypt implementation + + Closed #5030 -- md5: Forgot to update the code alignment in d84da52d +- sha256: Added SecureTransport implementation -- md5: Return CURLcode from the internally accessible functions +Daniel Stenberg (7 Mar 2020) +- lib1564: reduce number of mid-wait wakeup calls - Following 28f826b3 to return CURLE_OK instead of numeric 0. - -Daniel Gustafsson (15 Apr 2019) -- tests: Run global cleanup at end of tests + This test does A LOT of *wakeup() calls and then calls curl_multi_poll() + twice. The first *poll() is then expected to return early and the second + not - as the first is supposed to drain the socketpair pipe. - Make sure to run curl_global_cleanup() when shutting down the test - suite to release any resources allocated in the SSL setup. This is - clearly visible when running tests with PolarSSL where the thread - lock calloc() memory which isn't released when not running cleanup. - Below is an excerpt from the autobuild logs: + It turns out however that when given "excessive" amounts of writes to + the pipe, some operating systems (the Solaris based are known) will + return EAGAIN before the pipe is drained, which in our test case causes + the second *poll() call to also abort early. - ==12368== 96 bytes in 1 blocks are possibly lost in loss record 1 of 2 - ==12368== at 0x4837B65: calloc (vg_replace_malloc.c:752) - ==12368== by 0x11A76E: curl_dbg_calloc (memdebug.c:205) - ==12368== by 0x145CDF: Curl_polarsslthreadlock_thread_setup - (polarssl_threadlock.c:54) - ==12368== by 0x145B37: Curl_polarssl_init (polarssl.c:865) - ==12368== by 0x14129D: Curl_ssl_init (vtls.c:171) - ==12368== by 0x118B4C: global_init (easy.c:158) - ==12368== by 0x118BF5: curl_global_init (easy.c:221) - ==12368== by 0x118D0B: curl_easy_init (easy.c:299) - ==12368== by 0x114E96: test (lib1906.c:32) - ==12368== by 0x115495: main (first.c:174) + This change attempts to avoid the OS-specific behaviors in the test by + reducing the amount of wakeup calls from 1234567 to 10. - Closes #3783 - Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com> - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Reported-by: Andy Fiddaman + Fixes #5037 + Closes #5058 -Marcel Raad (15 Apr 2019) -- travis: use mbedtls from Xenial - - No need to build it from source anymore. - - Closes https://github.com/curl/curl/pull/3779 +- [Patrick Monnerat brought this change] -- travis: use libpsl from Xenial + mime: fix the binary encoder to handle large data properly - This makes building libpsl and libidn2 from source unnecessary and - removes the need for the autopoint and libunistring-dev packages. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes https://github.com/curl/curl/pull/3779 + Fixes #4860 + Closes #4833 + Reported-by: RuurdBeerstra on github -Daniel Stenberg (15 Apr 2019) -- runtests: start socksd like other servers - - ... without a $srcdir prefix. Triggered by the failures in several - autobuilds. - - Closes #3781 +- [Patrick Monnerat brought this change] -Daniel Gustafsson (14 Apr 2019) -- socksd: Fix typos + mime: do not perform more than one read in a row - Reviewed-by: Daniel Stenberg <daniel@haxx.se> - -- socksd: Properly decorate static variables + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Mark global variables static to avoid compiler warning in Clang when - using -Wmissing-variable-declarations. + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 667 checks mimepost using single-byte read callback with encoder. + New test 668 checks the end of part data early detection. - Closes #3778 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Fixes #4826 + Reported-by: MrdUkk on github -Steve Holme (14 Apr 2019) -- md(4|5): Fixed indentation oddities with the importation of replacement code +- [Patrick Monnerat brought this change] + + mime: latch last read callback status. + + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - The indentation from 211d5329 and 57d6d253 was a little strange as - parts didn't align correctly, uses 4 spaces rather than 2. Checked - the indentation of the original source so it aligns, albeit, using - curl style. + Fixes #4813 + Reported-by: MrdUkk on github -- md5: Code style to return CURLE_OK rather than numeric 0 +Marc Hoersken (7 Mar 2020) +- runtests: fix missing use of exe_ext helper function -- md5: Corrected code style for some pointer arguments +Daniel Stenberg (7 Mar 2020) +- [Ernst Sjöstrand brought this change] -Marcel Raad (13 Apr 2019) -- travis: update some builds to xenial + ares: store dns parameters for duphandle - Xenial comes with more up-to-date software versions and more available - packages, some of which we currently build from source. Unfortunately, - some builds would fail with Xenial because of assertion failures in - Valgrind when using OpenSSL, so leave these at Trusty. + With c-ares the dns parameters lives in ares_channel. Store them in the + curl handle and set them again in easy_duphandle. - Closes https://github.com/curl/curl/pull/3777 - -Daniel Stenberg (13 Apr 2019) -- test: make tests and test scripts use socksd for SOCKS + Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. - Make all SOCKS tests use socksd instead of ssh. + Fixes #4893 + Closes #5020 + Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com> -- socksd: new SOCKS 4+5 server for tests +- version: make curl_version* thread-safe without using global context - Closes #3752 + Closes #5010 + +- RELEASE-NOTES: synced -- singleipconnect: show port in the verbose "Trying ..." message +Marc Hoersken (7 Mar 2020) +- tests: use native Sleep function as fallback on Windows - To aid debugging better. + Reviewed-By: Daniel Stenberg + Closes #5054 -- [tmilburn brought this change] +- perl: align order and completeness of Windows OS checks - CURLOPT_ADDRESS_SCOPE: fix range check and more +Daniel Stenberg (7 Mar 2020) +- tool_cb_see: set correct copyright year range - Commit 9081014 fixed most of the confusing issues between scope id and - scope however 844896d added bad limits checking assuming that the scope - is being set and not the scope id. - - I have fixed the documentation so it all refers to scope ids. - - In addition Curl_if2ip refered to the scope id as remote_scope_id which - is incorrect, so I renamed it to local_scope_id. - - Adjusted-by: Daniel Stenberg - - Closes #3655 - Closes #3765 - Fixes #3713 + Follow-up to a39e5bfb9 -- urlapi: stricter CURLUPART_PORT parsing - - Only allow well formed decimal numbers in the input. +Marc Hoersken (7 Mar 2020) +- seek: fix fallback for missing ftruncate on Windows - Document that the number MUST be between 1 and 65535. + This fixes test 198 on versions of MinGW-w64 without ftruncate - Add tests to test 1560 to verify the above. - - Ref: https://github.com/curl/curl/issues/3753 - Closes #3762 + Reviewed-By: Daniel Stenberg + Reviewed-By: Marcel Raad + Closes #5055 -Jay Satiro (13 Apr 2019) -- [Jan Ehrhardt brought this change] +- config-win32: Windows does not have ftruncate - winbuild: Support MultiSSL builds +Daniel Stenberg (7 Mar 2020) +- pause: force a connection (re-)check after unpausing - - Remove the lines in winbuild/Makefile.vc that generate an error with - multiple SSL backends. + There might be data available that was already read off the socket, for + example in the TLS layer. - - Add /DCURL_WITH_MULTI_SSL in winbuild/MakefileBuild.vc if multiple SSL - backends are set. + Reported-by: Anders Berg + Fixes #4966 + Closes #5049 + +- socks5: switch state properly when the resolve is done - Closes https://github.com/curl/curl/pull/3772 + Regression from 4a4b63d (and #4907) + Reported-by: vitaha85 on github + Fixes #5053 + Closes #5056 -Daniel Stenberg (12 Apr 2019) -- travis: remove mesalink builds (temporarily?) +Jay Satiro (7 Mar 2020) +- libssh: Fix matching user-specified MD5 hex key + + Prior to this change a match would never be successful because it + was mistakenly coded to compare binary data from libssh to a + user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). - Since the mesalink build started to fail on travis, even though we build - a fixed release version, we disable it to prevent it from blocking - progress. + Reported-by: fds242@users.noreply.github.com - Closes #3767 + Fixes https://github.com/curl/curl/issues/4971 + Closes https://github.com/curl/curl/pull/4974 -- openssl: mark connection for close on TLS close_notify +Daniel Stenberg (6 Mar 2020) +- pause: bail out on bad input - Without this, detecting and avoid reusing a closed TLS connection - (without a previous GOAWAY) when doing HTTP/2 is tricky. + A NULL easy handle or an easy handle without an associated connection + cannot be paused or unpaused. - Reported-by: Tom van der Woerdt - Fixes #3750 - Closes #3763 - -- RELEASE-NOTES: synced + Closes #5050 -Steve Holme (11 Apr 2019) -- vauth/cleartext: Update the PLAIN login function signature to match RFC 4616 +Steve Holme (6 Mar 2020) +- unit1612: fixed the inclusion and compilation of the HMAC unit test + + Follow up to 3f74e5e6 to fix: - Functionally this doesn't change anything as we still use the username - for both the authorisation identity and the authentication identity. + - A typo in Makefile.inc where unit1611 was used instead + - Some compilation issues in unit1612.c - Closes #3757 + Closes #5024 -Daniel Stenberg (11 Apr 2019) -- test1906: verify CURLOPT_CURLU + CURLOPT_PORT usage +Daniel Stenberg (6 Mar 2020) +- pause: return early for calls that don't change pause state - Based-on-code-by: Poul T Lomholt + Reviewed-by: Patrick Monnerat + Ref: #4833 + Closes #5026 -- url: always clone the CUROPT_CURLU handle +Jay Satiro (6 Mar 2020) +- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - Since a few code paths actually update that data. + Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused + CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - Fixes #3753 - Closes #3761 + Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html + Reported-by: Felipe Gasper - Reported-by: Poul T Lomholt + Closes https://github.com/curl/curl/pull/5048 -- CURLOPT_DNS_USE_GLOBAL_CACHE: remove - - Remove the code too. The functionality has been disabled in code since - 7.62.0. Setting this option will from now on simply be ignored and have - no function. +- multi: skip EINTR check on wakeup socket if it was closed - Closes #3654 - -Marcel Raad (11 Apr 2019) -- travis: install libgnutls28-dev only for --with-gnutls build + - Don't check errno on wakeup socket if sread returned 0 since sread + doesn't set errno in that case. - Reduces the time needed for the other jobs a little. + This is a follow-up to cf7760a from several days ago which fixed + Curl_multi_wait to stop busy looping sread on the non-blocking wakeup + socket if it was closed (ie sread returns 0). Due to a logic error it + was still possible to busy loop in that case if errno == EINTR. - Closes https://github.com/curl/curl/pull/3721 + Closes https://github.com/curl/curl/pull/5047 -- travis: install libnss3-dev only for --with-nss build - - Reduces the time needed for the other jobs a little. - - Closes https://github.com/curl/curl/pull/3721 +Daniel Stenberg (6 Mar 2020) +- transfer: set correct copyright year range -- travis: install libssh2-dev only for --with-libssh2 build +- urldata: remove the 'stream_was_rewound' connectdata struct member - Reduces the time needed for the other jobs a little. + ... as it is never set anywhere. - Closes https://github.com/curl/curl/pull/3721 + Follow-up to 2f44e94ef + Closes #5046 -- travis: install libssh-dev only for --with-libssh build +- Revert "pause: force-drain the transfer on unpause" - Reduces the time needed for the other jobs a little. + This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) - Closes https://github.com/curl/curl/pull/3721 - -- travis: install krb5-user only for --with-gssapi build - - Reduces the time needed for the other jobs a little. + Clearly that didn't solve the problem correctly. - Closes https://github.com/curl/curl/pull/3721 + Reported-by: Christopher Reid + Reopens #4966 + Fixes #5044 -- travis: install lcov only for the coverage job - - Reduces the time needed for the other jobs a little. +- RELEASE-NOTES: synced - Closes https://github.com/curl/curl/pull/3721 + and bumped curlver.h -- travis: install clang only when needed +- MANUAL: update a dict-using command line - This reduces the GCC job runtimes a little and it's needed to - selectively update clang builds to xenial. + The 'web1913' database is now invalid, use 'gcide' instead. + +- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - Closes https://github.com/curl/curl/pull/3721 + Closes #3841 -- AppVeyor: enable testing for WinSSL build +- polarssl: remove more references and mentions - Closes https://github.com/curl/curl/pull/3725 + Assisted-by: Jay Satiro + Follow-up to 6357a19ff29dac04 + Closes #5036 + +Marc Hoersken (4 Mar 2020) +- tests: wrap ignored test failures in braces -- build: fix Codacy/CppCheck warnings +- tests: align some Windows sleep defines with each other + +- tests: try to make sleeping portable by avoiding select - - remove unused variables - - declare conditionally used variables conditionally - - suppress unused variable warnings in the CMake tests - - remove dead variable stores - - consistently use WIN32 macro to detect Windows + select does not support just waiting on Windows: + https://perldoc.perl.org/perlport.html#select - Closes https://github.com/curl/curl/pull/3739 + Reviewed-By: Daniel Stenberg + Closes #5035 -- polarssl_threadlock: remove conditionally unused code - - Make functions no-ops if neither both USE_THREADS_POSIX and - HAVE_PTHREAD_H nor both USE_THREADS_WIN32 and HAVE_PROCESS_H are - defined. Previously, if only one of them was defined, there was either - code compiled that did nothing useful or the wrong header included for - the functions used. +Daniel Stenberg (4 Mar 2020) +- runtests.1: rephrase how to specify what tests to run - Also, move POLARSSL_MUTEX_T define to implementation file as it's not - used externally. + Also mention the new tilde-prefixed way to ignore test results. - Closes https://github.com/curl/curl/pull/3739 + Reviewed-By: Marc Hoersken + Closes #5033 -- lib557: initialize variables +- cirrus-ci: disable the FreeBSD 13 builds - These variables are only conditionally initialized. + FreeBSD 13.0 is apparently close to a year away from a stable release + and has proven to cause intermittent builds failures recently. - Closes https://github.com/curl/curl/pull/3739 + Assisted-by: Dan Fandrich + Assisted-by: Fedor Korotkov + Fixes #5028 + Closes #5029 -- lib509: add missing include for strdup - - Closes https://github.com/curl/curl/pull/3739 +Version 7.69.0 (4 Mar 2020) + +Daniel Stenberg (4 Mar 2020) +- RELEASE-NOTES: 7.69.0 -- README.md: fix no-consecutive-blank-lines Codacy warning +- THANKS: from 7.69.0 - Consistently use one blank line between blocks. + Now sorted case insensitive + +Marc Hoersken (3 Mar 2020) +- ci/tests: fix escaping of testnames and disable proxy for CI APIs - Closes https://github.com/curl/curl/pull/3739 + Follow up to ada581f and c0d8b96 + Closes #5031 -- tests/server/util: fix Windows Unicode build +Jay Satiro (3 Mar 2020) +- cmake: Show HTTPS-proxy in the features output - Always use the ANSI version of FormatMessage as we don't have the - curl_multibyte gear available here. + - Show HTTPS-proxy in the features output for those backends that + support it: OpenSSL, GnuTLS and NSS. - Closes https://github.com/curl/curl/pull/3758 - -Daniel Stenberg (11 Apr 2019) -- curl_easy_getinfo.3: fix minor formatting mistake - -Daniel Gustafsson (11 Apr 2019) -- xattr: skip unittest on unsupported platforms + Prior to this change HTTPS-proxy was missing from the cmake features + output even if curl was built with it. Only cmake output was affected. + Both the library and tool correctly reported the feature. - The stripcredentials unittest fails to compile on platforms without - xattr support, for example the Solaris member in the buildfarm which - fails with the following: + Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html + Reported-by: David Lopes - CC unit1621-unit1621.o - CC ../libtest/unit1621-first.o - CCLD unit1621 - Undefined first referenced - symbol in file - stripcredentials unit1621-unit1621.o - goto problem 2 - ld: fatal: symbol referencing errors. No output written to .libs/unit1621 - collect2: error: ld returned 1 exit status - gmake[2]: *** [Makefile:996: unit1621] Error 1 + Closes https://github.com/curl/curl/pull/5025 + +Marc Hoersken (3 Mar 2020) +- ci/tests: Make it possible to still run but ignore failing tests - Fix by excluding the test on such platforms by using the reverse - logic from where stripcredentials() is defined. + This enables the development of a solution for the failing tests by + running them on CI while ignoring their result for the overall status. - Closes #3759 - Reviewed-by: Daniel Stenberg <daniel@haxx.se> + Closes #4994 -Steve Holme (11 Apr 2019) -- emailL Added reference to RFC8314 for implicit TLS +- README.md: add Azure DevOps Pipelines build status badge -- README: Schannel, stop calling it "winssl" +- ci/tests: Move CI test result creation above environment setup - Stick to "Schannel" everywhere - follow up to 180501cb. - -Jakub Zakrzewski (10 Apr 2019) -- cmake: clear CMAKE_REQUIRED_LIBRARIES after each use + This avoids using our test servers as proxy to the AppVeyor API. - This fixes GSSAPI builds with the libraries in a non-standard location. - The testing for recv() were failing because it failed to link - the Kerberos libraries, which are not needed for this or subsequent - tests. + Closes #5022 + +- ci/tests: Send test results to AppVeyor for status overview - fixes #3743 - closes #3744 + Closes #5021 -- cmake: avoid linking executable for some tests with cmake 3.6+ +Daniel Stenberg (3 Mar 2020) +- Revert "sha256: Added SecureTransport implementation" - With CMAKE_TRY_COMPILE_TARGET_TYPE set to STATIC_LIBRARY, the try_compile() - (which is used by check_c_source_compiles()) will build static library - instead of executable. This avoids linking additional libraries in and thus - speeds up those checks a little. + This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - This commit also avoids #3743 (GSSAPI build errors) on itself with cmake - 3.6 or above. That issue was fixed separately for all versions. + That commit broke test 1610 on macos builds without TLS. - Ref: #3744 + Closes #5027 -- cmake: minor cleanup +- dist: include tests/azure.pm in the tarball - - Remove nneeded include_regular_expression. - It was setting what is already a default. + Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 + Reported-by: Marcel Raad + +Steve Holme (3 Mar 2020) +- configure.ac: Disable metalink if mbedTLS is specified - - Remove duplicated include. + Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as + being one of the backends that metalink needs to be disabled for, I + seem to have included it in the list of allowed SSL/TLS backends in + comnfigure.ac :( - - Don't check for pre-3.0.0 CMake version. - We already require at least 3.0.0, so it's just clutter. + Closes #5013 + +- sha256: Tidy up following recent changes - Ref: #3744 + Reviewed-by: Daniel Stenberg + Closes #4956 -Steve Holme (8 Apr 2019) -- build-openssl.bat: Fixed support for OpenSSL v1.1.0+ +- sha256: Added WinCrypt implementation -- build-openssl.bat: Perfer the use of if statements rather than goto (where possible) +- sha256: Added SecureTransport implementation -- build-openssl.bat: Perform the install for each build type directly after the build +- sha256: Added mbedtls implementation -- build-openssl.bat: Split the install of static and shared build types +- sha256: Added GNU TLS gcrypt implementation -- build-openssl.bat: Split the building of static and shared build types +- sha256: Added GNU TLS Nettle implementation -- build-openssl.bat: Move the installation into a separate function +Jay Satiro (2 Mar 2020) +- curl_escape.3: Add a link to curl_free + + Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 -- build-openssl.bat: Move the build step into a separate function +- curl_getenv.3: Fix the memory handling description + + - Tell the user to call curl_free() to free the pointer returned by + curl_getenv(). + + Prior to this change the user was directed to call free(), but that + would not work in cases where the library and application use separate C + runtimes and therefore have separate heap memory management. + + Closes https://github.com/curl/curl/pull/5016 -- build-openssl.bat: Move the OpenSSL configuration into a separate function +Daniel Stenberg (2 Mar 2020) +- [Nick Zitzmann brought this change] -- build-openssl.bat: Fixed the BUILD_CONFIG variable not being initialised + md4: use init/update/final functions in Secure Transport + + We can use CC_MD4_Init/Update/Final without having to allocate memory + directly. - Should the parent environment set this variable then the build might - not be performed as the user intended. + Closes #4979 -Daniel Stenberg (8 Apr 2019) -- socks: fix error message +Marc Hoersken (2 Mar 2020) +- ci/tests: some MacOS builds randomly take longer than 20min -- config.d: clarify that initial : and = might need quoting [skip ci] +Daniel Stenberg (2 Mar 2020) +- multi_wait: stop loop when sread() returns zero - Fixes #3738 - Closes #3749 + It's unclear why it would ever return zero here, but this change fixes + Robert's problem and it shouldn't loop forever... + + Reported-by: Robert Dunaj + Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html + Closes #5019 -- RELEASE-NOTES: synced +- http: mark POSTs with no body as "upload done" from the start - bumped to 7.65.0 for next release + As we have logic that checks if we get a >= 400 reponse code back before + the upload is done, which then got confused since it wasn't "done" but + yet there was no data to send! + + Reported-by: IvanoG on github + Fixes #4996 + Closes #5002 -- socks5: user name and passwords must be shorter than 256 +- tests: disable 962, 963 and 964 on Windows - bytes... since the protocol needs to store the length in a single byte field. + These tests are also doing UTF-8 SMTP. - Reported-by: XmiliaH on github - Fixes #3737 - Closes #3740 + Follow-up to df207d2dd93b9e73 -- [Jakub Zakrzewski brought this change] +Marc Hoersken (2 Mar 2020) +- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer - test: urlapi: urlencode characters above 0x7f correctly +Daniel Stenberg (2 Mar 2020) +- configure: bump the AC_COPYRIGHT year range -- [Jakub Zakrzewski brought this change] +- [Steve Holme brought this change] - urlapi: urlencode characters above 0x7f correctly + tests: disable SMTP UTF-8 tests on Windows - fixes #3741 - Closes #3742 - -- [Even Rouault brought this change] + Fixes #4988 + Closes #4992 - multi_runsingle(): fix use-after-free - - Fixes #3745 - Closes #3746 - - The following snippet - ``` - - int main() - { - CURL* hCurlHandle = curl_easy_init(); - curl_easy_setopt(hCurlHandle, CURLOPT_URL, "http://example.com"); - curl_easy_setopt(hCurlHandle, CURLOPT_PROXY, "1"); - curl_easy_perform(hCurlHandle); - curl_easy_cleanup(hCurlHandle); - return 0; - } - ``` - triggers the following Valgrind warning - - ``` - ==4125== Invalid read of size 8 - ==4125== at 0x4E7D1EE: Curl_llist_remove (llist.c:97) - ==4125== by 0x4E7EF5C: detach_connnection (multi.c:798) - ==4125== by 0x4E80545: multi_runsingle (multi.c:1451) - ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) - ==4125== by 0x4E766A0: easy_transfer (easy.c:625) - ==4125== by 0x4E76915: easy_perform (easy.c:719) - ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) - ==4125== by 0x4008BE: main (in /home/even/curl/test) - ==4125== Address 0x9b3d1d0 is 1,120 bytes inside a block of size 1,600 free'd - ==4125== at 0x4C2ECF0: free (vg_replace_malloc.c:530) - ==4125== by 0x4E62C36: conn_free (url.c:756) - ==4125== by 0x4E62D34: Curl_disconnect (url.c:818) - ==4125== by 0x4E48DF9: Curl_once_resolved (hostip.c:1097) - ==4125== by 0x4E8052D: multi_runsingle (multi.c:1446) - ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) - ==4125== by 0x4E766A0: easy_transfer (easy.c:625) - ==4125== by 0x4E76915: easy_perform (easy.c:719) - ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) - ==4125== by 0x4008BE: main (in /home/even/curl/test) - ==4125== Block was alloc'd at - ==4125== at 0x4C2F988: calloc (vg_replace_malloc.c:711) - ==4125== by 0x4E6438E: allocate_conn (url.c:1654) - ==4125== by 0x4E685B4: create_conn (url.c:3496) - ==4125== by 0x4E6968F: Curl_connect (url.c:4023) - ==4125== by 0x4E802E7: multi_runsingle (multi.c:1368) - ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) - ==4125== by 0x4E766A0: easy_transfer (easy.c:625) - ==4125== by 0x4E76915: easy_perform (easy.c:719) - ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) - ==4125== by 0x4008BE: main (in /home/even/curl/test) - ``` +- formdata/mime: copyright year range update - This has been bisected to commit 2f44e94 - - Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14109 - Credit to OSS Fuzz + Due to the merge/revert cycle -- pipelining: removed +- Revert "mime: latch last read callback status." - As previously planned and documented in DEPRECATE.md, all pipelining - code is removed. + This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. - Closes #3651 - -- [cclauss brought this change] + Fixes #5014 + Closes #5015 + Reopens #4833 - tests: make Impacket (SMB server) Python 3 compatible +- Revert "mime: do not perform more than one read in a row" - Closes #3731 - Fixes #3289 - -Marcel Raad (6 Apr 2019) -- [Simon Warta brought this change] + This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. - cmake: set SSL_BACKENDS +- Revert "mime: fix the binary encoder to handle large data properly" - This groups all SSL backends into the feature "SSL" and sets the - SSL_BACKENDS analogue to configure.ac + This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. + +- altsvc: both h3 backends now speak h3-27 - Closes https://github.com/curl/curl/pull/3736 + ... also updated the HTTP3 build description for ngtcp2 accordingly. -- [Simon Warta brought this change] +- [Patrick Monnerat brought this change] - cmake: don't run SORT on empty list + mime: fix the binary encoder to handle large data properly - In case of an empty list, SORTing leads to the cmake error "list - sub-command SORT requires list to be present." + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes https://github.com/curl/curl/pull/3736 + Fixes #4860 + Reported-by: RuurdBeerstra on github -Daniel Gustafsson (5 Apr 2019) -- [Eli Schwartz brought this change] +- [Patrick Monnerat brought this change] - configure: fix default location for fish completions + mime: do not perform more than one read in a row - Fish defines a vendor completions directory for completions that are not - installed as part of the fish project itself, and the vendor completions - are preferred if they exist. This prevents trying to overwrite the - builtin curl.fish completion (or creating file conflicts in distro - packaging). + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Prefer the pkg-config defined location exported by fish, if it can be - found, and fall back to the correct directory defined by most systems. + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 664 checks mimepost using single-byte read callback with encoder. + New test 665 checks the end of part data early detection. - Closes #3723 - Reviewed-by: Daniel Gustafsson + Fixes #4826 + Reported-by: MrdUkk on github + +- [Patrick Monnerat brought this change] -Marcel Raad (5 Apr 2019) -- ftplistparser: fix LGTM alert "Empty block without comment" + mime: latch last read callback status. - Removing the block is consistent with line 954/957. + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Closes https://github.com/curl/curl/pull/3732 + Fixes #4813 + Reported-by: MrdUkk on github + Closes #4833 -- transfer: fix LGTM alert "Comparison is always true" +Steve Holme (1 Mar 2020) +- unit1651: Fixed conversion compilation warning - Just remove the redundant condition, which also makes it clear that - k->buf is always 0-terminated if this break is not hit. + 371:17: warning: conversion to 'unsigned char' from 'int' may alter its + value [-Wconversion] - Closes https://github.com/curl/curl/pull/3732 - -Jay Satiro (4 Apr 2019) -- [Rikard Falkeborn brought this change] + Closes #5008 - smtp: fix compiler warning +- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - - Fix clang string-plus-int warning. + tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, + The Win32 Crypto library and Apple's Common Crypto library. - Clang 8 warns about adding a string to an int does not append to the - string. Indeed it doesn't, but that was not the intention either. Use - array indexing as suggested to silence the warning. There should be no - functional changes. + If an TLS backend such as mbedTLS or WolfSSL is specified then the + following error is given during compilation along, with a load of + unresolved extern errors: - (In other words clang warns about "foo"+2 but not &"foo"[2] so use the - latter.) + Can't compile METALINK support without a crypto library. - smtp.c:1221:29: warning: adding 'int' to a string does not append to the - string [-Wstring-plus-int] - eob = strdup(SMTP_EOB + 2); - ~~~~~~~~~~~~~~~~^~~~ + Reviewed-by: Daniel Stenberg + Closes #5006 + +Marc Hoersken (1 Mar 2020) +- ci/tests: Update Azure DevOps pipeline job display names + + Make the configure step more descriptive and align others. + +- ci/tests: Fix typo in previous commit 597cf2 + +- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available - Closes https://github.com/curl/curl/pull/3729 + For security reasons the access token is not available to PR builds. + Therefore we should not try to use the DevOps API with an empty token. |