NAME

perlcdelta - what is new for cperl v5.27.2

DESCRIPTION

This document describes the differences between the cperl 5.27.1 and the cperl 5.27.2 releases.

If you are upgrading from an earlier release such as v5.24.2c, first read the perl526*cdelta documentation, which describes differences between v5.26.0c and v5.26.2c

Core Enhancements

Optimize static methods to subs

When a method call can be proven at compile-time to use a specific subroutine, optimize away the run-time method dispatch on a non-threaded perl. When it is for a XS subroutine, such as with e.g. use strict; which does BEGIN {require strict;} strict->import; the strict->import part is optimized from the slow entersub op to the enterxssub op, as if via strict::import("strict",...). The slow GV lookup in entersub cannot be converted to a faster CVref lookup, as this would break dynamic method redefinitions. See http://blogs.perl.org/users/rurban/2011/06/how-perl-calls-subs-and-methods.html. This is ~4% faster for each such method call. cperl-only.

Dump more COP fields

Dump the values of a COP $^H, %^H and the lexical warnings via op_dump() and it's XS module Devel::Peek. Improve the display of cop SEQ values and ranges, and padnames. Added a DEBUGGING cop_dump() helper, esp. useful for deleted PL_curcop's. cperl-only.

More unicode ops

Added the strong unicode fraction op / U+2044, which binds stronger than all other arithmetic ops, and the two unicode multiplication ops U+022C5 (dot operator) and U+2219 (multiplication dot), which binds the same as *. cperl-only.

delete on key/value slices

delete can now be used on key/value slices, returning the keys along with the deleted values. [perl #131328]

Unicode 10.0 is supported

A list of changes is at http://www.unicode.org/versions/Unicode10.0.0.

Re-enable warnings illegalproto

Parse prototypes for errors again. Either with :prototype() or as signature. Illegal prototypes are not stored. Most illegal protos are subsequently parsed as signature, but some are just wrong protos, such as '[$' missing the ']', or :prototype($_$).

In-place editing is now safer

Previously in-place editing would delete or rename the input file as soon as you started working on a new file.

Without backups this would result in loss of data if there was an error, such as a full disk, when writing to the output file.

This has changed so that the input file isn't replaced until the output file has been completely written and successfully closed.

This works by creating a work file in the same directory, which is renamed over the input file once the output file is complete.

Incompatibilities:

[perl #127663]

Initialisation of aggregate state variables

A persistent lexical array or hash variable can now be initialized, by an expression such as state @a = qw(x y z). Initialization of a list of persistent lexical variables is still not possible.

Full-size inode numbers

On platforms where inode numbers are of a type larger than perl's native integer numerical types, stat will preserve the full content of large inode numbers by returning them in the form of strings of decimal digits. Exact comparison of inode numbers can thus be achieved by comparing with eq rather than ==. Comparison with ==, and other numerical operations (which are usually meaningless on inode numbers), work as well as they did before, which is to say they fall back to floating point, and ultimately operate on a fairly useless rounded inode number if the real inode number is too big for the floating point format.

Changed warnings::register handling of all

If a scope used "use warnings 'all'" to turn on all warnings and then turned off some specific warnings, then that scope wouldn't get warnings for subsequently-registered warning categories, because its bitset doesn't extend to the bit controlling the new category.

Unlike all other warning categories, the bits for "all" used to be set only when there were no warning categories disabled; disabling any would also clear the "all" bits. That was supporting the dynamic nature of warnings::register extending the all-warnings bitset for new categories. This exception is now removed, so the meaning of the "all" bits is now the more obvious meaning, of indicating the default treatment that the scope wants for warnings not falling into any category known to the bitset. In warnings::warnif() et al, if the caller's bitset is found to be too short to have a bit for the relevant category, then the setting for the "all" category is used instead.

Because the length of a bitset is an integral number of bytes, but only two bits are used per category, the length of a bitset doesn't precisely indicate which categories had been registered by the time it was constructed. So the standard bitsets for the "all" category are now always filled to their byte length, with bits set preemptively for categories not yet registered that fall within the current bitset length.

When a warnings pragma operates on a bitset, it first expands it to the preferred length, by duplicating the "all" bits for the categories covered by the new length. It is careful to maintain the length when combining the bitset with the standard bitsets for categories. When a bitset is read from ${^WARNING_BITS} or from caller(), the standard pWARN_ALL setting is no longer expanded by the core to $warnings::Bits{all}, because the core's short WARN_ALLstring will now be expanded correctly just like any other bitset.

Fixes [perl #108778]

Security

[CVE-2017-12837] Heap buffer overflow in regular expression compiler

Compiling certain regular expression patterns with the case-insensitive modifier could cause a heap buffer overflow and crash perl. This has now been fixed. [perl #131582]

[CVE-2017-12883] Buffer over-read in regular expression parser

For certain types of syntax error in a regular expression pattern, the error message could either contain the contents of a random, possibly large, chunk of memory, or could crash perl. This has now been fixed. [perl #131598]

[CVE-2017-12814] $ENV{$key} stack buffer overflow on Windows

A possible stack buffer overflow in the %ENV code on Windows has been fixed by removing the buffer completely since it was superfluous anyway. [perl #131665]

Unsigned long stack access of printf format arguments

perl 5.27.5 started allowing unsigned long/4 stack indices for its sv arguments, and width and precision with the API even allowing unsigned long. Previously those numbers were safely typed to int. Revert that security problem in cperl. The maximal width and precision is also back to INT_MAX. See [cperl #344]

Incompatible Changes

Comma-less variable lists in formats are no longer allowed

Omitting the commas between variables passed to formats is no longer allowed. This has been deprecated since perl 5.000.

The :locked and :unique attributes have been removed

These have been no-ops and deprecated since perl 5.12 and 5.10, respectively.

\N{} with nothing between the braces is now illegal.

This has been deprecated since Perl 5.24.

Opening the same symbol as both a file and directory handle is no longer allowed

Using open() and opendir() to associate both a filehandle and a dirhandle to the same symbol (glob or scalar) has been deprecated since Perl 5.10.

Use of bare << to mean <<"" is no longer allowed

Use of a bare terminator has been deprecated since Perl 5.000.

Setting $/ to a reference to a non-positive integer no longer allowed

This used to work like setting it to undef, but has been deprecated since Perl 5.20.

Unicode code points with values exceeding IV_MAX are now fatal.

This was deprecated since Perl 5.24.

B::OP::terse no longer exists.

Use B::Concise::b_terse instead.

Use of inherited AUTOLOAD for non-methods is no longer allowed.

This was deprecated in Perl 5.004.

Use of strings with code points over 0xFF is not allowed for bitwise string operators

Code points over 0xFF do not make sense for bitwise operators.

Backslash no longer escapes colon in PATH for the -S switch

Previously the -S switch incorrectly treated backslash ("\") as an escape for colon when traversing the PATH environment variable. [perl #129183]

Subroutines no longer need typeglobs

Perl 5.22.0 introduced an optimization allowing subroutines to be stored in packages as simple sub refs, not requiring a full typeglob (thus potentially saving large amounts of memeory). However, the optimization was flawed: it only applied to the main package.

This optimization has now been extended to all packages. This may break compatibility with introspection code that looks inside stashes and expects everything in them to be a typeglob.

When this optimization happens, the typeglob still notionally exists, so accessing it will cause the stash entry to be upgraded to a typeglob. The optimization does not apply to XSUBs or exported subroutines, and calling a method will undo it, since method calls cache things in typeglobs.

... is a term in cperl, not a statement

perl5.27.6 restricted ..., the yada-yada operator, to be a statement only. cperl reserved ... for the ellipsis, i.e. unassigned varargs in signatures to be passed through. Which is not a statement, but a term. In cperl ... is a TERM. Before ... was parsed as an operator, also used for ranges.

This affects illegal code like:

    ... . "foo";
    ... if $a < $b;
    do{...}

Deprecations

Use of $1|perlfunc/vec EXPR,OFFSET,BITS> on strings with code points above 0xFF is deprecated.

Use of these is nonsensical, as vec is a bit-oriented operation, which operates on the underlying UTF-8 representation these strings must be in, and will likely give unexpected results.

Some uses of unescaped "{" are no longer fatal

Perl 5.26.0 fatalized some uses of an unescaped left brace, but an exception was made at the last minute, specifically crafted to be a minimal change to allow GNU Autoconf to work. This code is heavily depended upon, and continues to use the deprecated usage. Its use of an unescaped left brace is one where we have no intention of repurposing "{" to be something other than itself.

That exception is now generalized to include various other such cases where the "{" will not be repurposed. This is to get real experience with this more complicated change now, in case we need to issue a dot release if we find other things like Autoconf that are important to work around.

Note that these uses continue to raise a deprecation message.

Module removals

The following modules will be removed from the core distribution in a future release, and will at that time need to be installed from CPAN. Distributions on CPAN which require these modules will need to list them as prerequisites.

The core versions of these modules will now issue "deprecated"-category warnings to alert you to this fact. To silence these deprecation warnings, install the modules in question from CPAN.

Note that these are (with rare exceptions) fine modules that you are encouraged to continue to use. Their disinclusion from core primarily hinges on their necessity to bootstrapping a fully functional, CPAN-capable Perl installation, not usually on concerns over their design.

Locale::Codes

Performance Enhancements

Modules and Pragmata

Removal of use vars

The usage of use vars has been discouraged since the introduction of our in Perl 5.6.0. Where possible the usage of this pragma has now been removed from the Perl source code.

This had a slight effect (for the better) on the output of WARNING_BITS in B::Deparse.

Several core modules released on CPAN will not work anymore with perl versions prior to 5.6.

Updated Modules and Pragmata

arybase 0.14

Remove #ifdef USE_ITHREADS around MUTEX_* calls

attributes 1.14c

Remove deprecated no-op attributes :unique, :locked. Added another param to validate_proto.

Attribute::Handlers 1.00

Remove deprecated no-op attributes :unique, :locked

base 2.27c

Limit dotless-INC effect on base.pm with guard

B 1.71_08

Support PL_sv_zero

B::C 1.55_06

Support POK/PVLV REGEXP for v5.27.3/v5.27.2c VOL => volatile for v5.27.6/v5.27.2c

B::Concise 1.003c

Support PL_sv_zero

B::Deparse 1.43_04c

Support boolean optims. Better constant-dumping heuristics.

B::Terse 1.09_01

Remove internal links to B::Debug

B::Xref 1.07_01

Support sub refs

Carp 1.44c

Fix problems introduced by the partial EBCDIC support from version 1.35, and ISA constants. Not all stash elements are globs.

charnames 1.45

Remove obsolete pod about NBSP

Compress::Raw::Zlib 2.076

Updated internal zlib from 1.2.8 to 1.2.11

Silence gcc compiler warnings when -Wsign-compare is enabled #123688: Compilation warnings with clang

Silence gcc compiler warnings when -Wimplicit-fallthrough is enabled #123358: Compilation warnings with gcc-7.*

Windows uses -lzlib. Linux uses -lz #123699: wrong external zlib name used on Windows

Update zlib-src directory to use zlib 1.2.11 #123245: perl 5.26.1 is vulnerable to CVE-2016-9843, CVE-2016-9841, CVE-2016-9840, CVE-2016-9842

Don't allow offset to be greater than length of buffer in crc32.

Change my_zcalloc to use safecalloc. The link, https://github.com/madler/zlib/issues/253, is the upstream report for the remaining valgrind errors not already dealt with by 1.2.11. Using calloc in Zlib.xs for now as a workaround. #121074: valgrind errors in the test suite

Config::Perl::V 0.29_01
    * Add test for 5.26.1-RC1, and some cperl -V's
    * Fix signature with argument
    * Ignore ccache and compiler path in signature

    * Add default_inc_excludes_dot
constant 1.34
coretypes 0.03c

Each coretype is now a class.

Cpanel::JSON::XS 3.0239

Fixed locale-insensitive numeric conversion on older perls. Fixed compilation under windows.

Data::Dumper 2.167_02

Attempt to work around XS deparse issues on old perls.

Devel::Peek 1.27_03

Dump the values of a COP $^H, %^H and lexical warnings. Improved the test a bit.

use cv_set_call_checker_flags

Devel::PPPort 3.36_03

Silence PERL_CORE make

Digest::SHA 5.98

Remove deprecated -p option

DirHandle 1.05

Document as obsolete

Encode 2.93

lib/Encode/MIME/Name.pm t/mime-name.t Pulled: Add "euc-cn" => "EUC-CN" alias to Encode::MIME::Name https://github.com/dankogai/p5-encode/pull/124

lib/Encode/CN/HZ.pm lib/Encode/JP/JIS7.pm lib/Encode/MIME/Header.pm t/decode.t Pulled: Uninitialized value fixes #122 https://github.com/dankogai/p5-encode/pull/122

Makefile.PL Pulled: Fix -Werror=declaration-after-statement for gcc 4.1.2 https://github.com/dankogai/p5-encode/pull/121

experimental 0.019_01c

eval use feature, minor test fix.

ExtUtils::Constant 0.24_01

Minor updates from p5p

ExtUtils::CBuilder 0.280230

use vars => our

ExtUtils::Install 2.14

Minor updates, some taken from cperl.

ExtUtils::ParseXS 3.26_02
feature 1.49_01
fields 2.26c

Implemented "fields" in Mu returning fields objects, as in perl6 (there called Attributes).

Revert the conversion of a fields package to class. Only close the ISA as with base.

cperl-only.

File::Fetch 0.56

Switch test URL from www.cpan.org to httpbin.org

File::Glob 1.30

Skip a test when File::Glob not used. Avoid undefined behaviour in Copy() [perl #131746]

Filter::Util::Call 1.58

XSLoader, our. defer Carp. add filter_read_exact tests.

GDBM_File 1.16

Add a pod link

Hash::Util::FieldHash 1.20
I18N::Langinfo 0.14

Use new fcn Perl_langinfo()

if 0.0607

Doc how to import when you want to 'use MODULE ()'

IO::Handle 1.37
JSON::PP 2.97000_04

Minor updates, mostly docs. Upstream Boolean still did not work with Cpanel::JSON::XS.

libnet 3.11_01
Locale::Codes 3.55

Deprecate from core with v5.27.7, move to cpan.

Module::CoreList 20170815c
Module::Load::Conditional 0.68_01

Improve CPANPLUS: Fix [cperl #331], checking for cperl builtins. With a slightly adjusted patch from @voegelas.

mro 1.21

prevent mro.pm from loading DynaLoader

Mu 0.02c

Implemented "fields" in Mu returning fields objects. cperl-only.

Net::Ping 2.63

Keep v5.002 - v5.6 support

Removed outdated demo/fping from the documentation (RT #123750 by Steve Morris)

Added t/420_ping_syn_port.t (#4 by Julio Fraire) with fixes.

Added indices and crosslinks to the documentation

NEXT 0.67_01

Fix for GLOB stubs [cpan #123002].

O 1.03

Improve error message for bogus -MO=… arguments

ODBM_File 1.15
Opcode 1.40_02c
perl5db 1.52_01c

Support debugger cmds not requiring spaces

PerlIO::scalar 0.29

check invariant at compile time.

return EACCES on writing to a read-only scalar

Pod::HTML 2.2303c
Pod::Perldoc 3.2801

customize to fix formatters and pager misbehaviour

POSIX 1.78_03

internal cleanup of setlocale (unusable for cpan). get/setpriority constants

re 0.35_01

Faster. add DEBUGGING_RE_ONLY define

SelfLoader 1.24
Scalar::Util 2.49_09

fix find_rundefsvoffset logic Fix t/subname.t for unstrict names perl5.26 started defining $Config{usecperl} as 'UNKNOWN', hence this check is now unreliable.

Merge with 1.49 upstream: Ensure pairmap extends its stack correctly (davem) Fix name of List::Util::unpairs in its error messages

sort 2.03

no sort stable; SORTf_UNSTABLE flag

Storable 3.05_16

Try to fix readonly stacksize.h with perlbrew. [cperl #335]

Fix t/blessed.t for 5.10/5.12 threaded.

Restore max stacksizes for non-windows systems with proper system(SHELLSTRING) passing signals. Unify stack reserve to 32 across all platforms.

Symbol 1.08_01

Improved Symbol::delete_package for readonly and protected symbols and classes.

Term::ReadLine 1.17

fix for empty &STDERR files RT #132008

threads 2.18_01

Support PL_sv_zero. Don't Copy() null pointer

Time::Piece 1.3202
    - Fix AIX compile
    - Fix compile errors on MS
    - Use macro for buf sizes
    - Remove some compile warnings
    - SP off by 1 in _crt_localtime
Time::HiRes 1.9747_01

Drop 5.005 support. Replace DynaLoader with XSLoader.

Unicode::Collate 1.25

Drop 5.005 support. Replace DynaLoader with XSLoader.

Unicode::Normalize 1.26

Switch to XSLoader from Dynaloader

utf8 2.02c

Improved function documentation

version 0.9918_02c

Add updates from 0.9918: Add LAX_DECIMAL_VERSION, LAX_DOTTED_DECIMAL_VERSION, STRICT_DECIMAL_VERSION, STRICT_DOTTED_DECIMAL_VERSION regexes.

Add t/11_taint.t

VMS::Stdio 2.44_01

Drop VMS::stdio compatibility shim

XS::APITest 0.92_02
XS::Typemap 0.16

Removed Modules and Pragmata

B::Debug 1.25

Moved to CPAN. cperl only. perl5 will remove it with 5.30.

Documentation

New Documentation

PACKAGING

Notes and best practice for packaging perl 5

Changes to Existing Documentation

perlapi

perlop

perldata

perluniprops

perlintern

perlfilter

POSIX

Diagnostics

The following additions or changes have been made to diagnostic output, including warnings and fatal error messages. For the complete list of diagnostic messages, see perldiag.

New Diagnostics

New Errors

New Warnings

Changes to Existing Diagnostics

Configuration and Compilation

C89 requirement

Perl has been documented as requiring a C89 compiler to build since October 1998. A variety of simplifications have now been made to Perl's internals to rely on the features specified by the C89 standard. We believe that this internal change hasn't altered the set of platforms that Perl builds on, but please report a bug if Perl now has new problems building on your platform.

-Werror=pointer-arith

With GCC, -Werror=pointer-arith is now enabled by default, disallowing arithmetic on void and function pointers.

d_builtin_arith_overflow

Fixed d_builtin_arith_overflow detection and usage for 32bit systems with use64bitint. They do use the ll suffix, same as with WIN64. Now we can use this optimization. cperl-only.

New probes

Note that cperl ignores the new HAS_BUILTIN_{ADD,MUL,SUB}_OVERFLOW config keys as we use the better HAS_BUILTIN_ARITH_OVERFLOW for years already.

HAS_THREAD_SAFE_NL_LANGINFO_L

d_thread_safe_nl_langinfo_l

HAS_LOCALECONV_L

d_localeconv_l

HAS_MBRLEN

d_mbrlen

HAS_MBRTOWC

d_mbrtowc

HAS_MEMRCHR

d_memrchr

HAS_NANOSLEEP

d_nanosleep

HAS_STRNLEN

d_strnlen

HAS_STRTOLD_L

d_strtold_l

I_WCHAR

i_wchar

Removed probes and keys
ansi2knr
d_bcmp
d_bcopy
d_bzero
d_index
d_charvspr
d_dbl_dig
d_safebcpy
d_safemcpy
d_sanemcmp
d_sprintf_returns_strlen
d_strctcpy
d_strerrm
i_memory
i_values
i_varargs
i_varhdr
strings
d_memset

Beware: HAS_MEMSET is undefined since v5.28, but always considered to be defined. Because of C89. cperl kept the HAS_MEMSET preprocessor definition for backcompat with extensions which do use it. It is used in all ppport.h files to use memzero().

d_memchr

Ditto: Keep HAS_MEMCHR

d_memcmp

Ditto: Keep HAS_MEMCMP

d_memcpy

Ditto: Keep HAS_MEMCPY

d_memmove

Ditto: Keep HAS_MEMMOVE

d_strchr

Ditto: Keep HAS_STRCHR

d_strerror

Ditto: Keep HAS_STRERROR

d_vprintf

Ditto: Keep HAS_VPRINTF

d_volatile

Ditto: Keep HASVOLATILE

i_assert

Ditto: Keep I_ASSERT in perl.h

i_float

Ditto: Keep I_FLOAT in perl.h

i_limits

Ditto: Keep I_LIMITS in perl.h

i_math

Ditto: Keep I_MATH in perl.h

i_stdarg

Ditto: Keep I_STDARG in perl.h

i_stdlib

Ditto: Keep I_STDLIB in perl.h

i_string

Ditto: Keep I_STRING in perl.h

prototype

Ditto: Keep CAN_PROTOTYPE in perl.h

vaproto

Ditto: Keep CAN_VAPROTO, _V

git_version.sh

New generated shell config with the same content as git_version.h, for myconfig.

myconfig

Added git_branch and git_describe.

make_patchnum: touch the targets

avoiding needless miniperl recompilation on Config_git.pl, git_version.{sh,h}.

Testing

Platform Support

Discontinued Platforms

PowerUX / Power MAX OS

Compiler hints and other support for these apparently long-defunct platforms has been removed.

Platform-Specific Notes

Darwin

Perl now correctly uses reentrant functions, like asctime_r, on versions of Darwin that have support for them.

locale specific error messages are now thread-safe. cperl-only, [cperl #341]

CentOS

Compilation on CentOS 5 is now fixed.

UMIPS

CAN_PROTOTYPE is now always defined, which might cause problems with the system cc compiler. ccom: Error: ./mg.h, line 12: redeclaration of formal parameter, sv

ULTRIX 4

CAN_PROTOTYPE is now always defined, which might cause problems with the gcc-4.2 compiler.

Windows

Visual C++ compiler version detection has been improved to work on non-English language systems.

Internal Changes

Selected Bug Fixes

Known Problems

Acknowledgements

cperl 5.27.2 represents approximately 4 months of development since cperl 5.27.1c and contains approximately 250,000 lines of changes across 1,800 files from 50 authors.

Excluding auto-generated files, documentation and release tools, there were approximately 82,000 lines of changes to 630 .pm, .t, .c and .h files.

The following people are known to have contributed the improvements that became cperl 5.27.2:

Karl Williamson, David Mitchell, Reini Urban, Aaron Crane, Tony Cook, Father Chrysostomos, Zefram, Lukas Mai, Dagfinn Ilmari Mannsåker, Steve Hay, Nicolas Rochelemagne, Yves Orton, James E Keenan, Craig A. Berry, H.Merijn Brand, Chris 'BinGOs' Williams, Abigail, Todd Rinaldo, Dominic Hargreaves, Sawyer X, Matthew Horsfall, Aristotle Pagaltzis, Steffen Müller, Jarkko Hietaniemi, Daniel Dragan, Eric Herman, Smylers, Vitali Peil, David H. Gutteridge, Neil Bowers, John P. Linderman, Thomas Sibley, Graham Knop, E. Choroba, Andrew Fresh, Dmitry Ulanov, John Lightsey, Ricardo Signes, Tom Hukins, Ken Cotterill, Andy Lester, Jacques Germishuys, Eugen Konkov, Andy Dougherty, John Peacock, Nicholas Clark, Slaven Rezic, Harald Jörg, Dan Collins, John SJ Anderson.

The list above is almost certainly incomplete as it is automatically generated from version control history including the perl and cperl repos. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker and the cperl github issues.

Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.

For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.

Generated with:

    cperl Porting/acknowledgements.pl cperl-5.27.1..HEAD -c

Reporting Bugs

If you find what you think is a bug, you might check the articles recently posted to the comp.lang.perl.misc newsgroup and the perl bug database at https://rt.perl.org/ . There may also be information at http://www.perl.org/ , the Perl Home Page.

If you believe you have an unreported bug, please run the cperlbug program included with your release. Be sure to trim your bug down to a tiny but sufficient test case. Your bug report, along with the output of perl -V, will be sent off to perlbug@perl.org to be analysed by the Perl porting team.

If you think it's a cperl specific bug or trust the cperl developers more please file an issue at https://github.com/perl11/cperl/issues.

If the bug you are reporting has security implications which make it inappropriate to send to a publicly archived mailing list, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec For details of how to report the issue.

SEE ALSO

The Changes file for an explanation of how to view exhaustive details on what changed.

The INSTALL file for how to build Perl.

The README file for general stuff.

The Artistic and Copying files for copyright information.