The sad state of foldcase and string comparisons

You probably heard about case-folding or foldcase before. Unicode defines CaseFolding mappings for some upper-case characters, which in full casefolding mode will expand some exotic characters to larger sequences. In simple mode it will do 1:1 tolower() mappings. The perl documentation has this to say: fc Returns the casefolded version of EXPR. This is the internal function implementing the “\F” escape in double-quoted strings. Casefolding is the process of mapping strings to a form where case differences are erased; comparing two strings in their casefolded form is effectively a way of asking if two strings are equal, regardless of case.

Read More

Attribute arguments

perl5 had broken attribute handling forever perl5 attributes were invented to provide extendable hooks to attach data or run code at any data, and made for nice syntax, almost resembling other languages. E.g. my $i :Int = 1; sub calc :prototype($$) { shift + shift } There were a few number of builtin attributes, like :lvalue, :shared, :const, adding a flag to a function or data, and you could add package-specific for compile-time or run-time hooks to process arbitrary custom attributes.

Read More

strict names

Consistent identifier parsing rules perl5 and cperl older than 5.27.0 accepts any string as valid identifier name when being created under no strict 'refs' at run-time, even when most such names are illegal, and cannot be handled by most external modules. Even invalid unicode is allowed. cperl 5.26 fixed embedded NUL’s and invalid unicode identifiers illegal, and normalizes unicode identifiers in the parser. Since cperl 5.27.1 dynamically created names are treated the same way as when they are parsed.

Read More

strict hashpairs

perl5 optionally warns on odd hash elements my %h = (0,1,2); is legal code in perl5. The second pair is constructed with the undef value. With use warnings 'misc' it will warn at least. use warnings; my %h = (0,1,2); => Odd number of elements in hash assignment (WARNING only) perl6 throws on odd hash elements perl6 is sane and strict by default. my %h = (0..2); => Odd number of elements found where hash initializer expected: Found 3 (implicit) elements: Last element seen: 2 in block <unit> at <unknown file> line 1 cperl 5.

Read More

Automatic cperl deployments

Binary packages perl5 relies on external packagers to update and maintain packages for various distributions. It only provides source packages as tarballs. cperl does a bit better by also providing binary packages for all major platforms. See also Installation at the STATUS page. win32, win64, debian 7 i686, debian 8 amd64, centos 7 x86_64, centos 6 i686+x86_64 and darwin amd64. Packaging was done with this do-make-cperl-release script, leading to

Read More

Unicode Identifiers

Binary names with 5.16 With perl 5.16 added support for binary names, announcing it as support for unicode names. Unicode names were already supported since 5.8.4 with a negative length stored in the hash key of the symbol. Supporting binary names without any supporting measures opened huge security holes, as names are mapped 1:1 to filenames when searching for a package, and as we know the C API for files or names just ignores a \0, leading to inconsistencies.

Read More

cperl hash tables

The old perl5 hash table uses linked lists for its collisions, with slow out-of-cache pointer chasing and data overhead. unsorted flags at the end, while some flags are needed for compare. has questionable security measures to slow down all cases. seed ok, randomize iter maybe, but randomize the collisions and slow hash funcs is stupid. The security should be fixed with proper collision iteration, not by pseudo-security theatre upfront.

Read More

Worst perl5 bugs

A small list of the worst perl5 bugs, all fixed in cperl DoS It’s trivial to DoS a perl5 system. $a[9223372036854]=0; %a=(0..4294967296); Examples for a 64bit system, but also trivial on 32bit. It creates a huge array or hash, which runs out of memory in the VMM subsystem which eventually kills the process. cperl dies with “Too many elements”, here even at compile-time. No Hash Security Nothing is done against the root-cause of a hash flood denial of service attack with colliding keys, only some security theatre by using slower hash functions and slower collision resolution KEY_PERTURB_RANDOM.

Read More

p5p incompetence

So I was continously asked to explain why I call p5p too incompetent to design anything for the perl5 language and any internal VM features. So far I was reluctant to do so, because users will be as emberrassed as I am, and need a way to got forward. Now that I can offer a way forward it might be easier to publish the detailed criticsm on p5p’s incompetence.

Read More

The dangerous SipHash myth

SipHash claims that its “cryptographically strong pseudo random function” properties protects against hash table DoS flood attacks. This is wrong, because for a successful attack against a SipHash hash table with chained linked lists or linear probing it is enough to get the secret random seed, and then brute force create collisions, which is doable in <1s for 16k keys, 2m for 16k keys, and from 32k to 268M keys in 4 minutes.

Read More