Unit Testing Lots of Perl

Getting There from Here:
Unit testing a few generations of code.
Steven Lembark
Workhorse Computing
lembark@wrkhors.com

perl 5.8
Fun isn’t it? A generation of nothing.
No improvements in Perl.
No advances in CPAN.
No reason to upgrade.
20 years of nada...
Ask all the programmers stuck with 5.8.

perl 5.8
Q: Why are we stuck with 5.8?
A: /usr/bin/perl is 5.8
Q: Why are we stuck with /usr/bin/perl?
A: ???

perl 5.8
A: Because “it worked”.

perl 5.8
A: Because something else “might not” work.

perl 5.8
A: Because we can’t show that 5.X works.
(for some definition of Christmas > 5.8)

perl 5.X
Q: Why can’t we show that 5.X works?
A: Because Christmas came after 2002.

perl 5.X
A: Because we stopped testing our code.

perl 5.X
A: Because they stopped testing our code.

perl 5.X
A: Because we have worked around it for way too long.

perl 5.X
A: Because we have worked around it for way too long.
perlbrew, all of the other approaches to dodging
/usr/bin/perl.

OK how do we fix it?
Prove that “it works”.
With testing.
In one client’s case: Lots and lots of testing.

With testing.
75_000 modules worth of testing.

With testing.
75_000 modules worth of testing.
Q: But how did you write 75_000 unit tests?

With testing.
75_000 modules worth of automated testing.
A: Lazyness.

Syntax checks are the first pass.
Unit tests get a bad rap.
“They don’t really test anything.”

Syntax checks are the first pass.
Unit tests get a bad rap.
“They don’t really test anything.”
Except whether 75_000 modules compile.
Code that was written on 5.8.
That now to run on 5.20+.

How do we test that much code?
Metadata driven testing:
Encode data into a test.
The standard test is data-driven.
Tests each validate one small part of a module.

Q: How do you write that many tests?
A: Don’t.

Q: How do you write that many tests?
A: Don’t.
Symlinks are your friends.

Simple basic test: require_ok
Run “require_ok $module” for every *.pm.
Simple, basic, obvious test for syntax errors.
… missing modules.
… botched system paths.
… all sorts of things.

Simple basic test: require_ok
Run “require_ok $module” for every *.pm.
Simple, basic, obvious test for syntax errors.
Nice thing: $module can be a path.
Even an absolute path.

Passing a path
“require_ok $path”
One argument.
Q: How do we pass it?
A: In the basename of a test.

Basic unit test
use v5.30;
use Test::More;
use File::Basename;
my $base0 = basename $0, '.t';
my $s = substr $base0, 0, 1;
my $path = join ‘/’, split m{[$s]}, $base0;
require_ok $path;
done_testing
__END__

Basic unit test
Install
them
using
shell.
#!/bin/bash
dir=$(cd $(dirname $0)/../..; pwd -L);
cd “$dir/t/01-units”;
find $dir/lib -name ‘*pm’ |
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
ln -fs ../bin/01-unit_t ./$base;
done

Basic unit test
Install
them
using
shell.
From
./t/bin
#!/bin/bash
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
done

Basic unit test
Install
them
using
shell.
Path to
basename.
#!/bin/bash
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
done

Basic unit test
Now you
know if it
compiles. prove t/01-units;

Basic unit test
Now you
know if it
compiles.
A bit
faster.
prove --jobs=8 t/01-units;

Basic unit test
Now you
know if it
compiles.
At 3am
without
watching.
args=(--jobs=8 --state=save );
prove ${args[*]} t/01-units;

Basic unit test
Now you
know if it
compiles.
At 8am
knowing
what
failed.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units;

Basic unit test
Now you
know if it
compiles.
Missing
module
anyone?
prove ${args[*]} t/01-units
Can't locate Foo/Bar.pm in @INC (you
may need to install the Foo::Bar
module) (@INC contains:
/opt/perl/5.30/lib/site_perl/5.30.1/x8
6_64-linux

Basic unit test
Now you
know if it
compiles.
Fine:
Install
them.
prove ${args[*]} t/01-units |
grep ‘you may need to install the’ |
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm ;

Basic unit test
Now you
know if it
compiles.
Local
mirror
provides
pre-
verified
modules.
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm -M$local ;

Basic unit test
Now you
know if it
compiles.
Local
library
simplifies
auto-
install.
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm
-M$local -l$repo --self-contained ;

Basic unit test
Start with
a virgin
/opt/perl.
Keep
testing
until it’s
all use-
able.
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm
-M$local -l$repo --self-contained ;

Version control what you install
git submodule add blah://.../site_perl;
perl Makefile.PL INSTALL_BASE=$PWD/site_perl;
cpanm --local-library=$PWD/site_perl;
If anyone asks, you know what non-core modules have
been installed.
If anything breaks, check out the last commit.

What else can we do with a path?
Ever fat-finger
a package?
package AcmeWigdit::Config;
use v5.30;

Ever fat-finger
a package?
Paths define
packages.
require_ok $path or skip ... ;
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
isa_ok $pkg, ‘UNIVERSAL’
or skip “$pkg not ‘UNIVERSAL’”;
$pkg->can( ‘VERSION’ )
or skip “$path missing $pkg”, 1;

Ever fat-finger
a package?
Paths define
packages.
All defined
packages are
UNIVERSAL.
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
isa_ok $pkg, ‘UNIVERSAL’

Ever fat-finger
a package?
Paths define
packages.
VERSION is
UNIVERSAL.
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
is_ok $pkg, ‘UNIVERSAL’

Minor issue with paths
Packages are related to paths.
We know that now.
In 2000 not everyone got that.

What about:
package <basename>;
use lib <every directory everywhere>;
It works, but there are better ways...

One pattern: Overlapping product of evolution.
./lib/AcmeWig/
./lib/AcmeWig/Config.pm AcmeWig::Config

./lib/AcmeWig/
./lib/AcmeWig/Acme/Config.pm Acme::Config

./lib/AcmeWig/
./lib/AcmeWig/Acme/Config.pm Acme::Config
What’s the expected package?

Solving nested paths
Say we figure out the paths:
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/AcmeWig/External/Plastic
)

We can obviously iterate them...
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
)

Q: But how do we avoid duplicate tests?
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
)

A: Sort them by length.
qw
(
./lib/AcmeWig/Acme
./lib/AcmeWig
)

A: Sort them by length & exclude known paths.
qw
(
./lib/AcmeWig/Acme
./lib/AcmeWig
)

my %prune = ();
my $wanted = sub
{
-d && exists $prune{ $File::Find::dir }
? $File::Find::prune = 1
: $File::Find::Path->$install_test_symlink
};
for my $dir ( @dirs_sorted_by_length )
{
find $wanted, $dir;
$prune{ $dir } = ();
}

my %prune = ();
my $wanted = sub
{
-d && exists $prune{ $File::Find::dir }
? $File::Find::prune = 1
: $File::Find::Path->$install_test_symlink
};
for my $dir ( @dirs_sorted_by_length )
{
find $wanted, $dir;
$prune{ $dir } = (); # don’t revisit
}

Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.

Fix: Double the separator.
~path~to~repo~lib~AcmeWig~Acme~Config.pm.t

Fix: Double the separator.
~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t

Fix: ‘//’ works fine with require_ok.
/path/to/repo/lib/AcmeWig//Acme/Config.pm.t

Fix: Gives sub-path for module.
Acme~Config.pm.t

Fix: basename with s{~}{::}.
Acme::Config

Save the search dir’s length and substr in a ‘~’:
my $l = length $dir;
...
substr $symlink, $l, 0, ‘~’;

Skipping test groups
We started with 75_000 files.
Found some dirs we didn’t want to test.
Fix:
my @no_test = qw( … );
my %prune = ();
@prune{ @no_test } = ();

Net result
We were able to test 45_000+ files each night.
Found missing modules.
Found outdated syntax.
Managed to get it all working.
Largey with “require_ok”.

Knowledge is power
Unit tests are useful.
Provide automated, reproducable results.
No excuse for “It may not work in v5.30.”
We can know.
And fix whatever “it” is.

For some definition of Perl5
Perl7 is coming.
Adopting it requires showing that “it works.”
Which means finding where it doesn’t.
Unit testing all of CPAN.
Unit testing all of everything.

For some definition of Perl5
Perl7 is coming.
Adopting it requires showing that “it works.”
Which means finding where it doesn’t.
Unit testing all of CPAN.
Unit testing all of everything.
We just have to be lazy about it.

Units of your very own!
In case you don’t like pasting from PDF:
https://gitlab.com/lembark/perl5-unit-tests
If anyone wants to work on this we can release it as
App::Testify or Test2::Unitz …
Be nice to have a migration suite for Perl7.

Units of your very own!
A wider range of unit tests is shown at:
https://www.slideshare.net/lembark/path-toknowlege
With a little bit of work we could get them all on CPAN.

Unit Testing Lots of Perl

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Unit Testing Lots of Perl

Similar to Unit Testing Lots of Perl (20)

More from Workhorse Computing

More from Workhorse Computing (18)

Recently uploaded

Recently uploaded (20)

Unit Testing Lots of Perl