SlideShare a Scribd company logo
1 of 19
Download to read offline
Virtuous Lazyness
  For Your Data

  Doing it once and
knowing you've done it.

  Steve Lembark
Workhorse Computing
 lembark@wrkhors.com
Data wants to be free,
    it also wants to be very expensive.
●   Data you never use is free; accessing gets expensive.
●   Managing data's cost requires controlling its lifecycle.
●   Mis­managing the lifecycle causes problems:
    ●   Caching table lookups in forked httpd crashes your database.
    ●   Pre­processing messages files kills your startup times.
    ●   Reading XML configuration data you never use wastes most of 
        your testing budget.
    ●   Your tests fail because of a connection failure to a database 
        server that you don't use in the test.
●   Fixing these problems requires using lazy data.
False Lazyness
●   We have all see the two most common data 
    management stratagies:
    ●   You load everything at startup to avoid checking it every 
        time it is used.
    ●   You check everything every time before using it to avoid 
        loading it up front.
●   Both approaches ignore important knowledge:
    ●   You know when the data is needed.
    ●   You know that the data was loaded.
One alternative: Scalar Cache
●   $cache ||= read_cache_values;
    ●   Seems nice: You know that $cache is populated.
    ●   This requires dereferencing a hashref throughout the 
        program, which is expensive. 
●   What you'd rather do is just use $cache{ $foobar } 
    without having to check it every time.
●   Even checking if( ! %cache ) is expensive – arrays 
    are much cheaper to check.
True Lazyness:
                Do Something Once
●   Truly Lazy data means loading you data when you 
    need it and knowing that it is loaded.
●   Perl gives us – of course – more than one way:
    ●   Trampoline objects and subroutines.
    ●   “state” variables, introduced in v5.10.
●   Trampolines are flyweight objects – data structures or 
    subroutines that transform themselves when used.
●   State variables are assigned only once, at runtime, the 
    first time they are used.
Follow The Bouncing Object
●   Object::Trampoline – flyweight data you don't know 
    isn't there.
●   These delay calling the “real” constructor until the 
    object is actually used.
    ●   Your constructor gets called before the first method call.
    ●   At that point it can cache, parse, or compute whatever it 
        needs.
●   Spreads the cost of loading data set each over the 
    lifetime of a process.
Example: Delay Expensive XML
●   Use an initializer to read and parse the XML.
●   The Object::Trampoline calls your constructor once 
    to transform the object the first time you call a 
    method.
●   Parsing the XML is pushed off until you actually use 
    the data.
●   Requires using a hashref – and possibly methods – 
    to access the data.
package XML::Message;
use Object::Trampoline;
...

sub new
{
    Object::Trampline->install( 'XML::Message', $path );
}

sub install
{
    my $error = &construct;

    $error->initialize( @_ )
}

sub initialize
{
    my ( $err, $path ) = @_;

    %$err    = %{ XMLin $path => @lots_of_args };

    $err
}

# calling translate bounces the trampoline exactly once

sub translate { … }
Tramploline Subroutine
●   Similar to a trampoline object: A portion of the code 
    runs once and replaces itself.
●   One­shot code initializes the cache, which can now 
    be a simple hash.
●   Symbol::qualify_to_ref make this painless in Perl.
    ●   An anonymous sub manages the cache.
    ●   A named subroutine loads the cache, replaces itself with 
        the manager, and re­dispatches to the manager.
The Simplest Version
my %foo_cache = ();
                                          ●   Minimal code 
my $handler
= sub                                         includes:
{
   my $foo = shift;                           ●   Closed­over cache,
     %foo_cache{ $foo }
     Or die “Bogus foo: '$foo' unknown”
                                              ●   Subref to permenant 
};                                                cache manager,
sub do_something
{
                                              ●   Initial subroutine.
   %cache = initialize_foo_cache;
                                          ●   The initial subroutine 
     my $ref
     = qualify_to_ref 'do_something';         initializes, installs, 
     *$ref   = $handler;
                                              and re­dispatches.
     goto &$handler
}
BEGIN Blocks Are Cleaner
BEGIN
{
   my $name = 'foo_handler';           ●   The block isolates 
   my $ref   = qualify_to_ref $name;
   my %cache = ();                         cache, ref, and 
    my $handler                            sub variables; 
    = sub
    {                                      allows re­cycling 
       %cache{ $_[0] } or die ...
    };                                     the ref.
    *$ref
    = sub
                                       ●   This is also rather 
    {
       %cache   = init_the_cache;
                                           amenable to 
        *$ref   = $handler;
                                           installation by 
                                           module.
        goto &$handler
    }
}
Sub::Trampoline
sub install_trampoline
{
   my ( $name, $init, $mgr ) = @_;
                                       ●   Aside from the actual 
    my $caller = caller;
                                           assignment, init code is 
    my $ref
    = qualify_to_ref $name, $caller;
                                           identical.
    *$ref                              ●   Simply pass the name, 
    = sub
    {                                      manager, and init 
       $init->();
                                           assignment.
        &$ref   = $mgr;
                                       ●   The module can call 
        goto &$mgr
    }                                      $init, replace itself.
}
                                       ●   Caller defines the 
                                           cache and handler.
Using Sub::Trampoline
use Sub::Trampoline;

my %cache1   = ();
my @cache2   = ();

my $cache1_mgr
= sub { $cache1{ $_[0] } or croak "Unknown '$_[0]'" };

my $cache2_mgr
= sub { first { $_ eq $_[0] } @cache2 } };

my $init_cache1 = sub { %cache1 = select_from_hell }
sub init_cache2 { @cache2 = XMLin $nasty_messy_xml_struct }

install_trampoline( subone     => $init_cache1, $cache1_mgr );
install_trampoline( known      => &init_cache2, $cache2_mgr );

my $value1   = subone $key1;

if( known $value )
{ … }
else
{ carp “Unknown: '$value'” }
True One­Shot: Empty $handler
●   If you want to run something exactly once but don't 
    know where it might be called initially:
      my $manager = sub(){};
●   You can also substitute a trampoline object with a 
    constructor that does the work and no methods.
●   Calling the object once constructs it, after which the 
    classes constructor can stub itself.
●   Useful for sharing the cache variable: the init 
    populates it once and stubs itself to do nothing more.
Cycling The Cache
●   There are times when you want to purge and re­
    initialize the cache
●   A trampoline object with populate and use subs that 
    flip­flop can handle this easily.
●   Re­assigning a trampoline re­initializes the cache:

    $cache
    = Object::Trampoline->init_cache( $class => @argz )
    if $age > $time_max;
v5.10 Introduced “state” Variables
●   Scoped like a lexical.
●   Assinged once at runtime.
●   Maintain value within a single lexical context 
    throughout the program.
●   Assign the cache or assign a flag variable with the 
    side effect of populating the cache.
●   Currently supports only scalars.
Obvious case: Assign the cache.
●   Assign the cache at runtime:
    sub cache_mangler { state $cache = init_cache; … }
●   $cache will be assigned at runtime, the first time 
    cache_mangler is called.
●   The value will be retained between calls.
●   Catch: $cache is only available within 
    cache_mangler, not outside of it.
Initialize a Shared Cache
use v5.10;                       ●   Subs may want to share 
my %cache    = ();                   a cache.
sub init                         ●   $y and $z are assigned 
{
   %k or %k = …;                     at most once per 
}
                                     executison when foo or 
sub foo { state $y = init; … }       bar are called.
sub bar { state $z = init; … }

…
                                 ●   The sanity check in init 
                                     only needs to be handled 
my $foo = foo 'bletch';
my $bar = bar 'blort';               at most per subroutine.
Summary
●   True lazyness includes managing data.
●   Preloading it all or testing it at each step are not lazy.
●   Object::Trampoline provides one way.
●   Trampoline subroutines offer another approach.
●   v5.10 introduced state variables which provide a few 
    ways to initialize something once.

More Related Content

What's hot

PHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolvePHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolveXSolve
 
Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Fabien Potencier
 
SPL: The Missing Link in Development
SPL: The Missing Link in DevelopmentSPL: The Missing Link in Development
SPL: The Missing Link in Developmentjsmith92
 
Speed up your developments with Symfony2
Speed up your developments with Symfony2Speed up your developments with Symfony2
Speed up your developments with Symfony2Hugo Hamon
 
Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Kang-min Liu
 
Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)brian d foy
 
Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Lin Yo-An
 
Looping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsLooping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsMark Baker
 
PHP Performance Trivia
PHP Performance TriviaPHP Performance Trivia
PHP Performance TriviaNikita Popov
 
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Puppet
 
Learning Perl 6
Learning Perl 6 Learning Perl 6
Learning Perl 6 brian d foy
 
Zend Certification Preparation Tutorial
Zend Certification Preparation TutorialZend Certification Preparation Tutorial
Zend Certification Preparation TutorialLorna Mitchell
 
Rich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationRich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationKirill Chebunin
 
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoIntroduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoMasahiro Nagano
 
What's New in Perl? v5.10 - v5.16
What's New in Perl?  v5.10 - v5.16What's New in Perl?  v5.10 - v5.16
What's New in Perl? v5.10 - v5.16Ricardo Signes
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHPGuilherme Blanco
 

What's hot (20)

PHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolvePHPCon 2016: PHP7 by Witek Adamus / XSolve
PHPCon 2016: PHP7 by Witek Adamus / XSolve
 
Php & my sql
Php & my sqlPhp & my sql
Php & my sql
 
Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)Beyond symfony 1.2 (Symfony Camp 2008)
Beyond symfony 1.2 (Symfony Camp 2008)
 
SPL: The Missing Link in Development
SPL: The Missing Link in DevelopmentSPL: The Missing Link in Development
SPL: The Missing Link in Development
 
Speed up your developments with Symfony2
Speed up your developments with Symfony2Speed up your developments with Symfony2
Speed up your developments with Symfony2
 
Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)Good Evils In Perl (Yapc Asia)
Good Evils In Perl (Yapc Asia)
 
Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)Learning Perl 6 (NPW 2007)
Learning Perl 6 (NPW 2007)
 
Web 4 | Core JavaScript
Web 4 | Core JavaScriptWeb 4 | Core JavaScript
Web 4 | Core JavaScript
 
Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015Code Generation in PHP - PHPConf 2015
Code Generation in PHP - PHPConf 2015
 
Looping the Loop with SPL Iterators
Looping the Loop with SPL IteratorsLooping the Loop with SPL Iterators
Looping the Loop with SPL Iterators
 
PHP Performance Trivia
PHP Performance TriviaPHP Performance Trivia
PHP Performance Trivia
 
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...Replacing "exec" with a type and provider: Return manifests to a declarative ...
Replacing "exec" with a type and provider: Return manifests to a declarative ...
 
Web 8 | Introduction to PHP
Web 8 | Introduction to PHPWeb 8 | Introduction to PHP
Web 8 | Introduction to PHP
 
Learning Perl 6
Learning Perl 6 Learning Perl 6
Learning Perl 6
 
Zend Certification Preparation Tutorial
Zend Certification Preparation TutorialZend Certification Preparation Tutorial
Zend Certification Preparation Tutorial
 
Rich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 ApplicationRich Model And Layered Architecture in SF2 Application
Rich Model And Layered Architecture in SF2 Application
 
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 TokyoIntroduction to CloudForecast / YAPC::Asia 2010 Tokyo
Introduction to CloudForecast / YAPC::Asia 2010 Tokyo
 
PerlScripting
PerlScriptingPerlScripting
PerlScripting
 
What's New in Perl? v5.10 - v5.16
What's New in Perl?  v5.10 - v5.16What's New in Perl?  v5.10 - v5.16
What's New in Perl? v5.10 - v5.16
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHP
 

Similar to Lazy Data Using Perl

Object::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeObject::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeWorkhorse Computing
 
Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Jeff Carouth
 
Auto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK NodesAuto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK Nodesnihiliad
 
Lithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksLithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksNate Abele
 
Drupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionDrupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionPhilip Norton
 
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)arcware
 
PHP 5.3 Overview
PHP 5.3 OverviewPHP 5.3 Overview
PHP 5.3 Overviewjsmith92
 
Introducing PHP Latest Updates
Introducing PHP Latest UpdatesIntroducing PHP Latest Updates
Introducing PHP Latest UpdatesIftekhar Eather
 
Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)andrewnacin
 
The promise of asynchronous php
The promise of asynchronous phpThe promise of asynchronous php
The promise of asynchronous phpWim Godden
 
How Kris Writes Symfony Apps
How Kris Writes Symfony AppsHow Kris Writes Symfony Apps
How Kris Writes Symfony AppsKris Wallsmith
 
WordPress plugin #3
WordPress plugin #3WordPress plugin #3
WordPress plugin #3giwoolee
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8Alexei Gorobets
 
Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Workhorse Computing
 
Perl web frameworks
Perl web frameworksPerl web frameworks
Perl web frameworksdiego_k
 
Curscatalyst
CurscatalystCurscatalyst
CurscatalystKar Juan
 

Similar to Lazy Data Using Perl (20)

Object::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your CodeObject::Franger: Wear a Raincoat in your Code
Object::Franger: Wear a Raincoat in your Code
 
Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4Can't Miss Features of PHP 5.3 and 5.4
Can't Miss Features of PHP 5.3 and 5.4
 
Subroutines
SubroutinesSubroutines
Subroutines
 
Mojolicious
MojoliciousMojolicious
Mojolicious
 
Auto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK NodesAuto-loading of Drupal CCK Nodes
Auto-loading of Drupal CCK Nodes
 
Lithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate FrameworksLithium: The Framework for People Who Hate Frameworks
Lithium: The Framework for People Who Hate Frameworks
 
Drupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency InjectionDrupal 8 Services And Dependency Injection
Drupal 8 Services And Dependency Injection
 
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
10 Things Every Plugin Developer Should Know (WordCamp Atlanta 2013)
 
PHP 5.3 Overview
PHP 5.3 OverviewPHP 5.3 Overview
PHP 5.3 Overview
 
Magic methods
Magic methodsMagic methods
Magic methods
 
Introducing PHP Latest Updates
Introducing PHP Latest UpdatesIntroducing PHP Latest Updates
Introducing PHP Latest Updates
 
Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)Best Practices in Plugin Development (WordCamp Seattle)
Best Practices in Plugin Development (WordCamp Seattle)
 
The promise of asynchronous php
The promise of asynchronous phpThe promise of asynchronous php
The promise of asynchronous php
 
How Kris Writes Symfony Apps
How Kris Writes Symfony AppsHow Kris Writes Symfony Apps
How Kris Writes Symfony Apps
 
WordPress plugin #3
WordPress plugin #3WordPress plugin #3
WordPress plugin #3
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8
 
Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.Object Trampoline: Why having not the object you want is what you need.
Object Trampoline: Why having not the object you want is what you need.
 
Doctrine in FLOW3
Doctrine in FLOW3Doctrine in FLOW3
Doctrine in FLOW3
 
Perl web frameworks
Perl web frameworksPerl web frameworks
Perl web frameworks
 
Curscatalyst
CurscatalystCurscatalyst
Curscatalyst
 

More from Workhorse Computing

Wheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWorkhorse Computing
 
Paranormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpParanormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpWorkhorse Computing
 
The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.Workhorse Computing
 
Generating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlGenerating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlWorkhorse Computing
 
Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Workhorse Computing
 
BSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationBSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationWorkhorse Computing
 
BASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationBASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationWorkhorse Computing
 
The W-curve and its application.
The W-curve and its application.The W-curve and its application.
The W-curve and its application.Workhorse Computing
 
Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Workhorse Computing
 
Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Workhorse Computing
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Workhorse Computing
 

More from Workhorse Computing (20)

Wheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility Modules
 
mro-every.pdf
mro-every.pdfmro-every.pdf
mro-every.pdf
 
Paranormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpParanormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add Up
 
The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.
 
Unit Testing Lots of Perl
Unit Testing Lots of PerlUnit Testing Lots of Perl
Unit Testing Lots of Perl
 
Generating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlGenerating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in Posgresql
 
Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!
 
BSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationBSDM with BASH: Command Interpolation
BSDM with BASH: Command Interpolation
 
Findbin libs
Findbin libsFindbin libs
Findbin libs
 
Memory Manglement in Raku
Memory Manglement in RakuMemory Manglement in Raku
Memory Manglement in Raku
 
BASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationBASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic Interpolation
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 
Metadata-driven Testing
Metadata-driven TestingMetadata-driven Testing
Metadata-driven Testing
 
The W-curve and its application.
The W-curve and its application.The W-curve and its application.
The W-curve and its application.
 
Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.
 
Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.
 
Smoking docker
Smoking dockerSmoking docker
Smoking docker
 
Getting Testy With Perl6
Getting Testy With Perl6Getting Testy With Perl6
Getting Testy With Perl6
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
 
Neatly folding-a-tree
Neatly folding-a-treeNeatly folding-a-tree
Neatly folding-a-tree
 

Recently uploaded

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 

Recently uploaded (20)

WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 

Lazy Data Using Perl

  • 1. Virtuous Lazyness For Your Data Doing it once and knowing you've done it. Steve Lembark Workhorse Computing lembark@wrkhors.com
  • 2. Data wants to be free, it also wants to be very expensive. ● Data you never use is free; accessing gets expensive. ● Managing data's cost requires controlling its lifecycle. ● Mis­managing the lifecycle causes problems: ● Caching table lookups in forked httpd crashes your database. ● Pre­processing messages files kills your startup times. ● Reading XML configuration data you never use wastes most of  your testing budget. ● Your tests fail because of a connection failure to a database  server that you don't use in the test. ● Fixing these problems requires using lazy data.
  • 3. False Lazyness ● We have all see the two most common data  management stratagies: ● You load everything at startup to avoid checking it every  time it is used. ● You check everything every time before using it to avoid  loading it up front. ● Both approaches ignore important knowledge: ● You know when the data is needed. ● You know that the data was loaded.
  • 4. One alternative: Scalar Cache ● $cache ||= read_cache_values; ● Seems nice: You know that $cache is populated. ● This requires dereferencing a hashref throughout the  program, which is expensive.  ● What you'd rather do is just use $cache{ $foobar }  without having to check it every time. ● Even checking if( ! %cache ) is expensive – arrays  are much cheaper to check.
  • 5. True Lazyness: Do Something Once ● Truly Lazy data means loading you data when you  need it and knowing that it is loaded. ● Perl gives us – of course – more than one way: ● Trampoline objects and subroutines. ● “state” variables, introduced in v5.10. ● Trampolines are flyweight objects – data structures or  subroutines that transform themselves when used. ● State variables are assigned only once, at runtime, the  first time they are used.
  • 6. Follow The Bouncing Object ● Object::Trampoline – flyweight data you don't know  isn't there. ● These delay calling the “real” constructor until the  object is actually used. ● Your constructor gets called before the first method call. ● At that point it can cache, parse, or compute whatever it  needs. ● Spreads the cost of loading data set each over the  lifetime of a process.
  • 7. Example: Delay Expensive XML ● Use an initializer to read and parse the XML. ● The Object::Trampoline calls your constructor once  to transform the object the first time you call a  method. ● Parsing the XML is pushed off until you actually use  the data. ● Requires using a hashref – and possibly methods –  to access the data.
  • 8. package XML::Message; use Object::Trampoline; ... sub new { Object::Trampline->install( 'XML::Message', $path ); } sub install { my $error = &construct; $error->initialize( @_ ) } sub initialize { my ( $err, $path ) = @_; %$err = %{ XMLin $path => @lots_of_args }; $err } # calling translate bounces the trampoline exactly once sub translate { … }
  • 9. Tramploline Subroutine ● Similar to a trampoline object: A portion of the code  runs once and replaces itself. ● One­shot code initializes the cache, which can now  be a simple hash. ● Symbol::qualify_to_ref make this painless in Perl. ● An anonymous sub manages the cache. ● A named subroutine loads the cache, replaces itself with  the manager, and re­dispatches to the manager.
  • 10. The Simplest Version my %foo_cache = (); ● Minimal code  my $handler = sub includes: { my $foo = shift; ● Closed­over cache, %foo_cache{ $foo } Or die “Bogus foo: '$foo' unknown” ● Subref to permenant  }; cache manager, sub do_something { ● Initial subroutine. %cache = initialize_foo_cache; ● The initial subroutine  my $ref = qualify_to_ref 'do_something'; initializes, installs,  *$ref = $handler; and re­dispatches. goto &$handler }
  • 11. BEGIN Blocks Are Cleaner BEGIN { my $name = 'foo_handler'; ● The block isolates  my $ref = qualify_to_ref $name; my %cache = (); cache, ref, and  my $handler sub variables;  = sub { allows re­cycling  %cache{ $_[0] } or die ... }; the ref. *$ref = sub ● This is also rather  { %cache = init_the_cache; amenable to  *$ref = $handler; installation by  module. goto &$handler } }
  • 12. Sub::Trampoline sub install_trampoline { my ( $name, $init, $mgr ) = @_; ● Aside from the actual  my $caller = caller; assignment, init code is  my $ref = qualify_to_ref $name, $caller; identical. *$ref ● Simply pass the name,  = sub { manager, and init  $init->(); assignment. &$ref = $mgr; ● The module can call  goto &$mgr } $init, replace itself. } ● Caller defines the  cache and handler.
  • 13. Using Sub::Trampoline use Sub::Trampoline; my %cache1 = (); my @cache2 = (); my $cache1_mgr = sub { $cache1{ $_[0] } or croak "Unknown '$_[0]'" }; my $cache2_mgr = sub { first { $_ eq $_[0] } @cache2 } }; my $init_cache1 = sub { %cache1 = select_from_hell } sub init_cache2 { @cache2 = XMLin $nasty_messy_xml_struct } install_trampoline( subone => $init_cache1, $cache1_mgr ); install_trampoline( known => &init_cache2, $cache2_mgr ); my $value1 = subone $key1; if( known $value ) { … } else { carp “Unknown: '$value'” }
  • 14. True One­Shot: Empty $handler ● If you want to run something exactly once but don't  know where it might be called initially: my $manager = sub(){}; ● You can also substitute a trampoline object with a  constructor that does the work and no methods. ● Calling the object once constructs it, after which the  classes constructor can stub itself. ● Useful for sharing the cache variable: the init  populates it once and stubs itself to do nothing more.
  • 15. Cycling The Cache ● There are times when you want to purge and re­ initialize the cache ● A trampoline object with populate and use subs that  flip­flop can handle this easily. ● Re­assigning a trampoline re­initializes the cache: $cache = Object::Trampoline->init_cache( $class => @argz ) if $age > $time_max;
  • 16. v5.10 Introduced “state” Variables ● Scoped like a lexical. ● Assinged once at runtime. ● Maintain value within a single lexical context  throughout the program. ● Assign the cache or assign a flag variable with the  side effect of populating the cache. ● Currently supports only scalars.
  • 17. Obvious case: Assign the cache. ● Assign the cache at runtime: sub cache_mangler { state $cache = init_cache; … } ● $cache will be assigned at runtime, the first time  cache_mangler is called. ● The value will be retained between calls. ● Catch: $cache is only available within  cache_mangler, not outside of it.
  • 18. Initialize a Shared Cache use v5.10; ● Subs may want to share  my %cache = (); a cache. sub init ● $y and $z are assigned  { %k or %k = …; at most once per  } executison when foo or  sub foo { state $y = init; … } bar are called. sub bar { state $z = init; … } … ● The sanity check in init  only needs to be handled  my $foo = foo 'bletch'; my $bar = bar 'blort'; at most per subroutine.
  • 19. Summary ● True lazyness includes managing data. ● Preloading it all or testing it at each step are not lazy. ● Object::Trampoline provides one way. ● Trampoline subroutines offer another approach. ● v5.10 introduced state variables which provide a few  ways to initialize something once.