PERLDSC(1) Perl Programmers Reference Guide PERLDSC(1)
NAME
perldsc - Perl Data Structures Cookbook
DESCRIPTION
The single feature most sorely lacking in the Perl programming language prior to its 5.0 release was complex
data structures. Even without direct language support, some valiant programmers did manage to emulate them,
but it was hard work and not for the faint of heart. You could occasionally get away with the $m{$AoA,$b}
notation borrowed from awk in which the keys are actually more like a single concatenated string "$AoA$b", but
traversal and sorting were difficult. More desperate programmers even hacked Perl's internal symbol table
directly, a strategy that proved hard to develop and maintain--to put it mildly.
The 5.0 release of Perl let us have complex data structures. You may now write something like this and all of
a sudden, you'd have an array with three dimensions!
for $x (1 .. 10) {
for $y (1 .. 10) {
for $z (1 .. 10) {
$AoA[$x][$y][$z] =
$x ** $y + $z;
}
}
}
Alas, however simple this may appear, underneath it's a much more elaborate construct than meets the eye!
How do you print it out? Why can't you say just "print @AoA"? How do you sort it? How can you pass it to a
function or get one of these back from a function? Is it an object? Can you save it to disk to read back
later? How do you access whole rows or columns of that matrix? Do all the values have to be numeric?
As you see, it's quite easy to become confused. While some small portion of the blame for this can be
attributed to the reference-based implementation, it's really more due to a lack of existing documentation
with examples designed for the beginner.
This document is meant to be a detailed but understandable treatment of the many different sorts of data
structures you might want to develop. It should also serve as a cookbook of examples. That way, when you
need to create one of these complex data structures, you can just pinch, pilfer, or purloin a drop-in example
from here.
Let's look at each of these possible constructs in detail. There are separate sections on each of the
following:
· arrays of arrays
· hashes of arrays
· arrays of hashes
· hashes of hashes
· more elaborate constructs
But for now, let's look at general issues common to all these types of data structures.
REFERENCES
The most important thing to understand about all data structures in Perl--including multidimensional
arrays--is that even though they might appear otherwise, Perl @ARRAYs and %HASHes are all internally one-
This is actually the way almost all C multidimensional arrays work as well.
$array[7][12] # array of arrays
$array[7]{string} # array of hashes
$hash{string}[7] # hash of arrays
$hash{string}{'another string'} # hash of hashes
Now, because the top level contains only references, if you try to print out your array in with a simple
print() function, you'll get something that doesn't look very nice, like this:
@AoA = ( [2, 3], [4, 5, 7], [0] );
print $AoA[1][2];
7
print @AoA;
ARRAY(0x83c38)ARRAY(0x8b194)ARRAY(0x8b1d0)
That's because Perl doesn't (ever) implicitly dereference your variables. If you want to get at the thing a
reference is referring to, then you have to do this yourself using either prefix typing indicators, like
"${$blah}", "@{$blah}", "@{$blah[$i]}", or else postfix pointer arrows, like "$a->[3]", "$h->{fred}", or even
"$ob->method()->[3]".
COMMON MISTAKES
The two most common mistakes made in constructing something like an array of arrays is either accidentally
counting the number of elements or else taking a reference to the same memory location repeatedly. Here's the
case where you just get the count instead of a nested array:
for $i (1..10) {
@array = somefunc($i);
$AoA[$i] = @array; # WRONG!
}
That's just the simple case of assigning an array to a scalar and getting its element count. If that's what
you really and truly want, then you might do well to consider being a tad more explicit about it, like this:
for $i (1..10) {
@array = somefunc($i);
$counts[$i] = scalar @array;
}
Here's the case of taking a reference to the same memory location again and again:
for $i (1..10) {
@array = somefunc($i);
$AoA[$i] = \@array; # WRONG!
}
So, what's the big problem with that? It looks right, doesn't it? After all, I just told you that you need
an array of references, so by golly, you've made me one!
Unfortunately, while this is true, it's still broken. All the references in @AoA refer to the very same
place, and they will therefore all hold whatever was last in @array! It's similar to the problem demonstrated
in the following C program:
#include <pwd.h>
root name is daemon
The problem is that both "rp" and "dp" are pointers to the same location in memory! In C, you'd have to
remember to malloc() yourself some new memory. In Perl, you'll want to use the array constructor "[]" or the
hash constructor "{}" instead. Here's the right way to do the preceding broken code fragments:
for $i (1..10) {
@array = somefunc($i);
$AoA[$i] = [ @array ];
}
The square brackets make a reference to a new array with a copy of what's in @array at the time of the
assignment. This is what you want.
Note that this will produce something similar, but it's much harder to read:
for $i (1..10) {
@array = 0 .. $i;
@{$AoA[$i]} = @array;
}
Is it the same? Well, maybe so--and maybe not. The subtle difference is that when you assign something in
square brackets, you know for sure it's always a brand new reference with a new copy of the data. Something
else could be going on in this new case with the "@{$AoA[$i]}" dereference on the left-hand-side of the
assignment. It all depends on whether $AoA[$i] had been undefined to start with, or whether it already
contained a reference. If you had already populated @AoA with references, as in
$AoA[3] = \@another_array;
Then the assignment with the indirection on the left-hand-side would use the existing reference that was
already there:
@{$AoA[3]} = @array;
Of course, this would have the "interesting" effect of clobbering @another_array. (Have you ever noticed how
when a programmer says something is "interesting", that rather than meaning "intriguing", they're disturbingly
more apt to mean that it's "annoying", "difficult", or both? :-)
So just remember always to use the array or hash constructors with "[]" or "{}", and you'll be fine, although
it's not always optimally efficient.
Surprisingly, the following dangerous-looking construct will actually work out fine:
for $i (1..10) {
my @array = somefunc($i);
$AoA[$i] = \@array;
}
That's because my() is more of a run-time statement than it is a compile-time declaration per se. This means
that the my() variable is remade afresh each time through the loop. So even though it looks as though you
stored the same variable reference each time, you actually did not! This is a subtle distinction that can
produce more efficient code at the risk of misleading all but the most experienced of programmers. So I
usually advise against teaching it to beginners. In fact, except for passing arguments to functions, I seldom
like to see the gimme-a-reference operator (backslash) used much at all in code. Instead, I advise beginners
$aref->[2][2] # clear
$$aref[2][2] # confusing
That's because Perl's precedence rules on its five prefix dereferencers (which look like someone swearing: "$
@ * % &") make them bind more tightly than the postfix subscripting brackets or braces! This will no doubt
come as a great shock to the C or C++ programmer, who is quite accustomed to using *a[i] to mean what's
pointed to by the i'th element of "a". That is, they first take the subscript, and only then dereference the
thing at that subscript. That's fine in C, but this isn't C.
The seemingly equivalent construct in Perl, $$aref[$i] first does the deref of $aref, making it take $aref as
a reference to an array, and then dereference that, and finally tell you the i'th value of the array pointed
to by $AoA. If you wanted the C notion, you'd have to write "${$AoA[$i]}" to force the $AoA[$i] to get
evaluated first before the leading "$" dereferencer.
WHY YOU SHOULD ALWAYS "use strict"
If this is starting to sound scarier than it's worth, relax. Perl has some features to help you avoid its
most common pitfalls. The best way to avoid getting confused is to start every program like this:
#!/usr/bin/perl -w
use strict;
This way, you'll be forced to declare all your variables with my() and also disallow accidental "symbolic
dereferencing". Therefore if you'd done this:
my $aref = [
[ "fred", "barney", "pebbles", "bambam", "dino", ],
[ "homer", "bart", "marge", "maggie", ],
[ "george", "jane", "elroy", "judy", ],
];
print $aref[2][2];
The compiler would immediately flag that as an error at compile time, because you were accidentally accessing
@aref, an undeclared variable, and it would thereby remind you to write instead:
print $aref->[2][2]
DEBUGGING
Before version 5.002, the standard Perl debugger didn't do a very nice job of printing out complex data
structures. With 5.002 or above, the debugger includes several new features, including command line editing
as well as the "x" command to dump out complex data structures. For example, given the assignment to $AoA
above, here's the debugger output:
DB<1> x $AoA
$AoA = ARRAY(0x13b5a0)
0 ARRAY(0x1f0a24)
0 'fred'
1 'barney'
2 'pebbles'
3 'bambam'
4 'dino'
1 ARRAY(0x13b558)
0 'homer'
ARRAYS OF ARRAYS
Declaration of an ARRAY OF ARRAYS
@AoA = (
[ "fred", "barney" ],
[ "george", "jane", "elroy" ],
[ "homer", "marge", "bart" ],
);
Generation of an ARRAY OF ARRAYS
# reading from file
while ( <> ) {
push @AoA, [ split ];
}
# calling a function
for $i ( 1 .. 10 ) {
$AoA[$i] = [ somefunc($i) ];
}
# using temp vars
for $i ( 1 .. 10 ) {
@tmp = somefunc($i);
$AoA[$i] = [ @tmp ];
}
# add to an existing row
push @{ $AoA[0] }, "wilma", "betty";
Access and Printing of an ARRAY OF ARRAYS
# one element
$AoA[0][0] = "Fred";
# another element
$AoA[1][1] =~ s/(\w)/\u$1/;
# print the whole thing with refs
for $aref ( @AoA ) {
print "\t [ @$aref ],\n";
}
# print the whole thing with indices
for $i ( 0 .. $#AoA ) {
print "\t [ @{$AoA[$i]} ],\n";
}
# print the whole thing one at a time
for $i ( 0 .. $#AoA ) {
for $j ( 0 .. $#{ $AoA[$i] } ) {
print "elt $i $j is $AoA[$i][$j]\n";
}
}
HASHES OF ARRAYS
$HoA{$1} = [ split ];
}
# reading from file; more temps
# flintstones: fred barney wilma dino
while ( $line = <> ) {
($who, $rest) = split /:\s*/, $line, 2;
@fields = split ' ', $rest;
$HoA{$who} = [ @fields ];
}
# calling a function that returns a list
for $group ( "simpsons", "jetsons", "flintstones" ) {
$HoA{$group} = [ get_family($group) ];
}
# likewise, but using temps
for $group ( "simpsons", "jetsons", "flintstones" ) {
@members = get_family($group);
$HoA{$group} = [ @members ];
}
# append new members to an existing family
push @{ $HoA{"flintstones"} }, "wilma", "betty";
Access and Printing of a HASH OF ARRAYS
# one element
$HoA{flintstones}[0] = "Fred";
# another element
$HoA{simpsons}[1] =~ s/(\w)/\u$1/;
# print the whole thing
foreach $family ( keys %HoA ) {
print "$family: @{ $HoA{$family} }\n"
}
# print the whole thing with indices
foreach $family ( keys %HoA ) {
print "family: ";
foreach $i ( 0 .. $#{ $HoA{$family} } ) {
print " $i = $HoA{$family}[$i]";
}
print "\n";
}
# print the whole thing sorted by number of members
foreach $family ( sort { @{$HoA{$b}} <=> @{$HoA{$a}} } keys %HoA ) {
print "$family: @{ $HoA{$family} }\n"
}
# print the whole thing sorted by number of members and name
foreach $family ( sort {
@{$HoA{$b}} <=> @{$HoA{$a}}
Friend => "barney",
},
{
Lead => "george",
Wife => "jane",
Son => "elroy",
},
{
Lead => "homer",
Wife => "marge",
Son => "bart",
}
);
Generation of an ARRAY OF HASHES
# reading from file
# format: LEAD=fred FRIEND=barney
while ( <> ) {
$rec = {};
for $field ( split ) {
($key, $value) = split /=/, $field;
$rec->{$key} = $value;
}
push @AoH, $rec;
}
# reading from file
# format: LEAD=fred FRIEND=barney
# no temp
while ( <> ) {
push @AoH, { split /[\s+=]/ };
}
# calling a function that returns a key/value pair list, like
# "lead","fred","daughter","pebbles"
while ( %fields = getnextpairset() ) {
push @AoH, { %fields };
}
# likewise, but using no temp vars
while (<>) {
push @AoH, { parsepairs($_) };
}
# add key/value to an element
$AoH[0]{pet} = "dino";
$AoH[2]{pet} = "santa's little helper";
Access and Printing of an ARRAY OF HASHES
# one element
$AoH[0]{lead} = "fred";
# another element
for $i ( 0 .. $#AoH ) {
print "$i is { ";
for $role ( keys %{ $AoH[$i] } ) {
print "$role=$AoH[$i]{$role} ";
}
print "}\n";
}
# print the whole thing one at a time
for $i ( 0 .. $#AoH ) {
for $role ( keys %{ $AoH[$i] } ) {
print "elt $i $role is $AoH[$i]{$role}\n";
}
}
HASHES OF HASHES
Declaration of a HASH OF HASHES
%HoH = (
flintstones => {
lead => "fred",
pal => "barney",
},
jetsons => {
lead => "george",
wife => "jane",
"his boy" => "elroy",
},
simpsons => {
lead => "homer",
wife => "marge",
kid => "bart",
},
);
Generation of a HASH OF HASHES
# reading from file
# flintstones: lead=fred pal=barney wife=wilma pet=dino
while ( <> ) {
next unless s/^(.*?):\s*//;
$who = $1;
for $field ( split ) {
($key, $value) = split /=/, $field;
$HoH{$who}{$key} = $value;
}
# reading from file; more temps
while ( <> ) {
next unless s/^(.*?):\s*//;
$who = $1;
$rec = {};
$HoH{$who} = $rec;
for $field ( split ) {
($key, $value) = split /=/, $field;
$HoH{$group} = { %members };
}
# append new members to an existing family
%new_folks = (
wife => "wilma",
pet => "dino",
);
for $what (keys %new_folks) {
$HoH{flintstones}{$what} = $new_folks{$what};
}
Access and Printing of a HASH OF HASHES
# one element
$HoH{flintstones}{wife} = "wilma";
# another element
$HoH{simpsons}{lead} =~ s/(\w)/\u$1/;
# print the whole thing
foreach $family ( keys %HoH ) {
print "$family: { ";
for $role ( keys %{ $HoH{$family} } ) {
print "$role=$HoH{$family}{$role} ";
}
print "}\n";
}
# print the whole thing somewhat sorted
foreach $family ( sort keys %HoH ) {
print "$family: { ";
for $role ( sort keys %{ $HoH{$family} } ) {
print "$role=$HoH{$family}{$role} ";
}
print "}\n";
}
# print the whole thing sorted by number of members
foreach $family ( sort { keys %{$HoH{$b}} <=> keys %{$HoH{$a}} } keys %HoH ) {
print "$family: { ";
for $role ( sort keys %{ $HoH{$family} } ) {
print "$role=$HoH{$family}{$role} ";
}
print "}\n";
}
# establish a sort order (rank) for each role
$i = 0;
for ( qw(lead wife son daughter pal pet) ) { $rank{$_} = ++$i }
# now print the whole thing sorted by number of members
foreach $family ( sort { keys %{ $HoH{$b} } <=> keys %{ $HoH{$a} } } keys %HoH ) {
$rec = {
TEXT => $string,
SEQUENCE => [ @old_values ],
LOOKUP => { %some_table },
THATCODE => \&some_function,
THISCODE => sub { $_[0] ** $_[1] },
HANDLE => \*STDOUT,
};
print $rec->{TEXT};
print $rec->{SEQUENCE}[0];
$last = pop @ { $rec->{SEQUENCE} };
print $rec->{LOOKUP}{"key"};
($first_k, $first_v) = each %{ $rec->{LOOKUP} };
$answer = $rec->{THATCODE}->($arg);
$answer = $rec->{THISCODE}->($arg1, $arg2);
# careful of extra block braces on fh ref
print { $rec->{HANDLE} } "a string\n";
use FileHandle;
$rec->{HANDLE}->autoflush(1);
$rec->{HANDLE}->print(" a string\n");
Declaration of a HASH OF COMPLEX RECORDS
%TV = (
flintstones => {
series => "flintstones",
nights => [ qw(monday thursday friday) ],
members => [
{ name => "fred", role => "lead", age => 36, },
{ name => "wilma", role => "wife", age => 31, },
{ name => "pebbles", role => "kid", age => 4, },
],
},
jetsons => {
series => "jetsons",
nights => [ qw(wednesday saturday) ],
members => [
{ name => "george", role => "lead", age => 41, },
{ name => "jane", role => "wife", age => 39, },
{ name => "elroy", role => "kid", age => 9, },
],
},
simpsons => {
series => "simpsons",
nights => [ qw(monday) ],
members => [
{ name => "homer", role => "lead", age => 34, },
# here's a piece by piece build up
$rec = {};
$rec->{series} = "flintstones";
$rec->{nights} = [ find_days() ];
@members = ();
# assume this file in field=value syntax
while (<>) {
%fields = split /[\s=]+/;
push @members, { %fields };
}
$rec->{members} = [ @members ];
# now remember the whole thing
$TV{ $rec->{series} } = $rec;
###########################################################
# now, you might want to make interesting extra fields that
# include pointers back into the same data structure so if
# change one piece, it changes everywhere, like for example
# if you wanted a {kids} field that was a reference
# to an array of the kids' records without having duplicate
# records and thus update problems.
###########################################################
foreach $family (keys %TV) {
$rec = $TV{$family}; # temp pointer
@kids = ();
for $person ( @{ $rec->{members} } ) {
if ($person->{role} =~ /kid|son|daughter/) {
push @kids, $person;
}
}
# REMEMBER: $rec and $TV{$family} point to same data!!
$rec->{kids} = [ @kids ];
}
# you copied the array, but the array itself contains pointers
# to uncopied objects. this means that if you make bart get
# older via
$TV{simpsons}{kids}[0]{age}++;
# then this would also change in
print $TV{simpsons}{members}[2]{age};
# because $TV{simpsons}{kids}[0] and $TV{simpsons}{members}[2]
# both point to the same underlying anonymous hash table
# print the whole thing
foreach $family ( keys %TV ) {
print "the $family";
print " is on during @{ $TV{$family}{nights} }\n";
print "its members are:\n";
references are to be represented on disk. One experimental module that does partially attempt to address this
need is the MLDBM module. Check your nearest CPAN site as described in perlmodlib for source code to MLDBM.
SEE ALSO
perlref, perllol, perldata, perlobj
AUTHOR
Tom Christiansen <[email protected]>
Last update: Wed Oct 23 04:57:50 MET DST 1996
perl v5.16.3 2013-03-04 PERLDSC(1)
Back to main site | Back to man page index