Core Function Pack

From Sputnik Wiki
Jump to: navigation, search 
Pack( <format>, <args> )

Contents

Description

Pack data into a binary array.

Pack given arguments into a binary string according to format.

Parameters

format

The format string consists of format codes followed by an optional repeater argument. The repeater argument can be either an integer value or * for repeating to the end of the input data.

For a, A, b, B, h, H the repeat count specifies how many characters of one data argument are taken, for @ it is the absolute position where to put the next data, for everything else the repeat count specifies how many data arguments are consumed and packed into the resulting binary string.

For P the repeat counter specifies how many characters to read from the memory pointer.

For u the repeater counter does nothing it will fully encode the whole string and fully decode the whole thing it finds later.

For z the repeat counter specifies the encoding to use when packing/unpacking the string.

For p P t T their byte size depends on if Sputnik is running in 32 or 64 bit.

You may place spaces in the format string and they will be stripped automatically so you can use them to make things more readable.

Currently implemented formats are: 

Code 	Description
a 	NUL-padded string (ASCII)
A 	SPACE-padded string (ASCII)
Z 	NUL-padded string (ASCII) a NUL-terminator will be added to the end regardless
b 	A bit string (ascending bit order inside each byte, like the Vec() function)
B 	A bit string (descending bit order inside each byte)
h 	Hex string, low nibble first
H 	Hex string, high nibble first
c	signed ASCII char
C 	unsigned ASCII char
U 	signed UNICODE char (always 16 bit, machine byte order)
W 	unsigned UNICODE char (always 16 bit, machine byte order)
s 	signed short (always 16 bit, machine byte order)
S 	unsigned short (always 16 bit, machine byte order)
n 	unsigned short (always 16 bit, big endian byte order)
v 	unsigned short (always 16 bit, little endian byte order)
i 	signed integer (machine dependent size and byte order)
I 	unsigned integer (machine dependent size and byte order)
l 	signed long (always 32 bit, machine byte order)
L 	unsigned long (always 32 bit, machine byte order)
q 	signed quad (64-bit) value (always 64 bit, machine byte order)
Q 	unsigned quad (64-bit) value (always 64 bit, machine byte order)
N 	unsigned long (always 32 bit, big endian byte order)
V 	unsigned long (always 32 bit, little endian byte order)
f 	float (machine dependent size and representation)
d 	double (machine dependent size and representation)
p 	A pointer to a null-terminated string
P 	A pointer to a fixed-length string
t 	A signed pointer
T 	A unsigned pointer
u 	A uuencoded string
z7 	string encoded as UTF8 with 1-byte null terminator
z6 	string encoded as UTF7 with 1-byte null terminator
z5 	string encoded as UTF16 with 2-byte null terminator
z4 	string encoded as BigEndianUnicode with 2-byte null terminator
z3 	string encoded as UTF32 big endian with 4-byte null terminator
z2 	string encoded as UTF32 with 4-byte null terminator
z1 	string encoded as ASCII with 1-byte null terminator
z0 	string encoded as ASCII without a null terminator
x 	NUL byte
X 	Back up one byte
@ 	NUL-fill to absolute position

args

One or more variables to be used in the packing.

If you place an array in the arguments it will act as if you placed each element into the arguments this is recursive and it will add everything it finds in every array.

Return Value

Success: Returns the new binary variable.

Failure: Returns null.

Remarks

The return is a binary variable that is basically a byte[] array wrapped in a class so it is not a string by any means however that does not mean you cant read it as a string. If you place the binary variable in a print statement or cast it with (string) it will become a string it would be if it was an ASCII string containing them bytes.

If the binary variable you have should wants to be Unicode instead of ASCII you should check out BinaryToStr( ) function.

In Perl the Pack() and Unpack() actually read/write to strings directly where as in Sputnik the output of a Pack() is actually a binary variable that consists of a byte[] array it can be edited with the Binary___() functions and even cast to a string (will be ASCII or if you use BinaryToStr() you can explicitly demand it treated as Unicode you could even force it as raw ASCII like (ASCII)$mystring).

Note that Sputnik stores variables as signed/unsigned as needed which can be confusing if you are a PHP user using this Pack function basically when the format says Unsigned it is literal and the variable will be unsigned/signed as specified in both the packing and unpacking.

This also means if the original value as an Int64 and you packed it with "i" then unpack it with "i" the new value will be an Int32 and not an Int64 since Sputnik sees no need to magically unpack everything to highest data type so "f" will actually give you a Float and not a Double and so on of course that doesn't mean you can't cast it to a double when getting from the return array.

Be aware that if you do not name an element, an empty string is used. If you do not name more than one element, this means that some data is overwritten as the keys are the same

Warning: when packing multiple pieces of data, * only means "consume all of the current piece of data". That's to say 

print pack("A*A*", $one, $two);

packs all of $one into the first A* and then all of $two into the second. This is a general principle: each format character corresponds to one piece of data to be packed.

Example

Using Unpack() to do a simplistic SubStr() on an ASCII string 

Function _SubStr ( $what, $where, $howmuch )
{
	return unpack("x$where/a$howmuch", $what);
}
printr _SubStr("UberFoX", 2, 3);
# Prints
# erF


Example of packing multiple ASCII strings 

$bin = Pack("z1z1z1", "Cat", "Dog", "Fox");
$unpacked = Unpack("z1/z1/z1", $bin);
printr $unpacked;
# Prints
# ARRAY
# {
#         [0] => Cat
#         [1] => Dog
#         [2] => Fox
# }

Example of packing multiple ASCII strings and unpacking them by name 

$Name = "UberFoX";
$ID = "yay?";
$RandomText = "Hello world!";
$bin = Pack("z1z1z1", $Name, $ID, $RandomText);
$unpacked = Unpack("z1Name/z1ID/z1RandomText", $bin);
printr $unpacked;
# Prints
# ARRAY
# {
#         [Name] => UberFoX
#         [ID] => yay?
#         [RandomText] => Hello world!
# }

Example of packing multiple ASCII strings and unpacking them using List 

$Name = "UberFoX";
$ID = "yay?";
$RandomText = "Hello world!";
$bin = Pack("z1z1z1", $Name, $ID, $RandomText);
List ($TheName, $TheID, $TheRandomText ) = Unpack("z1/z1/z1", $bin);
echo "Name is: $TheName\n";
echo "ID is: $TheID\n";
echo "RandomText is: $TheRandomText\n";
# Print
# Name is: UberFoX
# ID is: yay?
# RandomText is: Hello world!

Same as above but why stop at just text? Lets add an Int and Float in there 

$Name = "UberFoX";
$ID = "yay?";
$RandomText = "Hello world!";
$RandomInt = 7000;
$RandomFloat = 133.7;
$bin = Pack("z1iz1z1d", $Name, $RandomInt, $ID, $RandomText, $RandomFloat);
List ($TheName, $TheInt, $TheID, $TheRandomText, $TheFloat ) = Unpack("z1/i/z1/z1/d", $bin);
echo "Name is: $TheName\n";
echo "ID is: $TheID\n";
echo "RandomText is: $TheRandomText\n";
echo "Int is: $TheInt\n";
echo "Float is: $TheFloat\n";
# Print
# Name is: UberFoX
# ID is: yay?
# RandomText is: Hello world!
# Int is: 7000
# Float is: 133.7

Example of using UUEncode 

# This will read/write the string as it is in Sputnik so
# it handles Unicode/ASCII just fine no problems
my $enc = pack("u", "Testy");
print "Value Encoded '$enc'\n";
$dec = unpack("u", $enc, 3);
print "Value Decoded '$dec'\n";
# Prints
# Value Encoded '%5&5S='D`
# '
# Value Decoded 'Testy'

Example of using UUEncode multiple times with other data as well 

# Encode some text to UUEncode and add some integers and floats 
# in for the fun of it
my $enc = pack("uiud", "Testy", 777, "Cat", 133.77);
print "Value Encoded '$enc'\n";
$dec = unpack("u/i/u/d", $enc);
print "Value Decoded:\n";
printr $dec;
# Prints
# Value Encoded '%5&5S='D`
#         ♥  #0V%T
# q=
# ×£¸`@'
# Value Decoded:
# ARRAY
# {
#         [0] => Testy
#         [1] => 777
#         [2] => Cat
#         [3] => 133.77
# }

Example of using a pointer to a null-terminated string 

# What this does is obtain the memory address where the string is
# located and packs it using the systems pointer size which is
# 4 bytes for 32-bit systems and 8 bytes for 64-bit systems
# It does not allocate a new string it simply links to one that
# is already there.
# It will also accept IntPtr and UIntPtr variables as
# valid input.
# This pointer could be ASCII if you are using IntPtr, etc
# However it will most likely link to a Unicode (UTF8) string.
# When "p" reads in Unpack() it will stop at the first
# null-terminator it finds if you wish to continue and grab
# as much data as you need see "P" below this example
 
$val = "Cat";
my $bin = pack("p", $val);
my $str = Unpack("p", $bin, 3);
print( "Value is: $str\n" );
# Prints
# Cat

Example of using a pointer to a fixed-length string 

# What this does is obtain the memory address where the string is
# located and packs it using the systems pointer size which is
# 4 bytes for 32-bit systems and 8 bytes for 64-bit systems
# It does not allocate a new string it simply links to one that
# is already there.
# It will also accept IntPtr and UIntPtr variables as
# valid input.
# This pointer could be ASCII if you are using IntPtr, etc
# However it will most likely link to a Unicode (UTF8) string.
# When "P" reads in Unpack() it will stop where the repeater says to
# it will ignore null terminators and continue capturing characters
# until the repeater count is used up.
# If you use * with "P" example "P*" it will act like you just used
# "p*" see below
 
# NOTE - It uses a neutral reading method so it will create the string
# as ASCII/Unicode depending on what it finds so if you set it to read
# 5 chars and it is a pointer to an ASCII string chances are it will
# create the string with the correct ASCII characters. However if the
# pointer leads to a UNICODE string chances are high it will create a
# UNICODE string of course it could fail miserably and create a UNICODE
# string when it was supposed to be all ASCII but that is unlikely.
# If such a thing becomes a problem i will add a way to explicitly say
# what the bytes should be considered.
 
# This will read until it hits a null terminator
$val = "Cat";
my $bin = pack("p", $val);
my $str = Unpack("P*", $bin, 3);
print( "Value is: $str\n" );
# Prints
# Cat
 
# This will read until its used up the 2 repeater counter
$val = "Cat";
my $bin = pack("p", $val);
my $str = Unpack("P2", $bin, 3);
print( "Value is: $str\n" );
# Prints
# Ca
 
# This will read until its used up the 10 repeater counter
$val = "Cat";
my $bin = pack("p", $val);
my $str = Unpack("P10", $bin, 3);
print( "Value is: $str\n" );
# Prints
# Cat   ??♦
# Reading beyond a valid string is dangerous and could cause a crash
 
# Here a string containing binary data is used
# then packed and unpacked with it being intact
# and no loss of information
$val = BinaryToStr(pack("iii", 100, 200, 300));
$len = StrLen($val);
echo( "Value is: $val\n" );
echo( "Length: " . $len . "\n" );
my $bin = pack("p", $val);
my $str = Unpack("P$len", $bin, 3);
echo( "Value is: $str\n" );
echo( "Length: " . StrLen($str) . "\n" );
echo( $val == $str ? "Orig is same as new\n" : "ERROR\n" );
# Prints
# Value is: d   È   ,☺
# Length: 12
# Value is: d   È   ,☺
# Length: 12
# Orig is same as new

Example of using an array with an array embedded as the argument 

$array = array( "Cat", "Dog", "FoX", array( "hmm", "lol" )  );
 
my $ret = pack("A*A*A*A*A*", $array);
print "$ret\n";
# Prints
# CatDogFoXhmmlol

Same as above but this time with normal arguments along side arrays 

$array = array( "Cat", "Dog", "FoX", array( "hmm", "lol" )  );
 
my $ret = pack("A*A*A*A*A*A*A*", "hello", $array, "world!");
print "$ret\n";
# Prints
# HelloCatDogFoXhmmlolworld!

So pack an array of Integers to binary you would do 

# Of course this would work with anything not just Integers
 
# Make array of Integers
$array = array( 100, 200, 300, 400, 500, 600  );
my $Count = Count($array);
# Pack the array of Integers
my $bin = pack("I$Count", $array);
# Unpack back to an array of Integers
my $newArray = unpack("I$Count", $bin);
printr $newArray;
# Prints
# ARRAY
# {
#         [0] => 100
#         [1] => 200
#         [2] => 300
#         [3] => 400
#         [4] => 500
#         [5] => 600
# }
# This time lets unpack it to a named list
# We could unpack more but lets just get the first 3
List ( $Int0, $Int1, $Int2 ) = unpack("I3", $bin);
echo "Int0 = $Int0 | Int1 = $Int1 | Int2 = $Int2";
# Prints
# Int0 = 100 | Int1 = 200 | Int2 = 300

Reading and writing ASCII 

my $vec = pack("A*", "Hello");
printr $vec;
printr unpack("A*", $vec, 3);
# Prints
# {BINARY:5}
# {
#         [0] => 72
#         [1] => 101
#         [2] => 108
#         [3] => 108
#         [4] => 111
# }
# Hello

Reading and writing ASCII in bits 

my $vec = pack("A*", "Hello");
my $chars = unpack("A/A/A2", $vec, 3);
print( "First letter: " . $chars[0] . "\n" );
print( "Second letter: " . $chars[1] . "\n" );
print( "Third and forth letter: " . $chars[2] . "\n" );
# Prints
# First letter: H
# Second letter: e
# Third and forth letter: ll

Using Pack() to format ASCII text in a readable way with everything correctly spaced out 

// Make an array of people
my $People = array(array("Name", "ID", "Birth", "Sex", "Race"));
push($People, array("Mike", 0, "1980", "M", "German"));
push($People, array("Tommy", 1, "1985", "M", "French"));
push($People, array("David", 2, "1950", "M", "Jewish"));
push($People, array("Sukara", 3, "1990", "F", "Japanese"));
push($People, array("Mary-Kate", 4, "1987", "F", "American"));
 
// Print them all in a nice formatted way
foreach($People as $Person)
{
	my List ( $Name, $ID, $Birth, $Sex, $Race ) = $Person;
	my $Sorted = pack("A12 A4 A7 A5 A*", $Name, $ID, $Birth, $Sex, $Race);
	echo "$Sorted\n";
}
# Prints
# Name        ID  Birth  Sex  Race
# Mike        0   1980   M    German
# Tommy       1   1985   M    French
# David       2   1950   M    Jewish
# Sukara      3   1990   F    Japanese
# Mary-Kate   4   1987   F    American

Using Unpack() to read from formatted ASCII text and extract information from it similar to Regex but much more reliable for this specific task 

# Lets imagine this variable was loaded from a file
# It contains information that is using specific spaces
# to keep it organized so it can be displayed on the
# company computers properly
# We are going to read this information and process it
# You could use regexp or splitting but they would be
# impractical so lets use Unpack() instead
my $data = 
"
Date      |Description                | Income|Expenditure
01/24/2001 Zed's Camel Emporium                    1147.99
01/28/2001 Flea spray                              24.99
01/29/2001 Camel rides to tourists      235.00
01/29/2001 Tourist camel feedings       100.10
";
 
my $TotalIncome = 0.0;
my $TotalExpenditure = 0.0;
foreach(Lines($data) as $line)
{
	if(isEmptyOrNull(trim($line)))
		continue; # Skip useless entries
	my List( $date, $desc, $income, $expend  ) = unpack("A10/x/A27/x/A7/A*", $line);
	$income = trim($income); # Trim it so we can do arithmetics on it
	$expend = trim($expend); # Trim it so we can do arithmetics on it
	$TotalIncome += $income;
	$TotalExpenditure += $expend;
	# Print it just so prove we are reading it correctly other than that
	# it is pointless
	echo "Log: '$date' | '$desc' | '$income' | '$expend'\n";
}
echo "Total income is: $TotalIncome\n";
echo "Total expenditure is: $TotalExpenditure\n";
# Prints:
# Log: 'Date' | 'Description' | 'Income' | '|Expenditure'
# Log: '01/24/2001' | 'Zed's Camel Emporium' | '' | '1147.99'
# Log: '01/28/2001' | 'Flea spray' | '' | '24.99'
# Log: '01/29/2001' | 'Camel rides to tourists' | '235.00' | ''
# Log: '01/29/2001' | 'Tourist camel feedings' | '100.10' | ''
# Total income is: 335.1
# Total expenditure is: 1172.98

Example of choosing names for KEY in the return array 

$binarydata = "\x04\x00\xa0\x00";
$array = Unpack("cchars/nint", $binarydata);
// The resulting array will contain the entries
// "chars" with value 4 and "int" with 160. 
printr $array;
// Prints:
// ARRAY
// {
//         [chars] => 4
//         [int] => 194
// }

Same as above 

$binarydata = "\x04\x00\xa0\x00";
$array = Unpack("c2chars/nint", $binarydata);
// The resulting array will contain the entries
// "chars1", "chars2" and "int". 
printr $array;
// Prints:
// ARRAY
// {
//         [chars1] => 4
//         [chars2] => 0
//         [int] => 49824
// }

Packing a bunch of values at once 

$binarydata = Pack("nvc*", 0x1234, 0x5678, 65, 66);
// The resulting binary string will be 6 bytes long
// and contain the byte sequence 0x12, 0x34, 0x78, 0x56, 0x41, 0x42. 
printr $binarydata;
// Prints:
// {BINARY:6}
// {
//         [0] => 18
//         [1] => 52
//         [2] => 120
//         [3] => 86
//         [4] => 65
//         [5] => 66
// }

Using * to pack all the remaining objects with the same specifier 

printr pack("C*",80,72,80);

String to Hex and back again 

function H2Str( $hex ) 
{
	return pack('H*', $hex);
}
 
function Str2H( $str )
{
	return unpack('H*', $str, true);
}
$txt = 'This is test';
$hex = Str2H( $txt );
$str = H2Str( $hex );
echo "${txt} => ${hex} => ${str}\n";

Display the ASCII character codes for an entire string 

echo join (unpack('C*', 'abcdef'), ' ');
// 97 98 99 100 101 102

Display the UNICODE character codes for an entire string 

my $Str = 'こんにちは';
say join (unpack('U*', pack('U*', split($Str, ''))), ' ');
// 12371 12435 12395 12385 12399

Convert a string into a binary array and back again: 

$arr = BinaryFromStr("Hello World!");
foreach ($arr as $i)
{
	println($i);
}
$str = Unpack("A*", $arr, true);
println($str);

Convert a double into a binary array and back again: 

$arr = Pack("d", 777.42);
foreach ($arr as $i)
{
	println($i);
}
$str = Unpack("d", $arr, 3);
println($str);

Convert an int into a binary array and back again: 

$arr = Pack("i", (int)777);
foreach ($arr as $i)
{
	println($i);
}
$str = Unpack("i", $arr, 3);
printr($str);

Convert a string into a hex and back again: 

$str = "Hello World!";
println("Original String: " . $str);
$hex = Unpack("H*", $str, true);
println("Hex String: " . $hex);
$strf = Pack("H*", $hex);
println("Normal String: " . $strf);

Convert a string into a hex and back again (using a bit of Regexp too) 

Function StrToHex ( $Str )  
{
	$Str = unpack('H*', $Str, 3);
	# Operationally convert the hex characters
	# to upper case
	$Str =~ tr/a-z/A-Z/;
	return $Str;
}
Function HexToStr ( $Hex )
{
	$Hex =~ s/([\dA-Fa-f][\dA-Fa-f])/pack("C", dec($1))/eg;
	return $Hex;
}
my $Hex = StrToHex("Hello world!");
echo "Hex: $Hex\n";
my $Str = HexToStr($Hex);
echo "Str: $Str\n";

Using pack() to check if the system is Little-Endian or Big-Endian 

# A hex number that may represent 'abyz'
my $abyz = 0x6162797A;
# Convert $abyz to a binary string containing 32 bits
# Do the conversion the way that the system architecture wants to
switch (pack ('L', $abyz))
{
    # Compare the value to the same value converted in a Little-Endian fashion
    case pack ('V', $abyz):
        echo 'Your system is Little-Endian.';
        break;
    # Compare the value to the same value converted in a Big-Endian fashion
    case pack ('N', $abyz):
        echo 'Your system is Big-Endian.';
        break;
    default:
        echo "Your system 'endian' is unknown.";
        break;
}
Retrieved from "http://ubersoft.org/Sputnik/wiki/index.php/Core_Function_Pack"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox