A Message in a Socket – Interprocess Communications by Example

Introduction

I couldn’t help but feel like a luddite as I sat down to write down this article. Who works on Flash applications in 2016 any more? Aren’t we all done with it now?

The Flash Player might have its own set of problems, which is a topic for another discussion. The ActionScript language, however, is fun to work with. It is syntactically similar to Java, has a class-based inheritance model, a choice between static or dynamic typing, or both, within the same code base, namespaces, and a host of easy to use and powerful APIs for 2D and 3D graphics, network operations, animation, audio processing, XML and text processing, and more. Frameworks like the open source Flex SDK make it possible to do even more interesting things on top of this.

But, more importantly, we are heavily invested into the Flash ecosystem for the VMX platform at Vertika. We have 9 years of effort and 40,000 lines of code written in ActionScript and MXML, which would be very expensive to port over to a new platform without a very compelling business reason. So here we are today.

Unfortunately, the Flash Platform also comes with its idiosyncrasies, one of which is addressed in this article.

For those losing interest at the mention of Flash, the good thing about this client architecture is that it can easily be ported over to literally any other platform with little change and still work. And the server-side stuff is written in C#. Collectively, this solution covers a plethora of different concepts such as socket programming, data structure design, a custom-built lightweight ORM framework and design patterns.

Raison D’être

The Great Catsby is a rather upset feline. He is fond of company and loves nothing more than to kick back with a saucerful of heavy cream milk and a stack of Calvin & Hobbes comics with his human pals. But they just won’t let him out of his cage. Catsby, you see, is a 300 kilogram Bengal Tiger at the zoo. And no matter how much he pleads, they are not going to let him out and explore the city.

It’s a rather bleak state of affairs for the poor cat.

The guys at Adobe (then Macromedia) had the same ideas when they designed the Flash Player, which was designed primarily as a browser plug-in to play specially authored content served off websites. And as every competent security-minded person knows, it can be a very bad idea to allow strange websites to gain unrestricted access to your computer. Even if every ActionScript programmer pinky-swore not abuse their users’ trust, the guys at Macromedia were not going allow them free rein over the computers running the Flash Player. Hence, Flash content runs in a sandbox with several restrictions, one of them being no direct access the file system.

This is a tough situation for many programmers who need any kind of runtime application logging. No file system access means there is no persistent data storage on the client.

It is not that there’s no logging at all. The Flash Player Debugger exposes a very limited, plain-text log file access that is disabled by default, and requires a configuration flag in a specific plain-text file in order to be activated. The state of this flag cannot be queried or controlled from within the Flash Player. Extracting the file is a task and a half because it’s buried deep within the file system hierarchy. The log file is shared among all Flash Player applications which are executed on that computer. While there is no possibility of data conflicts, filtering and extraction of the data that is relevant to your application becomes a challenge. Finally, since this is a plain-text file, there is no structured information storage facility available here other than what can be achieved with plain text files (such as comma-separated values).

Did I mention that even this facility is only available on a special Debugger edition of the Flash Player that most people don’t use?

In effect, it’s impossible to have any kind of reliable, persistent runtime logging on a production computer. A third-party intervention is essential. And this is the challenge that is addressed in this article – a logging framework that allows Flash content to delegate the task of persistent storage to another process, which runs at a higher privilege level and has access to the file system. The persistence process can be running locally on the same computer, or on another computer. The mechanism works identically as long as the two computers are able to establish a connection with each other over standard sockets.

Hi! It’s me, your brother!

Parents are like the operating system of the family. They spawn child processes, manage resources and schedule time for tasks. A well-organised family is like Unix. Every child gets their own space, and nobody gets into each other’s way. When they have to communicate, they send messages to pre-designated points. These could be a cell phone, a sticky note on the fridge or a whiteboard in the hallway. They could also speak across the dinner table, but nobody does that in 2016. So that’s out.

Speaking over the cell phone is more versatile because it works whether the person is at home or away at work, out running an errand, or at a party. A message on a sticky note is only delivered when the person returns back home and checks the fridge.

Operating systems also offer similar means to communicate between processes. You could send a message to the recipient over a socket, or write to a file that the recipient checks periodically. Since we have already discussed earlier that writing to files is out, that leaves us with socket communications.

Fortunately, the Flash Player has an API to communicate over TCP sockets. It works identically when communicating with a different device or another process on the same computer where the Flash application is being run. The Flash Player Socket API can only make client-side calls. It cannot listen for connections from another process. Therefore, the external application must play the role of a server and begin listening for connection requests from the Flash Player on a known port number. Once the connection is established, the two processes can send messages back and forth.

This mechanism can be utilised to create an external logging tool that listens for messages from a Flash Player client, and logs them to a persistent storage such as hard disk. Since the socket API works locally as well as remotely, the logging server can be situated at an offsite location, and as long as the two computers can communicate with each other, log messages will be successfully delivered and logged.

Flex Logging API

A lot of work that is needed to establish a robust logging API has already been implemented as part of the Flex SDK in form of the Logging API, which is loosely modelled on its Java framework equivalent. This API provides a convenient way to create structured log messages with severity levels and named categories. A log message is triggered by simple API calls, and is printed to a destination by a target object. Targets can be programmed to listen for messages at a certain severity level (and above), as well as certain categories only. For example, low-latency logging such as an in-memory temporary store can be used for diagnostic messages which are triggered more often, while infrequent but critical errors can be dispatched to remote persistent storage.

The SDK ships with two in-built logging targets – LineFormattedTarget and TraceTarget – which provide basic logging facilities to the application. TraceTarget prints the message to the Flash Builder or fdb console and extends LineFormattedTarget.

The Flex Logging API is documented elsewhere in sufficient detail. However, I’ll cover it briefly in this article to provide context to the reader for how it is used.

The API is centred around four types.

  1. mx.logging.Log
  2. mx.logging.ILogger
  3. mx.logging.LogEvent
  4. mx.logging.ILoggingTarget

The SDK ships with a mx.logging.AbstractTarget class that implements the ILoggingTarget interface. Developers can extend this class to build their own custom logging target implementations.

How the Flex Logging API Works

The ILogger interface provides methods to send messages to one or more targets. The developer doesn’t create the logger instance directly. Rather, they call upon the getLogger static method of the Log class to retrieve an ILogger implementation instance. The getLogger method takes a string parameter called category, which can be used to filter log messages down to a particular sub-system within an application.

The category is conventionally set to the fully-qualified name of the class that calls the ILogger API.

The ILogger interface also exposes methods to perform log operations at five different severity levels, and a generic log method that takes the severity level as a parameter.

The severity levels in increasing order are DEBUG, INFO, WARN, ERROR and FATAL.

A logging target receives the log message and prints it to the destination medium. The logging target has a property called level, which determines the severity of messages that it will receive.

The target receives all messages of its assigned severity level and below. For example, the target instance in the example above would be able to receive DEBUG and INFO messages.

A logging target instance also has a filters property of type array that contains the categories which this instance should listen for.

This snippet enables the target to receive all log messages whose category is set to “com.notadesigner.Example”. The filters array can contain more than one category.

The target is actually of greatest interest in this article, because it is this class that performs the message printing. In order to send the message to an external process, we need to build a custom logging target that is able to dispatch messages over a socket.

com.notadesigner.sockPuppet.SocketTarget

The SocketTarget class extends AbstractTarget rather than LineFormattedTarget, because the latter merges all the message fields into a single string, which we don’t want. By keeping them as separate fields of their native types, we get maximum control over the structure of the message, as well as the least amount of memory usage.

We begin by instantiating the class.

The constructor takes two parameters for the host name and the port number on which the socket server is already ready and running. We will gloss over the details of performing the connection and maintaining it.

When a message is received, the logging framework triggers the logEvent method of the SocketTarget instance. This is defined as an empty method in the AbstractTarget class that takes a parameter of type LogEvent. By overriding this method, the SocketTarget instance can then dispatch the message to the server.

Making It Real

Before Daryl got into the programming business, he used to wait at the local deli. His job was to take orders from diners and pass them on to the kitchen, and serve the completed order after it was prepared by the kitchen staff. Daryl’s task was simple. He would jot down a list of orders from the table and pass it on to the chef. The chef would then prepare whatever items came up in the queue, and ring up the waiting staff once it was ready. The staff picked the completed order and served it fresh and hot to their patrons.

You want a burger and fries? Sure! Footlong with everything? On its way, sir. Need coffee with extra cream? Here it comes.

The food was good and footfalls were high. The entire team loved their job and did their best to stick to the process because it was so efficient. It abstracted away the underlying complexity of preparing a meal with all its customisations into a simple, mechanical task, while still retaining all necessary details that the kitchen staff would need to build an order.

In the years that followed, Daryl was on his way to become an accomplished software developer of some fame. It was during this period that he learned about design patterns in software engineering and came across the command pattern. In his mind, he could relate immediately.

A command is a reified method call. – Bob Nystrom, Game Programming Patterns

I find Bob’s explanation to be much more relateable than the longer definition defined in the GOF book. There’s a beauty in its terseness. To understand it, of course you need to understand what reified means. To borrow another quote from Bob’s book

“Reify” comes from the Latin “res”, for “thing”, with the English suffix “–fy”. So it basically means “thingify”, which, honestly, would be a more fun word to use.

So there we are. The command pattern converts an abstract item like a deli order into a real, fresh, piping hot meal. I’m sure even reading about this makes you hungry. Maybe you would like to repeat the experience right about now, and issue a command to your local delivery joint for a large pizza with Coke. Go ahead. I’ll wait.

Are you back? Let’s continue.

Returning to the task at hand, we are now at a stage in the logging operation where the message is to be dispatched to the socket server process. The Flex framework already provides all the necessary structured information in an event handler – the logEvent method – that is overridden by the SocketTarget class. It is a matter of sending this information to the external process, so that it can be logged to disk. Reify the log, if you please. We need some infrastructure in order to implement this mechanism.

A Standard for Commands

The invention of the assembly line provided a massive boost to human progress during the industrial revolution. It was a break from the previous practice of having few master-craftsmen assembling a product from start to finish. By delegating each part of the assembly to a single individual, the human skill required was greatly diminished and productivity soared. Rather than having one exceptionally good worker build a single sword in a day, factories were able to assemble dozens by assigning several average-skilled workers, each handling only one aspect of the assembly process. All workers just followed one instruction – assemble the product. But what each did in response to this instruction was different. Somewhat akin to what we do with the ICommand interface.

This interface unifies all commands so that they can be guaranteed to have a single method for execution. The ActionScript implementation looks like this.

The obvious follow up is why have a command interface at all when the only operation we need is to trace messages. However, it’s a small leap from where we are now (sending plain-text messages for printing) into communicating more elaborate commands such as enabling and disabling logging, force-flushing the log to disk, clearing the log or sending other structured diagnostic information such as internal data structures. It becomes easier to swap out one command for another, much like the assembly line, if they all must adhere to a common interface. All these commands can be triggered from the Flash application by instantiating an appropriate command class instance, serialising it, and dispatching the result over the connection.

The details of the execution of the command are specific to the command. The common operation is serialisation of the message from a Flash object to something that the server can read. For this, we use a byte-array because it is compact and efficient to send over a socket.

Trace generates a certain kind of layout in the byte array, which contains the string to be traced in the message log. Another possible message might be to toggle the state of the logging operation itself. In this case, the byte array could only require a Boolean value to indicate the toggle state. A more complex diagnostic message might send across complete object instances after serialisation, along with information on how they are to be deserialised again.

While message specifics are different, they all share a common structure called the message header. The header in the current version of this API requires the following elements.

  1. Message length (32-bit integer) – The total length of the message in bytes (including the header and all parameters).
  2. Instruction (byte) – An 8-bit integer value that contains the instruction code. Each command has a unique instruction number associated with it. This makes the message more compact than sending the human-readable command name as a string.
  3. Message timestamp in UTC (32-bit integer) – The date and time that the message was created in the Flash Player, encoded as seconds since the UNIX epoch. The time is calculated relative to UTC and must be converted into local time as an additional step by the receiver of the message, if needed.
  4. Number of parameters (32-bit integer) – A count of the number of parameters attached to the message.

The instruction for the Trace command is assigned the value 0×02. The byte array needed to trace the string “Hello” at severity level INFO is shown in the table below. Each cell represents 1 byte.

0×00 0×00 0×00 0×20
Message Length
0×02
Instruction
0×57 0×CD 0×AC 0×31
Timestamp
0×00 0×00 0×00 0×03
Parameter Count
0×00 0×04
String Length
0×49 0×4e 0×46 0×4f
Severity Level (INFO)
0×00 0×04
String Length
0×4d 0×61 0×69 0×6e
Category (Main)
0×00 0×05
String Length
0×48 0×65 0×6c 0×6c 0×6f
Message (Hello)

The first 4 bytes contain the message length (32 bytes). The next 1 byte contains the instruction code (0×02). Four bytes are allocated to sending the timestamp in seconds (1,473,096,753 seconds since 1 January 1970). The next 4 bytes contain the number of parameters attached to this message (3). The contents of the rest of the message are specific to the Trace command. It is made up of 3 data fragments that encode a string prefixed by its length as a 16-bit integer. A string is an array of characters. C-style strings mark the end of the array by placing a null character in the last position of the array. When a piece of code needs to perform any string operation, it walks the length of the array until it arrives at the null terminating character. Alternative to this are length-prefixed strings, or Pascal strings, which prefix the length of the character array at the beginning. Pascal strings do not require a null terminating character as the length of the string is already known beforehand.

This kind of data structure has an advantage of speed over null-terminated strings. Any operation that requires the length of the string can peek into the beginning of the sequence, making it a constant time operation. Finding the length of null terminated strings, on the other hand, is a linear operation because it requires walking the entire byte array to locate the null terminator. Longer strings take more time, shorter strings take less.

The writeUTF method of the ActionScript ByteArray class uses Pascal strings. Programmers do not have a choice in this matter. Therefore, all commands, present as well as new ones in the future, use this same structure to encode strings.

A Pregnant Pause

Once the message has been serialised into a byte array, the send method of the Socket class is used to dispatch the message over the network.

At this point, the message is out of the realm of the Flash Player and is crossing boundaries into the .NET runtime. If the server at the other end receives the message in full (which is more or less guaranteed due to the TCP stack), it can be parsed, deserialised and acted upon by the server. If the message fails for any reason, the Socket class raises an error event within the Flash Player for the programmer to handle. For brevity, we ignore that aspect and proceed with the assumption that the message has been received successfully and in full. Transparent to the ActionScript programmer, the server dispatches an ACK response to acknowledge receipt of the message. There is nothing more that the Flash Player has to after this point and it can return back to its steady state.

Keep watching this space for part two of this article that explores receiving the message on the server, interpreting it into its component parts, then acting upon the instruction contained in it.

What’s in a Name?

Naming has been acknowledged in many texts as a difficult aspect of practical computer programming. The problem with assigning appropriate names for entities is that it is not quite as cut and dry as most other aspects of the craft. The rules are not very objective. Choosing a good name requires applying a certain amount of heuristics drawn out of experience and intuition.

Good names are easy to overlook because they stay out of the way. But a smart programmer can learn much about the kind of naming schemes to avoid simply by having to maintain code with poor quality naming conventions. Badly named variables can make this task so difficult that there might be some merit in the idea of exposing programmers to such a maintenance exercise specifically to teach them this valuable skill.

We look over a simple example of how poor naming conventions inhibit the design, and conversely, how a well chosen name can make short work of extending the application in the future.

This signature declares a method that takes an integer spoke count as input and returns a Wheel instance. Fred calls the method in the following manner.

This gets flagged by his pair programmer Dave for using a magic number. So Fred changes the code.

It isn’t the most elegant snippet they have encountered. But both Fred & Dave agree that this code no longer uses a magic number. So they commit the code and go home.

Two years later, the specifications change. The client can now build lightweight wheels with just 16 spokes. Fred is no longer on the team. Susan, the maintenance programmer, takes up the task and inspects the code. She finds the line where the number of spokes is declared and edits it.

Technically, the code is fixed. But it has deteriorated a bit more in quality and maintainability.

A few months later, the client sees resurgent demand from their customers for high-spoke count wheels due to their durability and higher resilience to handle bad surfaces. So they wish to make the number of spokes a customisable option that can either be 32 or 16. Two weeks into the modification they also release a new 12-spoke wheel. The code now looks like this.

This is a very minor, localized problem in the code. It has relatively negligible impact on the performance of the application. Dozens of minor warts such as this fester in any sizeable code base. Programmers are human too, and make mistakes sometimes. Maybe the deadline inched too close and they had to roll out right that minute. The key is to identify and fix those code smells as soon as they’re encountered. Queuing them up for a grand refactoring project risks turning these minor issues into full-blown architectural challenges.

Things often get trickier when such names are used in giant mudballs with extremely wide scope, such as singleton objects or global buckets. Nobody wants to read the code surrounding all 28 references to the constant THIRTY_TWO, without which making the change would be extremely hazardous. So THIRTY_TWO continues to live on in the code base, forever doomed to contain just half of what it denotes.

This type of problem can be nipped in the bud very easily by simply spending a moment thinking about the names of objects. In this case, the method signature itself provides a hint for the correct name of the parameter – spokeCount. Then Fred’s iteration would have looked like this.

When Susan comes in to add two new spoke count options, she realizes that the constant SPOKE_COUNT already exists. This nudges her into defining two more constants – LOW_SPOKE_COUNT, MID_SPOKE_COUNT. If Susan uses a reasonably modern IDE, it would also allows her to rename the old constant to HIGH_SPOKE_COUNT.

Suddenly, by simply choosing a name that better reflects the purpose of the entity rather than its value, the application programmer has made its intended usage clearer, and nudged future maintenance programmers towards using similar appropriate names to new code that they write.

A Guide to Effective Version Control

We have come a long way since 2008 when I wrote this introductory article about version control systems. Their rising popularity in the past few years is a notable sign of maturity in the software development community. Whereas they were once a bastion of only “serious” development operations, now even app developers working alone use them regularly.

In this article, I talk about how a team can use a version control system to its advantage by describing a preferred way to store the files in the repository and some processes to complement the structure.

There are myriad ways of organizing a version control repository. It is a sophisticated file system, after all. A well-structured repository can be a valuable asset to a software team by creating a clearly defined workspaces for programmers, opening up the opportunity to create a continuous integration pipeline, and standardizing deployment.

Repository Layout

In my preferred layout, the trunk is used for the main development line. In addition, branches are created for releases and feature sandboxes. This style of organization is loosely based on trunk-based development.

The Trunk

All the primary development of the project occurs in this directory. The trunk is further organized into sub-directories for every project that constitutes the final product. For example, a product might consist of separate web and desktop components. Or it might have a dependency on a separate static or dynamic library. The exact arrangement of this is determined by the programmers and release engineers working together. All these projects reside side-by-side in the trunk.

A feature or bug fix is considered open until the code has not been committed to the trunk. Hence, all developers have access to this directory and must make it a regular part of their daily work.

trunk

Maintenance Branches

Once the product reaches a milestone and is deemed ready for release, the release engineer is responsible for generating a binary and deploying the product. But the first step is to make a maintenance branch. This branch is a snapshot of the trunk that can be used by developers to provide bug-fixes to an already deployed product without accidentally releasing new features.

It is named according to whatever convention the team decides (I prefer a numeric x.y scheme).

branch

If bugs are discovered in the product post release, they are patched in the trunk and merged into the active maintenance branch. Other developers, meanwhile, can safely continue adding new features on the trunk without inadvertently bringing them into the current deployed versions.

A bug-fix release is made out of the maintenance branch after a bug is patched.

It is important that bug fixes should originate in the trunk and later be merged into the branch rather than the other way around. If the fix is made in the branch and the developer forgets to merge it back into the trunk, it can cause a regression later down the line. By patching in the trunk, the developer uses the process to automatically ensure that patches exist in both places.

Tags

Release binaries should always be made out of a clean export from the repository. But which revision should one export? If you’re creating a maintenance branch, you could use its HEAD revision. But you’re out of luck if you need to generate a previously released build again after subsequent edits have been made to the branch. Unless you tagged the branch before generating a binary.

A tag is a branch that everyone agrees to never edit. It is given a unique name from the rest of its siblings. My preference is to go the x.y.z scheme, where the x and y components of the name come from the maintenance branch from which it derives its lineage.

tag

The z component of the tag name is set to 0 when the first maintenance branch is made. This number is incremented whenever subsequent tags are made out of the maintenance branch.

Using the Repository

Once the structure is ready, some processes need to be put in place in order to utilize it. The typical developer role has it easy – keep committing error-free code into the trunk, and make sure the working directory is updated from the trunk as often as possible.

Feature Releases

A new feature release begins when one of the managers green-lights it. The repository is locked down temporarily and a maintenance branch and a tag are created from the head revision of the trunk. These two copies are named according to the version convention in place for the project.

This maintenance branch supersedes all previously generated branches, which may even be deleted from the repository. The repository is then unlocked and developers can go back to making commits in the trunk.

Maintenance

When bugs are discovered in a released product, the developers investigate in the trunk and attempt to fix it there. If found and fixed, they notify the release engineer who then cherry-picks the relevant commits and merges them into the current maintenance branch.

The maintenance branch is tagged after it is deemed ready for release. The binary generation and deployment process remains the same – clean export from the newly created tag, followed by compilation and deployment.

If a bug cannot be located in the trunk, the developer may have to look for it in the maintenance branch and fix it if it is found there. In this case, the bug fix may be merged back into the trunk if deemed necessary.

Release Generation

The release engineer’s work is not done yet. The release binaries are yet to be created from the newly created tag. For this, the tag is exported from the repository into an empty directory. This is important so that no unnecessary files are accidentally added into the binaries. This step cannot be emphasized enough.

All binaries must be generated from a clean and complete export from the repository.

Generating binaries from partially exported directories runs a high risk of including incorrect or outdated files into the binary. This inaccuracy can be the source of impossible-to-reproduce bugs or regressions. At the minimum, it is a sign of a poorly managed process.

Once the binary is generated, it is packaged to its final consumption form – an archive or installer for distribution – then deployed to wherever it is needed (such as a file server or web server).

The binaries and installers may be added into the repository for posterity.

Asynchronous AIR Native Extensions

What could be worse than attempting to build an image processing application in ActionScript? While the language has its positives, handling large data sets quickly is not one of them. This was the dilemma we were up against in a recent assignment. And the images themselves were expected to be hot off the camera sensors. At a time when even tiny point-and-shoot cameras regularly reach 12 megapixels, our target audiences of photographers and studios were expected to hit us with a minimum of 24 megapixels.

That works out to 72 megabytes of data per image. Now multiply that by a few hundred images and you know why we were very concerned. The task in question was not very complicated – just resizing images. But the volume of data ensured that even this mundane operation took forever to complete, without any feedback to the user because ActionScript runs on a single thread.

We flirted a bit with ActionScript Workers, but they were an incomplete solution. Our UI thread became available again, but the processing still remained unacceptably slow.

Our fallback came in the form of Adobe’s Native Extension API, that allows ActionScript applications running in the Adobe Integrated Runtime to access libraries written in C or similar languages.

Well, that’s that then. This task was easy enough. Bang out some functions to resize and encode images and make a call from ActionScript. Since it was native, it would be fast and the user would never even notice the pause.

Unfortunately, it didn’t turn out that straightforward.

While this operation became faster by an order of magnitude, it still stuttered when loading really high resolution images. And when multiplied by a hundred or so photos to be batch processed, the frequent freezing of the UI was very obvious.

So it was back to the drawing board.

Asynchronous Processing

The native extension API offers the ability to dispatch events from the native code back into the runtime. This has been designed precisely for such situations. The extension spawns a new thread that handles the processing and returns control back to the runtime on the UI thread. After the computations are complete, the native extension dispatches an event that the runtime listens for.

To illustrate, let us implement a simple native extension that runs a timer and notifies the runtime when the allocated time is up. For simplicity, this timer will run for ten seconds. It can be made to run for arbitrary durations by passing the interval value as a parameter to the native side. We’ll call this extension the Ten Second Timer. Its ActionScript class definition is as follows.

This class extends EventDispatcher. Collectively, the client, TenSecondTimer and the native code set up a chain. The client listens for events from TenSecondTimer, which in turn subscribes for events from the native code. When the native code dispatches an event, TenSecondTimer creates its own Event instance and dispatches it. The client thus receives an indirect notification from the native code through the TenSecondTimer class.

On the native side, the start function is implemented with the function signature required of all native API functions.

When this function is invoked by the runtime, it spawns a new thread (using the pthread API in this case) and immediately returns control back to the runtime. A reference to the waitForDuration function is passed to the pthread_create function. The newly created thread executes that function.

The waitForDuration function calls the usleep API that suspends the thread for 10 seconds. The CPU wakes this thread again after this duration has elapsed, and the function dispatches an event through FREDispatchStatusEventAsync.

In order to notify the runtime, the native coded needs to maintain a reference to the context. The context is passed as a parameter by the native extension API to the context initializer function. This function must store the reference somewhere that is accessible to the thread. A global variable works nicely in a pinch.

The runtime then kicks back into action, and passes on the event notification to the TenSecondTimer class. The context_statusHandler method is triggered, which in turn dispatches a complete event for the client to handle.

This pattern of triggering actions in the runtime from the native code can be used for a variety of other tasks that require asynchronous execution. The function invoked by the thread can perform whatever task may be required of it in place of the usleep call.

In our case, we implemented the native extension method to resize images asynchronously. Since loading the original image was taking the most amount of time, it went straight into the separate thread. The thread also took care of resizing the image after it was loaded and saving the resized file back to disk.

Packaging AIR Native Extensions

Native extensions are a handy improvement to the AIR ecosystem. There are numerous benefits of being able to to drop into some C code for processor-intensive tasks or extending the Flash API into domains outside of the standard library provided by Adobe.

But anybody who has had to deal with developing their own extensions knows that the packaging process is less than straightforward. It requires maintaining at least two different code trees – one for the native portion of the extension, and the other for the ActionScript library that glues the client application to the native portion. The two projects have to be compiled separately – the ActionScript library through the compc tool, and the native portion with its relevant compiler. Finally, both binary outputs have to be packaged into a single ANE file, which is equivalent in structure and function to a SWC.

In addition to all these, the ADT tool which packages the ANE has its own quirks with relative paths (learned at the school of hard knocks), which are best dealt with by placing all the files required for the extension into a single directory rather than scattered across different directories on the hard drive. Which means copying all the various output binaries and XML files into a single directory every time they are changed.

Running all these activities by hand is time consuming and error prone. However, they are mechanical tasks that computers are well-suited to handle. The following batch script is designed for a Windows build chain that compiles both source trees and packages them into an ANE for consumption into the client application.

  1. Delete the previously generated binaries to begin with a thoroughly clean slate.
  2. Create a new designated build directory. This is the directory where your binaries are copied into and packaged into an ANE.
  3. Compile the ActionScript library using the compc tool. It becomes much easier if you pass a flex-config.xml file as a parameter to the tool rather than passing every setting as a parameter to the command line.
  4. Copy the output SWC into the build directory. Alternatively, the compc tool has an -output parameter that can be used to specify the location and filename of the generated binary.
  5. Extract the library.swf file from the SWC generated in the previous step into the build directory. Unfortunately, Windows does not ship with a native command line utility to extract files from an archive. Alternatives such as 7-zip help fill up that gap.
  6. Compile the native code using your compiler toolchain of choice.
  7. Copy the native runtime library along with a previously authored descriptor.xml into the build directory.
  8. Package all these files into an ANE using the ADT tool. Invoke the ADT command from inside the build directory.