Return to the main Texapp page

Texapp Advanced Usage: The Texapp API

The Texapp API is relatively new and is subject to change. While the API is anticipated to remain mostly stable, it will not be frozen until 1.0.0. Similarly, for users who have already developed or use TTYtter extensions and bots, the Texapp API (while ultimately based on it) has already diverged in several fundamental respects and may do so again. Check back on this page for changes from version to version until the API is final. If you are porting TTYtter extensions, consult this brief comparison of APIs.

The Texapp API is your way of integrating custom functionality inside of Texapp, turning it from just a client into a fully realized ADN platform. Using the Texapp API, you can almost totally customize how the client interacts with you and displays posts and direct messages, add your own custom commands to the client, drive Texapp as a completely scripted command line tool, or use Texapp as a rapid-deployment engine to construct ADN bots. Texapp takes care of posting and fetching posts and PMs, and your application can supersede as much or as little functionality as it needs to. You don't even need to write loops or handle any actual transactions with ADN unless you really want to.

Texapp's architecture

Texapp uses a bifurcated model: asynchronous timeline and private message channel fetches occur on a background process which receives IPC commands from the foreground process where the user interacts. The IPC link is bidirectional, so the foreground can ask the background for information on posts and PMs it's collected, as well as synchronize behaviour through both real-time signals and deferred command processing.

In the sections below, certain options apply specifically to either one environment or the other, and certain operations are affected by IPC. However, the net goal of the API is to make this plumbing largely transparent. Just be aware of it when we look at some of the deeper guts.

Relevant command line options

Here are the command line options relevant to the API (see the full list of command-line options):

-exts=[extension to load,extension to load,...]
Specifies a list of (recommended) fully qualified filespecs, separated by commas, which are passed to the multi-module loader (we'll discuss the loader below). Essentially the extensions are directly required by Texapp and become part of it. Your extension must return a true value at the end (an idiom like 1; as the last line will suffice nicely). The order of extensions is salient, as it determines their load order and call order (we'll discuss that when we get to the API reference).

The actual "methods" and globals you can and must use are discussed below, but I strongly advise reading this whole page first.

-extpref_*=[option] (optional)
These options become instantiated inside Texapp by name and can be used to pass additional arguments relevant to your extension (e.g., -extpref_myextension_logfile=foo.log, which then can be seen as the global $extpref_myextension_logfile). Good practice indicates that you give yourself as unique a namespace as reasonable with these; although the option name might get long, they are globals, so you should try to avoid any collisions. You can put these in .texapprc, though they cannot be set with /set or displayed with /print (but they can be accessed, like anything else, with /eval).

Environment variables can also accomplish a similar effect, but using -extpref_* avoids any unusual restrictions your OS might set on your environment, which are usually more restrictive than your command line, and keeps it clutter-free. Plus, the current value of any global with $extpref_ is written out to .texapprc for you when Texapp's state is saved, so these variables can act as a permanent store (read on).

-daemon (optional)
Forces Texapp to run as a "detached" process in the background (the PID of the new background process is reported). With no extensions loaded, this is as if one were running the regular background update process, but without a console process (so updates continue to appear, but you must use a command line incantation like texapp -status=... or something similar to actually post, as you are in your shell and not Texapp). You must kill the process manually to shut it down.

If an extension is defined, then this is the basis of how to construct a fully automated bot (we'll talk more about shortly), as Texapp will periodically at the specified interval automatically fetch posts and PMs and hand it to your extension(s). In this situation, there is no foreground; your extension runs as part of the background, which has some minor limitations on functionality.

You might also specify -silent for independent bots.

-script (optional)
Forces Texapp into a fully deterministic "scripted" mode, where fetches do not occur automatically, user activity is suppressed, -silent and -noansi are forced on, and all activity is entirely determined by the extension(s) loaded. Generally not particularly useful without an extension specified, -script mode would be used for driving Texapp entirely from the command line, or as a tool to be called from things like cron jobs and CGIs. In fact, see below for -runcommand, which is the perfect option for those sorts of tasks.

This mode is so important for Texapp that extensions can specifically request whether they want -script on or off. Because -script requires all fetches to be specifically requested by the extension, -script and -daemon are incompatible.

-runcommand=[command to run] (optional)
Runs a single command as if you had typed it at the terminal. Implies -script.

I have always found it easiest to learn by example, so let's look at some simple but useful examples before we get to the API reference. It will make a lot more sense if this is your first time. Before we do, however, consider this next section before rolling up your sleeves ...

Before getting started: Simple command line queries, piping to Texapp, and -runcommand

Sometimes you don't have to write an extension at all. If you are simply requesting data that you can get with regular Texapp commands, then simply piping commands to Texapp would be enough. Since this way you are completely controlling what you want to fetch and do, you should use the -script option. The examples below assume you already have your API keyfile created in the default location.

For example, suppose you just wanted to find out the last 20 posts of user dalton and then grep it for something:

echo "/again dalton" | texapp -script | grep -i banana

Notice that this uses -script. The -script argument, as you will recall, disables automatic updates, sets -silent, and disables ANSI colour and other settings inappropriate for not-a-terminal.

In the above example, if you're simply interested in fetching specific types of posts or filtering out others, a -filter option and a set of hashtags could be all you require. See Command-line options.

Obviously, you are not limited to single commands; you could have an entire command file if you like, and just pass that in with redirection or as an argument (texapp -script my_script_file). However, you can make your requests very compactly if you are only sending a single command by using -runcommand. Instead of fetching your timeline like this,

echo "/a" | texapp -script

you can make the request without the pipe by using it as a parameter to -runcommand:

texapp -runcommand="/a" # this is faster too

which, as you remember from above, sets -script automatically and runs the single command you want for you in one step. As another example, consider what's in my crontab: texapp -runcommand="/replies"

If the command you want to pass to -runcommand requires a menu code, you can simply provide it a post ID or a PM ID instead (to pass a PM ID, prepend the ID with d to disambiguate it, since the ID numbers can overlap).

Similarly, posting from the command line can either be done by piping posts to Texapp, or using -status (and/or -hold to make the post bulletproof, and/or -silent to make it shut up):

echo "Eating other ADN clients for lunch. Yowza." | texapp -script

echo "Eating other ADN clients for lunch. Yowza." | texapp -status=-

texapp -status="Eating other ADN clients for lunch. Yowza."

Scripting is limited to only what commands the console understands, since basically you're driving the console. Similarly, it is difficult to hook error responses for any given particular command since the console does not offer this behaviour and thus scripted applications cannot be considered bulletproof. Thus, if you need to change Texapp's behaviour in a way the console does not support, or need to write custom behaviour for enhanced reliability or fallbacks, you will need to use an extension -- which brings us to our first training-wheels, yet useful, example.

A first example: Displaying more post metadata

Texapp throws away a lot of (in this context) irrelevant metadata by default when it formats posts, but suppose you're interested in seeing a little more under the surface. Here is an example of a more florid way of displaying posts. Place this into a separate file (example name: spammyposts.pl) and invoke it with texapp -exts=spammyposts.pl or wherever you stored it:

$handle = sub {
        my $ref = shift;
        &std($ref->{'id'}. " ". # BUT SEE &sto BELOW!
                &descape($ref->{'user'}->{'name'}). " ==>".
                &descape($ref->{'text'}). "\n");
	return 1;
};

This introduces the general way of hooking into the Texapp API: rather than having regular Perl subroutines, instead you assign anonymous subroutine references to specific global scalars (listed below). This particular subroutine reference, $handle, is called for every post that is to be displayed and is handed a hash reference containing the individual fields of the post. If you don't define a handler for a particular Texapp API subroutine reference, then the default is automatically used for you so that you only need to define the custom behaviour you want.

The subroutine then pulls out the fields it wants to display, which are simply keys to the hashref (see the ADN API Documentation for what fields are available in JSON). Notice that the text fields are passed to a special internal function &descape, which is a convenience function that Texapp uses internally (and exposes to its extensions) for implementing UTF-8/UTF-16 support and converting character entities and other metadata.

Having built everything into a string, it calls the built-in function &std() which emits into a buffer. On regular synchronized intervals, this buffer is emptied to the output device (for this function, to the UI/UX filehandle; the built-in function &sto(), in contrast, emits to a buffer emptied to a designated output filehandle). More about that in a minute.

The handler returns 1 to tell texapp that one logical post was handled. If it had declined to handle it for some reason, it should return zero.

Note that the extension just ends. Any other setup that needs to be done should be done by the extension before it exits. Because an anonymous subroutine reference is considered a "true" value, this small extension does not need the 1; idiom at the end.

With this extension running, suddenly Texapp's background updates take a new form:

5279876 App.net Staff ==>>> @ben: Beautiful day in SF, beautiful night
for a meetup. Tonight: App.net meetup in San Francisco at Bloodhoud on
Thursday, May 2, from 7-9 pm. http://bloodhoundsf.com/
5279878 elyse ==>@shawnking Me too! I'm kinda sad I lost touch with all of
you nice people .. // @andrewh @formfireglass 
5279896 Matt ==>Could I please be removed from this thread.
5279898 Trine ==>@charlesg that looks amazing! 

SUPER IMPORTANT LITTLE UNEXPECTED SNAG: &sto() versus &std(): when do you use them?

What happens if you decide to use this with -script? You're about to get a rather rude surprise:

texapp -exts=spammyposts.pl -runcommand="/a" # NO OUTPUT!

Nothing prints out!

The reason this doesn't work is where the extension sends its data. When you use &std() to emit to the UI/UX buffer, when -script is in effect, all data sent to that buffer is suppressed so that you only get data, not all the UI and UX stuff.

So, what do you do? Change the extension to use &sto() and this will fix your problem, and works both when in the interactive client and from the command line with -script. Use &sto() for the actual results of a transaction, and &std() for everything else, and you should be good. This is the convention we will use for the rest of these examples.

(Historically, TTYtter used two explicit filehandles, $stdout and $streamout, which are the UI/UX filehandle and the data filehandle respectively, and extensions printed directly to them. If you are porting a TTYtter extension to Texapp, you should convert it to use these functions and avoid directly manipulating these filehandles. Note, however, that emptying the buffer is deferred to "safe" points, meaning it may be delayed by up to several seconds. If your extension demands immediate output and can't wait for the buffer to be flushed, you can still emit your data directly to the filehandle, but doing so may interfere with screen updates or cause problems with the prompt in readline mode.)

An "enhanced" first example: An ADN filter

Suppose you are only interested in one particular subject, let's say, bananas. It should be entirely obvious that you can filter on any term just by using a regular expression search on the relevant JSON field. Yes, you could just grep the output or use the -filter option, but this is another way:

$handle = sub {
        my $ref = shift;
	my $text = &descape($ref->{'text'});
	return 0 if ($text !~ /banana/i);
        &sto($ref->{'id'}. " ".
                &descape($ref->{'user'}->{'name'}). " ==>".
                &descape($ref->{'text'}). "\n");
	return 1;
};

Note that posts that do not match up are not printed, and a zero is returned to alert Texapp that the post was declined. Only if it actually is accepted (in this case for printing) is one returned.

The idea of "accepting" a post is more closely examined in the third example.

A second example: Adding a mapping command to Texapp

Another popular thing to do with Texapp is add custom commands. So here is one that is slightly, even, useful, defining a /gmap command that takes a post's menu code, looks it up, and (if there is geolocation information) opens a browser with that location in Google Maps. Save it to a file like googlemap.txt and add it on with -exts=googlemap.txt (or, if you want our previous example running at the same time, combine the extensions together with -exts=spammyposts.pl,googlemap.txt).

$addaction = sub {
	my $command = shift;

	if ($command =~ s#^/gmap ## && length($command)) {
		my $post = &get_post($command);
		if (!$post->{'id'}) {
			&std("-- sorry, no such post (yet?): $command\n");
			return 1;
		}
		my $lat = $post->{'_texapp_latitude'};
		my $long = $post->{'_texapp_longitude'};
		if (!length($lat) || !length($long)) {
			&std(
		"-- sorry, no geoinformation in that post.\n");
			return 1;
		}
		&openurl("http://maps.google.com/maps?q=${lat},${long}");
		return 1;
	}

	return 0;
};

Let's look at a few of the new things we've introduced here:

Obviously, you can make a whole bunch of little single-command extensions and mash them together with -exts=... to expand your Texapp command vocabulary any way you like.

A third example: An ADN logger

Naturally, you are not restricted merely to output. For those inclined towards blackmail, you can create a "logger" that not only displays the posts it receives, but neatly organizes them into files:

&setvariable('savequit', 1);
$last_id = $extpref_logger_last_id if ($extpref_logger_last_id);
$store->{'last_id'} = $last_id;
$extension_mode = $EM_SCRIPT_OFF;

$handle = sub {
        my $ref = shift;

        my $sn = &descape($ref->{'user'}->{'username'});
        my $string = &descape($ref->{'created_at'}) .
                " \@$sn " .
                &descape($ref->{'user'}->{'name'}) .
                " says: " .
                &descape($ref->{'text'}) . "\n";

	$sn =~ s#/#-#g;
        open(S, ">>", "${sn}.adn") || # combine strings for 5.005
                die("can't open ${sn}.adn for append: $!\n");
        binmode(S, ":utf8") unless ($seven);
        print S $string;
        close(S);
        &defaulthandle($ref);
	return 1;
};

$conclude = sub {
	$store->{'last_id'} = $last_id;
	&defaultconclude;
};

$collectsave = sub {
        $extpref_logger_last_id = &getbackgroundkey('last_id');
        &std("logger: saving last_id = $extpref_logger_last_id\n");
};

This extension not only displays the posts you receive, but it also organizes them into individual [screenname].adn files with date stamps and full names. This more elabourate example also illustrates some other standard things:

Make the above into an automated logging bot "instantly"

It should be obvious that this could be made into a background bot simply by removing the line that calls &defaulthandle (so nothing is actually displayed), starting texapp with -exts=... and -daemon, and then letting the bot just slurp posts out into files in the background. Presto, instant logging bot! Remember that even though no logical posts are being displayed, they are still being accepted, so $handle should still return 1.

Because -daemon runs forever, at least in theory, and therefore does not autosave (because you'll kill it when it's done, and it may never get a chance), you may want to write in some code in the background to keep track of state in another way such as writing the last ID to a file and then loading that instead.

A fourth example: An ADN parrot

First, a word of warning: please don't actually run this, you will irritate a lot of people! This is a very silly example, but it will give you a basis on how to create interactive applications. It is intentionally broken so that it can't be used as is, but yet serve an educational purpose.

This extension creates a ADN parrot, which is to say any post it can see, it will post again. To avoid an endless loop, it determines the user it is running as and won't parrot back something it itself has said.


$handle = sub {
        my $ref = shift;
        my $sn = &descape($ref->{'user'}->{'username'});
        return if ($sn eq $whoami);

        my $string = "\@$sn " . &descape($ref->{'text'});
        die("broken");
        &updatest($string);
        &defaulthandle($ref);
	return 1;
};

Most of this we have seen before, except for the global $whoami, which represents the current user screen name, and the subroutine &updatest, which is used to send a new post for the current user. It should be obvious to the reader that making a more interactive system is just a matter of parsing the text of the post, and then posting out a smarter or at least less aggravating response.

Fourth example redux: Making the parrot use private messages instead

Or, you can make it much, much less aggravating by having it only talk to people who actually directly message it -- hence, surprise surprise, private messaging support.

Private messaging operation through channels is handled similarly to regular posts but instead through the $pmhandle method. ADN, because PMs are channel oriented, requires PMs to be passed in the context of a channel. To improve performance, Texapp uses contexts, which are core channel objects that have been pre-cached and populated with internal state. You need to pass the context to Texapp so that it also has the relevant metadata.


$pmhandle = sub {
        my $ref = shift;
	my $context = shift;

        my $sn = &descape($ref->{'user'}->{'username'});
        return if ($sn eq $whoami);

        my $string = "\@$sn " . &descape($ref->{'text'});
        die("broken");
        &updatest($string, undef, undef, $context);
        &defaultpmhandle($ref, $context);
	return 1;
};

We'll document the extra parameters we passed to &updatest below, but the changes necessary were relatively minor; we just had to make sure we sent the context along to everything that needed it. Assuming that your bot account is not filtering PMs from users it doesn't follow, this should "just work" for anyone who chooses to interact with it.

An exercise for the reader: as written, just like in our logger example, every time the bot starts it will go through its most recent PMs all over again, even if it had already processed them previously. Change this example to use a bookmark as well (hint: $last_pm).

API reference

Let's now get into the technical details.

The multi-module architecture

Texapp is designed around a cooperative multi-module architecture, which allows it to load, manipulate and maintain multiple extensions at once limited only by memory. It was designed to be similar enough to the long-obsolete TTYtter single extension system so that many extensions would require no or minimal adjustment to function in a multi-module world, and to use the same general concepts of redefining subroutine references to override desired portions of the Texapp API, while still allowing the expansion of Texapp's capabilities in any way the user wants.

When Texapp starts up, it completes its own initialization and then enters the multi-module loader. The loader examines each extension in the -exts=... option, in the order they are specified. The extension is then required into Texapp, allowing it to execute its initialization code and define its API subroutine references, and then each subroutine reference it defines (of the known API subroutine references) is examined and assigned to a dispatch table to be called by the multi-module dispatch. Error checking is also done at this point to prevent multiple extensions from hooking API subroutines that cannot be shared (more on that in the next subsection) and to enforce the extension's requirement for the -script option.

During Texapp's execution, when it reaches an API hook point, it enters the multi-module dispatch. The dispatch goes through each of the defined subroutine references for the API hook in the order defined (therefore, in the order specified in -exts=...) and calls up to, though not necessarily, all of them, and finally the default method if any of the subroutine references called it during their execution. The default method is only ever called once in actuality, even if all the extensions requested it to be called.

What you can do, what you can't do, what you should do and what you must not do in multi-module land

Generally, most extensions don't even have to worry about the internal implementation. As long as they stick to their own namespace and behave in a "rational" manner, the details are generally irrelevant. However, there are some benefits and restrictions that go with the territory.

Superclassable subroutine references

These subroutine references can be used to replace or augment Texapp behaviour. Falling back on the default behaviour is optional, but is always available using the &default* subroutine (e.g., the default $handle routine can be called with &defaulthandle). The default routine expects to be called with the same arguments the "super-routine" was called with. Only the default routine for this particular method should be called within subroutine references, e.g., only call &defaulthandle within $handle. Also, default routines are terminal: if you desire to call the default method, it should be the last thing your reference calls before returning a value or terminating.

References are also marked with whether they're typically called in the background, foreground or both. In theory, if your extension runs commands or makes posts in the background, almost all of these functions could be run in background context, but as a rule of thumb command and interactive references are foreground only. If your routine needs to figure out where it's being called, the global $is_background is set to true if you are inside the background process.

$addaction (argument 1: command line) (maskable) (foreground)
Called after initial commandline processing by the default console to allow the implementation of custom commands or to override internal commands (except /quit, etc., which obviously probably shouldn't be overridden). If the routine returns 0, then Texapp assumes that the routine does not want or recognize the command line it was provided, and continues with processing. If the routine returns 1, then Texapp assumes that the routine accepted (or at least wants to suppress further processing of) the command line for its own internal processing, and no further processing is done.

Default behaviour is to return 0. This routine is one-way, i.e., if you rewrite the command line within $addaction, Texapp will discard it and resume with its own copy. If you want to actually alter the command line itself and have Texapp process that (e.g., macro or alias substitution), look at $precommand.

This API reference is maskable -- the first extension to return 1 masks all subsequent extensions from receiving the command, including the default handler.

$autocompletion (argument 1: text to be completed, argument 2: state of current command line, argument 3: position within line of text to be completed) (exclusive) (foreground)
Called, if readline mode is enabled, by the operating Term::ReadLine::* driver whenever TAB completion is needed. An array of fully-qualified likely choices is expected as a return value. For an example of how such a routine would operate, look at &defaultautocompletion. You should be familiar with Term::ReadLine to make the most of this hook.

This API reference is exclusive for technical reasons.

$collectsave (no arguments) (foreground)
Called when the application state is being saved, either with an explicit /save command, or the application is shutting down in an orderly fashion and savequit is on. The extension should collect its data, including calling the background for state if needed, and place it in the $extpref_ variables if it wants to save its state. These variables should be scalars -- do not use them for references. All extensions are called in evaluation order and the operation is not atomic, so a well-behaved extension should not change state here or it may affect extensions downstream.

$conclude (no arguments) (foreground/background)
Called at the end of each cycle of post processing. Default behaviour is to display the count from -filter, if any posts were discarded and a count was requested. Return value, if any, is discarded.

$eventhandle (argument 1: hash reference) (maskable) (foreground/background)
This is reserved for future expansion.

$exception (argument 1: exception number, argument 2: exception text) (foreground/background)
Called when a non-fatal exception is received during processing of posts. Argument 2 is guaranteed to be human readable text corresponding to argument 1, which comes more or less from this table:

(Numbering gaps are for historical reasons.) The exception number code is provided simultaneously to facilitate localization or custom notification. Exceptions passed to $exception are designed to be informative only, as Texapp can recover from these errors and automatically try again. Fatal errors are raised immediately and the extension does not receive notification for technical reasons. Default behaviour is to print the error text to standard output. Return value, if any, is discarded.

By default, $exception is not maskable and all extensions that are loaded get notified. However, if you want to suppress error reporting for some reason, you can make $exception maskable with -exception_is_maskable (or exception_is_maskable=1 in your .texapprc), and return 1 from your extension to prevent the error condition from further propagation. This is not recommended unless you know what you're doing, because you can suppress error reporting to the client and not be aware of server status. "Don't Blame Texapp."(tm)

$getpassword (no arguments) (foreground)
This is reserved for future expansion.

$handle (argument 1: hash reference, [optional] argument 2: origination) (maskable) (foreground/background)
Called when a post is to be displayed or otherwise handled in some manner. The keys of the hash reference are based on those specified by the ADN API. Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text "true" and "false". The routine, naturally, is not obligated to generate any output if it desires. Default behaviour is to display the post formatted to standard options (using &standardpost [below]) with screen name and post text, and with the post's menu code prepended. For success, the number of "logical posts" handled (almost always one) should be returned. If the post was declined for processing, a zero value should be returned.

If you like, you can pass your hash reference to &standardpostinteractive for the "default" formatting, which will return a string formatted according to the return value of $posttype and whatever standard options are set (such as -timestamp, -ansi, -wrap, etc.).

An optional origination argument may also be passed, giving the routine information about the user command the post originated from. This allows your routine to distinguish old and new posts reliably. Origination classes defined currently are a null string, meaning a new post; replies, meaning posts from the /replies command; or again, meaning old posts (usually from the /again command). The distinction is important; replies does not appear on new replies, but only on replies that you ask for. This is on purpose to ensure that API activity results are consistent and match up. Note that again overrides replies, and since /again can sometimes pull up new posts, the originator is blanked on purpose for those new ones so they are properly seen as new. The complexity here is mostly intended for those clients who want to distinguish old and new posts, or posts that a user requested versus posts that were automatically fetched, and handle them differently. If you actually want to change how Texapp classifies posts internally, regardless of their age, see $posttype.

This API reference is maskable -- if an extension returns zero, then all subsequent extensions will not be passed the post. Only if the extension returns 1 will it be passed to subsequent extension references. However, the default method will always be called if this extension or any prior to it had called it during this run of the dispatch, and if no extension filters the post, notifications will be sent even if the default method is not called.

$heartbeat (no arguments) (background)
Called at the beginning of each automatic refresh cycle (in either daemon or interactive mode). Default behaviour is to do nothing additional. Return value, if any, is discarded.

$main (no arguments) (exclusive) (foreground)
This is Texapp's main loop. Default behaviour is to operate the console, i.e., initialize the history, print an initial prompt, and then accept data line by line from standard input until a terminating command is received or the input stream ends. Return value, if any, is discarded, and Texapp will terminate completely when the routine is exited.

If you redefine this subroutine reference, then your extension has complete control of the application. Nothing says your main loop actually has to take user input, by the way -- it can do its own thing and ignore standard input completely, and even drive Texapp itself with internal commands using &ucommand.

This hook can also be used to run code after initialization of the client but prior to accepting user input. If so, your code should end with something like goto &defaultmain; to transparently return control to the default handler.

This API reference is exclusive for technical reasons.

$pmconclude (no arguments) (foreground/background)
Called at the end of each cycle of private message processing. Default behaviour is to do nothing additional. Return value, if any, is discarded.

$pmhandle (argument 1: hash reference, argument 2: context reference) (maskable) (foreground/background)
Called when a private message is to be displayed or otherwise handled in some manner. The keys of the hash reference are based on those specified by the ADN API. Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text "true" and "false". The routine, naturally, is not obligated to generate any output if it desires. Default behaviour is to display the PM formatted to standard options (using &standardpminteractive [below]) with the sender's name, time stamp provided by ADN, and the text of the direct message. For success, the number of "logical PMs" handled (almost always one) should be returned. If the DM was declined for processing, a zero value should be returned.

If you like, you can pass your PM and context hash references to &standardpm or &standardpminteractive for the "default" formatting, which will return a string formatted according to whatever standard options are set (such as -timestamp, -wrap, -ansi, etc.)

This API reference is maskable -- if an extension returns zero, then all subsequent extensions will not be passed the private message. Only if the extension returns 1 will it be passed to subsequent extension references. However, the default method will always be called if this extension or any prior to it had called it during this run of the dispatch, and if no extension filters the PM, notifications will be sent even if the default method is not called.

$posttype (argument 1: hash reference, argument 2: screen name, argument 3: post text) (maskable) (foreground/background)
Called to determine what class a post should be (which should be returned as a string); PMs are discovered separately (there is no corresponding $pmtype). The standard post classes are me reply search default follow alien subthread; you could define other classes for use, say, with the notification framework (discussed below) or with your own custom $handle routine. To fall back on the standard post class selection, simply return &defaultposttype($ref, $sn, $post) for ones you don't want to classify yourself.

You can also return a set of classes, separated by /s, such as reply/alien/search, which says this post meets the criteria for all three. However, only one notification is sent if any of the classes trigger a notification. Note that by default me never has other classes associated with it. Order is salient, i.e., the most significant or highest priority class should be specified first.

When multiple classes are fed to &standardpostinteractive, the priority is me > reply > follow for selecting the ANSI colour. If the post falls into none of those, the first colour of the remaining classes that is not OFF is selected, or default.

This API reference is maskable, but in a slightly unusual way -- any extension reference that does not call the default method (and returns an explicit post type) is considered to mask all subsequent references, as it is assumed routines calling the default method are signaling they do not know how or want to classify this post.

$precommand (argument 1: command prior to processing) (foreground)
Called as soon as a command is received for processing, even before history substitution. Allows you to implement your own preprocessing. The new command should be returned as a single response, and is subject to things like % substitution and so on. Default behaviour is to just return the same command without further pre-substitution. Although you could also attempt to intercept and handle custom commands here too, $addaction is a better choice for that purpose as it is maskable.

This API reference is telescoped: the output of prior extensions is fed to any subsequent ones. The order of evaluation is, as always, determined by their order in -exts=....

$prepost (argument 1: post prior to posting) (foreground/background)
$postpost (argument 1: post after posting) (foreground/background)
These two are paired, so they are listed together. $prepost is called when a post is about to be URL-encoded and sent, allowing you to implement your own post preprocessor (such as, say, a translation or shortening service). The new post should be returned as a single response. After the post is posted, $postpost is called with the final post (which barring an act of God or cosmic radiation should be the same as what $prepost returned), which is useful for tools such as loggers. Default behaviour for the former is to simply return the same post without further pre-substitution, and for the latter, to do nothing.

$prepost is telescoped: the output of prior extensions is fed to any subsequent ones. The order of evaluation is, as always, determined by their order in -exts=....

$prompt ([optional] argument 1: information only) (exclusive) (foreground)
Called every time a prompt is to be displayed by the console. Default behaviour is to display Texapp>, followed by a separating space. If ANSI colour is enabled, the prompt is displayed in cyan.

The prompt is only printed by the default handler if -readline is not enabled (otherwise the prompt is maintained by ReadLine). If optional argument 1 is true, then the prompt and its screen width (which may not be the same as its length) are returned as a list for interested subroutines. When the prompt is printed or requested, the wordwrap subroutine is hinted to use it in calculations by setting global $wrapseq to zero.

This API reference is exclusive for technical reasons.

$shutdown (no arguments) (foreground)
Called when a normal shutdown is occuring, viz., a shutdown initiated by the user or by Texapp as part of expected operation. Unexpected shutdowns such as &screech intentionally do not call this routine, so your extension should be prepared to handle that possibility. Default behaviour is to do nothing additional. Return value, if any, is discarded.

$userhandle (argument 1: user object reference) (maskable) (foreground/background)
Called to display a user object. This is the routine that displays the two-line user text from /followers and like-minded commands.

This API reference is maskable -- the first extension to return 1 is considered to have handled the display of the object. An extension can simply passively observe by merely returning 0, indicating it declines to show it (or wishes to pass it on). If no extensions return 1, meaning none of them elected to terminally display the user object, then the default routine is called.

Writing custom notification drivers and using custom post classes

Your API extension can define a custom notification driver, which is handled differently from the above. In general, the argument to -notifytype=... is turned into a function name using the notifier_ prefix, e.g., -notifytype=growl indicates that the subroutine notifier_growl should be called for notifications. Thus, if you wanted to define, say, a notifier_email subroutines for E-mail notifications, then you invoke it with -notifytype=email after including it with -exts=....

The notification subroutine is called in two ways: with no arguments (or more accurately, with a single argument of undef) during initialization, and thereafter with three arguments: the class (as determined by $posttype), the post string as processed by &standardpost as a convenience, and the actual reference. If the class is actually "PM," the reference is an arrayref consisting of the PM reference and the context reference, and the string is processed through &standardpm; otherwise, it is the actual post reference. Your routine will then handle the notification, and return. Return value, if any, is discarded.

The default drivers included (&notifier_growl and &notifier_libnotify) are instructive examples. Both are formatted in the same basic way: during initialization they seek out their required utilities (growlnotify and notify-send respectively) and store them in an appropriate global, and then as posts are passed to the notifier they then pass them off to their dependent utility with the correct command line arguments on standard input.

Neither of the built-in drivers do anything special with the class currently. However, your driver may decide to in fact do special things with each class, and your customized $posttype method, if you choose to write one, can tag posts with new classes that your custom notification routine can handle (say, a class peter for posts from your friend Peter; for those posts, $posttype would return 'peter', and you would add peter to your list of classes in -notifies=...). This is entirely supported, and you can of course simply fall through to &defaultposttype for other posts that are not from Peter to get default behaviour otherwise.

Note that notification routines are called only for new posts; posts identified as old do not get passed to the notifier to avoid ping-ponging.

Output routines

To remind you, use these routines wherever possible instead of directly emitting to Texapp's output filehandles:

&std (argument 1: text string)
Emits a string to standard output, used for UI and UX. If this call is made in a background context, the actual display is deferred to a safe point, which may delay actually printing the string by several seconds. If this call is made in a foreground context, the display is immediate. If -script is on, or any option that implies -script such as -runcommand=..., no data is actually displayed.

&sto (argument 1: text string)
Emits a string to the output stream, used for data and results, including for -runcommand=... and -script. If this call is made in a background context, the actual display is deferred to a safe point, which may delay actually printing the string by several seconds. If this call is made in a foreground context, the display is immediate.

Library routines

These routines are explicitly designated as available for calling from a user application. Other routines may also be utilized, but are not guaranteed to maintain compatible naming or calling convention in future versions. They are not overridable.

&descape (argument 1: data, [optional] argument 2: entity mode)
General text decoding routine. This single entry point is responsible for HTML ampersand-entity decoding, converting escaped UTF-8 and UTF-16 characters into their correct/desired form, and returning the de-escaped data. You should call this routine if you reasonably expect the string to contain such data, or you are preparing it for user display. If the data contains UTF-8 or UTF-16 entities and UTF-8 is disabled with -seven, then the entities are rendered as dots ('.'). The processed data is returned.

If optional argument 2 is true, then UTF-8 entities are rendered in HTML "ampersand" form, even if UTF-8 is "off." This is particularly useful for web applications. &descape will also convert many ampersand-escaped entities into ASCII as well, unless argument 2 is true (in which case it assumes that you wish them to remain ampersand-escaped like the UTF-8/16 entities will be).

&getbackgroundkey (argument 1: key)
Asks the background process for the specified key in $store. This is useful for using Texapp's built in IPC to get the state of the background process, or to get data from an extension running in the background process. Only a string is returned; pre-serialize your data yourself. The IPC protocol truncates keys to 15 characters. See &sendbackgroundkey for more information on this feature. You can only make this call from the foreground.

&get_pm (argument 1: menu code)
&get_pm_context (argument 1: menu code):
&get_post (argument 1: menu code)
This family of library functions returns a hash reference corresponding respectively to the PM, context of the PM, or post specified by the menu code given, or undef if not found. If the call does not require IPC (e.g., the post is in background memory and the request is in background context), that actual hashref will be returned; otherwise, the console will ask the background and make a "fake" smaller hash reference with only essential fields. The background cannot ask the foreground for foreground menu codes. You should only rely on the essential fields listed here; you are not guaranteed to get the entire post structure. If the returned reference has an undefined or zero ID field, you should also assume the requested reference is invalid and/or does not exist, and not use it further.

If you are querying a private message, only the ID (as $key->{'id'}), channel ID ($key->{'channel_id'}), sender ($key->{'user'}->{'username'}), source string ($key->{'source'}->{'name'}), menu code (_texapp_menu_select), whether there are entities (_texapp_has_entity_links), the geolocation info, if any (_texapp_latitude, _texapp_longitude), creation time (created_at) and text are guaranteed part of the reference. The /dump command will show you these fields as well as the associated PM context (below). You generally need both the context and the PM to do something useful with the private message.

If you are querying a post, only the ID (as $key->{'id'}), reply-to information (reply_to), repost information, if any ($key->{'repost_of'}->{'id'}), number of replies (num_replies), number of stars (num_stars), the thread ID (thread_id), number of reposts (num_reposts), source string ($key->{'source'}->{'name'}), the internal Texapp tag and payload ($key->{'_texapp_tag'}->{'type'} and $key->{'_texapp_tag'}->{'payload'}), the classes the post was determined to fall under (_texapp_classes), menu code (_texapp_menu_select), whether there are entities (_texapp_has_entity_links), the geolocation info, if any (_texapp_latitude, _texapp_longitude), creation time (created_at) and text are guaranteed part of the reference. The /dump command will show you these fields.

If you are querying the PM context for a PM, it consists of id, owner and users, the latter of which is an arrayref. Remember: a context is not the same thing as a PM channel. To turn a channel into a context, use &pmchanneltocontext. The /dump command will show you these fields for a given PM. You generally need both the context and the PM to do something useful with the private message.

&getvariable (argument 1: variable name [as string])
The explicitly designated getter for user settings. Call this function with a variable name as a string (as specified in the list of command-line options), and the variable will be returned to you. Illegal requests will return undef. While you can access most settings variables directly, this is now deprecated, as there are virtual settings you can only get from this getter routine and there will be more in future versions.

This is the same routine the /print command calls. See also &setvariable.

&grabjadn (argument 1: URL, [optional] argument 2: last ID, [optional] argument 3: don't authenticate, [optional] argument 4: number of items to request, [optional] argument 5: tag object, [optional] argument 6: get annotations, [optional] argument 7: before ID)
&grabjson (argument 1: URL, [optional] argument 2: last ID, [optional] argument 3: don't authenticate, [optional] argument 4: number of items to request, [optional] argument 5: tag object, [optional] argument 6: get annotations, [optional] argument 7: before ID)
&grabjson asks URL for a JSON data source, calls &parsejson on it to turn it into a variable structure (q.v.), and returns either a reference to a hash, reference to an array, or a scalar, or an undefined reference if there was a problem. As with &parsejson, severe or unexpected parsing errors will simply return undef (if -verbose is on, the syntax tree and debugging information will be emitted). Note that the URL need not be a ADN URL; you can call this on any JSON source and get a ref back, if Texapp can parse it. If optional argument 2 is true, then a since_id= is added to the request for you. If argument 3 is true, then authentication information is not sent with the request (for those JSON APIs that don't need the token to be sent; you could leak the bearer token to a malicious service if you're not careful here). If argument 4 is specified and non-zero, then a count= is added to the request for you (you are not guaranteed to get that number, but you are guaranteed to not receive more than that number); otherwise, the default count is provided. If argument 5 is specified and not undef, _texapp_tag for the JSON result is set to that tag. Generally this should be a string. If argument 6 is true, annotations are requested with include_annotations=1. If argument 7 is specified and non-zero, then a before_id= is added to the request for you. Your API on the other end should know what to do with these if you ask.

The &grabjadn form is a convenience when you know you're talking to ADN on the other side (as an aid to TTYtter authors and lazy programmers) which unwraps the ADN response envelope for you. If you call it in a list context, it returns an array with the data portion of the JSON and the meta portion (see the ADN developer docs). If you call it in a scalar context, the returned format depends on whether this ADN endpoint returns an array or a mapping and/or hash. If an array, &grabjadn simply returns the data arrayref, which may be undef; the meta information is thrown away. If a hash/mapping, &grabjadn embeds the meta information in the data hashref as {'meta'} and returns the data hashref. If an error occurs, if called in array context, undef and the meta object are returned; in scalar context, just the meta object.

Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text "true" and "false".

If you just want the contents of that URL without any parsing, then use ...

&graburl (argument 1: URL, [optional] argument 2: POST data)
Uses Texapp's user agent to execute an HTTP GET to fetch the desired URL and returns the contents as a string scalar without further interpretation. If optional argument 2 is specified, then it is used as POST data and the request is sent as a POST instead. You should make sure you have done all required encoding prior to calling this routine.

&openurl (argument 1: URL)
Opens the URL in the user's browser according to the current -urlopen settings.

&parsejson (argument 1: JSON text)
Interprets JSON into a Perl variable structure, returning either a reference to a hash, reference to an array, or a scalar, or an undefined reference if there was a problem (if the JSON is not parseable, the data and syntax tree will be dumped in -verbose mode). Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text "true" and "false".

This is the same routine &grabjson/&grabjadn calls (and &postjson, for that matter). In general, however, it is more efficient and less trouble to just use &grabjson to fetch from a URL and parse it in one step than it is to grab the URL with &graburl and feed it to &parsejson.

&pmchanneltocontext (argument 1: channel object)
If you somehow wind up with an ADN channel object (for example, you wrote a driver to handle other kinds of PM types), you need to turn it into a context if you want to use any of the built-in routines; this routine does all the precomputation, using cached information if available. Note that it is assumed that the channel is immutable, which may not be true for unusual use cases, and a warning will be emitted if a mutable ACL is discovered. Returns a Texapp context object or undef if a problem occurred.

&postjson (argument 1: URL, argument 2: POST data)
Makes a POST request to the specified URL. If the data is null, it may be converted into a GET depending on your user agent, so pass something if you want to be safe. Always sends authentication information if available. Just like &grabjson, this routine has all the parsing quirks of &parsejson, which it also calls.

&savestate (no arguments)
Saves the state of Texapp to ~/.texapprc while copying the old one to ~/.texapprc~, even if savequit is not set, just as if the user had typed the /save command. You cannot call this function from the background, or by extension -daemon mode, since in that situation there is no foreground; for extensions that must save their state in the background, you will need to implement your own means of saving state such as emitting data to a separate file and loading it back during initialization of your extension.

&screech (argument 1: error text)
Emit error text to standard output (ring the bell if supported), and kill and shutdown immediately. Used as an escape hatch for unsafe situations, or fatal errors. Note that this bypasses $conclude and $shutdown, and as such, no further notification is given to any extension that a fatal condition has occurred. This routine does not return.

&sendbackgroundkey (argument 1: key, argument 2: string)
Sends the string to the background process using Texapp's built-in IPC facility, which stores it in the calling extension's $store hashref with argument 1 as the key. Only strings are supported; serialize your funky structured data your own self. If you pass a null second argument, then the key is set to undef. Keys are truncated to 15 characters due to the Texapp's IPC protocol requirements. The foreground can use the analogous &getbackgroundkey to fetch as well (q.v.).

This function can be used to implement a form of RPC via message passing if you need to have operations executed by your extension in the background. Your extension in the background has something hooked into $heartbeat that looks at a predefined place, like $store->{'command'}, for a command; it does an operation based on that, and saves the result into $store->{'result'}. In the foreground, your extension would use &sendbackgroundkey to send a command string to key command, and pick up the result from key result with &getbackgroundkey either asynchronously or block and busy wait until a result is received.

Keys are local to the extension store, so you don't have to worry about other extensions stomping on your namespace. You can only call this function (and &getbackgroundkey) in the foreground.

&sendnotifies (argument 1: post reference, [optional] argument 2: origination)
&sendpmnotifies (argument 1: PM reference, argument 2: PM context reference)
These functions can be used by an extension to independently raise notifications, or send generated data through the notification system, especially if the extension wants to block the post or PM from getting downstream or displayed it in a different fashion but still wants to send it to a notifier. They should be passed with the same arguments as &defaulthandle and &defaultpmhandle.

In TTYtter, extensions generally had to drive these library functions themselves if they overrode the default behaviour, even if they never actually filtered the post or PM. In Texapp, that specific use case is no longer necessary: if the post is never filtered, the notification will still be sent, even if the default handler is never called.

&setvariable (argument 1: variable name [as string], argument 2: variable value [anything], [optional] argument 3: interactive)
The explicitly designated setter for user settings. Call this function with a variable name as a string (as specified in the list of command-line options) and the value to store into it. This setter will then trigger whatever side effects need to occur and synchronize whatever state needs to be synchronized with the background process. Errors are logged to $stdout; success returns 0 and failure returns 1. If optional argument 3 is true, then success is also logged to $stdout.

&setvariable can set read-only variables -- but only during the multi-module loader phase (i.e., only during the initialization step of your extension). Once Texapp has started up and handed control to the main loop, then read-only variable settings are locked and may not be changed without a restart. Please be nice and don't try to write to them directly anyway; this is not supported, not socially friendly, and almost certainly won't work in future versions.

This is the same routine the /set command calls. See also &getvariable.

&standardpost, &standardpostinteractive (argument 1: hash reference)
&standardpm, &standardpminteractive (argument 1: hash reference, argument 2: context reference)
These routines return the default pre-formatted string according to any user-specified arguments for a post or PM, based on the hash(es) passed to it.

The *interactive versions emit the menu code embedded in the text if needed, suitable for directly emitting to the user with &sto. The other versions instead emit a stand-alone format. Usually the *interactive form is the one you want, since it will figure out everything for you.

The other versions also allow two additional optional arguments (not the *interactive forms): a "no colour" form (if non-zero, ANSI codes are not interpolated, even if ANSI mode is enabled), and number of characters to wrap the first portion (mostly used for prompt computations). If you pass them to the *interactive calls, they are ignored.

The &standardevent* family is not yet implemented in this version of the Texapp API.

&ucommand (argument 1: console command)
Executes a command as if you had typed it at the console, even if you have hooked $main or are not using the console. This is mostly useful for running user commands with consequences (but while you could use it for the /set command as well, for example, &setvariable is a bit more transparent for that specific purpose). You cannot use &ucommand during your extension's initialization, however, because there is not enough state information instantiated yet to run commands; doing so will cause an error to be raised by the multi-module loader.

&updatest (argument 1: status text, [optional] argument 2: interactive mode, [optional] argument 3: in reply to post ID, [optional] argument 4: PM specifier [see below], [optional] argument 5: repost ID)
Update the current user's status with status as a post. If optional argument 2 is true, any error condition will also be displayed on standard output. If optional argument 3 is specified and non-zero, then the post will have its reply-to information set to that post ID. If optional argument 4 is specified, then the post is converted to a private message to the specified user (see below). If optional argument 5 is specified, then the post is converted to a repost of that post ID; the status text should still be set so that prompts in interactive mode function.

Argument 4 can be in one of a number of formats. You can, for example, directly pass it a PM context, and then the status will become a PM to the channel encoded by that context. If you pass it an arrayref of user names, or a single string with a single user name, then a channel will be opened to those users (creating a new one if necessary) and the status will become a PM to that channel.

A status value is returned: zero if the post was successful, or a return code dependent on cURL if not (see documentation). Return code 96 indicates that full-screen editing was attempted, but had a critical failure. Return code 97 indicates the user decided not to post (refused by -verify or -slowpost, or on-screen editing was cancelled or unsuccessful). Return code 99 indicates that the subprocess could not even start, possibly due to change in the filesystem or permissions.

Exposed globals

These globals are explicitly designated as available or permissible for user operation and manipulation. Globals specific to the extension itself should always be stored as (permanent) $extpref_* prefix or keys within the hash reference $store; manipulating other globals could interfere with normal Texapp operation, and are not guaranteed to retain the same function or name in future versions.

For globals that are instantiated by user command-line options and/or /set, you should use &getvariable and &setvariable as their getter/setter respectively. While you can try to access them directly, they are not guaranteed to be current, and setting some options have side effects that simply modifying the globals directly will not replicate. The variable key(s) for these functions are the same as the named command-line option and/or runtime variable; see above for a more detailed description.

$Texapp_VERSION
The current major/minor version of Texapp (currently a string containing a float value). This doesn't include the patch level; see the next variable for that.

$Texapp_PATCH_VERSION
The current patch level of this release of Texapp, loaded into a separate scalar. Mostly a historical holdover from TTYtter. This is an integer, so $Texapp_VERSION 0.5 and $Texapp_PATCH_VERSION 1 indicates version 0.5.1.

$extpref_*
Any command line option added by the user or placed in .texapprc starting with extpref_ is turned into a global (so -extpref_bletch_foobar=5000 sets $extpref_bletch_foobar to 5000). By definition these are external to Texapp and it does not do anything with them further other than maintaining them in .texapprc if they are /saved; their behaviour is fully defined by your extension. Because these are global, you should reasonably make sure that the name you choose for your option will not trample on the namespaces of other extensions the user may have loaded.

$whoami
The current screen name in use. It is not immediately instantiated until authentication is complete and the client main loop has started, so you should not assume it will be populated during your extension's initialization. You should treat this global as read-only: changing $whoami does not necessarily change the credentials sent by Texapp. If you are authenticating with OAuth, this is only set if -status is not set, because Texapp doesn't need your screen name simply to post with an OAuth keyfile.

$parent $child
The PID of the parent and child processes. In interactive mode, both are defined; in daemon mode, only the latter (the parent ID becomes zero). Neither should be modified, or processes may not get properly terminated.

$is_background
If true, the currently executing code is in the background process. Do not modify this value, or the application may crash or behave adversely.

$last_id
The last/highest post ID so far processed. It starts at zero, but may be advanced to skip posts as $handle (or &defaulthandle where not specified) is only called for new posts, viz., posts with an ID higher than $last_id. Even if $handle returns zero for an arbitrary post, that post's ID is still considered for $last_id.

$last_pm
Analogously, the last/highest private message ID so far processed. Its behaviour is exactly the same as $last_id, including starting from zero on startup, and even if $pmhandle returns zero for an arbitrary PM, that PM's ID is still considered for $last_dm.

$lastposted
The last successful post (empty if no posts have been made). This is not carried from session to session.

$streamout $stdout
The two file handle references handling, respectively, the results of a command and information and prompts. Avoid driving these directly; use &sto and &std respectively to send strings to them in a safe manner.

$CCme $CCreply $CCpm $CCsearch $CCprompt $CCwarn $CCdefault $CCalien $CCsubthread
The surface manifestation of the -colour* command line options, viz., the actual printable terminal sequences. You should use this rather than specifying an ANSI sequence when displaying a particular post class.

Porting TTYtter extensions to Texapp

Most TTYtter extensions will mostly work with minimal changes. Assuming they are compatible already with TTYtter 2.1, the following changes must be made (this is not an exhaustive list):

Send comments and six-packs of Mr Pibb to ckaiser@floodgap.com, or return to the main Texapp page.


Cameron Kaiser