TTYtter Advanced Usage

Please note that certain options or API operations are limited to certain minimum versions. The API does not exist in TTYtter versions prior to 0.4. Operations requiring a minimum version of TTYtter later than this will be specified.

Here are the command line options relevant to the API:

-lib=[library path] (optional)

Specifies the library to use for custom handlers and setup. The library is directly required by ttytter and becomes part of it. Your library must return a true value at the end (an idiom like 1; as the last line will suffice nicely). If the path of the specified library is not guaranteed to be in @INC, a fully qualified filespec would be a good idea.

The actual "methods" and globals you can and must use are discussed below, but I strongly advise reading this whole page first.

-daemon (optional)

Forces ttytter to run as a "detached" process in the background (the PID of the new background process is reported). If no library is given, then this is as if one were running the regular background update process, but without a console process (so updates continue to appear, but you must use a command line incantation like ttytter -status=... or something similar to actually post, as you are in your shell and not TTYtter). You must kill the process manually to shut it down.

If a library is defined, then this is the basis of how to construct a fully automated bot. We'll talk more about this below.

-twarg=[argument] (optional, 0.7.0+)

Specifies a user-defined argument string, allowing you to pass data to your extension library from outside of TTYtter.

If your bot is passive and doesn't need to post, it may be helpful and quite a bit faster to run it in anonymous mode with -anonymous (optional, 0.7.0+). This will not work for the examples below that generate tweets.

I have always found it easiest to learn by example, so let's look at three simple but useful examples before we get to the API reference. It will make a lot more sense if this is your first time.

Before getting started: Scripting (0.8+)

echo "/again api_updates" | ttytter -script -anonymous | grep -i banana

The -script argument, as you will recall, disables automatic updates, sets -silent, and disables ANSI colour and other settings inappropriate for not-a-terminal.

Obviously, you are not limited to single commands; you could have an entire command file if you like, and just pass that in with redirection or as an argument (ttytter -script my_script_file). Remember that if you want to wait for all processes to finish emitting data before ending the script, you must end your file with /end (or end-of-file will be seen as /quit and all pending queued asynchronous commands will be cancelled).

Scripting is limited only to what commands the console understands, since basically you're driving the console. Similarly, it is difficult to hook error responses for any given particular command since the console does not offer this behaviour and thus scripted applications cannot be considered bulletproof. Thus, if you need to change TTYtter's behaviour in a way the console does not support, or need to write custom behaviour for enhanced reliability or fallbacks, you will need to use an extension -- which brings us to our first training-wheels, yet useful, example.

A first example: Displaying more tweet metadata

$handle = sub {
        my $ref = shift;
        print STDOUT ($ref->{'id'}, " ",
                &descape($ref->{'user'}->{'name'}), " ",
                &descape($ref->{'user'}->{'location'}), " ==> ",
                &descape($ref->{'text'}), "\n");
	return 1;
};

This introduces the general way of writing handlers: rather than having regular Perl subroutines, instead you assign anonymous subroutine references to specific global scalars (listed below). This particular subroutine reference, $handle, is called for every tweet that is to be displayed and is handed a hash reference containing the individual fields of the tweet.

A peculiarity of the way ttytter handles JSON is that it adds certain unambiguous quote-escapes to aid it in interpretation. For those fields that are likely to have quote-escapes (free text fields, particularly), the library function &descape is used to turn ttytter's internal quote-escapes back into regular quotes. It is not done automatically for you for reasons of efficiency, since it is not needed for fields you will not be displaying or digesting in your routine.

The handler returns 1 to tell ttytter that one logical tweet was handled. If it had declined to handle it for some reason, it should return zero.

Note that the library just ends. Any other setup that needs to be done should be done by the library before it exits. Because an anonymous subroutine reference is considered a "true" value, this small library does not need the 1; idiom at the end.

With this library running, suddenly ttytter's background updates take a new form:

92780182 Jenny How Malaysia ==> nice cold morning.. feels so lazy.... :)
92780192 Clifford Dog Desierto de Sonora Mexico ==> @ingridipity: huy que mal, espero que sea Cerveza Ligera ;)
92780202 Kelly Sims Torrance, Ca ==> also, the showcase on ee site is cool to see how others are using it

You can also use this with a script to fetch tweets and format them in a particular fashion, suitable for piping to something else; remember, using -script and -lib=... together is completely valid! This exercise is left for the reader.

An "enhanced" first example: A Twitter filter

$handle = sub {
        my $ref = shift;
	my $text = &descape($ref->{'text'});
	return 0 if ($text !~ /banana/i);
        print STDOUT ($ref->{'id'}, " ",
                &descape($ref->{'user'}->{'name'}), " ",
                &descape($ref->{'user'}->{'location'}), " ==> ",
                $text, "\n");
	return 1;
};

Note that tweets that do not match up are not printed, and a zero is returned to alert TTYtter that the tweet was declined. Only if it actually is accepted (in this case for printing) is one returned.

The idea of "accepting" a tweet is more closely examined in the next example.

A second example: A Twitter logger

$Lib_master = "$ENV{'HOME'}/twt.bookmark";
if(open(S, $Lib_master)) {
        $last_id = 0+scalar(<S>);
        print STDOUT "LIB: init last id: $last_id\n";
        close(S);
}

$handle = sub {
        my $ref = shift;
        return 0 if ($ref->{'user'}->{'protected'} eq 'true');

        my $sn = &descape($ref->{'user'}->{'screen_name'});
        my $string = &descape($ref->{'created_at'}) .
                " \@$sn " .
                &descape($ref->{'user'}->{'name'}) .
                " says: " .
                &descape($ref->{'text'}) . "\n";

	$sn =~ s#/#-#g;
        open(S, ">>", "${sn}.twt") ||
                die("can't open ${sn}.twt for append: $!\n");
        binmode(S, ":utf8") unless ($seven);
        print S $string;
        close(S);
        &defaulthandle($ref);
	return 1;
};

$conclude = sub {
        print STDOUT "LIB: writing out: $last_id\n";
        if(open(S, ">$Lib_master")) {
                print S $last_id;
                close(S);
        } else {
                print STDOUT "LIB: failure to write: $!\n";
        }
};

This library not only displays the tweets you receive, but it also organizes them into individual [screenname].twt files with date stamps and full names. It will not log users who are protected.

This more elabourate example also illustrates some other standard things:

• All globals used by the library should start with $Lib_. (You could also do stuff with packages but this is simpler, and TTYtter is guaranteed never to use any global starting with that prefix.) Except for globals that are explicitly declared as user-manipulable, don't clobber or mess with other globals.

• Any initialization routine can just go at the top, without having to be in an organized subroutine.

• This library manipulates an important TTYtter global called $last_id, which is the ID of the last/highest tweet received. ttytter maintains it for you automatically. The reason why this routine cares about it is that initially, when $last_id is zero, it will fetch the last twenty tweets and this could make the logs stutter with repeated tweets if the client has to stop and start. For this reason, $last_id is saved in a file ~/twt.bookmark and automatically set to the correct value of the last tweet logged when the library starts. (I'll talk about how $last_id gets automatically recorded in a second.)

• The $handle subroutine reference looks at the 'protected' value to see if this user should be recorded. Note that as a side effect of how Perl is used to evaluate the JSON data, rather than using logical true and false, the JSON reference instead uses the literal text values true and false. Don't trust this for fields that aren't guaranteed to be Boolean. Please note that checking 'protected' is considered mandatory behaviour for Twitter API-compliant applications where private data may be handled or stored. If the user is protected, the tweet is dropped and zero is returned to indicate the tweet was declined.

• The $handle subroutine reference looks at the global $seven to determine if UTF-8 should be turned on. This is good social behaviour.

• The $handle subroutine reference calls a subroutine &defaulthandle to display the tweet. Every "overloadable" function has a corresponding default* subroutine to get the default behaviour. You are under no obligation to call it, but it is always available.

• Now to how $last_id gets automatically saved: the $conclude subroutine reference. This reference is called at the end of a "pulse" of tweets. Note that for technical reasons, there is no guarantee that a cleanup routine will be called properly when ttytter terminates, so this is the way around this problem. When the tweets are done printing each time, this $conclude reference then writes it out to ~/twt.bookmark.

Make the above into an automated logging bot "instantly"

A third example: A Twitter parrot

This library creates a Twitter parrot, which is to say any tweet it can see, it will tweet again. To avoid an endless loop, it determines the user it is running as and won't parrot back something it itself has said. (This example, therefore, will not work in anonymous mode.)

die("I can't run anonymously") if ($anonymous);

# in 0.7.0+ you can use the global $whoami to get the username instead
# of splitting $user
($Lib_dontecho, $Lib_crap) = split(/:/, $user, 2);

$handle = sub {
        my $ref = shift;
        my $sn = &descape($ref->{'user'}->{'screen_name'});
        return if ($sn eq $Lib_dontecho);

        my $string = "\@$sn " . &descape($ref->{'text'});
        die("broken");
        &updatest($string);
        &defaulthandle($ref);
	return 1;
};

Most of this we have seen before, except for the global $user (be nice: the user's password is here too, so behave please), and the subroutine &updatest, which is used to send a new status for the current user. It should be obvious to the reader that making a more interactive system is just a matter of parsing the text of the tweet, and then tweeting out a smarter or at least less aggravating response.

Third example redux: Making the parrot use direct messages instead

[...]
$dmhandle = sub {
	[...]
        my $sn = &descape($ref->{'sender'}->{'screen_name'});
	[...]
        my $string = "D $sn " . &descape($ref->{'text'});
	[...]
};

Here, we query the screen name from the sender field, reply using the Twitter D command in a standard post, and return 1 to tell TTYtter that the direct message was accepted and acted upon. Note that only a minimal change in logic is required to make this happen on direct messages instead of public tweets and vice versa. The "anti-loop" logic isn't really needed here, but is nice to account for and won't harm anything either.

To make such bots effective and the communication bidirectional, essentially you must be following everyone who follows you. This can be done with the Twitter API in a programmatic, delayed fashion, but it's more efficient and faster to have Twitter do this work for you automatically; this can be arranged by speaking with the Twitter developers (now is a good time to subscribe to twitter-development-talk).

An exercise for the reader: as written, just like in our logger example, every time the bot starts it will go through its most recent 20 DMs all over again, even if it had already processed them previously. Change this example to use a bookmark as well (hint: $dmconclude and $last_dm).

Tying it all together: a publicly-accessible TTYtter-powered application

The result is Twitpher (gopher URL), a gateway that reads the public timeline or a specified user's timeline and displays it as a formatted gopher menu. If your browser cannot handle gopher URLs, here is a HTTP proxied link to see what it looks like.

 _          _ _        _            
| |___ __ _(_) |_ _ __| |_  ___ _ _ 
|  _\ V  V / |  _| '_ \ ' \/ -_) '_|
 \__|\_/\_/|_|\__| .__/_||_\___|_|  
Gopher-Twitter   |_|   gateway

Reload for most current tweets from the public timeline.  	 	 	
Click a tweet to see more from that user.

Here is the source for the library, which will open in a new window so you can refer to it. It requires 0.7.0+.

Twitpher uses TTYtter as its client rather than running as a bot. It accomplishes this in the following ways:

• It obtains arguments from the gopher executable stub. The stub passes the argument using -twarg, although you could also use an environment variable for a similar effect. Twitpher then sanitizes the username to prevent various kinds of naughtiness.

• It establishes its custom environment, including forcing anonymous mode on and overriding several options like -seven, which is usually not helpful in Gopher.

• It dups standard output to a custom filehandle it writes its output to so that output is limited purely to what the library generates, and -silent suppresses the rest. (This is possible because the code in TTYtter to squelch standard output is intentionally placed after the library's initialization.) However, because the library name is printed before it can specify the -silent option, the gopher executable stub passes -silent too to make it quiet. Note that dupping standard output is no longer needed in 0.8.0 to achieve this effect -- see the DUPSTDOUT filehandle.

• It implements a small gopher library, using anonymous subroutine references stored in $Lib_ globals.

• It takes over the console and drives TTYtter itself with the $console subroutine reference, which is described below. Twitpher takes advantage of the asynchronous nature of the console by setting a timeout with sleep. In the background, if the timeline was successfully downloaded or an error is propagated, then after the tweets are displayed (if any), $conclude kills the parent process to end execution and the timeout is not reached. On the other hand, should the background process hang waiting for a response, the timeout will occur and the hijacked console will display a message and instead kill the child background process to end execution.

• Handlers are set for direct messages (in this case to ignore them, and none should be received in any case), tweets and error conditions.

If you've created an application or bot that uses TTYtter, I would love to know about it and possibly include it in a list of TTYtter-powered applications. Please send a description to ckaiser@floodgap.com.

API reference

Superclassable subroutine references

The API is not available for TTYtter versions prior to 0.4. Subroutine references are specified with minimum supported version; if none is given, it is available in any version from 0.4 higher.

$conclude (no arguments)

Called at the end of each cycle of tweet processing. Default behaviour is to do nothing additional. Return value, if any, is discarded.

$console (no arguments) (0.6+)

Master loop to manage the console. Default behaviour is to initialize the history, print an initial prompt, and then accept data line by line from standard input until a terminating command is received or the input stream ends. Return value, if any, is discarded, and TTYtter will terminate completely when the routine is exited.

This hook can also be used to run code after initialization of the console but prior to accepting user input. If so, your code should end with something like goto &defaultconsole; to transparently return code to the default handler.

$dmconclude (no arguments) (0.6+)

Called at the end of each cycle of direct message processing. Default behaviour is to do nothing additional. Return value, if any, is discarded.

$dmhandle (argument 1: hash reference) (0.6+)

Called when a direct message is to be displayed or otherwise handled in some manner. The keys of the hash reference are based on those specified by the Twitter API. Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text true and false. The routine, naturally, is not obligated to generate any output if it desires. Default behaviour is to display screen name, time stamp provided by Twitter, and the text of the direct message. For success, the number of "logical DMs" handled (almost always one) should be returned. If the DM was declined for processing, a zero value should be returned.

In 0.8.0+, you can pass your hash reference to &standarddm for the "default" formatting, which will return a string formatted according to whatever standard options are set (such as -timestamp, -ansi, etc.)

$exception (argument 1: exception number, argument 2: exception text)

Called when a non-fatal exception is received during processing of tweets. Argument 2 is guaranteed to be human readable text corresponding to argument 1, which comes more or less from this table:

• 1: timeout or no data
• 2: Twitter error or status message received
• 3: hit rate limit (0.5.x only, deprecated in 0.6+)
• 4: unexpected HTTP return code not caught by other exceptions (0.8.2+)
• 10: JSON cut (unexpected end)
• 11: JSON null list (missing or corrupt array reference)
• 12: JSON absolute null list (list with no members received) (0.5.x only, deprecated in 0.6+)

The exception number code is provided simultaneously to facilitate localization or custom notification. Exceptions passed to $exception are designed to be informative only, as TTYtter can recover from these errors and automatically try again. Fatal errors are raised immediately and the library does not receive notification for technical reasons. Default behaviour is to print the error text to standard output. Return value, if any, is discarded.

Twitter is now using a generalized error reporting method to indicate server-based exceptions, including hitting the API rate limit. All of these errors are swallowed up under code 2; to distinguish them, check the error text. Although the old rate-limit trigger message is still parsed for, and can still theoretically generate code 3, this message has been replaced by the new reporting convention in practise.

Code 12 is no longer reported as a null list can be valid in some contexts. However, this is generally transparently intercepted silently for you in contexts that it is not.

$handle (argument 1: hash reference, [optional] argument 2: origination)

Called when a tweet is to be displayed or otherwise handled in some manner. The keys of the hash reference are based on those specified by the Twitter API. Note that as a side effect of Perl's interpretation of the JSON, logical true and false in Boolean fields are rendered as literal text true and false. The routine, naturally, is not obligated to generate any output if it desires. Default behaviour is to display screen name and tweet text. For success, the number of "logical tweets" handled (almost always one) should be returned. If the tweet was declined for processing, a zero value should be returned.

In 0.8.0+, you can pass your hash reference to &standardtweet for the "default" formatting, which will return a string formatted according to whatever standard options are set (such as -timestamp, -ansi, etc.)

In 0.8.1+, an optional origination argument is also passed, giving the command that caused the tweet to be displayed. Origination classes defined currently are a null string, meaning a new tweet; replies meaning tweets from the /replies command; or again meaning old tweets usually from the /again command). The distinction is important; replies does not appear on new replies, but only on replies that you ask for. This is on purpose to ensure that API activity results are consistent and match up. Note that again overrides replies, and since /again can sometimes pull up new tweets, the originator is blanked on purpose for those new ones so they are properly seen as new. The complexity here is mostly intended for those clients who want to distinguish old and new tweets, or tweets that a user requested versus tweets that were automatically fetched, and handle them differently.

$heartbeat (no arguments) (0.6+)

Called at the beginning of each automatic refresh cycle (in either daemon or interactive mode). Default behaviour is to do nothing additional. Return value, if any, is discarded.

$precommand (argument 1: command prior to processing) (0.8.2+)

Called as soon as a command is received for processing, even before history substitution. Allows you to implement your own preprocessing. The new command should be returned as a single response, and is subject to things like % substitution and so on. Default behaviour is to just return the same command without further pre-substitution.

$prepost (argument 1: tweet prior to posting) (0.8.2+)
$postpost (argument 1: tweet after posting) (0.8.2+)

These two are paired, so they are listed together. $prepost is called when a tweet is about to be URL-encoded and sent, allowing you to implement your own tweet preprocessor (such as, say, a URL-shortening service). The new tweet should be returned as a single response. After the tweet is posted, $postpost is called with the final tweet (which barring an act of God or cosmic radiation should be the same as what $prepost returned), which is useful for tools such as loggers. Default behaviour for the former is to simply return the same tweet without further pre-substitution, and for the latter, to do nothing.

$prompt (no arguments) (0.7+)

Called every time a prompt is to be displayed by the console. Default behaviour is to display TTYtter>, followed by a separating space. If ANSI colour is enabled, the prompt is displayed in cyan.

Library routines

&descape (argument 1: data, [optional] argument 2: Unicode mode)

Remove internal quote-escaping for entity data, convert escaped UTF-8 characters into their correct form, and return the de-escaped data. If the data contains UTF-8 entities and UTF-8 is disabled with -seven, then the entities are rendered as dots ('.'). If optional argument 2 is true, then UTF-8 entities are rendered in HTML "ampersand" form, even if UTF-8 is "off." This is particularly useful for web applications. Starting in version 0.5.1, &descape will also convert many ampersand-escaped entities into ASCII as well, unless argument 2 is true (in which case it assumes that you wish them to remain ampersand-escaped like the UTF-8 entities will be). The processed data is returned.

&screech (argument 1: error text)

Emit error text to standard output (ring the bell if supported), and kill and shutdown immediately. Used as an escape hatch for unsafe situations, or fatal errors. Note that this bypasses $conclude, and as such, no further notification is given to the library that a fatal shutdown has occurred. This routine does not return.

&updatest (argument 1: status text, [optional] argument 2: interactive mode)

Update the current user's status with status. If optional argument 2 is true, any error condition will also be displayed on standard output. A status value is returned: zero if the post was successful, or a return code dependent on Lynx or curl if not (see respective documentations). Return code 99 indicates that the subprocess could not even start, possibly due to change in the filesystem or permissions.

&standardtweet, &standarddm (argument 1: hash reference) (0.8+)

These routines return the default pre-formatted string according to any user-specified arguments for a tweet or DM respectively indicated by the hash reference passed it. See $handle and $dmhandle respectively.

Exposed globals

$TTYtter_VERSION

The current major/minor version of TTYtter (currently a float value [so version 0.7 is represented as 0.7]). This doesn't include the patch level; see the next variable for that.

$TTYtter_PATCH_VERSION

The current patch level of this release of TTYtter, loaded into a separate scalar for backwards compatibility; although this was not defined prior to 0.5.1, the patch level of previous versions can be safely assumed to be patchlevel 0, and as such a nice idiom is to use (0+$TTYtter_PATCH_VERSION) which is guaranteed to give zero for old versions and will not bug out on current ones. This is an integer, so $TTYtter_VERSION 0.5 and $TTYtter_PATCH_VERSION 1 indicates version 0.5.1.

$url $lynx $curl $pause $user $seven $lib $daemon $verbose $script $ttytteristas $avatar $superverbose $hold $status $update $dmurl $frurl $dmpause $silent $anonymous $ansi $noansi $timestamp $uurl $rurl $wurl $maxhist $twarg

These are the manifestations of the valid command line options, set to their respective specified values (or 1 for true Boolean values). For the meanings of these options, see the main page.

$twarg is special as it is explicitly designated as user-defined, i.e., it's your way of passing external data or arguments into your code. Don't trust it, do sanitize it if you're not the source. It is available in 0.7+.

$parent $child

The PID of the parent and child processes. In interactive mode, both are defined; in daemon mode, only the latter (the parent ID becomes zero). Neither should be modified, or processes may not get properly terminated.

$last_id

The last/highest tweet ID so far processed. It starts at zero, but may be advanced to skip tweets as $handle (or &defaulthandle where not specified) is only called for new tweets, viz., tweets with an ID higher than $last_id. Even if $handle returns zero for an arbitrary tweet, that tweet's ID is still considered for $last_id.

$last_dm

Analogously, the last/highest direct message ID so far processed. Its behaviour is exactly the same as $last_id, including starting from zero on startup, and even if $dmhandle returns zero for an arbitrary DM, that DM's ID is still considered for $last_dm.

$lasttwit (0.8.1+)

The last successful tweet (empty if no tweets have been made). This is not carried from session to session.

DUPSTDOUT (0.8+)

The DUPSTDOUT filehandle is defined in 0.8+ and allows you to emit to standard output even if everything else is being suppressed with -silent. As its name suggests, it is a simple dup(3) of standard output. -silent forces output to DUPSTDOUT, but only from &defaulthandle and &defaultdmhandle; everything else still goes to STDOUT.