Idées Distribuées – Distributed Ideas

Sorry, this entry is only available in Français.

Sorry, this entry is only available in Français.

As for my previous article, I would like to talk about a not well known subject: How to color your console?

As administrator, and in particular when my eyes are tired, I am happy to read “OK” in green and “Not OK” in red. If you google with this subject, you will find many pages with simple tricks, but few with good explanations.

The purpose of this article is not to give hints about colorization, but to explain how it works and the main differences between a Linux terminal and a Windows console.

Linux terminal

History

A long time ago, a time that the youngest can not know, a computer terminal referred to an hardware device that was used for entering data into (a keyboard), and displaying data (a screen) from a computer. Somehow the end of a network.

VT100 was a video terminal. Its detailed attributes became the de facto standard for terminal emulators.
It communicated with its host system over serial lines using the ASCII character set and control sequences (a.k.a. escape sequences) standardized by ANSI. The VT100 was also the first Digital mass-market terminal to incorporate “graphic renditions” (blinking, bolding, reverse video, and underlining) as well as a selectable 80 or 132 column display.

Today we use mostly virtual terminals (in Linux, the first six virtual terminals provide a text terminal with a login prompt to a Unix shell) or emulators as xTerm that provides VT102 compatible terminals.

Here few escape sequences commonly used by VT100 compatible terminals.

How to colorize my VT100 terminal

The following lines should show “OK” in green and “Not OK” in red.

echo -e '\033[32m OK \033[0m'
echo -e '\033[31m Not OK \033[0m'

The previous script mingles Bash and VT102 escape sequences. Bash interprets and replaces "\033" by the escape ASCII code. Then the VT102 emulators interpret "<ESC>[32m" as set the foreground color to green.

Portability

A good script should work not only on your computer but everywhere. So we should test the capabilities of our environment before relying on it. I found a lot of examples on Internet that perform tests based on $SHELL or $TERM environment variables. From my experience, these are not a good practices as I will explain.

Shell capabilities

$SHELL contains the login Shell, not the current Shell. Therefore scripts that use it are often buggy and people that use several Shells (like me) are very unhappy.

Simple is beautiful! I advice to do a non-portable script or to insert directly the escape character into your script in order to be independent of the Shell (its implementation of the escape characters and on its possible echo builtin command).

Good text editors show special characters, but it is painful to write "<ESC>" on a keyboard (on Windows you can press "ALT" and type "0027" on the keypad...)

echo '<ESC>[32m OK <ESC>[0m'
echo '<ESC>[31m Not OK <ESC>[0m'

Terminal capabilities

A script should check the terminal capabilities before to use VT102 color sequences. We can use the $TERM environment variable and the "test -t fd" conditionnal expression. But it is not easy.

$TERM can take a lot of different values (aterm, xterm, cygwin, etc.) Therefore it is difficult to create an exhaustive list of all terminals that understand the color sequences. Elsewhere, if you are bound to a pseudo-terminal it is not always possible to know the real capabilities of the real end-terminal. And I already experienced few bugs so...

Finally, I think the best practice is to implement a color parameter as the ls command:

• never means the script will not generate VT102 color sequences
• always means the script will generate VT102 color sequences
• auto is where you try to implement a smart default behavior

function print_help() {
	cat <<EOF
# ...
       --color[=WHEN]
              control  whether color is used to distinguish file types.  WHEN
              may be 'never', 'always', or 'auto'
# ...
EOF
}

#...

if [ $color = always -o $color = auto -a -t 1 ]
then
	function print_KO() { echo '\033[31mKO\033[0m'; }
	function print_OK() { echo '\033[32mOK\033[0m'; }
else
	function print_KO() { echo 'KO'; }
	function print_OK() { echo 'OK'; }
fi

Windows console

History (source Wikipedia)

MS-DOS 1.0 did not support the ANSI or any other escape sequences. Only a few control characters (CR, LF, BS) were interpreted.

MS-DOS 2.0 introduced the ability to add a device driver for the ANSI escape sequences – the de facto standard being ANSI.SYS. This continued to work through Windows 98. Extreme slowness and the fact that it was not installed by default made usage by software almost non-existent.

Console windows in Windows versions based on NT (Windows NT 4.0, Windows 2000, Windows XP, Windows Server 2003, Windows Vista and Windows Server 2008) do not support ANSI escape sequences at all. Softwares can manipulate the console with the ioctl-like system Console API.

Few examples

Many script languages (Perl, Tk, and so on) provide library based on the Console API to change the color of a Windows console. If you use them, the commands to change colors is interlaced with the text output.

Another solution is to use a library that provides an ANSI support. So POSIX guys can keep their habbits.

So how you would color your Windows console depends on the programming langage you chose and its helper libraries.

MS-DOS

MS-DOS is a very poor language (not only for color). We can change the color of the entire Windows console, but we cannot change the color of a particular line.

REM MS-DOS example
REM Creates a Black background with light bright green text
COLOR 0A

My advice, do not use MS-DOS to colorize your output or do not use MS-DOS at all if you can.

PowerShell

I'm not completely a fan of PowerShell because of some strange syntaxes, supposed to make life simpler. But it is a very powerful language and, obviously, we can change color output easily.

Here is a powershell function that I use in my daily work. It is able to print a string with ANSI escape sequences to control colors. I use it basically when I run a remote command on a Linux machine.

### Author: Cyril Martin (mcoolive)

function Write-ColoredHost
{
	Param(
		[switch]$noNewLine,
		[String]$separator=" ",
		[System.ConsoleColor]$foregroundColor = $HOST.UI.RawUI.ForegroundColor,
		[System.ConsoleColor]$backgroundColor = $HOST.UI.RawUI.BackgroundColor
	)
	BEGIN {
		$fgc, $bgc = $foregroundColor, $backgroundColor
	}
	PROCESS {
		$token, $coloredTokens = ([Regex]"\033\[").split($_)
		Write-Host $token `
				   -noNewLine -separator $separator `
				   -foregroundColor $fgc `
				   -backgroundColor $bgc
		foreach ($coloredToken in $coloredTokens) {
			$colorCodes, $token = ([Regex]"m").split($coloredToken, 2)
				([Regex]";").split($colorCodes) | foreach-object {
					switch -regex ($_) {
						"^0{1,2}$" { $fgc, $bgc = $foregroundColor, $backgroundColor }
						"^30$"     { $fgc       = "Black"                            }
						"^31$"     { $fgc       = "Red"                              }
						"^32$"     { $fgc       = "Green"                            }
						"^33$"     { $fgc       = "Yellow"                           }
						"^34$"     { $fgc       = "Blue"                             }
						"^35$"     { $fgc       = "Magenta"                          }
						"^36$"     { $fgc       = "Cyan"                             }
						"^37$"     { $fgc       = "White"                            }
						"^39$"     { $fgc       = $foregroundColor                   }
						"^40$"     {       $bgc =                   "Black"          }
						"^41$"     {       $bgc =                   "Red"            }
						"^42$"     {       $bgc =                   "Green"          }
						"^43$"     {       $bgc =                   "Yellow"         }
						"^44$"     {       $bgc =                   "Blue"           }
						"^45$"     {       $bgc =                   "Magenta"        }
						"^46$"     {       $bgc =                   "Cyan"           }
						"^47$"     {       $bgc =                   "White"          }
						"^49$"     {       $bgc =                   $backgroundColor }
					}
				}
			Write-Host $token `
					   -noNewLine -separator $separator `
					   -foregroundColor $fgc `
					   -backgroundColor $bgc
		}
		if (!$noNewLine) { Write-Host }
	}
}

### A little test
$plink= # path of plink (see PuTTY)
$linuxServer= # hostname
$login= # username
&"$plink" -ssh $login@$linuxServer ls --color=always / | Write-ColoredHost

Sorry, this entry is only available in Français.

Sorry, this entry is only available in Français.

Sorry, this entry is only available in Français.

Sorry, this entry is only available in Français.

Question : what will happen with the following code ?

List suits = ...;
List ranks = ...;
List sortedDeck = new ArrayList();

for (Iterator i = suits.iterator(); i.hasNext(); )
    for (Iterator j = ranks.iterator(); j.hasNext(); )
        sortedDeck.add(new Card(i.next(), j.next()));

A : nothing, the code will run without error
B : the code cannot be compiled
C : the code cannot be executed
D : an exception will be thrown at runtime

=== Response : Here the code could be compiled and executed without errors. But we 
will potentially get a NoSuchElementException because the the next method is being 
called too many times on the “outer” collection (suits). It is being called in the 
inner loop for both the outer and inner collections, which is wrong. We could reach 
the end of the "outer" collection before the "inner" one.

I’ve faced an error recently on a project that I visited where varargs usage caused a weird issue.

Well I guess everybody knows what Varargs is.
Basically varargs will allow you to specify a method that can accept optional several arguments. Defining a method that accept varargs is as simple as

public void doSomething(String... values) {
// do something here
}

allows users to call the method without any parameters or with any number of parameters:

public void callDoSomething() {
doSomething();
doSomething("a","b","c");
}

Well that’s a very convenient way to pass a kind of array or arguments a method can work with, it also allows you to be very flexible on method usage for optional or variable parameters.
So what’s the trick ? When java compile the code it actually creates a method that takes an array as a parameter:

public void doSomething(String[] values) {
// do something here
}

and then from the client would compile the code in the following way:

public void callDoSomething() {
doSomething(new String[0]);
doSomething(new String[]{"a", "b", "c"});
}

Well that’s sounds great, and this makes things a bit easy to write… So what’s the risk here ?

First is that of course most of people doesn’t know that Java generates an array and people may think that it’s a kind of runtime feature of the JDK.
And what’s the issue when you don’t understand things ? Well you may then not understand the associated risks:

Let’s say that I’m working in a team A and I have to deliver a jar tool to another team B.

I first released a version of my jar that contains the following class:

public void doSomething() {
// do something here
}

The B team guy is using my great doSomething method using the following method:

public void callDoSomething() {
doSomething();
}

In a next release I add a new optional parameter to my method thanks to Java VarArgs. That way people doesn’t have to change their code to keep using my method and other people can benefits from extended parameters if they want to benefit from the new behavior!

public void doSomething(String... params) {
// do something here
}

B team for some other reasons want to use my new version so they update their project’s dependency to use my new jar.
B team are good guys, they have automated tests that covers all their application and especially the interraction with A team jar, everything is running fine.

In order to not send the full application to their many servers (B team deliverable is heavy) they decide to update only my jar, that was the only update!
At runtime and even if everything is compiling correctly they will get the following error:

java.lang.NoSuchMethodError: fr.fastconnect.java.blog.varargs.MyProviderClass.doSomething()V
at fr.fastconnect.java.blog.varargs.UsingClass.main(UsingClass.java:10)

Well so just keep that in mind, nothing is magic so be careful. And from a provider perspective remember that even adding a varargs may in fact sounds like breaking somehow the API!

Sorry, this entry is only available in Français.