Monday, December 7, 2009

VMWare server 2 "empty" screen with "Loading..." in the title

I've installed the VMWare server, and as many, hit the issue of an empty screen.

It appears to happen upon the subsequent logins. Obviously "empty screen" can be caused by a lot of errors, but in my case it seems to be some SSL-related issue, in /var/log/vmware/hostd.log I saw:

[2009-12-07 11:58:12.150 'Proxysvc' 1093462352 warning] SSL Handshake on client connection failed: SSL Exception:
[2009-12-07 11:58:12.316 'Proxysvc Req00102' 1081919824 warning] PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)
[2009-12-07 11:58:12.573 'Proxysvc Req00101' 1081919824 warning] PendingServerStrm: client write-completion error: SSL Exception: error:00000001:lib(0):func(0):reason(1)

What seems to help - is holding the Control key and hitting the "reload" button when I am on the page that does not show anything.

Then the page reloads fully and I get the login prompt to login into the VM.

Hope this is useful to someone.

I later tried on the ff-3.5, and there Ctrl-Shift-R for reload, apparently did the trick as well.

Tuesday, November 3, 2009

Recording sound and video from a webcam

I've found a lot of questions on the internet on how to record the sound and video at the same time from the webcam. Here's the magic incantation that worked well for me (it assumes you do not have any video capture devices besides the webcam and already have one ALSA soundcard besides the mic of the webcam - if not, adjust accordingly). This is a single long line:

mencoder tv:// -tv driver=v4l:width=320:height=240:device=/dev/video0:fps=30:forceaudio:alsa:adevice=hw.1 -ovc lavc threads=2 -ffourcc xvid -oac mp3lame -lameopts cbr=128 -o recording.avi

EDIT: with a purchase of the Sony Playstation Eye camera, the above does not work all that well at all - at the high rate my mencoder misses too many frames. Something I found that is a bit better:

gst-launch v4l2src ! 'video/x-raw-yuv,width=640,height=480,framerate=(fraction)25/1' ! mux. alsasrc ! audioconvert ! 'audio/x-raw-int,rate=44100,channels=2' ! mux. avimux name=mux ! filesink location=test.avi

This lacks the compression, and the voice is a bit dragging compared to video - but gives more or less the 30fps that the webcam sends.

Sunday, June 14, 2009

configure: error: C++ preprocessor "/lib/cpp" fails sanity check

A fairly odd message, especially given that 'which cpp' shows another location.

The solution is:

sudo aptitude install build-essential

Thursday, May 28, 2009

F*ctor - a 0-dimensional Turing tarpit.

Following a post of one of my friends to the http://manage.yore.ma/eso/, I felt a bit inspired,
so I wanted to write up something fun. So, here's the experiment - an imaginary language which holds the entire program and data memory in a single variable!

The F*ctor machine has only other elements are a single instruction, and a single variable which holds the Program, and Data, and the machine state, in just a single natural number X. The "single instruction", however, is a bit difficult to describe, so I will attempt to describe it in terms of smaller "micro-operations" - they do not really change nor are visible to the user, they are here mostly for the sake of explanation. The temp variable in the pseudocode is just for the sake of ease of comprehension.

The machine can really efficiently calculate Prime(N) (being the Nth prime, that is Prime(0) == 2; Prime(1) == 3, Prime(2) == 5, etc.)

Also it can count how many times is the given number divisible by some prime: CountPrime(V, P); CountPrime(2,2) == 1, CountPrime(9,3) == 2; CountPrime(13,3) == 0.

Finally, the third operation is the destructive divide: DivideBy(V, P) - try to integer-divide V in-place by a prime P ith no remainder, and return True if we were successful, and False if no remainder-free division is possible.

The remaining operations that comprise the single cycle are some trivial arithmetics.

the single cycle of execution:


one_cycle(X) {
local CurrentP = CountPrime(X, 3);
while(DivideBy(&X, 2)) {
if(not DivideBy(&X, Prime(CountPrime(X, Prime(CurrentP))))) {
return X * 3 * 3;
}
}
return X * 2^CountPrime(X, Prime(CurrentP)) * 3;
}


So running the program consists of repeatedly executed "X = one_cycle(X)".

If we take a closer look, then this is nothing more than a folded version of Reverse-subtract and skip if borrow single-instruction machine, with "memory contents" at location N being the number of times the X is divisible by Nth prime number. As it is visible, the accumulator is memory-mapped at 0 (see these divisions/multiplications by 2?) and the program counter is memory-mapped at 1 (see the multiplications either by 3 or by 9 ?)

So, unless I made a bug somewhere in the logic, we have a Turing-complete absract machine squashed into a single variable.

Saturday, May 16, 2009

A "nod" X11 linux interface

Today I wanted to spend the evening doing something different and fun, and came up with an idea to play around with some of the opencv's functions. Well, did not play that much - just slightly tweaked the facedetect.c, to get what I called a "nod interface".

When you compile and launch the program, it pops up the screen with the video. If all goes well and you are sitting in front of your camera, you should see a circle around your face. If you do not see it - move closer and/or get a smoother background.

The few lines of pretty silly code that I've added does the following: calculates the two averages of the last N points of the center of the detected face object - "fast" (N=4) and "smooth" (N=8).

If you don't move your head the delta between these will be almost 0 - it gets forced down to zero by a simple check against the threshold.

If you nod - then obviously the "fast" average moves down faster than the "smooth" one - so if you nod energetic enough, the computer will notice it and will send a few "arrow down" synthetic events via the XTest X11 extension. If you manage to "nod up" (unnatural, but somewhat doable) - then it will send the equivalent number of "arrow up" events.

The counter-movement (returning the head back after the nod) is compensated by another simple logic - it's needed to have at least 4 "zero-delta" cycles in order to let the detected delta through.

The end result is that with some amount of luck you might manage to read things from your laptop hands-free. Should be good for reading the /. threads :-)

this file is the tweaked facedetect.c - the long line to feed to the compiler is in the comments in the beginning.

Depending on the speed of your computer, you might need to adjust some of the thresholds.

Enjoy!

Thursday, May 7, 2009

Hercules DJ Console RMX linux midi drivers

Yay! Open source and all. Get the MIDI out of the RMX.

Grab them here

Saturday, May 2, 2009

A very compact renderer

Been a while since I babbled something here.

So to ensure I'm still alive and kicking - here's something neat: a renderer in under 600 lines of code.

Minilight has a bunch of implementations in various languages.

A very nice toy.

Saturday, April 4, 2009

Linear interpolation code for smoothing out the Z-movement

Today I've got a dead-simple but pretty cute piece of code in. It has to do with smooth avatar movement when walking. The map of heights is stored with the resolution of 1 meter, while in the viewer it's smoothed out. This creates a "staircase"-like effect when walking on the slope.

The simple solution which I went to is to do the linear interpolation, with the following
approach. First we look at the heightmap "from the top" - just looking at its X and Y coordinates. And logically we split the X-Y space into a bunch of 1-sized squares. So we always
have our x,y point for which we are calculating, inside one of these small tiles. So we can zoom into this tile and take further look:


(x0,y0) (x0+1, y0)
| /
v v
+-----+---> X
|\ 1 /|
| \ / |
|4 + 2|
| / \ |
|/ 3 \|
+-----+ <-- (x0+1, y0+1)
|
v
Y


in this picture - the corners of the square are the integer X, Y values which have the data.

The point in the center is the "virtual" point which is used to split the square into four triangles - it's x,y coordinates are simply X0+0.5, Y0+0.5.

Then we can pretty simply find which of the triangles holds our "interesting" point - now it's all dead simple - find the equation of the plane that passes through the three points the coordinates of which we know (the Z coordinate of the "center" point is the average of the Z coordinates of all 4 corners) - and then simply substitute the X, Y into the equation, to get Z straight away.

I don't have walking animations yet - for now I simply set the Z coordinate of the avatar, which gets its butt approximately at ground level :-) While running around, if one would measure by the proportions to the real world, the staircase effect is reduced to almost unnoticeable ~10cm if we relate to the real-world body proportions. And I hope that once I get the avatar's butt out of the ground, it should be even better visually.

Now, the obvious question - why 4 triangles, instead of 2 ? Because if you take the following layout:

0---*
| |
*---0


and the "*" is "high" an "0" is "low" - depending on the choice to cut this X-Y box, you get either a pit, or a spike. 4 triangles handle this case noticeably better.

Monday, March 30, 2009

Cosimus now with avatar shapes and clothing

Today I took a brute-force approach to solving the missing avatar clothing problem that
I was chasing for a few days - simply by creating my own set of the default clothing and wearables.
It's just the standard set of items which are uploaded by the viewer in the case it does not find them - so for the secondlifers, it's Ruth in white clothing :)

Alongside, I've put a very simple moving logic - only forward, in the direction that the avatar faces. This required some rudimentary support for something the called "scene" in opensim - or at least my interpretation of it. For now, I am taking the assumption "Agent == Avatar", it probably will change sometime in the future. However, I already put the "clientstack-independent scene" atop the SMV one - to represent the qualities that are (or should be) fundamental to any avatar system - X, Y, Z position, AgentID - I'll use the UUIDs for representing the "entity" for now, and the "label" - some text that identifies the avatar as seen in-world.

The rest of the platform-dependent details go into a table called "Blob" which is at the moment SMV-specific data about the avatar (like, local object ID, which is needed for the viewer, as each avatar has its own local id to be drawn correctly).

Possibly exposing a "local avatar id" might have been a better move for the platform-independent record, and having the clientstack-specific lookup tables.. But I will see in the future what is better. In any case, each clientstack registers its own event handler functions, which are called per-avatar in case the need arises (this indeed raises some additional possibilities for optimization in the future).

Alongside the minor infra mods, the loginserver config is now in the config file (main.lua), as well as the serializable state to facilitate trivial no-outage restarts is inside a separate file.

Interestingly enough, there seems to be something odd with the object update packet - the omvviewer tends to hard-hang the laptop - I am not sure which part is causing that. And somehow I managed to get some green text atop the avatar at one occasion, which was funny.

As the vacation time is over now for me, most definitely the speed of the development will suffer - I'll only have some late nights, which are also good to be used for sleep, but I am pretty happy with the progress so far.

Sunday, March 29, 2009

Cosimus - a little toolkit for metaverse experiments

The past two weeks I've been on holiday - so this translated for doing various things that I did not have time to do before. One of those things was completing the chunk of coding needed to at least start with my little metaverse experiment.

The result is at http://github.com/ayourtch/cosimus/tree/master - and will suffer the periodic commits from now on, depending on my so called "spare time" :)

It's a start of the simulator code for the 3d simulator, for now I've started with the SecondLife(tm) protocol, as it's something I had in the beginning of the works.

For now it is a custom Lua interpreter, with a few libraries linked in and initialized by default, and the application code in prototype phase is being written in Lua. This gives the much faster speed of the development compared to C (a long while ago, while checking the speed of development on Perl vs. C I found about 10x difference - in favour of Perl, of course. I did not do any personal measurements now, but from a gut feeling it should be about the same). Much later, as needed, I'll push the performance-critical pieces of the code downwards to C.

For now the thing does not do all that much - these few days I've been mostly dealing with the inventory/asset code, appears there are some magic tweaks still needed - from the behaviour it seems like I am triggering some data corruptions in the client inventory code..

From the cool features that are already there - the inventory/asset library loading is done from zipfiles, which means easier handling of them for the consumer. Another fun feature - or rather design choice - is that all the communication via the components that is possible to be networked in the distributed setup, will be also networked in the standalone setup, with the simple servers doing the job from within the executable itself. So it is "batteries included", and a single code path to debug.

...Now, back to coding :-)

Wednesday, March 11, 2009

Compiling zettair on ubuntu

The first compile throws a bunch of errors, due to missing zlib, among them:

‘Z_NO_FLUSH’ undeclared

solution - sudo apt-get install zlib1g-dev, then rerun configure script, and all compiles fine.

Tuesday, March 10, 2009

Zettair is back alive

Hooray! Zettair folks down in Oz have revived the development. "in-development" version now online.

Let's see what we have (with a brutal diff -rc across the directories :-) Disclaimer: this is not necessarily a complete list, at least in the autoconf part - the diff was fairly boring and long.
Alternatively, I might have missed it in the set of whitelist changes or did not get a clue it was important...:)

configure script got what looks a 64-bit "kfreebsd". Probably inherited with a newer version of autotools, but an interesting beast nonetheless, if I read it well, it is here. GNU userland running on top of freebsd kernel.. The other tweaks belong to dragonflybsd. I heard about Hammer filesystem, but haven't tried the OS yet. Might be interesting. All depends on the H/w support.

index.h:

option INDEX_NEW_VOCAB is gone, INDEX_NEW_PARSEBUF (dictate how large the postings hashtable is) appeared.

INDEX_NEW_NO_OFFSETS - build the index without offsets. Probably not very interesting for me...

INDEX_NEW_QTHREADS - how many query threads to allow. Now I wonder if this allows for simultaneous query and indexing - that'd be sweet. My experiments with the previous version running separate processes were not terribly successful. With threads indeed it might be a different story.

some other similar parameter changes to index_load (as opposed to index_new)

index_cleanup is gone.

INDEX_SEARCH_DYNAMIC_RANK - provide the text describing the rank to be used as well as the rank parameters. Interesting, probably a bit too advanced for my needs.

autogenerated code for the metrics is obviously now parametrized.

mutexes for various pieces (For multithreading)

vocab_vector structure is simplified (looks like thanks to getting rid of multiple files)

a lot of multithreaded-oriented code

a hacko-fix in makeindex_append_docno to get around the fact that zettair can't handle docno-s bigger than a pagesize

vocabury operations seem to be simplified - previously the memory management was more sophisticated.

This completes the list - again, very probably I missed something important - if so, drop me a note in the comments.

Monday, March 9, 2009

LLSD scanning with ragel-made parser.

I've written a while ago about the Ragel.

Today evening (literally in about 4 hours), I tried it and cooked a semi-functional parser for LLSD XML serialization. Yes, the code below was cranked from scratch in just a few hours :)

What's nice is that it is can parse the data by chunks of practically any size, so could be used to grab the data over a slower TCP connection. And - the base64 decoding is included! Things that would need to be done in order to make it usable include proper error handling and a better approach to handle possibly large data - right now it is kind of dumb...

Here's the result of running it over the sample XML from the draft:


$ ./a.out `cat ../sample.llsd`
Array start
Integer: '42'
UUID: '6bad258e06f04a87a659493117c9c162'
Map start
String: 'cold'
Map member with key 'hot':'cold'
Undef
Map member with key 'higgs_boson_rest_mass':''
URI: 'https://example.org/r/6bad258e-06f0-4a87-a659-493117c9c162'
Map member with key 'info_page':'https://example.org/r/6bad258e-06f0-4a87-a659-493117c9c162'
Date: '2008-10-13T19:00.00Z'
Map member with key 'status_report_due_by':'2008-10-13T19:00.00Z'
Map end
Array end
result = 1


And here goes the .rl code in case you're curious (maybe you need to "view source" in case it gets garbled):


#include
#include
#include

typedef struct llsd_scanner {
char *accum;
int accum_size;
int accum_index;

char *key_accum;
int key_accum_size;
int key_accum_index;

unsigned char *bin_accum;
int bin_accum_size;
int bin_accum_index;
unsigned char b64chars[4];

// ragel variables
int *stack;
int stack_size;
int top;
int act;
int cs;
unsigned char *ts, *te;

} llsd_scanner_t;



static const char cd64[]="|$$$}rstuvwxyz{$$$$$$$>?@ABCDEFGHIJKLMNOPQRSTUVW$$$$$$XYZ[\\]^_`abcdefghijklmnopq";

static void llsd_scanner_putb64char(llsd_scanner_t *scanner, int index, unsigned char v)
{
v = (unsigned char) ((v < 43 || v > 122) ? 0 : cd64[ v - 43 ]);
if( v ) {
v = (unsigned char) ((v == '$') ? 0 : v - 61);
}
if (v) {
scanner->b64chars[index] = (unsigned char) (v - 1);
} else {
scanner->b64chars[index] = v;
}
}

static void b64decodeblock( unsigned char in[4], unsigned char out[3] )
{
out[ 0 ] = (unsigned char ) (in[0] << 2 | in[1] >> 4);
out[ 1 ] = (unsigned char ) (in[1] << 4 | in[2] >> 2);
out[ 2 ] = (unsigned char ) (((in[2] << 6) & 0xc0) | in[3]);
}


static int append_binchars(llsd_scanner_t *scanner)
{
if (scanner->bin_accum_index + 4 < scanner->bin_accum_size) {
b64decodeblock(scanner->b64chars, &scanner->bin_accum[scanner->bin_accum_index]);
scanner->bin_accum_index += 3;
return -1;
} else {
printf("Bin accumulator overflow!");
return 0;
}
}

static void clear_accum(char *accum, int *accum_index)
{
*accum_index = 0;
}

static int append_accum(char *accum, int accum_size, int *accum_index, unsigned char c)
{
if (*accum_index < accum_size-1) {
accum[(*accum_index)++] = c;
return -1;
} else {
return 0;
}
}

static void terminate_accum(char *accum, int *accum_index)
{
accum[*accum_index] = 0;
}




%%{

machine LLSDScanner;

access scanner->;



action terminate_accum {
terminate_accum(scanner->accum, &scanner->accum_index);
printf("Accumulated: '%s'\n", scanner->accum);
}
action clear_accum {
clear_accum(scanner->accum, &scanner->accum_index);
clear_accum(scanner->bin_accum, &scanner->bin_accum_index);
}
action append_accum {
append_accum(scanner->accum, scanner->accum_size, &(scanner->accum_index), fc);
}

action print_key_accum {
terminate_accum(scanner->key_accum, &scanner->key_accum_index);
// printf("Key Accumulated: '%s'\n", scanner->key_accum);
}

action clear_key_accum {
clear_accum(scanner->key_accum, &scanner->key_accum_index);
}
action append_key_accum {
append_accum(scanner->key_accum, scanner->key_accum_size, &scanner->key_accum_index, fc);
}

action convert_string {
terminate_accum(scanner->accum, &scanner->accum_index);
printf("String: '%s'\n", scanner->accum);
}
action convert_uuid {
terminate_accum(scanner->accum, &scanner->accum_index);
printf("UUID: '%s'\n", scanner->accum);
}
action convert_uri {
terminate_accum(scanner->accum, &scanner->accum_index);
printf("URI: '%s'\n", scanner->accum);
}
action convert_base64 {
terminate_accum(scanner->bin_accum, &(scanner->bin_accum_index));
printf("Binary Accumulated: '%s'\n", scanner->bin_accum);
}
action convert_date {
terminate_accum(scanner->accum, &(scanner->accum_index));
printf("Date: '%s'\n", scanner->accum);
}
action convert_integer {
terminate_accum(scanner->accum, &(scanner->accum_index));
printf("Integer: '%s'\n", scanner->accum);
}
action convert_real {
terminate_accum(scanner->accum, &(scanner->accum_index));
printf("Real: '%s'\n", scanner->accum);
}
action convert_undef {
terminate_accum(scanner->accum, &(scanner->accum_index));
strcpy(scanner->accum, "");
printf("Undef\n");
}

action assign_map_member {
printf("Map member with key '%s':'%s'\n", scanner->key_accum, scanner->accum);
}

one_or_two = /[12]/ @append_accum;
zero_or_one = /[01]/ @append_accum;
zero_to_nine = /[0-9]/ @append_accum;
zero_to_two = /[0-2]/ @append_accum;
zero_to_three = /[0-3]/ @append_accum;
zero_to_five = /[0-5]/ @append_accum;
dot = '.' @append_accum;
dash = '-' @append_accum;
colon = ':' @append_accum;
dot_or_colon = /[.:]/ @append_accum;
letter_T = 'T' @append_accum;
letter_Z = 'Z' @append_accum;

date_fullyear = one_or_two . zero_to_nine . zero_to_nine . zero_to_nine;
date_month = zero_or_one . zero_to_nine;
date_mday = zero_to_three . zero_to_nine;
time_hour = zero_to_two . zero_to_nine;
time_minute = zero_to_five . zero_to_nine;
time_sec = zero_to_five . zero_to_nine;
time_secfrac = dot . zero_to_nine;
full_date = date_fullyear . dash . date_month . dash . date_mday;
partial_time = ( time_hour . colon . time_minute . colon . time_sec . time_secfrac | time_hour . colon . time_minute . dot_or_colon . time_sec );
date_time = full_date . letter_T . partial_time . letter_Z;

maybe_space = space*;

some_string_char = /[^<]/ @append_accum;
some_string = some_string_char*;

some_key_string_char = /[^<]/ @append_key_accum;
some_key_string = some_key_string_char*;

# base64 parsing. Base64 characters come in series of 4

base64_char = [ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+=/];
action base64_one { llsd_scanner_putb64char(scanner, 0, fc); }
action base64_two { llsd_scanner_putb64char(scanner, 1, fc); }
action base64_three { llsd_scanner_putb64char(scanner, 2, fc); }
action base64_four { llsd_scanner_putb64char(scanner, 3, fc); }
action base64_four_chars { append_binchars(scanner); }

base64_first_char = base64_char @base64_one;
base64_second_char = base64_char @base64_two;
base64_third_char = base64_char @base64_three;
base64_fourth_char = base64_char @base64_four;

base64_four_chars = maybe_space . base64_first_char . maybe_space . base64_second_char . maybe_space .
base64_third_char . maybe_space . base64_fourth_char @base64_four_chars;

base64_string = base64_four_chars+;

tag_start = maybe_space . '<';
tag_end_nospace = ' tag_end = maybe_space . tag_end_nospace;

llsd_tag = /llsd>/i;

undef_tag = /undef\/>/i;
boolean_tag = /boolean>/i;
integer_tag = /integer>/i;
real_tag = /real>/i;
uuid_tag = /uuid>/i;
string_tag = /string>/i;
date_tag = /date>/i;
uri_tag = /uri>/i;
binary_tag = /binary/i . (space . /encoding="base64"/i)? . '>';

key_tag = /key>/i;
map_tag = /map>/i;
array_tag = /array>/i;

action call_array {
printf("Array start\n");
fcall array_element;
}
action call_map {
printf("Map start\n");
fcall map_element;
}
action return_from_map {
printf("Map end\n");
fret;
}
action return_from_array {
printf("Array end\n");
fret;
}

real_number = /[0-9]*/ @append_accum;
integer_number = /[0-9]*/ @append_accum;

hex_char = /[0-9a-fA-F]/ @append_accum;
uuid_val = hex_char{8,8} . '-'? . hex_char{4,4} . '-'? . hex_char{4,4} . '-'? . hex_char{4,4} . '-'? . hex_char{12,12};


action error { printf("Error!\n"); }
some_element = tag_start . (undef_tag @convert_undef
| boolean_tag @clear_accum . ('true'|'false') . tag_end . boolean_tag
| integer_tag @clear_accum . integer_number . tag_end . integer_tag @convert_integer
| real_tag @clear_accum . real_number . tag_end . real_tag @convert_real
| uuid_tag @clear_accum . uuid_val . tag_end . uuid_tag @convert_uuid
| string_tag @clear_accum . some_string . tag_end_nospace . string_tag @convert_string
| date_tag @clear_accum . date_time . tag_end . date_tag @convert_date
| uri_tag @clear_accum . some_string . tag_end . uri_tag @convert_uri
| binary_tag @clear_accum . base64_string . tag_end . binary_tag @convert_base64
| array_tag @call_array
| map_tag @call_map
);



array_element := (some_element)* . tag_end . array_tag @return_from_array;

map_key_start = tag_start . key_tag @clear_key_accum;

map_key_end = '
map_member = map_key_start . some_key_string . map_key_end . some_element @assign_map_member;

map_element := (map_member)* . tag_end . map_tag @return_from_map;

body = (some_element)?;

xml_preamble = /xml[^>]*/i . '>';

llsd_body = tag_start . ( '?' . xml_preamble . tag_start )? . llsd_tag . body . tag_end . llsd_tag;

main := llsd_body @{ res = 1; };

}%%

%% write data;


void llsd_scanner_free(llsd_scanner_t *scanner)
{
if(scanner->stack) { free(scanner->stack); }
if(scanner->key_accum) { free(scanner->key_accum); }
if(scanner->accum) { free(scanner->accum); }
if(scanner->bin_accum) { free(scanner->bin_accum); }
free(scanner);
}

llsd_scanner_t *llsd_scanner_alloc(int key_accum_size, int accum_size, int bin_accum_size, int stack_size)
{
int failure = 0;
llsd_scanner_t *scanner = calloc(1, sizeof(llsd_scanner_t));
if (scanner) {
scanner->bin_accum = malloc(bin_accum_size);
if (scanner->bin_accum) {
scanner->bin_accum_size = bin_accum_size;
scanner->accum = malloc(accum_size);
if (scanner->accum) {
scanner->accum_size = accum_size;
scanner->key_accum = malloc(key_accum_size);
if (scanner->key_accum) {
scanner->key_accum_size = key_accum_size;
scanner->stack = calloc(sizeof(int), stack_size);
if(scanner->stack) {
scanner->stack_size = stack_size;
%% write init;
} else {
failure = 1;
}
} else {
failure = 1;
}
} else {
failure = 1;
}
} else {
failure = 1;
}
if (failure) {
llsd_scanner_free(scanner);
scanner = NULL;
}
}
return scanner;
}

int llsd_scanner_run(llsd_scanner_t *scanner, char *chunk)
{
char *p = chunk;
char *pe = chunk + strlen(chunk);
int res = 0;
// printf("Running parser on: %s, curr state: %i\n", chunk, scanner->cs);
%% write exec;
if (scanner->cs && res == 0) {
res = -1;
}
return res;
}


int main( int argc, char **argv )
{
int res = 0;
int i = 1;
llsd_scanner_t *scan;
if ( argc > 1 ) {
scan = llsd_scanner_alloc(1000, 1000, 32768, 100);
while(i < argc) {
res = llsd_scanner_run(scan, argv[i]);
if (res == 0) {
printf("Error on chunk %d\n", i);
i = argc;
} else {
}
i++;
}
llsd_scanner_free(scan);
}
printf("result = %i\n", res);
return 0;
}


Wednesday, February 25, 2009

Preprocessor directives for ruby (not!:)

At first I thought to do some more link-whoring - as I found precisely the search terms that are in the title of this post to lead to this blog... But then I though I will limit the on-topic information on the matter:

1) You don't need a preprocessor for Ruby
2) If you think you do, read up on metaprogramming - _Why's article will be a good start to spend a good time thinking.

But, this lead me back to the CPP - the preprocessor. Of course, beyond the Obfuscated C contest entries that are of parlor entertainment value only, there is some more to that.

Take a look at this one.

Then get it for infinite fun:


cvs -d:pserver:anonymous@chaos-pp.cvs.sourceforge.net:/cvsroot/chaos-pp login
cvs -z3 -d:pserver:anonymous@chaos-pp.cvs.sourceforge.net:/cvsroot/chaos-pp co -P chaos-pp
cvs -z3 -d:pserver:anonymous@chaos-pp.cvs.sourceforge.net:/cvsroot/chaos-pp co -P order-pp


And then go check out and mess with the examples:

the .c file looks like:


.....
int main(void) {
printf
("The 500th Fibonacci number is "
ORDER_PP(8stringize(8to_lit(8fib(8nat(5,0,0)))))
".\n");
return 0;
}

//>
// Unless you are already convinced, you should check, by
// preprocessing this example, that the actual parameter to
// `printf' in the above code is just a single string that
// contains the 500th Fibonacci number in base 10.
//
// If you know the order of growth of Fibonacci numbers, you
// probably realized that the result isn't exactly small.
.....



Now, let's try it (stderr forwarded to the null for brevity, as it seems to spit some errors):


:~/tmp/order-pp/example$ cpp -I../inc fibonacci.c 2>/dev/null | tail
return fib_iter(n, 0, 1);
}
# 63 "fibonacci.c"
int main(void) {
printf
("The 500th Fibonacci number is "
"139423224561697880139724382870407283950070256587697307264108962948325571622863290691557658876222521294125"
".\n");
return 0;
}


Now, let's look at another example - average.c

you define the macro AVERAGE as:


#define AVERAGE(...) \
((ORDER_PP(8seq_for_each_with_delimiter \
(8put, \
8emit(8quote(+)), \
8tuple_to_seq(8quote((__VA_ARGS__)))))) / \
ORDER_PP(8to_lit(8tuple_size(8quote((__VA_ARGS__))))))


and use it in your C code like:


int main(void) {
printf(AVERAGE(3, 1, 4, 1, 5, 9, 2, 6, 5) == 4
? "OK.\n"
: "ERROR!\n");

return 0;
}


The code post-cpp looks like:


int main(void) {
printf(((3 + 1 + 4 + 1 + 5 + 9 + 2 + 6 + 5) / 9) == 4
? "OK.\n"
: "ERROR!\n");

return 0;
}


Of course, it's only simple things like this that are so nice looking... Any nontrivial definitions cause giant single-liners - the worst nightmare for the debuggers and maintainers - so I would personally torture anyone who used this in the production code.. :)

Nonetheless - pretty impressive stuff.

The FONC project & potion

While messing around looking if anyone did anything with potion language, that I wrote about a few days ago, found this interesting link:

http://wiki.ruby-lang.nl/amsterdamrbwiki/show/The+FONC+project - The Fundamentals OF New Computing. Has some interesting reading on research in programming languages.

Potion itself is rather fun, though of course my microbenchmark was a bit of an over-joyous :)

Yeah, while it looks to be pretty damn fast, LuaJIT is only about 1.7 times slower. Nonetheless, I'll play with it more, it is an interesting thing to mess around with - at least the internals. Syntax I am not so sure...


Foo = Object delegated
Foo def("xx", (x): "foo\n".)

Bar = Foo delegated
Bar def("yy", (x): "123\n".)

a = Foo allocate
b = Bar allocate

a xx print
b xx print
b yy print


Running this with potion will give you three lines "foo", "foo", and "123" - the middle one being the method of "Bar", "inherited" from "Foo". Did not dig on how to glue multiple mixins - _why wrote that it's incomplete, I'll see maybe some other day.

Saturday, February 21, 2009

What the world is interested in...

The other day, while searching for a "how to" on something, I became amused with the autosuggestions I got after typing in the "how to" into the search input editor.

Assuming the results there are sorted by the interest from the users (which is something pretty obvious to do), this is the apparent hit parade of the things
that the people are wondering about:

1. how to tie a tie

Ha. I wonder whether this stems from the fact that the people don't really wear a tie every day, so this knowledge gets into the dark corners of the forgotten, or the science of tying the tie is as complicated as actually spelling it. I've last had to use this search maybe half a year ago when I in fact needed to wear the tie to some place. The caching is a good optimization here - tie the tie once, and keep it tied. Saves a lot of time.

2. how to kiss

Something is wrong. This one was supposed to land on the first position IMO - as the potential audience for the ties is at most is half as big as the potential audience for the kisses. So, I can conclude that the sets are actually very much intersecting and it's the same folks who don't know how to tie a tie. Others either already know it or are practicing it off-line. Indeed, practice is the key here. Instead of optimizing, just brute-force it. The process is fun.

3. how to get pregnant

Now the mystery of the bias for the previous one is probably starting to dissolve - that second half is actually not wasting the time on paying too much attention to the preamble, and is much more result driven. I'd think that here the practice is also a key. Also, there's something about the days of the month and such. But with sufficient practice the distribution is going to be across all the days of the month, so it should not be that important.

4. how to make money

assuming your looks are ok - after all, the tie is an important part of it! - indeed, money might be of use sometimes. I think it's a simple economics - find the area with most demand that is unsatisfied, and provide a supply there. Even better - create the demand out of thin air - that's how you make new markets. The most creative and spectacular that readily comes up, is the one million pixel page. In a nutshell - create 1000x1000 image, and start selling it in blocks of 10x10. Hundred bucks each. I am in awe.

5. how to lose weight

I won't comment much on this one - it's outside of my experience. Although, a full long weekend of dedicated nonstop gaming with not much food inbetween gets quite a few kilos off. I accidentally tried that once - the result was pretty funny (needless to say I was dead tired after that). But I think that's not something anyone should follow - talk to your doctor, or something. Eat lighter food, do sports, and such.

6. how to draw

I wonder if this is about drawing as in a form of art or it about making sure that no-one wins. My best advise - find an artist nearby who is willing to bear with you, and use the results from the (4) above to pay for the lessons. Practice makes miracles. But it aint going to be easy, if you want to get decent that is.
My wife's an artist and she literally spent years doing that thing. (oh, if you wanna get lessons, maybe I can ask, drop me a note :)

7. how to make a website

Hire a professional.

8. how to cook a turkey

Ah, this is a much more appropriate one. Though, no opinion here - I'm in europe and prefer chicken anyway. Or beef, for that matter.

9. how to write a resume

I'd do it in a couple of steps: first achieve a bunch of great things. Then write them down. Though, it's not a quick process. But should surely work.

10. how to lose friends and alienate people

Oh, that's easy - just write something outright silly like I did in this post, and it might work. Or go watch the film and learn from there.

Wednesday, February 18, 2009

Sketching network diagrams - entirely in javascript, in your browser!

This is a bit of shameless self-advertising - I figured that if I don't advertise a little bit, I never know if there are some people that might benefit of this thing.

Basically, if you want to scratch a network with a couple of routers and a cloud, now you do not need to start visio, dia, or anything else for that matter: just go to http://employees.org/~andin/network/, draw the network, and snap the picture from the screen.

Of course, it's all web2.x enabled and jazz, so you can save the layout as JSON, and load it later. Even, in a demonstration of a blatant violation of the security, you can pull in the JSON from another domain.

A little bit of history: as I use linux at work pretty much exclusively, I was always curious to have something simple to draw the diagrams. Dia is an excellent program, but I wanted something even simpler.

And so when I saw the Draw2D library I thought "ah-ha!".

The editor is not way too much more than a combination of the existing components - you can see the ingredients on its page.

Well, enough of me talking, go try it and let me know :-)

Monday, February 16, 2009

Potion for x86

Looking around, I stumbled on something interesting. Potion.
The source has an example:

fib = (n):
if (n <= 1): 1. else: fib (n - 1) + fib (n - 2)..
fib (40)

Interesting. Looks a bit like OCaml/Haskell. Sort of.
A microbenchmark for calculating fibonacci number in a very inefficient way :-)
Let's time it:

$ time ./potion example/fib.pn
real 0m17.924s
user 0m17.909s
sys 0m0.008s

Now I'll try the similar-looking lua code:

function fib(n)
local ret = 1;
if (n > 1) then
ret = fib(n-1) + fib(n-2)
end
return ret
end
fib(40)

Timing it:

$ time lua fib.lua
real 2m18.566s
user 2m18.089s
sys 0m0.048s
$

Now, with all my non-love to microbenchmarks, this does look quite interesting.
If this baby has zero-based arrays, I might get seduced :-)

Tuesday, February 10, 2009

One-shot makefile to install lua, prosody, and associated libs...

I've a couple of systems that I do not want to touch the "system-wide" packages on, so I hacked the following makefile to do a one-shot install of prosody. Pretty darn ugly, but does its job. Maybe could be of use to someone...(NB: beware of horribly long lines that are wrapped by the display!)

EDIT:

moved it to http://github.com/ayourtch/prosody-makefile as blogger is much better suited for text.

Monday, February 9, 2009

The FOSDEM'09 memories of a yellow T-shirt...

...or - "how we had a lot of fun and a little bit of pressure at FOSDEM 2009".

I planned to attend Fosdem, but I was pretty delighted to hear that Jerome managed to get our employer Cisco to provide the wireless networking gear for the event, so jumped in to grab the opportunity to give back to the community, and meet the new friends.

It was a lot of fun - thanks to Jerome, Peter, Seb, Antonio, and of course to Fosdem organizers, staff, and indeed the visitors - folks, you rock! :)

We had about 60 access points scattered around the area, so the wireless controller proved to be a very handy beast (it reduced the number of entities to actually "manage" to 3).

Of course, being first time and all, it was quite a few hiccups - ranging anywhere from cables to configs to users "helping" by misconfiguring their gear to do an Ad-Hoc network with the same SSID, and me being slow like a snail in the morning :)

The hardest things for me were escaping from the office at 4:30pm on Friday, and waking up before 10am on Saturday.

But hopefully we might repeat the experiment next year and try some improvements - the first on the wishlist being the faster-than-halfduplex-10mbps uplink to internet - if we manage to borrow the toys again :-)


Correction: "60" was the number of "unknown/rogue APs", we had only 37.
Duh!

Sunday, February 8, 2009

It's a small world after all...

When we were enjoying the last hours of warm spanish weather in Barcelona on 29th January after Networkers, a few of us went to Parc Guell. It was pretty much closed, so we had a beer in a cafe which was conveniently situated here to take the profit from the wave of the visitors. And then on the way back I heard a man playing medieval Lute. Inside the ancient column-rich area - so the acoustic qualities of the area were fantastic.

Having devoted a nonzero portion of my life to the classical guitar, I can fully appreciate the richness of this instrument - and maybe given its specific non-mainstream nature, the only music I get a chance to hear with it has quite a quality. As we did not have the time to listen, I bought a CD right there. And today I finally got to slide it in... and it's pure joy. So, looking at the cover of the CD I find the name of the person, and punch it in the facebook - "Hey, maybe you do not remember me. You play great. How can I get more?". Amazing how simple it is.

Possibly some 10 years from now we will be able to get connected with someone via even a less material token of information - maybe just by recalling their image from our memory. And the world might become a much happier place then.

Or not.

Friday, February 6, 2009

Refactoring - when I don't want to do something else

Refactoring.

With my hobby code, I've caught myself thinking refactoring mostly when I realise that the next chunk of code I am going to write is going to be inherently ugly or large and boring. A very good recipe which I found against that is to put the thing on the shelf for a week or two, and let the brain work in the background - the power of the subconscious "coprocessor" is very well suited for that. Usually in this kind of time frame I'd have the "eureka" moment which unlocks the process and I happily hum along till the next "bore-block".

Maybe that't the reason why I don't code for food.

Sunday, February 1, 2009

Finally getting a bit more of IPv6...

While I was at Networkers, Eric has sent me a nice teaser - a Facebook app to test the IPv6 connectivity. I've played with "getting IPv6" a while ago, but wanted to give it another try. Yesterday I spent a long while mucking with OpenVPN + a tunnel to my internet host that would connect to V6 via 6to4. Slow and painful - if running 6to4, I'd rather run it locally... Alas - the DSL box does not have enough memory to run a V6-enabled IOS.

As a result, I've now split the single-head setup into dual-head, with one (older) router functioning as a modem + bridge, and the other one running the PPPoE client and doing all the cool IPv6 stuff.

I'll write up in a bit more details on how I have it in the upcoming days (leave comment if I want it sooner!), meantime a menu of nice links:

Saturday, January 31, 2009

Troubleshooting lenovo's wireless with Vista: plot thickens

In the previous post, I came to a conclusion that it was the wireless driver update which sent my wireless on a new vista laptop south. Today I finally got some time to debug it a bit further...

..And it turns out the previous conclusions were, well... not entirely correct. Ubuntu livecd certainly has very little to it from Vista - yet I got only 400kbps performance - while on the wired connection I got 4Mbps. Though, the connectivity appeared smoother, but maybe it is the more tolerant Linux TCP stack. So, the conclusion is that it is OS-independent, and how I got "good" performance in the first place is a mystery.

The access point I'm using was an almost 10 years old noname 802.11b. Given that all the neighbouring ones are 802.11g, I think mine is probably driving the neighbours' data rates down, so probably a good time to go and get something newer - and write how it looks after that.

Update:

Even though I did not try the G access point, this link might explain it - at least the symptoms match very well: http://ubuntuforums.org/archive/index.php/t-1032617.html
Now, how in the world that happened for both Vista and Linux - I don't know :-)

Indeed updating the driver from the linuxwireless seemed to alleviate the problem for linux at least.

Wednesday, January 21, 2009

Interfacing Lua with C and hacking Lua syntax

After my initial experiments, I played last weekend a bit more with the lua+c integration - and indeed it is quite nice and flexible. My fears about the 1-based array indexing so far turn out to be a bit oversized - the biggest problem this would bring is when porting something from a 0-based language, when writing a new code from scratch it is in fact pretty straightforward (and the "for i,v in ipairs(array) do" iterator helps to avoid thinking about indices altogether).

What's especially cool is the freedom it allows - to code parts in the high-level language, and parts in low-level C, and intermix them as needed.

As a result, over the past weekend I've ported some "hobby" XML-parsing HTTP code over form Ruby to Lua with very little effort.

Joyed with such an ease, for the work (reproducing a setup involving the signaling of a couple dozen thousand IP phones) I've written in just over a day a little helper code, which includes Lua itself + convenient framework to abstract all of the socket-handling details into much more highlevel structures, something like:


function server_read(sock, data)
print("Server received ", string.len(data), "bytes:", data)
send(sock, "I got your data!\n")
end

function server_closed(sock)
print("Client disconnected!")
end

function server_incoming_connect(parent, remotehost, remoteport, newindex)
local server_methods = {
read = server_read;
close = server_closed
}
return server_methods
end

function server_main(process, number)
tcp_listen("0.0.0.0", 12345, { connect = server_incoming_connect })
end



So, all of the socket low-level code (poll cycle, checking where the data has arrived, etc.) is hidden inside the compiled C blob, with the higher-level logic easily adjustable. Quite yummy :-)

Overall it looks like Lua is definitely there to stay in my toolbox, which now will mainly contain Lua, C, and Ruby (though the biggest major advantage of the latter for now is the richness of the libraries and the lack of parentheses).

Speaking of parentheses, I experimented with getting rid of parens in Lua - alongside with more Ruby-esque "def" and "elsif" - but in the end decided to throw it into garbage - primarily because of the danger of the confusion in the brain because of somewhat-similar-yet-quite-different semantics of the resulting "language" vs. Ruby, as well of some of the inconsistencies that arised as a result, i.e.:


foo bar baz


would be the same as foo(bar(baz)), and baz, to be called, would need explicit parens. As well - a single-handed statement "baz" on the line would cause a syntax error. These all are pretty easily fixable (though then I'd need to have a way to specify that I want a function itself, not its call), but would break a lot of backward compatibility with Lua, which I wanted to have - hence not worth it.
I'll better live with "elseif". :-) But the entertainment value of this experiment was certainly worth it.

Monday, January 12, 2009

Open, standards based video player for the web ?

http://www.openvideoplayer.com/ - seems nice, I wish I would be a lawyer to read up on the license, as it's obviously not just something as "easy" as a BSD or GPL...

Thursday, January 8, 2009

Initial impressions of Lua

During the long weekend, I've spent a few hours on messing up with Lua - after some readings I finally got seduced enough to give it a try.

As an exercise, I went over porting part of the reader of the "message template" file (the latter I grabbed from libopenmv) that I previously wrote in Ruby (about 15K of code to generate the semi-functional Ruby class definitions). The port now reads the defininions into a bunch of interconnected tables, however it's not too big either - just about 3K:

First, a helper method (I miss ruby String.split() somewhat):


function string.gather(str, pat)
local tab = {}
for match in str:gmatch(pat) do
table.insert(tab, match)
end
return tab
end


Then goes my experiment with an iterator-within-a-coroutine-in-OO-like-thing to get the tokens in batches:


TemplateReader = function(filename)
local worker = function(fname)
for line in io.lines(fname) do
-- get rid of comments, [rl]trim, remove trailing and multispaces
local line2 = line:gsub("//.*$",""):gsub("^%s*","")
:gsub("%s+$",""):gsub("%s+"," ")
local tokens = line2:gather("([^%s]+)")
if(#tokens > 0) then
coroutine.yield(tokens)
end
end
end

local me = {}
me.fname = filename

me.coworker = coroutine.create(worker)
me.tokens = function(me)
local ok, tok = coroutine.resume(me.coworker, me.fname)
return tok
end

return me
end



And then go the "conceptual piece" readers. They were in separate classes in Ruby version, but given the pretty similar structure for all the packets, I think I might not need it.

Here we go, in reverse order. The bottom-most part is the actual "main" body:


tr = TemplateReader(filename)

version = tr:tokens()
print("version: " .. version[2])
packets = {}
for packet in PacketTemplateRead, tr do
print("Packet obtained: " .. packet.name)
table.insert(packets, packet)
end

print("done, total: " .. #packets)


Now, let's see how we read the packets:



PacketTemplateRead = function(tr)
local start = tr:tokens()
local pt = nil
local finished = false

if (start and #start == 1 and start[1] == "{") then
local pkt_info = tr:tokens()
pt = {}
pt.blocks = {}
pt.name = pkt_info[1]
repeat
local tok = tr:tokens()
if tok[1] == "{" then
table.insert(pt.blocks, PacketBlockTemplateRead(tr))
else
finished = true
end
until finished
elseif start then
print("start[1] is: " .. start[1])
print("foobar!")
end
return pt
end



So, let's see how we read the block...


PacketBlockTemplateRead = function(tr)
local b = {}
local toks = tr:tokens()

b.name = toks[1]
if toks[2] == "Single" then
b.count = 1
elseif toks[2] == "Multiple" then
b.count = toks[3]
elseif toks[2] == "Variable" then
b.count = -1
else
print("Unknown block count " .. toks[2]
.. " for block " .. toks[1])
-- FIXME: need an exception ?
end

b.fields = {}
for field in PacketFieldRead, tr do
table.insert(b.fields, field)
end
return b
end


The only thing that is left mysterious, is the packet field, which is the "name / type / length" kind of thing:


PacketFieldRead = function(tr)
local toks = tr:tokens()
local field = nil
if not (toks[1] == "}") then
field = {}
field.name = toks[2]
field.type = toks[3]
field.len = toks[4]
-- print("Got field name: " .. field.name .. " type " .. field.type)
end
return field
end


This reads the message template visually several times faster than Ruby, in barely noticeable fractions of a second on my laptop.

Which *may* make this a reasonable candidate not to store the generated code anywhere at all, and just keep the message template file. Though, my experiences with the Ruby autogenerated in-memory code were that in case there's an exception inside that code, it's a royal PITA to debug - so the question of how to do error handling would need a better thought.

Overall impressions of translating this from looking at the Ruby version:

1) 1-based arrays are a pain for the C/Ruby/...-infected zero-based brain to switch to.
In fact, I did create a couple of bugs precisely while retyping usage of the elements of the array. And there's hardly twice than that places where I use the indices :) So - this is a big warning sign to myself. Just that it is not easy - not that "0" or "1" based are better or worse in all cases.

2) having functions as first class citizens feels really easy-going after Ruby.

3) coroutines are cool. the yield/resume couple interaction creates the feeling that the world is something you can turn inside-out and back - and these two do precisely that. Very odd feeling. Though I would not say that my use of coroutine-within-the-iterator is a terribly good idea.

4) table-based OO feels a big bit like in Javascript.

5) you notice "not x == y". I find it more readable compared to ~=, and less mentally confusing with =~ (Perl-ism to match on a regexp).

6) finally a minor one but probably the most annoying - after being used to write print "Var: #{x}\n" in Ruby, it is pretty bad to have to write print("Var: " .. x .. "\n") in Lua. And the main annoyance is the parens. I know they do create the ambiguity and pain for the parser, but the lack of them is so conweeeeenient! :)

Overall, seems to be pretty usable and sweet language, and given its speed, tiny size of the base kit, and the ease of integration with C, as well as a JIT compiler, which in my microbenchmarks did show pretty nice results - I like it

p.s. don't ask me why I mention Python only here. I am still trying to wrap my inner nerves around the whitespace thing.

A haiku on usability of security dialogs which resist the screenshots.

the vista update.
dialog. "details" - hex: {...-DEAF}.
thoroughly puzzling.

How I shot myself in the foot. Twice. During yoga - so my foot was on my forehead....

If you were expecting to see a real story here - indeed there is. But it would be of no entertainment value if I did not have the joy of wearing both the "patient" and the "doctor" hats - as well as taking the roles of the "independent 3rd party" at times to observe.

Here goes the true story a winded "case", happened to yours truly.

Step0: I have to admit, the phone socket that I mostly use (for DSL only), is a totally oxidized piece of junk, which I am perpetually too lazy to replace because everything works fine if I half-unplug the DSL connector from the wall - and then it becomes a classic "if it ain't broken, don't...".

Step1: my DSL is dead. Dead for for pretty long time, but I'm not much bothered about fixing it, as the majority of the time I spend at this time period at home is sleeping, I can survive that without connectivity :-) Though first I did do a fair share of usual fiddling of the wall connector during the initial phases of troubleshooting. But the line is dead. Real dead(tm). So, especially since the initial failure happened to coincide with potential DSL headend upgrade at the ISP (as they were widely marketing the much higher speeds than before) - all looks as if it's a proverbial man who was patching the stuff downstairs, and cut the wrong wire ;)

Step2: after some administrative wiggling, which is uninteresting for the purposes of this story, the DSL gets finally fixed - it comes up on friday while I am at work. Then I verify during the weekend that "yes it works" and signal the happiness and thanks to the ISP support folks. Well, it kinda works at half speed, but again, for connectivity mostly during sleep that's enough, so I do not complain. :-) And then as well - the glass that is half full is much better than a totally empty one. :-)

[ here a week or so passes, meantime the reader is directed to click on the advertisements on /. as a pastime ]

Step3: friday - clean-up party at home. Things get moved, vacuum-cleaned, watered, polished, and a lot of other funny activities. To try to handle the trauma, I hide in the corner with connectivity. At some point the latter disappears. Trying to fiddle with the socket - no way.. Oh well, it's pretty late anyway, so why bother.. time to sleep.

Step4: saturday - some shopping time. No time to fiddle. But, as a reward for my good behaviour throughout the year, I get a shiny new Lenovo N500 laptop with Vista on it. Curiously I haven't touched Windows-based systems for anything beyond simple install for quite a while, so it's entertaining and will have its own post at some point. But the internet is still unstable. So some evening fiddling with socket + amplification settings in the config - so-so.

Step5: sunday - the moment of truth - I manage to get the full speed! And the Vista is really connected and flying! wow! Now its time to install firefox, cygwin, and all the other command-line toys that I will for sure need in order for this laptop to be of any use. I leave the download of some gigabytes on, and we go for a walk for a few hours. When we arrive back home, everything is downloaded, installed, I am a happy camper. There's this box that asks for a reboot, after some postponing, I reboot it, and leave to install the locally downloaded cygwin - and then go to watch some TV meanwhile and hack on some stuff in the background.

Step6: I notice the internet is not too good again. Argh. The most interesting thing - the pings to my default gateway over wireless work apparently fine, the download of the ubuntu image from the ISP-local mirror gives about 200kbytes/sec instead of 400+ that it should - which is anyway bearable, and can be attributed to the socket which maybe I *should* replace now. The cool part though that anything beyond the local ISP network is just plain broken. Horrible latencies of 1.5+ second (the ping to ISP's DNS server is 22milliseconds), and huge packet loss. Traceroute shows some oddities, so I make the conclusion that they did not like me for that cygwin update, and I ate all the quota of the month, after which they normally rate-limit the traffic to 64K. Well, maybe this is done for some reason only for external traffic.. odd but believable.

Step7: today early morning, I go to use my desktop, which is an oldie gentoo box, and notice that *everything flies* !!! Wow, must be that this socket has fixed itself again - lets check the Vista box... hmm - it does not work still! Can't be *so* precise timing, but let's check two simultaneously - indeed it's Vista specific.
Not being fully awake, only the search reflex is functioning. So I look up for "Vista internet slow". Which turns out with lots of forums with voodoo-like manipulations, uninstallations of antivirus software, recommendations to remove the SIMMs one by one, and suggestions to uninstall updates.

Step8: *updates*!!! That reboot was insisted upon by the update! I totally forgot about this given the flurry of other non-computer events that happened this holiday weekend. Ok, now at least we know "what changed" (something that I use as an introductory joke - "because I know nothing changed, but it's interesting to try to find out whether there's any occurence where asking this question explicitly will be of use" :) Now, we can also think "what's different between these two boxes". Hmm... linux-vista... nope, I *know* vista was working for sure, just fine :-) wired-wireless... good point, let's try wired... and it works fine! Using wireless... still dialup-like and worse performances. So, it's got to be bound to wireless... Let's check - maybe my good ol' access point went kaput ? No, the wife can use her wireless with no problem (though she has an oldie iBook which survived a surgical hard drive replacement by yours truly :)

Step9: The most plausible theory, that so far explains almost all the observations:

  • vista update and lenovo wireless drivers did not like each other
  • the packet loss which is the effect of this bug is somehow proportional to RTT (????) - which is a bold statement in itself, but at least explains why the throughput to ISP mirror was halved (small latency), and to anything on the internet was almost nil (bigger latency), and why I did not see anything with the default gateway testing (pretty much no latency)



Of course, the jury is still out to figure out what's the matter with wireless (and whether to install 38 more updates that the windows update is proposing... (Half of them are named "Windows Update #KB-number", and going manually over all of them via manual retyping the KB#s into the browser is a pretty boring way to spend time. oops, Vista will have its own post, I promised).

Anyway, after such a long preamble, a few observations-reminders both to myself, and to you, if you made it reading till here :-)

1) Life is too short to keep a note "what changed". That's why noone ever "admits" - not because of some magic stubbornness, unlike some BOFHs think, but just because there are better things to do.

2) One persistent issue will psychologically distract the attention even if there is really another one with a slightly different symptom - I clearly was blaming the latent L1 instability a bit too readily.

3) The combination of several smaller issues amplifies the effect and makes the resulting problem look quite more "interesting".


All three are something which I knew for quite a few years already, but they packed themselves so nicely into a "case study" - which, since it happened to myself, I could freely share.

Update:



There's more to the story.