Monday, January 31, 2011

JavaScript: The Surprising Parts

This post is a summary of the the things that surprised me as I've been reading several books to update my knowledge of JavaScript (and practices that have changed since 2004/2005).  While this info would have been big news to most and worth writing articles about back then, it begs the question as to why I should blog about it given that now there are several good books available on the subject. There are two self-canceling reasons; (1) this blog will act as a cheat sheet for me, and (2) the very act of writing it will make me more likely not to need a cheat sheet.  While there are many small things that I didn't happen to know (or remember knowing), I am writing here about the things that actually surprised me, and they tended to be either big or small surprises, so I will organize them that way.

Small Surprises
  • [JGP, p.6] undefined is not a reserved word!
  • [JGP, p.107] undefined is not a constant; it is a global variable which can be redefined! (so is NaN).
  • [JGP, p.103] Reserved words can be used as object property names (unless accessed via dot notation)! Even stranger, an empty string is a legal property name! (as in obj[""]=123;
  • [JGP, p.7] integer types are really double under the hood!
  • [JGP, p.10, 102] Nested block statements (i.e. {...}) do not define a new variable scope!
  • [JGP, p.29] If a method is called as a simple function, this is not undefined but rather defined as the global object!
  • [JGP, p.36, 102] Local variables exist before they are declared! (which is why they may as well all be declared at the beginning of a function)
  • [JGP, p.105] The arguments array is not an array! (For that matter, arrays are not arrays!)
  • [JGP, p.109] The == operator does type coercion and the === operator does not. But the type coercion is done so poorly that it is too dangerous to use == at all in general!
  • [JGP, p.102] The return statement will return undefined if the return value expression does not start on the same line as the return keyword!
  • [HPJ, p.2] Browsers block loading anything else while the JavaScript inside a script tag executes! 
  • [HPJ, p.7] Scripts can be dynamically loaded by creating script tags via DHTML, and those scripts don't block loading other elements (like images, etc).
  • [HPJ, p.20] Global variables are not the fastest to access (compared to variables in other scopes), as would be the case in compiled languages, but are in fact the slowest since all the other scopes are searched first.
  • [JGP, p.112, HPJ, p.156] Bitwise operators are either really fast, or really slow, depending on who is judging. Crockford, in Appendix B: Bad Parts, say bitwise operators are "very slow", whereas, Zakas, in Use the Fast Parts, say bitwise operators are "incredibly fast"!

Large Surprises
  • [JGP, p.37, HPJ, pp.21-26] Closures imply Cactus Stacks.  I didn't really understand in a deep way what closures were, and JGP didn't help (nor any other JavaScript articles/books for that matter). After all, Pascal had nested function definitions where the inner procedures had access to the outer local variables, but that didn't make closures.  What really made me understand was realizing that JavaScript was like Scheme with regard to closures. (See JavaScript is Scheme not Java below). As I wrote in a comment to someone else's blog attempting to explain JavaScript closures...
    I think it would be more accurate to say right up front that our normal notion of a “stack” (as used to hold parameters and local variables) does not apply in languages like JavaScript (and Scheme, etc). Unlike Pascal, Ada, C, C++, Java, etc., JavaScript has only “heap with garbage collection"; all those “stack frames” to hold parameters and local variables are just more objects on the heap. When no more references to each exist, they individually get garbage collected. This means that there is no special “saving the stack frame” after the outer function has returned for closures to use…they all just get garbage collected like anything else does when all references to it are deleted.
    BTW, the only way I ever grokked this was after learning Scheme by watching the UC-Berkeley Comp Sci 61A class on iTunes U. Except for the EARRRRLY Fortran on CDC supercomputer days, where recursion was not supported and the hardware instruction set did not implement stacks, I had not experienced a language that wasn’t fundamentally stack-oriented (well ok, there was that funny Lisp language in the exotic languages class). The 61A class video explained (with nice pictures) how the “stack frames” are instead “environments” that chain to each other (just like JavaScript objects chain via Prototypes). There is a global environment with the global vars, plus a new environment for each function (to hold params and local vars), and instead of a stack of frames on a hardware stack, it is a linked list of environments on the heap.
    It probably also requires explaining right up front that unlike the Pascal, Ada, C, Java, etc languages (where functions are defined at COMPILE-TIME), JavaScript, Scheme, etc languages define functions at RUN-TIME. Because of this, there is always an "environment" that is active at the time of function creation/definition, namely, the call-stack that exists while the code creating a function is executed. [EXCEPT, of course that there is no stack... There is a great way to visualize how the stack is really a tree at http://en.wikipedia.org/wiki/Spaghetti_stack].
    A reference to this “current environment” chain is saved as a part of the function definition. In Java-like languages this makes no sense because there is no “current stack or environment” at compile time when the function was defined/compiled.
    SO, when a function gets called, instead of “pushing” its params/local vars on top of THE stack, it “pushes” it on top of an environment chain that was saved with the function definition. And since it isn't a real (hardware) stack, but actually sitting in the heap, the function can refer to “local variables” of its “outer” function long after that outer function has returned.
  • [JGP, pp.40-57]  JavaScript is Scheme not Java.  Having accidentally watched the entire 61A course that teaches Scheme and functional programming (mentioned above) before ever reading JGP, I immediately recognized the exotic techniques of JGP chapter 4 (currying, etc) as paralleling the topics in 61A. [BTW, I originally watched 61A because I wanted to learn about this Hadoop buzzword I kept coming across in Job Ads. When I looked it up and saw that it had something to do with Map/Reduce (whatever that was), and that this iTunes class had a section on Map/Reduce, I tried to watch just those lessons. It soon became clear that I needed to start from the beginning of the course to learn Scheme itself.] Learning the culture and toolkit of functional programming, which goes way beyond simply using function pointers or having functions be objects, I could see that the new style of programming JavaScript was not very strange Java, but fairly normal Scheme.  Back in 2005, I was aware of, but avoided, many of those alternate ways that JavaScript could be twisted to define classes, objects, functions, methods, inheritance, etc, BECAUSE, why would you want to make your  JavaScript code look so un-Java-like!?! If one embraces JavaScript as Scheme, it is not a weird syntax but simply a different programming paradigm: functional programming with lambdas and lexical scoping.

    One of the many differences between functional and non-functional programming is the cognitive "weight" of functions.  In Fortran thru Java (and JavaScript as I had been using it), methods and functions are fairly heavy weight things that get full blown API documentation (e.g. JavaDoc/jsDoc), and lots of whitespace around them, and are mostly not defined in a nested scope.  In functional programming, functions definitions are like water, and are created, used, and abandoned in as casual a fashion as Strings are in "normal" languages. Function definitions within function definitions in Scheme are as common as string concatenations in Basic. SO, one has to get over the feeling that it is expensive overkill to define a function just to wrap another function so that a one variable closure can be created.
[JGP] Crockford, Douglas. JavaScript: The Good Parts, O'Reilly, 2008
[HPJ] Zakas, Nicholas. High Performance JavaScript, O'Reilly, 2010

Saturday, January 8, 2011

Adding Cross-browser support to the Gravey Framework

In order to get back into the Web 2.0 development world, I have recently taken on the project of upgrading the Gravey framework to run on browsers other than Internet Explorer. This post is a chronicle of what I encountered/learned getting its sample applications to also run on iPad 4.2, Safari 5.0, Chrome 8.0, Firefox 3.6, SeaMonkey 2.0, and IE 8.0 (while not breaking IE6 on Win2K). [Note that the issues listed herein may have an IE-centric viewpoint, but it is only because the original code base naively used circa-2005 IE-only techniques, and therefore the news for Gravey is how everybody else does it now.]
The Gravey framework (gravey.org) is a JavaScript-only set of code that supports building Rich-Internet-Apps (RIA) using AJAX where the logic completely resides in the browser. The server-side need only provide a REST-style API for data persistence (which can be a simple as the file:: protocol accessing local xml files). The framework includes GUI widgets, automated domain object persistence, in-browser XML/XSL processing, and complete undo/redo capability.  In short, it was intended to allow RIA development in the style of "fat GUI" development in Java without requiring any particular server side technology (e.g. J2EE, .Net, LAMP, whatever).  The new version 2.5 of Gravey now runs on several browsers (including Safari on the iPad) without requiring any particular client-side technology (e.g. jQuery, Prototype, whatever).
Because it was back in 2005, when I was learning DHTML/OO-JavaScript/AJAX/CSS/REST on the fly, while building Gravey for use in several internal applications for a top-5-in-the-USA bank, and because their requirements only mandated (and their schedule only allowed) Internet Explorer 5.5 as the target browser, Gravey v1.0/v2.0 required IE5+, and were only tested on IE5/IE6 in Win2K/WinXP.

Since Gravey is built in layers, I had originally intended to merely retrofit the bottom layer to use jQuery to make short work of cross-browser compatibility.  However, I immediately encountered enough problems with jQuery that I was required to really learn the issues anyway, and so I ultimately fixed all of the problems in Gravey directly, and as a result, it still has no dependence on any other frameworks.

• AJAX / XML / XSL Issues

Issue:  jQuery doesn't cover in-browser XSL processing
Gravey and its example apps use in-browser processing of the XML returned from AJAX calls via XSL (also loaded from the server).  It turns out that the XSL processing API is browser-specific, and because jQuery didn't handle it, I was going to have to learn how to fix it directly anyway...

Issue:  IE doesn't do XSL processing like everyone else
In order to support older IE browsers, the existing Gravey code that uses ActiveX is required, but other browsers need to use the standard DOM APIs.
  • IE really wants XSL files to be loaded via Msxml2.DOMDocument even though XML files can be loaded via Msxml2.XMLHTTP. So, once the XML DOM has been loaded via AJAX, the IE XSL processing looks like...
    var xmlDom = XHR.responseXML;
    var xslDom = new ActiveXObject("Msxml2.DOMDocument");
        xslDom.async = false;
        xslDom.load( xslURL );
    if (xslDom.parseError.errorCode != 0)
        alert("Error loading["+xslURL+"]:" + xmlDom.parseError.reason);
    var resultStr = xmlDom.transformNode( xslDom );
  • The other browsers accomplish this via the standard API...
    var xmlDom  = XHR1.responseXML;
    var xslDom  = XHR2.responseXML;
    var xslProc = new XSLTProcessor();
        xslProc.importStylesheet( xslDOM );
    var resultStr = 
xslProc.transformToDocument( xmlDOM ).documentElement.textContent;
  • BTW, when I used an XMLSerializer to generate the result string in the above code, it gave me quotes around the result, so using the .textContent was simpler and did not include extraneous quotes.
  • IE also requires the XML file to be loaded via Msxml2.DOMDocument when the files are local. In other words, when the .html page is loaded from local disk via the "file::" protocol instead of via "http::" from a web server, IE fails attempting to load the xml unless it is via Msxml2.DOMDocument. Otherwise, for local files, the AJAX callback will be given readyState==4 and status==0, instead of status==200. At this point in Firefox/others, the file has loaded ok, but in IE it will have failed.
Issue:  Some browsers can't handle <xsl:import>
While good ole IE6 can handle an XSL file including another one via the <xsl:import> directive, new browsers like Safari and Chrome can't.  I never did find a reasonable fix, so I bit the bullet and just copied the imported XSL directly into the top level XSL file. Not pretty but it works.  If I had a larger system to worry about, I would preprocess, on the server side, the XSL file to include the imports before sending it back to the browser's AJAX request.

Issue:  Firefox doesn't call your AJAX callback on synchronous requests
Parts of Gravey performed AJAX requests with async=false with no problem until Firefox came along.  It turns out FF doesn't call your callback function like everyone else on synchronous requests. For a fix, I simply changed to async=true because I was already processing the responses via callback instead of inline code anyway! [ Back in 2005, I had thought that synchronous requests might be faster, or higher priority, than async requests, but experience has taught me otherwise.]

••• DOM Issues •••

Issue:  jQuery doesn't like funny characters in your element IDs.
This problem I ran into immediately because Gravey uses all sorts of special characters (e.g. ".", "#", "_", ":", etc), and it took a while to track down a fix.  By that time, I had taken to fixing Gravey directly.  But there is a work-around...
Given that ID="tar.foo.bar", replace the following...
    $(ID).whatever...
with...
    $( jq(ID) ).whatever...
where...
    function jq(ID){ return '#' + ID.replace(/(:|\.)/g,'\\$1'); }

Issue:  Only IE looks up Names as well as IDs in getElementById()
Gravey originally created HMTL elements identified by the "name" attribute because it worked just fine in IE when doing document.getElementById("foo").  Other browsers don't, SO, I changed all those .name references to .id references.

Issue:  innerHTML doesn't equal what it was just set to
Early on, I came across speculation that browser security might strip away inline JavaScript when setting the .innerHTML of some element. Being paranoid, I put in code to check if the .innerHTML I just set, was different than what I just set it to. To my surprise, it was quite often literally different, but not functionally different. In other words, the before and after strings did not match because the quotes might have been changed, the attributes might be in different orders, and the upper/lowercase of the tags might have been switched.  I never caught browser security stripping off anything, but the .toString() of the innerHTML could be completely reworked compared to my original HTML string.

Issue:  Saving data between page loads in window or navigator properties is no longer reliable
In the olden days of IE6 on Win2K, one could create properties on the navigator object that would allow communications between page loads.  With modern browsers this isn't reliable.  Gravey used this ability to save a reference to data between the time an AJAX request was made and when the callback was later invoked on the reply event. Luckily, I was only doing this because I didn't understand JavaScript "closures" enough to use one instead.  The Gravey 2.5 code uses a closure to save data in the callback function to be accessed when that function is later invoked on the reply event.

If I really needed client-side storage of data in between page loads or between pages, I would hope to wait until the near future when HTML5 is supposed to have local storage to share fat data between pages. There is an article here that tests this new feature out on a the usual suspect browsers.

Issue:  Some browsers change the placement of span and legend tags
In tracking down all the event handling problems, I came across a bug in some browsers (e.g. Firefox/Safari/SeaMonkey, but not Chrome/IE) where I created a <legend> tag inside of a <span> tag (via setting innerHTML), but in Firefox (via Firebug) I could see that it put the legend outside and after the span!  Luckily, I could live with the <span> being inside of the <legend>, and all the browsers seem to handle that ok.

Issue:  IE8 changes behavior depending on where a page is served up from
I was trying to use a simple trick to determine if the browser is IE (and which version of IE) via conditional compilation as shown here at quirksmode.org. Unfortunately, IE8 on WinXP would compile the comments and define the flags when the page was loaded from my test server on my LAN.  The exact same code would NOT compile when served up from my Internet web site.  Go figure. I tried adding a DOCTYPE to invoke quirks mode (which I had not specified before), but that made no difference. I wound up using the code below to distinguish between the cases that I needed to determine: pre-IE7, IE-in-general, IE-style-AJAX, Firefox-style-AJAX.
In the home web page...
    <!--[if lte IE 6]>
    <script type="text/javascript">
      //NOTE: THIS LOOKS COMMENTED OUT BUT IT IS ACTIVE IN IE !!!
      var Pre7IE = true;
    </script>
    <![endif]-->

In the javascript...
    function IE_Pre7(){ return (typeof(Pre7IE)=="undefined") ? false : Pre7IE; }
    function FF_AJAX(){ return document.implementation
 
                            && document.implementation.createDocument; }
    function IE_AJAX(){ return IE_Pre7() || !FF_AJAX(); }
    function Is_IE  (){ return '\v'=='v'; }


••• Display and Styling Issues •••

Issue:  Handling of partial escape sequences is browser-specific
A bug in Gravey went undetected because IE quietly ignored partial escape sequences like "&nb". The bug generated partial escape sequences in element text (e.g. <div>foo&nb</div>) and browsers like Safari and IE quietly display "foo".  Other browsers like Firefox and Chrome display "foo&nb". I fixed the bug so that only full sequences like "&nbsp;" would be generated.

Issue:  Special character escape sequences are browser and OS-specific
Originally, Gravey used special characters via the Microsoft "symbol" font, so I had a goal to replace them with standard HTML characters. E.G. replace...
    '<font face="Symbol">&#101;</font>'
with... 
    "&epsilon;"

The set of graphic characters like arrows, and bullets, and greek letters, etc can not be all rendered by all browsers.  Unfortunately, the intuitive idea that if an older browser can render a character, its newer brethren would also, is alas not true.  Most browsers, including IE6 on Win2000, can render the black star (&#9734;) and white star (&#9733;) characters, but IE8 on WinXP can't!  Some characters can be displayed when given the hex charcode, but not the name (e.g. &rArr;). And some characters that you would think are part of a set are not all implemented together. E.G. Of the up/down/right/left arrow set (see shapes), IE6 can render the right arrow but not the down arrow.

Issue:  Fieldset Legend rendering is browser-specific
Given the same style settings, the Legends of Fieldsets render differently on different browsers. Gravey had used the attribute "text-align:center" which didn't do anything on IE, but it shifts the entire legend display to the center (instead of the left that I want) on non-IE browsers. I removed the attribute. Unfortunately, the "vertical-align: middle" attribute DOES do something I want in IE but not the other browsers.  It was needed to get the gif images (being used for expanded/collapsed icons) to line up with the title text inside the legend box.  After a lot of fiddling with transparent pixels around the icons and legend padding attributes, I gave up and switched to graphic arrow characters (ala Google results).  Unfortunately, I ran into all of the special character problems listed earlier, and still had to specify a different set of characters (wingdings) for IE6 than specified for the other browsers.

Issue:  CSS doesn't easily replace all HTML presentational attributes
While the original Gravey used some external CSS, there was still a lot of inline style specs, and old-time HTML presentational markup and "spacer" gifs. I set out to replace them all with CSS, but since I was using IE specific attributes like cellspacing, cellpadding, bordercolordark and bordercolorlight it was not trivial. Sadly, some IE attributes like table "cellspacing" can't be replaced with CSS. Others like cellpadding, bordercolordark and bordercolorlight can but with great effort. Luckily, Gravey used a cellspacing and cellpadding of zero, some of which can be specified with "border-collapse:collapse". Also, the bordercolorlight/dark colors being used created an effect similar to { border-style: outset }.

Issue:  Background color gradients are browser-specific
The gradient coloring Gravey used was IE specific. Other browser specific style settings had to be added. E.G. the following...
    body {
      filter:progid:DXImageTransform.Microsoft.Gradient
        (GradientType=0,StartColorStr='#f8f8f8',EndColorStr='#802222');
    }
gets replaced with...
    body {
      background: -webkit-gradient(linear, left top, left bottom, from(#f8f8f8), to(#802222));
      background: -moz-linear-gradient(top, #f8f8f8, #802222) no-repeat;
      filter:progid:DXImageTransform.Microsoft.Gradient
        (GradientType=0,StartColorStr='#f8f8f8',EndColorStr='#802222');
    }

Issue:  Tooltip rendering is browser-specific
Tooltips are used extensively by Gravey, and for images it used the .alt text. It turns out that non-IE browsers don't render image .alt text, only .title text, so I changed all the alts to titles.
There is also a known bug in Firefox where tooltips are not displayed on disabled buttons. As a workaround, the "readOnly" attribute can be used, however, when setting the readonly attribute rather than the disabled attribute, I could never get the CSS styling to kick in, even though I tried the selectors ".fooClass[readonly]" and "input[readonly]" and "input:read-only". So, I wound up dynamically setting (via JavaScript) the style for both readonly/not-readonly explicitly via classnames (ala .foo and .fooRO).

Issue:  Cursor styles are browser-specific
Gravey extensively used the "hand" and "not-allowed" cursor shapes which turn out to be IE-specific.  The "hand" cursor can be changed to the standard "pointer" to get the same effect. But while many browsers implement the non-standard "not-allowed", Firefox on the Mac does not and has no real equivalent. Because there is a visual difference between the default cursor and the "pointer" on Firefox, I live with it as is.

Issue:  Disabled button styles are browser-specific
Gravey expects the rendering of GUI widgets to reflect their enabled/disabled status. Some browsers stop showing disabled buttons differently if their foreground color is set to black (as Gravey was doing and IE not caring).  I was able to simply stop setting the foreground color since black was the default color anyway).

Issue:  Chrome displays popup menu items in a strange order
There is a strange (but known?) bug in Chrome where it displays items in a pop-up menu in a weird order that is different than the order shown in all the other browsers. I give up and document it as a known Chrome bug.

Issue:  Cross-browser manipulation of element visibility is complicated
Gravey originally got away with making chunks of HTML visible or not via...

    e.style.visibility = visibleFlag ? "visible"  : "hidden";
    e.style.position   = visibleFlag ? "relative" : "absolute";
    e.style.display    = visibleFlag ? "inline"   : "none";

However, to do this with other browsers the original visible state needs to be saved and reused later when restoring visibility.  Code like the following is needed...
    if (visibleFlag) showElem(e); else hideElem(e);

    function hideElem( e )
    {
      if ( e.style.display === "none" ) return;
      e.oldDisplay = e.style.display;
      e.style.display = "none";
      e.style.visibility = "hidden";
    }
    function showElem( e )
    {
      e.style.visibility = "visible";
      if ( e.oldDisplay ) e.style.display = e.oldDisplay;
      else if ( e.style.display==="none") e.style.display = "";
    }

Also, some browsers have a problem unless the HTML is rendered visible before making it hidden.

Issue:  Pop-up windows are no longer reliable
In the early IE5 days, the cursor would not reliably change to the "please wait" icon when told to, so as a workaround, Gravey used a little "please wait" popup window that was launched and later killed.  Unfortunately, modern browsers suppress popup windows, so, that code won't reliably work anymore. If the cursor STILL can not be reliably set in this modern age, then a replacement "please wait" popup window would have to be done via fancy CSS floating DIVs as I've seen elsewhere.

••• Event Handling Issues •••

Issue:  Event object location is browser-specific
In IE, the event object may be accessed in event handler functions by the global variable "event". Most other browsers pass the event object as a parameter to the event handler function and it is not available as a global variable.
  • Thus, code like the following...
    <body onKeyPress="onKey()">
    ...
    function onKey(){ alert(event...); }
    must be changed to... 
    <script> 
      document.onkeypress = onKey;
      function onKey(e){ e = e || window.event; alert(e...); }
    </script> 
  • Another consequence of event handlers being passed the event as a parameter is that the calling signature has to be changed when passing your own parameters to the event handler. Even if you don't care about the event arg itself, the other parameters need to be shifted down otherwise the formal parameter list won't match the actual parameter list. For example, Gravey generates new HTML elements by building a string and setting the innerHTML.
    So, change code like this...
        e.innerHTML = "<select onchange='return foo("+myArg+")'>";
        ...
        function foo( myArg ){ alert(myArg); }

    to this...
        e.innerHTML = "<select onchange='return foo(event,"+myArg+")'>";
        ...
        function foo( e, myArg )
    { alert(myArg); }
Issue: Not all body tag event handlers are attached to the "window" object

When moving the specification of event handlers from inline in the <body> tag to being set in JavaScript, one might think that they all go on the window event, but not so.  Thus, this tag...
    <body onKeyPress="onKey()" onBeforeUnload="onBUL()" onLoad="onLoad()">
gets replaced with...
    <script>
        document.onkeypress   = onKey;
        window.onbeforeunload = onBUL;
        window.onload         = onLoad;
    </script>

Issue:  Keyboard event objects are browser-specific
In keyboard-related events, some browsers supply the key pressed via the event's "keyCode" attribute, whereas others via the "which" attribute.
  • Thus code that reads keys...
      function onKeyPress(){ alert( event.keyCode ); }
    must be changed to...
      function onKeyPress( e ){
     
       e = e || window.event;
        var code = e.keyCode || e.which;
       
    alert( code );
      }

     
  • Thus code that sets keys (like keystroke filtering)...
      event.keyCode = char;
    must be changed to...
      e = e || window.event;
      if (e.keyCode) e.keyCode = char; else e.which = char;
     
  • Secondarily, some browsers supply a code that directly reflects the use of control keys, etc, whereas others supply a code for the basic key and require also looking at other attributes that flag the use of any modifier keys.  Thus, to detect a control-z key, the following...
      function onKeyPress(){ if ( event.keyCode==26 ) ... }

    must be changed to...
      function onKeyPress( e ){
     
       e = e || window.event;
        var code = e.keyCode || e.which;
        if ( code==26 || code==122 && e.ctrlKey ) ...

      }
Even then, Chrome on the Mac will not generate an event at all when control-y is pressed, even though control-z works fine.  So, after not finding any info about this Chrome quirk, I gave up and documented the use of shift-control-y as a workaround.

Issue:  Modifier keys on mouse click are browser-specific
Gravey and its example apps use "control-clicks" (ie. holding down the control key while clicking) in several places. Unfortunately, on the Mac, control-click activates the context menu (ala back-click). So, for it, allow the "command" key to be used as an alternate. Unfortunately, the DOM3 standard indicator for this key is the event.metaKey flag and Internet Exploder does not even implement that attribute, so references to it will be undefined. So, this code...
  function onClick(){
    var specialFlag = event.ctrlKey;
  }
has to be changed to...
  function onClick(e){
    e = e || window.event;
    var cmdKey = (typeof e.metaKey=="undefined") ? false : e.metaKey;
    var specialFlag = e.ctrlKey || cmdKey;
  }

Issue:  Some "are you sure you want to leave?" techniques are browser-specific
In order to ask if the user really wants to leave the web page, only some browsers recognize setting the event object's "returnValue" attribute to the desired message.  However all browsers tested (except for a known bug on the iPad) will recognize returning the string message from the onbeforeunload event handler.  Thus, code like this...
    function onBeforeUnLoad(){
      if (unsavedData) event.returnValue = "Are you sure?";
    }
gets replaced with code like this...
    function onBeforeUnLoad(){
      if (unsavedData) return "Are you sure?";
    }

Issue: Getting a reference to the event's target element is browser-specific
For example, to blur the element that is the target of a keyboard event, the original code...
    event.srcElement.blur();
has to be changed to (according to here in quirksmode.org)...
    var targ = e.target || e.srcElement;
    if (targ.nodeType == 3) // defeat Safari bug
        targ = targ.parentNode;
    targ.blur();

Issue:  Focus and Blur and Change events are browser-specific
There is a known problem: "Safari and Chrome do not fire focus/blur events when the user uses the mouse to access checkboxes, radios or buttons. Text fields, textareas and select boxes work correctly."
Hmmm...this is difficult because the entire architecture of Gravey is based on blur events triggering datamodel updates. Lets see if I can fake it for the offending browsers with the change event (but only the offending ones since onchange is apparently buggy on IE).  Actually because of what I am doing on blur is idempotent, I can just make both onchange and onblur invoke the same event handler (and also because of idempotency I dont have to worry about the unpredictable ordering of blur and change events in different browsers).

Saturday, August 28, 2010

Neural Nets, Vagueness, and Mob Behavior

In response to the following question on a philosophy discussion board, I replied with the short essay below and reproduce it here.
"It was then that it became apparent to me that these dilemmas – and indeed, many others – are manifestations of a more general problem that affects certain kinds of decision-making. They are all instances of the so-called ‘Sorites’ problem, or ‘the problem of the heap’. The problem is this: if you have a heap of pebbles, and you start removing pebbles one at a time, exactly at what point does the heap cease to be a heap?"
VAGUE CONCEPTS
This leads to the entire philosophy of "vagueness". i.e. are there yes/no questions that don't have a yes/no answer? Are some things like baldness vague in essence, or, is our knowledge merely incomplete? e.g. we don't know the exact number of hairs on your head, and/or, we don't know/agree on the exact number of hairs that constitutes the "bald" / "not bald" boundary?

NEURAL NETS
My personal conclusion is that there ARE many vague concepts that we have created that are tied to the way our brains learn patterns (and, as a side effect, how we put things into categories). In contrast to rational thought (i.e. being able to demonstrate logically step by step our conclusions), we "perceive" (ala Locke/Hume/Kant) many things without being able to really explain how we did it.

In Artificial Intelligence, there are "neural network" computer programs that simulate this brain-neuron style of learning. They are the programs that learn how to recognize all different variations of a hand-written letter "A" for example. They do not accumulate a list of shapes that are definitely (or are definitely not) an "A", but rather develop a "feel" for "A"-ness with very vague boundaries. They (like our brains) grade a letter as being more or less A-like. It turns out that this technique works much better than attempting to make rational true/false rules to decide. This is the situation that motivates "fuzzy logic" where instead of just true or false answers (encoded as 1 or 0), one can have any number in-between, e.g. 0.38742 (i.e. 38.7% likely to be true).

WISDOM OF THE CROWD?
Because each person has their own individually-trained "neural net" for a particular perception (e.g. baldness, redness, how many beans are in that jar?), we each come up with a different answer when asked about it. However, the answers do cluster (in a bell-curve-like fashion) around the correct answer for things like "how many beans".  This is what led Galton to originally think that there was "wisdom in the crowd".  This idea has been hailed as one of the inspirations for the new World Wide Web (aka Web 2.0). The old idea was that McDonalds should ask you if "you want fries with that?" to spur sales. The new Web 2.0 idea is that Amazon should ask you if you want this OTHER book based on what other people bought when they bought the book you are about to buy. I.E. the crowd of Amazon customers know what to ask you better than Amazon itself.

The problem is that there are many failures of "crowd wisdom" (as mentioned in that Wikipedia page in the link above). My conclusion is that most people advocating crowd wisdom have not realized that it is limited to "perceptions". Many Web 2.0 sites are asking the crowd instead about rational judgments, expecting them to come up with a better answer than individuals. The idea of democracy (i.e. giving you the right to vote) has been confused with voting guaranteeing the best answer, no matter the question. In fact, Kierkegaard wrote "Against The Crowd" almost 200 years ago where he recognized that individuals act like witnesses to an event, whereas people speaking to (or as a part of) a crowd, speak what we would now call "bullshit" because they are self-consciously part of a crowd. We can see this in the different results of an election primary (a collection of individuals in private voting booths) versus Caucuses where people vote in front of each other.  So, Web 2.0 sites (Facebook, MySpace, blog Tag Clouds, etc) that allow people to see the effect on other people of what they are saying, are chronicling mob mentality rather than collecting reliable witness reports.

BTW, I have written several blog posts related to vagueness, for example:
http://existentialprogramming.blogspot.com/2010/03/model-entities-not-just-their-parts.html

Wednesday, June 9, 2010

Fuzzy Unit Testing, Performance Unit Testing

In reading Philosophy 101, about Truth with a capital "T", and the non-traditional logics that use new notions of truth, we of course arrive at Fuzzy Logic with its departure from simple binary true/false values, and embrace of an arbitrarily wide range of values in between.

Contemplating this gave me a small AHA moment: Unit Testing is an area where there is an implicit assumption that "Test Passes" has either a true or false value.  How about Fuzzy Unit Testing where there is some numeric value in the 0...1 range which reports a degree of pass/fail-ness? i.e. a percentage pass/fail for each test.  For example, testing algorithms that predict something could be given a percentage pass/fail based on how well the prediction matched the actual value.  Stock market predictions, bank customer credit default prediction, etc come to mind.  This sort of testing of predictions about future defaults (i.e. credit grades) is just the sort of thing that the BASEL II accords are forcing banks to start doing.

Another great idea (if I do say so myself) that I had a few years ago was the notion that there is extra meta-data that could/should be gathered as a part of running unit test suites; specifically, the performance characteristics of each test run.  The fact that a test still passes, but is 10 times slower than the previous test run, is a very important piece of information that we don't usually get.  Archiving and reporting on this meta-data about each test run can give very interesting metrics on how the code changes are improving/degrading performance on various application features/behavior over time.  I can now see that this comparative performance data would be a form of fuzzy testing.

Thursday, April 8, 2010

Too cool for school (or, history anyway)

There has been a flurry of articles and blog posts in reaction to Oracle's Jeet Kaul who, at EclipseCon 2010, said "We need to get the younger generation interested and excited [about Java] just like I was" and "I would like to see people with piercings doing Java programming".  There have been some exchanges over "cool" languages and whether "cool" is a good thing.  I contributed the following 2-cents on a JavaWorld discussion...

"Cool" led to most computer security problems in the world today

I have been around long enough to have seen several generations of "cool" languages overtaking established ones.  Unfortunately, the new language was sometimes a big step backward.

A case in point was when C swept the world, replacing the line of languages like Pascal and the stillborn Ada.  Because Unix was starting to get out of the lab and into microcomputers, C gained visibility.

BIG PROBLEM: Pascal (and others) could tell how big an array was, and hence could stop execution when someone attempted to write past the end.  C was unable to do this (and because of the language design there was no way to "fix it" in the compiler).  This is the security hole that is at the root of all computer virus exploits.

If developers chose new languages (or systems) based on technical merit rather than "cool", we would be decades ahead of where we are today.

As a side note/rant: I have always thought that "cool" was effected by political zeitgeist.  It seems more than coincidence that European/Govt-regulated/strong-typing languages (e.g. Pascal/Ada) were replaced by get-the-compiler-regulation-off-the-programmers-back ethic of C at the same time Ronald Reagan was selling everyone on the same notion about government.  Let the programmer do dangerous things (like convert an array into a pointer) because he knows better...(sounds like let the bankers do whatever they want because they know better than the regulators who after all only learn from past mistakes...wimps!)

Tuesday, March 30, 2010

Moore's Paradox. I'm just saying!

The popular phrase "I'm just saying" has been around long enough for most people to have heard it, but not long enough for it to be well-documented as to where it originated.  I heard a great stand up comic bit about it in the 1980's by Paul Reiser. There are several blog sites that muse over its origin and solicit theories:
It turns out that the most common definition of the phrase exhibits a logical paradox from Philosophy.  The book "this sentence is false" is a collection of philosophical paradoxes, and it describes Moore's Paradox (as developed by G.E. Moore).  I summarize it as follows:
Normally, everything that can be said about the world can be said by anyone. I can say the moon is made of green cheese, and you can say it.  The state of the world described by me can equally be described by you with no logical paradox...EXCEPT... I can say that the moon is made of green cheese, and I can say that you do not believe that the moon is made of green cheese, but YOU can not say the same thing.  I.E. you can not say that X is true and at the same time say that you do not believe that X is true.  Note that you are not saying that you could be wrong in your belief, you are be saying that you believe both that X is, and is not, true at the same time. A logical contradiction.
However, whenever you use the phrase "I'm just saying!", you are in effect performing Moore's paradox.

Thursday, December 18, 2008

Where Am I?

In I am a Strange Loop[1], Doug Hofstadter ponders where one's "self" is located while being mentally absorbed by a situation that is located in a different place than one's body is currently residing. A simple example being that of reading Jane Austin while sitting in a chair. Another example being remote-controlling a robot on the moon. He asks the question "Where Am *I*" (where "I" is his shorthand for soul/self/consciousness).

It reminded me of my very first days exploring the World Wide Web in 1994.  I explained to my family, as I gave them a guided tour of my new toy "Netscape", that we could "visit" places all around the world!  Look, here we go to the South Pole[2] or Australia[3]!  Because in those early days, the web server and the content were actually physically in those places, and because the browser was hardly more than a remote terminal program, it really was like remote logging in to computers around the world, which felt very much like being there. That made switching from one site to another feel like teleporting instantly from one continent to another.

These days, the content about a place, versus the server serving the content, versus the location of the many cached copies (e.g. Akamai), and so forth have blurred "where am I" so much as to be meaningless and not even contemplated anymore.  But in the early days, there was a real sense of "I am in Antarctica now!".

[1] "I am a Strange Loop",2007, Hofstadter, Basic Books
http://www.kinderwijs.nl/artikelen.asp?postid=188

[2] http://www.usap.gov/videoClipsAndMaps/spwebcam.cfm
[3] http://www.radioaustralia.net.au/