[Ed. Note. This is part one of a series found on "Existential Programming, the blog": "A Rose is a Rose is (not) a Rose"]
It delights me to find out that what I thought had been a particular nugget of wisdom, specific to building Identity matching computer systems, actually has a deep principle at work. While working on one of these systems, I learned the strategy of NOT merging all variations of an individual identity's name/address/phone/etc into a single canonical version. It turns out that the need to keep, and assign a unique key to, every variation of identity data (as opposed to only the "canonical" one) has deep roots in language itself...
While reading the Intellectual Devotional (which I highly recommend), I came across its page about "Philosophy of Language" and it had an immediate resonance with a project at my current client. The page describes the "problem of reference" where ideas about what a name "means" have been debated and changed over time.
One theory says that "names" don't have any meaning, in and of themselves, they merely refer to some thing that has meaning. Hence, Shakespeare's quote "A rose by any other name would smell as sweet" summarizes the position that the word "rose" is not meaningful, and could be exchanged with any other word that refers to the thing "rose". That is why "gulaab" (the Urdu word for rose) can work just as well for speakers of Urdu.
Another more modern theory though, says that names not only refer to some thing, they also carry the connotation of "in what sense" is the thing being referenced. The book illustrates the example of Superman and Clark Kent both being names for the same thing (the being Superman), but they are not interchangeable. Clark Kent (mild mannered reporter) has a work address of the Daily Planet whereas Superman (superhero able to leap tall buildings) does not. It matters which name is used when talking about Superman.
So, in the same way that Clark Kent and Superman both refer to different aspects of the same entity, and are thus not interchangeable, a computer system managing legal entity identity data can not translate name/address variations into a single entity ID# when those variations actually refer to different aspects of the entity. For example, if there is data that is specific to a particular store branch, that branch needs its own well-known ID# even though it is only a portion of a single legal entity. Further, since legal entity names are not unique (I own two different corporations with the identical legal name), the entire name/address/phone/etc combination needs managing rather than separate "alternate name" lists. It is also not sufficient to support alternate name/address records merely as search aids that still ultimately result in the ID# of the entity-as-a-whole. Otherwise one would loose track of the fact that we were talking about Clark Kent, not Superman.
Saturday, December 22, 2007
Friday, November 30, 2007
Odometer Game Redux
Well, after 35 years of pondering what I thought was an abstract mathematical puzzle, my "odometer game" has found a real-world application!
It turns out that my notion of "remarkable" numbers [i.e. numbers that are so remarkable that if the driver saw his odometer sitting on that number he would either honk his horn or point it out to his passengers] are just the ticket for finding "fake" ID numbers.
My current contract at a major bank found me looking for suspect ID numbers, Tax IDs, phone numbers, etc. in various customer databases. The bank employees entering this information would often get around the fact that these fields were "required" via entry of syntactically legal digit strings that were none the less meaningless. After viewing a few of these it quickly became obvious that they were related to my notion of remarkableness. Actual values found included: 0, 121212121, 000000000, 999999999(9), 111111111, 111111112, 222222221, 888888889, 188888888, 0999999999, 589999999, 255511555 (?)
So, rather than an explicit list of IDs to put on a watch list (as I was asked to find), it became clear that a better answer would have been to use an evaluation function that reported the remarkableness score for each value. A cutoff point could then be established to filter out suspicious values. Alas, while I have casually pondered the mathematics involved in scoring the remarkableness of a number, I've never actually tried to program it. But, now it has become more than an obscure puzzle, and shows signs of having "real world" value!
It turns out that my notion of "remarkable" numbers [i.e. numbers that are so remarkable that if the driver saw his odometer sitting on that number he would either honk his horn or point it out to his passengers] are just the ticket for finding "fake" ID numbers.
My current contract at a major bank found me looking for suspect ID numbers, Tax IDs, phone numbers, etc. in various customer databases. The bank employees entering this information would often get around the fact that these fields were "required" via entry of syntactically legal digit strings that were none the less meaningless. After viewing a few of these it quickly became obvious that they were related to my notion of remarkableness. Actual values found included: 0, 121212121, 000000000, 999999999(9), 111111111, 111111112, 222222221, 888888889, 188888888, 0999999999, 589999999, 255511555 (?)
So, rather than an explicit list of IDs to put on a watch list (as I was asked to find), it became clear that a better answer would have been to use an evaluation function that reported the remarkableness score for each value. A cutoff point could then be established to filter out suspicious values. Alas, while I have casually pondered the mathematics involved in scoring the remarkableness of a number, I've never actually tried to program it. But, now it has become more than an obscure puzzle, and shows signs of having "real world" value!
Saturday, October 13, 2007
The Odometer Game
Dear Dr. Douglas Hofstadter,
Having been a fan of yours since GEB (I had you sign my copy in '83 at UC Santa Cruz), I have always wanted to write to you about an "odometer game" I concocted about 1973 which touches upon several of your favorite themes: patterns, their recognition, and "human" vs "machine" intelligence. Following Hofstadter's law, it has taken longer to write you than I ever thought it would. ;-)The thought-provoking part is imagining how a computer would ever "play" the game. It would involve mathematically defining "remarkable" odometer numbers, where "remarkable" is defined as any number that would intuitively cause the driver to remark to the other passengers: "Hey, look at the odometer!". The more likely a number is to cause a driver to say that, the higher its "remarkableness".
I've casually pondered the math for this for years. Let me know if you have already solved it.
sincerely,
Bruce Wallace
The Odometer Game
As many people have done over the years, I honked my horn when my odometer rolled over to all zeros [000000] (back when that only took 100,000.0 miles to do so). Later, when I put on another 11,111.1 miles [111111], I decided that an odometer reading of all ones was also worthy of a honk (a bit of a geek whimsy).
Since I had many long boring drives between college and parents, I came up with a little diversion which was to honk (or otherwise take note of) any "remarkable" odometer reading [where "remarkable" was any number that would make a driver point it out to the passengers].
I even accumulated imaginary points that mirrored the amount of "remarkableness" of the number. But, soon I realized I needed a reason to keep from simply taking note of EVERY number in my quest to build up my point total. I thought that maybe some function balancing total-points vs average-points-per-honk was needed. And to make things more sporting, I should lose points if I missed any numbers in a "pattern" once I had taken points for that pattern. In other words, if I took points for [000000] and [111111], I would lose points if I missed [222222]. (I.E., don't start a pattern if you aren't going to keep it up!)
So, [000000] was definitely remarkable, and so was [111111], [222222], [333333], etc. (Hmmm... [111111] seems less remarkable than [000000], and [222222] thru [888888] all seem less remarkable than either [000000] or [111111]...should they all get the same points?). Then came [123456]. And while less remarkable, [234567], [345678], etc. all seem pretty good.
Palindromes are very good, but [123321] seems more remarkable than [394493] or [825528]. And [121212] & [123123] are very good, but less so [838383] & [378378]. While [010101] and [999999] beat out [898989] & [888888] respectively, all seem good enough to take the points.
Round numbers like [010000], [020000], [030000], etc. seem nice because the pattern is anchored with [000000]. Actually, a number like [000000] meets lots of patterns at once: [aaaaaa], [ababab], [abccba], [abcabc], etc. (in addition to being the ultimate round number), so, it gets LOTS of points.
The Puzzle
Why are some numbers (i.e. digit strings) instinctively more "remarkable" than others? How would one model this mathematically? Patterns seem part of the answer, but a readily recognizable pattern is in [192837] even though it would seem very unlikely for a driver to make a passenger take note of that number/pattern.
And why are [000000], [111111], & [999999] all more "remarkable" than [222222] thru [888888]? Why is [123456] more than [012345], but [121212] and [010101] seem more of a toss up? Is the "simplicity" of the pattern the crux of "remarkableness"? How would one describe that "simplicity" mathematically (especially when 0 and 1 and 9 seem somehow more "simple" than 2 thru 8)? What grammar "parses" this string language?
Monday, May 21, 2007
Language Plateaus in Evolution?
In reading[1] about the different levels of human language competency that plateau at various ages (6, puberty, etc), it made me wonder if those capabilities mirrored those of our ancestors at various stages of evolution. Just as a human embryo looks like amphibians, etc as it is developing (in a mirror of DNA development over the ages), maybe language skill levels that jump in quantum leaps mirror primate evolution?
[1] Introducing Chomsky, John Maher, Judy Groves, Icon Books, 1996.
Saturday, November 29, 2003
The reXume/GanttMagic webapps can now be embedded...
The reXume and GanttMagic web apps have been modified such that they can be configured to process a hardwired reXume file rather than only uploaded files.
My hand coded Interactive Resume web site ( http://rexume.dyndns.org/resume ) is being replaced with a new version based on the std. reXume and GanttMagic web apps ( http://rexume.dyndns.org/reXume/RBWrexume.html ) that also acts as a demonstration of using reXume for personal resume sites.
The WAR files are currently deployed on JRun 3.0 but have been tested on Tomcat 4.1 as well.
My hand coded Interactive Resume web site ( http://rexume.dyndns.org/resume ) is being replaced with a new version based on the std. reXume and GanttMagic web apps ( http://rexume.dyndns.org/reXume/RBWrexume.html ) that also acts as a demonstration of using reXume for personal resume sites.
The WAR files are currently deployed on JRun 3.0 but have been tested on Tomcat 4.1 as well.
Monday, October 6, 2003
Announcing RDDL file for GanttMagic XML Namespace...
There is a first major stab at publishing an RDDL file documenting the XML namespace I've defined for the GanttMagic XML input data. It is at http://www.polyglotinc.com/NS/GanttMagic/.
Soon (hopefully) there will be a similar one for the reXume data XML namespace [ http://www.polyglotinc.com/NS/reXume ].
Soon (hopefully) there will be a similar one for the reXume data XML namespace [ http://www.polyglotinc.com/NS/reXume ].
Sunday, September 21, 2003
How to Pick a Science Fair Experiment
Presented is a set of intuitive classroom assignments that lead students
to pick better quality science fair projects, and in the process, teaching
them the "Scientific Process" and how it is something that they already
intuitively know.
Students are often impeded with the notion that "thinking like a scientist"
is so different from the way they normally think that they don't know how
to proceed. The given approach starts students with intuitive activities
with which they are already familiar then shows them how to merely "fine tune"
their thinking rather than viewing science as "speaking a foreign language".
In a nutshell, I propose that choosing a topic for a science fair project
and designing its experiment properly are the equivalent to the intuitive
activities of making a list of “Whenever THIS then THAT” statements and then
choosing the best “bar bet” that can be constructed from the list.
One intuitively makes observations and constructs theories to explain them
whenever a “Whenever THIS then THAT” statement is made. One intuitively
evaluates the quality of a science experiment whenever one evaluates the
quality of a bet. I.E. the same things that make a good bet make good science.
Namely:
* - you think you know something about the world that the other guy
doesn’t know (otherwise he won’t take the bet if he knows it too)
* - you think you understand it well enough to explain it (otherwise it is
not a safe bet)
* - you think you can demonstrate a prediction about it (i.e. the bet itself)
in such a way that the outcome will be clear (otherwise there will be
an argument over who won the bet).
* - you have controlled the conditions of the demonstration (otherwise it
makes the outcome of the bet be affected by things you can’t predict).
* - someone else can perform the bet if need be (otherwise people will think
you’ve rigged the game).
-------------------
The Approach
-------------------
0) Preview the Scientific Process to students
a) observations about the world are made
b) theories are created that explain those observations
c) predictions are made from those theories
d) experiments to verify the predictions are made
e) the results of the experiments either confirm the
predictions and therefore the theory, or they don't,
in which case one goes back to (b) taking into account
the new observations made by this experiment.
0) Preview some factors that make “good science”
*) observations are new
*) observations are surprising
*) theories make predictions that can be tested
*) experiments produce results that are conclusive
*) experiments are repeatable by others
1) Observations and Theories
[Use the intuition students exhibit, to both make observations
and generate theories to explain them, whenever they use
sentences like "Whenever THIS then THAT".]
(a) Assignment 1: make a list of things you know/believe about the world
“Everyone make a list of things that they have noticed (or been told
or read) about the world.
(At least 5 things and extra credit for each extra item up to 20 items).
The items in the list should all be in one of the following forms:
* - Whenever THIS happens (or not),
then THAT seems to always happen (or not).
* - Whenever THIS situation exists (or not),
then THAT seems to always happen (or not).
Examples of form:
* - I've noticed that whenever I let go of something I'm holding,
it seems to always fall to the ground.
* - I've noticed that whenever I drop a rock in water,
it never floats.
* - I've noticed that whenever people are with their friends,
they are louder than when they are alone or with strangers.
* - I've been told that whenever plants get too much water,
they die.”
(b) follow up discussion to Assignment 1: review how students made both
observations *and* theories to explain them.
* - By putting beliefs (whether inspired by direct observation or
having been informed by others) into the form "when this then that",
it filtered out simple observations that had no theory attached.
E.G. "I noticed that the sky is usually blue" is an observation, but
there is no theory to explain it. In order to say "if A then B",
one had to already have enough of an idea about cause and effect to
make the statement, whether the statement itself was correct or not.
But people/students makes statements like this all the time and
therefore are producing theories whether they realized it or not.
2) Experiment Selection and Design
[Use the intuition students exhibit, to choose interesting science
experiments and to design them to produce a clear result, whenever
they use sentences like "I'll bet you!".]
(a) Assignment 2: make bets out of the theories
Part 1: Everyone take their list of observations, and for each one, make
a bet out of it.
Examples:
* - I'll bet you that if I let go of this ball,
it will fall to the floor.
* - I'll bet you that if I throw this rock in the water,
it won't float.
* - I'll bet you that if we measure the noise level of 3 friends
eating together in the lunchroom it will be louder than if we
measure 3 people eating together that don't know each other.
* - I'll bet you that if I give this houseplant way more water
than the gardening book says it should get, it will die.
Part 2: Take your list of bets and rate each one for the factors below.
Add these factors together to get the quality score for each bet.
For safety, add the practicality factor 3 times instead of once.
The higher the score, the better the quality.
* - How non-obvious is this? (i.e. will anyone take this bet?)
Rank from 1 to 10 where 10 is "nobody knows this but me"
and 1 is "everyone on the planet knows this"
* - How well do you understand your theory (i.e. how sure of
the bet are you?). Rank from 1 to 10 where 10 means "I'm
sure I'll win the bet" and 1 means "I'm just guessing
what will happen."
* - How practical is it? (i.e. is there a way to actually
make a bet out of this?) Rank from 1 to 10 where 10 means
"this is easy to perform" and 1 means "this will take
a UN resolution to actually do."
* - How obvious will the outcome be? (i.e. how obvious will
it be who won the bet?) Rank from 1 to 10 where 10 means
"obvious result" and 1 means "we'll be in an argument
all day over who won, was it fair, is it a do-over, etc."
Part 3: Make any changes you can to the description/design of each
bet to improve its quality score before settling on the final
quality score for each bet. Take the top 3 bets and rank
each for the following factors:
* - How well can I control things that might affect the result?
(i.e. will I lose the bet because of something I can't
predict or control?). Rank from 1 to 10 where 10 means
"nothing should foul up the works if I specify when/where/
how/etc" and 1 means "every time the air conditioner
comes on it blows down my house of cards".
What conditions can be added that will make it more of a
sure bet? After adding them, make a final rank for this
factor.
* - How well can I describe the procedure? (i.e. how easy will
it be for a 3rd party to perform the bet?) Rank from 1 to
10 where 10 means "even a trained monkey could do this
correctly" and 1 means "I'm the only one who can ever
make this work".
How can the procedure and description be simplified and
improved such that others can get the same results every
time? After making the improvements, make a final rank for
this factor.
Part 4: Add the factors from part 3 to those from part 2 for the top
3 bets and pick the one with the highest quality score as your
choice for an experiment.
to pick better quality science fair projects, and in the process, teaching
them the "Scientific Process" and how it is something that they already
intuitively know.
Students are often impeded with the notion that "thinking like a scientist"
is so different from the way they normally think that they don't know how
to proceed. The given approach starts students with intuitive activities
with which they are already familiar then shows them how to merely "fine tune"
their thinking rather than viewing science as "speaking a foreign language".
In a nutshell, I propose that choosing a topic for a science fair project
and designing its experiment properly are the equivalent to the intuitive
activities of making a list of “Whenever THIS then THAT” statements and then
choosing the best “bar bet” that can be constructed from the list.
One intuitively makes observations and constructs theories to explain them
whenever a “Whenever THIS then THAT” statement is made. One intuitively
evaluates the quality of a science experiment whenever one evaluates the
quality of a bet. I.E. the same things that make a good bet make good science.
Namely:
* - you think you know something about the world that the other guy
doesn’t know (otherwise he won’t take the bet if he knows it too)
* - you think you understand it well enough to explain it (otherwise it is
not a safe bet)
* - you think you can demonstrate a prediction about it (i.e. the bet itself)
in such a way that the outcome will be clear (otherwise there will be
an argument over who won the bet).
* - you have controlled the conditions of the demonstration (otherwise it
makes the outcome of the bet be affected by things you can’t predict).
* - someone else can perform the bet if need be (otherwise people will think
you’ve rigged the game).
-------------------
The Approach
-------------------
0) Preview the Scientific Process to students
a) observations about the world are made
b) theories are created that explain those observations
c) predictions are made from those theories
d) experiments to verify the predictions are made
e) the results of the experiments either confirm the
predictions and therefore the theory, or they don't,
in which case one goes back to (b) taking into account
the new observations made by this experiment.
0) Preview some factors that make “good science”
*) observations are new
*) observations are surprising
*) theories make predictions that can be tested
*) experiments produce results that are conclusive
*) experiments are repeatable by others
1) Observations and Theories
[Use the intuition students exhibit, to both make observations
and generate theories to explain them, whenever they use
sentences like "Whenever THIS then THAT".]
(a) Assignment 1: make a list of things you know/believe about the world
“Everyone make a list of things that they have noticed (or been told
or read) about the world.
(At least 5 things and extra credit for each extra item up to 20 items).
The items in the list should all be in one of the following forms:
* - Whenever THIS happens (or not),
then THAT seems to always happen (or not).
* - Whenever THIS situation exists (or not),
then THAT seems to always happen (or not).
Examples of form:
* - I've noticed that whenever I let go of something I'm holding,
it seems to always fall to the ground.
* - I've noticed that whenever I drop a rock in water,
it never floats.
* - I've noticed that whenever people are with their friends,
they are louder than when they are alone or with strangers.
* - I've been told that whenever plants get too much water,
they die.”
(b) follow up discussion to Assignment 1: review how students made both
observations *and* theories to explain them.
* - By putting beliefs (whether inspired by direct observation or
having been informed by others) into the form "when this then that",
it filtered out simple observations that had no theory attached.
E.G. "I noticed that the sky is usually blue" is an observation, but
there is no theory to explain it. In order to say "if A then B",
one had to already have enough of an idea about cause and effect to
make the statement, whether the statement itself was correct or not.
But people/students makes statements like this all the time and
therefore are producing theories whether they realized it or not.
2) Experiment Selection and Design
[Use the intuition students exhibit, to choose interesting science
experiments and to design them to produce a clear result, whenever
they use sentences like "I'll bet you!".]
(a) Assignment 2: make bets out of the theories
Part 1: Everyone take their list of observations, and for each one, make
a bet out of it.
Examples:
* - I'll bet you that if I let go of this ball,
it will fall to the floor.
* - I'll bet you that if I throw this rock in the water,
it won't float.
* - I'll bet you that if we measure the noise level of 3 friends
eating together in the lunchroom it will be louder than if we
measure 3 people eating together that don't know each other.
* - I'll bet you that if I give this houseplant way more water
than the gardening book says it should get, it will die.
Part 2: Take your list of bets and rate each one for the factors below.
Add these factors together to get the quality score for each bet.
For safety, add the practicality factor 3 times instead of once.
The higher the score, the better the quality.
* - How non-obvious is this? (i.e. will anyone take this bet?)
Rank from 1 to 10 where 10 is "nobody knows this but me"
and 1 is "everyone on the planet knows this"
* - How well do you understand your theory (i.e. how sure of
the bet are you?). Rank from 1 to 10 where 10 means "I'm
sure I'll win the bet" and 1 means "I'm just guessing
what will happen."
* - How practical is it? (i.e. is there a way to actually
make a bet out of this?) Rank from 1 to 10 where 10 means
"this is easy to perform" and 1 means "this will take
a UN resolution to actually do."
* - How obvious will the outcome be? (i.e. how obvious will
it be who won the bet?) Rank from 1 to 10 where 10 means
"obvious result" and 1 means "we'll be in an argument
all day over who won, was it fair, is it a do-over, etc."
Part 3: Make any changes you can to the description/design of each
bet to improve its quality score before settling on the final
quality score for each bet. Take the top 3 bets and rank
each for the following factors:
* - How well can I control things that might affect the result?
(i.e. will I lose the bet because of something I can't
predict or control?). Rank from 1 to 10 where 10 means
"nothing should foul up the works if I specify when/where/
how/etc" and 1 means "every time the air conditioner
comes on it blows down my house of cards".
What conditions can be added that will make it more of a
sure bet? After adding them, make a final rank for this
factor.
* - How well can I describe the procedure? (i.e. how easy will
it be for a 3rd party to perform the bet?) Rank from 1 to
10 where 10 means "even a trained monkey could do this
correctly" and 1 means "I'm the only one who can ever
make this work".
How can the procedure and description be simplified and
improved such that others can get the same results every
time? After making the improvements, make a final rank for
this factor.
Part 4: Add the factors from part 3 to those from part 2 for the top
3 bets and pick the one with the highest quality score as your
choice for an experiment.
Subscribe to:
Posts (Atom)
