Pages

Friday, August 28, 2009

AVEE!!!

It seems that idea of autonomous vehicles communicating with each other to improve traffic flow has been under development!

http://www.sciencedaily.com/releases/2009/08/090824204832.htm

Now just add the marketplace idea, and you've got the Avee! hurray!

(search this blog for previous posts on "avee," it was an idea I was really exited about a year ago. It involves creating an online marketplace where owners of autonomous cars can exchange rides with non-owners.)

Monday, August 24, 2009

Using statistics to learn about myself

I'm often at a loss when people ask me what kind of music I like. I listen to a lot of music, and I get all befuddled trying to answer. If I were to let my itunes statistics speak, this is what they'd say:

Artists most played:
Smashing Pumpkins
Yeah Yeah Yeahs
Alkaline Trio
Architecture In Helsinki
Maurizio Pollini (Chopin)
The White Stripes
Regina Spektor
Yo-Yo Ma (Bach)
Hum
Feist
(Rachmaninov and Prokofiev should also be up here, but since they have relatively few peices, they don't make the top ten in terms of song frequency)

Top ten albums by playcount:
Chopin: Etudes Opp. 10 & 25
Joshua Bell: Violin Works By Prokofiev & Shostakovich
Bach: The Cello Suites
Begin to Hope (Bonus Track Version)
Siamese Dream
Show Your Bones
Fingers Crossed
Fever to Tell
Vampire Weekend
Crimson

Insurance Costs

I commented on Steve Peterson's blog that I opposed heath care reform in principal, on the grounds that no designed system can produce equivalent output to the spontaneous competitive order.

This is a little rash; there are aspects of health care that can be legitimately reformed to remove the influence of planning, and that would potentially have great effect. This article discusses what seems obvious: legalizing the selling of insurance across state borders. I was, frankly, unaware that the restriction existed. But now that I know, it seems a pretty compelling alternative suggestion for lowering costs.

Saturday, August 22, 2009

Insight from Hayek

Hayek posits that in order for a planned economic system to yield an equivalent level of production to a competitive system, it would have to be similarly able to process information. The technical capabilities, uses, condition, and location of each individual piece of productive equipment would need to be quantified and collected in a single place where it could be operated upon. This would be a system of differential equations on the order of hundreds of thousands of variables (edit: more like hundreds of billions of variables: "productive equipment" must include every tool, part, machine, natural resource, and human available in the system, each with varying levels of many characteristics), all in constant flux. Setting aside the problems of information collection, which are formidable in themselves, the solution of such systems of equations are insolvable in real time, even on the largest conceivable digital computers. Such problems are NP complete. (edit: we should also note that since humans are included as productive resources, all of their actions must also necessarily be planned, something undesirable if freedom is a value.)

Note that the stated goal above was merely to emulate what the competitive marketplace already does. How does the competitive marketplace accomplish this apparently impossible feat of data collection and calculation?

The answer must be that the price system itself is an instantiation of a system of differential equations via matrix methods. We've seen from linear algebra how systems of equations can be represented in matrix form, and we've seen from neuroscience and learning theory how neural networks can instantiate functions of equations and matrices of data, and thereby do matrix transformations that translate input into predictions and action. My contention is that economic networks similarly represent information in the weights and frequencies of transactions, that firms similarly integrate the input information (demand) and turn it into prediction (capacity) and action (supply). I believe this is a novel hypothesis, a consistent elaboration of previous claims of market efficiency.

The system can solve itself in real time because every element of it is both a processing mechanism and a memory mechanism; its parallel distributed processing. This is also the reason that it can be flexible; as information changes, only the effected portions of the network are recalculated. But since all information is constantly changing, the entire network is eternally in flux. This probably means that it's solution is never at the absolute maximum; indeed a single maximum almost definately doesn't exist. If all exogenous change were to cease (an nonsensical idea anyway) then the system would probably reach a maximum, but it would still probably be in flux as it could slide across all the other possible maximum values as well.

The next question then becomes: why does the system exhibit this maximizing behavior? Given that its composed of simple computational units who are unaware of the global maximizing goal, why did the network come to instantiate its maximizing function, rather than some other function? I imagine that the answer is recursive: the network is maximizing because its components are maximizing; they are maximizing because their subcomponents are maximizing, and so on down to the most basic level possible.

Monday, August 17, 2009

Car Selling

Two people have called about buying my car since I put the add up yesterday! And I'll only loose 4K on it (hurray?).

It's interesting to note how difficult its been to motivate myself to do the few things I needed to do to get it sold. Namely get the title cleared, fix the few paint blemishes, spray some WD40 on the hood release, and fix the mirror. I know very rationally that I don't need it, but I guess the idea of not owning a motor vehicle gives me pause. The virtue of motor-vehicle ownership has been so deeply ingrained in me that I have to make a conscious effort to counteract it. Its an interesting example where a value-judgment that I've consciously and rationally made conflicts with my unconscious values.

Paying it off and seeing the hit to my bank account helped overcome that particular irrationality, certainly.

Hurray for biking! I'm going to Highlands.

Horray! Step on the Road to Serfdom Avoided!

So it turns out the winds of change aren't blowing as strongly as some thought, in terms of health care.
Chances Dim for a Public Plan
This is good! But I note that they're talking about "compromise," which means we can expect at least a little damage to seep through.

Here's another voice that's sweet to my ears:
We Don't Spend Enough on Health Care
The main points: The money spent on health goods and services goes to other people; primarily Americans. Health care is a sector that can't very easily be outsourced to other countries, but its growth can be restricted; and when that happens people will have no choice but to seek it elsewhere (as evidenced by the 400K+ people who visit America every year to get treatment) or forgo treatment. I'm not of the protectionist bent; I don't think its bad to buy things from other people in other countries. But it would be good for our country to have a good industry to support itself with, and what better industry than heath care? We could probably include education in there as well.

It's good that we spend lots of money on something positive! Hopefully this debate will be buried within the next few weeks, maybe we can start discussing constructive ideas, like freeing markets from interference.

Another interesting consequence: the republican party that's become so worthless and incoherent has solidified around opposition to this sort of government control. Maybe they can drop their socially conservative elements and become something worthwhile? Probably not, but maybe.

Saturday, August 15, 2009

Externalities & Cash for Clunkers

Oh-ho! Here's an externality to the CARS program I hadn't considered:

"ST. LOUIS, MO (KTVI-FOX2now.com) - The reports of a recovery may be greatly exaggerated. Both consumer confidence and retail sales dropped last month, indicating a recovery from the recession is still a long ways off. Analysts had expected retail sales to rise, spurred by new car sales because of the cash for clunkers program. Apparently consumers cut spending on everything but cars."

So the cost of encouraging the populace to subsidize a dying industry is that all other industries are hurt, eh? Man-o-man, why is it so easy for politicians to pretend like they can get something for nothing?

Here's a suggestion: before any government intervention in the market can be executed, it must include a statement detailing what harm it will cause to balance out its intended benefit. And to be realistic, we should be sure that the stated harm is greater than the stated benefit. Then, if as a public we really think the intervention is still worthwhile, we can execute it.

Control From On High

As a matter of principal, decisions should be made at the level closest to the problem, by the people who have the most information about the problem and who are most affected by its resolution.

Agree?

Read the following WSJ op-ed on the government's proposed reform of medical services, and judge for yourself how well it meets the principle above.

http://online.wsj.com/article/SB10001424052970204409904574350370729883030.html

Perhaps "reform" aught to be put in "scare quotes," eh?

[To summarize the article; the government proposes reducing health care spending by creating guidelines for what procedures doctors should give to their patients. Originally this included the decision to remove life support from patients, but that was so widely criticized that it had to be withdrawn.]

Friday, August 14, 2009

On Indentured Servitude

It's interesting how strongly people currently seem to react against the phrase "indentured servitude," yet how willing they seem to be to put themselves into that position. I'm referring to the acceptance of financial debts, especially for non-productive assets.

Thursday, August 13, 2009

I <3 XKCD

Maybe my favorite comic:

http://xkcd.com/242/

Tuesday, August 11, 2009

On the Healthcare Issue: What Issue?

This is, quite frankly, the first reasonable public discussion I've seen about the debate around nationalizing health care since... as long as I can remember. Hurray for Dean Kamen!

http://www.popularmechanics.com/science/health_medicine/4327012.html?page=1

To summarize: the "healthcare crisis" is a manufactured issue. Health care is more expensive now because it is better. People who aren't willing to pay high prices for the breaking technology have the option of using the previous generation of technology, which becomes cheaper every time a new technology replaces it. And the pre-technology "treatment" (comfort you while you die) is still available to anyone who eschews advancement through technology.

Focusing on the fairness of distribution of current resources is counter-productive, and risks being destructive. Especially when fairness is given the Rawlsian "what kind of world would we choose if we could start from scratch and re-make everything" interpretation. Ethicists aught to focus on the world that we do live in, or else what they do can't be classified as science.

Now, that's not to say we should give up on creating a better world. Its to say that our efforts to improve the world should be guided by the same standards as science: hypothesize, test, and verify. Public actions that cannot be put in that framework should not be executed. Note that that includes most public actions.

Monday, August 10, 2009

Bridge Theory - Neuroscience, Economics, Intellegince, and Information Theory

My fundamental hypothesis is that both economics and cognitive science can be integrated into some version of information theory or computer science. This has been stirring in my head for a long time, ever since I learned about the justifications for the efficient market hypothesis. Now, with my recent learning, I think I'm almost ready to start connecting my observations and conjectures.

Here's one: Churchland shows how the Purkinje cells can easily be seen as performing transformations of matrices; it follows clearly from their structure. Can firms in marketplaces be similarly interpreted? Here's my rationale for that bizarre leap:

A firm has many inputs: its suppliers and customers. The prices at each input convey some information about the outside world (moreover, the prices for each commodity are the same across firms, adjusted for transportation cost), these are the "synaptic weights." The relative frequency of the inputs' stimulation is the positive or negative value of the element of the matrix. The input vector is conveyed through the firm's inputs (again, the suppliers and customers), is transformed according the the weights, and is summed and output as the firm's net profit. By considering whole industries, we can get output vectors. Thus: economies do matrix transformations.

This, on the surface, appears to be the same as what we observe in Purkinje cells. Can we therefore conclude that markets have the potential for cognition, assuming that these matrix transformations underlie cognition in biological neurons?

Why is this important as a theory, and not just a metaphor? Part of the reason economics is so "dismal" as a science is that its impossible to test economic theories; all studies must be purely observational (though one might argue that the failure of the socialist "experiments" consisted of "testing"). Free markets seem to serve human ends quite nicely, and while free-market advocates have been around for more than a century, their arguments are not universally accepted by the relevant decision makers. Indeed, we're currently in an environment where free market ideas appear particularly unfashionable. If we could bridge the theories of economics and cognitive science, showing that there is a meaningful identity in the information processing capacities of brains and markets, we could do real scientific testing on simple information processing models. We could test the hypothesis that de-centralized control yields the best possible informational processing capabilities. I would actually expect result to be tautological, if the "parallel-distributed processing" model we recognize to be at work in human brains is accepted to be at the root of economic processes as well.

Maybe then the preponderance of evidence would be overwhelming enough to restrain the controlling hand; finally make the case for limited government. Many have tried, could this be the ultimate argument? Presumptuous of me to even suggest, but nevertheless, I think that it is worth exploring, and that the result could be important.

Sunday, August 9, 2009

WHOOOH!!! Insight from Paul Churchland

Upon Dr. Bickle's reccomendation, I'm reading Paul Churchland's 1989 book A Neurocomputational Perspective, and lo! more gems! One of the questions I prepared for Dr. Bickle (but havn't yet posed to him) is whether matrix multiplication could have a neurological implementation, and Churchland gives it to me on p 99 of his book. Its so simple!

One question I have on Churchland's interpretation: he notes on p 99 that the values of the input vector are coded by the relative change in the frequency of the neuron's firing, as compared with its resting baseline. He later brings up the problematic existence of dedicatedly excititory and inhibitory synapses (p 184), and notes that they don't change sign like artificial neural network synapse weights can. The question: wouldn't a reduction in the relative firing rate of an inhibitory neuron be the equivalent of an increase in the rate of an excititory neuron? Does that mean that they are interchangeable?

In a different vein, I also thought of what might be a neat experiment to help probe the dimensionality of language and what kinds of matrix transformations are going on when people use language. Its pretty simple: just ask subjects to brainstorm single-syllable words and record them in the order they occur (while measuring the time between each word). If the dimensions of each word's concept could be non-arbitrarily determined, one could do a relatively straightforward analysis on what sorts of transformations need to happen to connect a word to the previously given word (or set of words).
Why use monosyllabic words? Partly to reduce the complication, partly because concepts that are more important to survival tend to be represented in smaller words, so presumably you'd get a sample of more deeply salient concepts, and the connections between them could be explored.
I did a small scale study (n=2; myself and Rick), and it the results were interesting. There were quite a few different ways the concepts seemed to get transformed; on the semantic, phonic, and letter level.
A problem in doing the study for real would be in deciding which dimensions any given word falls; there might not be a non-arbitrary way to do so, since there's no guarantee that each word is represented identically in each person's language space. A different approach might be to map the transformations between concepts and try to impute the space in which they exist using that map... is that a sensible idea? I think that would involve some fancy math that I'm presently unaware of.

Musical illusions

When I was younger, I was really really into Metallica. I'd put my speakers on either side of my bed, facing toward my head, and zone out. It was magical. I'm doing the same thing again now, and I've got to wonder if some of my enthrallment comes less from the the music itself, and more from the way its mixed across the stereo channels. It seems that all the instruments are preferentially left or right mixed, and the vocals are equal in both. When I close my eyes, the effect seems to be to create an illusion of open space in the darkness in front of me.

I think its probably similar to the experience people report getting from binaural beats; but I haven't been terribly impressed with what I've heard there.

Wednesday, August 5, 2009

Quesitons for Dr. Minai and Dr. Bickle

I'm organizing my thoughts on what I'd like to ask my professors, here they are:

Regarding the basics of neural implementation of mathematics:


Is there evidence at the cell level of operations other than vector subtraction being performed in neurons?

What operations, via what cellular mechanisms?

Can the "negative echo" phenomenon observed in saccades be generalized as the inverse of the input, rather than simply the negative? (eg: This would lay the ground work for division through reciprocal multiplication; which would allow for a cellular mechanism of averaging inputs. )

Is "inverse" validly the same concept in addition and multiplication, just applied to different operators? (eg: are negatives and reciprocals fundamentally related by the concept of "inverse"?)

Do the concepts of inversion and transposition have relevance at the cell level?

Are neural representations always row or column vectors, or does the concept of a matrix or a tensor have relevance to neural representations?

Are claims of Bayesian learning trees being implemented in neurons substantiated and/or plausible?


Regarding implementation of vector subtraction to do analogy-making in a simulation:


Suppose that each time we move from point to point in vector-space, both the inverse of the last point and the directional vector for getting there are stored in memory.

When the point and the direction are stored, they are given a high activation value, and this value decays each time we move to a new point.

At each step, all of the previous points and directions are candidates for use in the next action. The next action is determined stochastically by randomly choosing from the contents of the memory, weighted by each of the entries' current activation value.

New target points might come into the system (as if via sensory mechanisms), and they would be given an activation weight just like the previously visited points. Since they will come in with a high activation while all other weights are decaying, they will most likely (but not definately!) be used in the next step.

If a point (whether newly received or previously visited) is selected via the stochastic process, then the inverse of the current point will be added to the new point, the directional vector between them will be found and stored in memory, and the focus will shift to the new point via the directional vector.

If a previously used directional vector is selected via the stochastic process, the system will move from its current point using the previously determined direction, and it will find a new point and store it in memory.

Each time a point is visited, or a directional vector is used, its activation will increase to its maximum.

The more frequently traveled or visited a vector is, the more slowly its activation will decay and the higher the floor of its activation will become (ie: long term potentiation)

Points observed by the sensory process will receive extra activation and obtain LTP if they have been previously visited.

Highly activated vectors might have some spontaneous oscillatory behavior; their activation might spike back to a high level without external input (eg: the "return to origin" attentional vector). This would keep the system from straying too far off into irrelevant territory.


Motivation for this approach:
If the relationships between concepts are stored as abstractly (as vectors), then they should be able to be applied to other concepts (by acting on points). This could lead to the discovery of concepts (points) that have not yet been observed by the sensory mechanisms. If concepts discovered by this process are later confirmed to exist by observation, then they can be regarded as the confirmation or falsification of a hypothesis, and thus should be regarded as a relevant concept and be preserved for future use via LTP. The stochastic process simulates the parallelism of biological neural networks; I'm unaware of a better way of programming this.

Arithmetic operations in neurons

Bickle's "negative echo" mechanism provides a neurological basis for subtraction, and we already know how neurons do addition. How about the other operators? Can we find evidence of multiplication and division in neural systems? I suppose if multiplication was discovered, we might hypothesize the existence of an "reciprocal echo" just like the negative echo, and this would satisfy the definition of division (since dividing by x is the same as multiplying by 1/x (or x^-1)).

More generally: the negative of a number is its "additive inverse," the reciprocal of a number is its "multiplicative inverse." A new question then arises: can Bickle's "negative echo" be re-interpreted more generally as just the inverse of the original stimulus? That would take care of division... assuming multiplication is already taken care of. I hope the use of the word "inverse" in both cases by mathematicians is more deeply seated than just an arbitrary convention.

How then might neurons implement multiplication? Maybe through a cascade effect: if an activated neuron causes many other neurons to become activated, that seems pretty similar to multiplication.

Another question would be whether neurons follow the same matrix multiplication rules as our artificial math. That is to say: in order to multiply two vectors in matrix algebra, their inner dimensions must be the same. Thus, a 1x2 row vector must be transposed into a 2x1 column vector if you want to multiply it with another 1x2 row vector. (See this wikipedia article if this summary is insufficient. ) Is the concept of transposition relevant for neurons? What does it mean to transpose a vector when the vector's dimensions represent the degree to which an object possesses a property?

Ahhh, and what place does the inverse of a matrix have in this vector-space theory of cognition? An inverse matrix is one such that when it is multiplied by the original matrix, you get the identity matrix (where the diagonal is all ones). I asked earlier if "inversion" was the general mechanism that could be abstracted from Bickle's "negative echo" phenomenon; is the same principle in operation here? I note that I slipped from thinking of solely row vectors to matrices in general... is this justifiable in biological neurons?

"Why is any of this interesting," one might ask? Division is interesting because it would allow vector averaging to be implemented in neural networks, which is hypothesized as the mechanism for integrating the output of many smaller networks into a single output, such as an action. Multiplication and division together, I think, are also requirements for more sophisticated learning methods, like Bayesian updating. If this is true, I could draw a direct line of reasoning between Bickle and Hawkins. Mix that in with lessons from Hofstadter and Baum... maybe there's something interesting and new.

In re-reading this post, I realize that I'm focusing solely on neurons doing calculations, and I haven't thought much about their dual-role as memory storage mechanisms... the two roles are deeply intertwined, so I need to make sure I don't artificially separate them.
Some additional comments on Matt's response to my original cash for clunkers complaints:

From Matt:
"This issue, while interesting, is, as you say, a drop in the bucket compared to the changes that would occur through retooling auto factories, research and development, and the private and public reorganization of health care that is imminent."

Matt,

I think the cash for clunkers issue is interesting as a small test case of central government's ability to effect changes it deems desirable. This is especially important considering the imminent public health reorganization, as nationalizing health care will be an incredibly sensitive, complex, and delicate undertaking. What degree of confidence should we have that it will be well-executed, efficient, and non-wasteful? If the cash for clunkers "drop" is a sample of what's in the rest of that "bucket," we can expect "poor results."

To reiterate from my response comment: I'm reacting to this issue because it is small enough and blatant enough to be understandable, whereas the other bailouts and wars are so huge as to be incomprehensible in scope. This is small enough for everyone to understand.

Tuesday, August 4, 2009

Incredible, outrageous, and obscene

I was mildly upset when I heard about the government's plan to buy old cars from people for more than their fair market value, as I mentioned a few posts ago. The question of what they do with them when they're done never occurred to me, and apparently it didn't occur to them until recently either:

http://online.wsj.com/article/SB124934376942503053.html#mod=whats_news_free?mod=igoogle_wsj_gadgv1

They're taking working, useful vehicles and destroying them. Cars that are less than a decade old! That should attract the attention of any person who's concerned with resource conservation. Does it really make sense that we're destroying vehicles that get 18mpg to replace them with freshly built vehicles that get slightly more? Remember that there's an environmental impact for building new vehicles, as well as junking the old ones.

Its interesting to note that in this case, environmentalists agendas seem to have trumped redistributionist ones; it seems like there's a pretty compelling case for giving the cars the government bought to low income folks who could use cheap transportation. And, for a review of the evidence that justifies the environmental motivation, see this article.

Monday, August 3, 2009

Discussion with Bickle

So Dr. Bickle had some positive responses to the questions I emailed him. He suggested that I read Paul Churchland to get some hints at answers on the "dimensionality of language" questions, and I just ordered the book. He was skeptical of my suggestion that the subtraction paradigm could be used to find new conceptual points, and he suggested that I try to work it out. This is consistent with what he said in the 2000 paper, so it doesn't come much of a surprise. I'm going to dive into Matlab to try to work it out.

Ali Minai also responded to my questions, saying he'd be happy to meet with me, and that my interests are in line with what he works on.

So there's an outside chance that I'll be able to stay in Cincinnati and work on a PhD. I don't have any qualms moving to another place, but I'm really coming to like Cincinnati and the community that is developing here.