Monday, November 16, 2009

Cognitive Science Applications Statement of Purpose

Cognitive Science Application: Statement of Purpose TJ Murphy

My motivations for wanting to study cognitive science can be summarized in three points. First, I want to understand the fundamental mechanisms of cognition and creativity in humans, and how these mechanisms might be generalized to other systems. Second, I would like to work in applying the findings of the first point to technologies and theoretical tools that can help humanity cope with the growing complexity of the environment we are creating. Third, I believe that success in the second point is vital for long-term human survival and prosperity, as the potential for catastrophic disaster increases with each new and more powerful technology we create. I believe that these are surmountable challenges, and I would like to devote my life's work to helping overcome them. In this letter I will briefly discuss what I have done to prepare myself for this course of action, then relate my present vision for what form that course might take.

My formal undergraduate background is in Finance, where I studied how markets represent and process information about the environment and distribute resources efficiently (to the degree in which they do). Informally, I pursued learning in philosophy and ethics. The synthesis of these disciplines led me to develop research interests in the emergence of learning and adaptive behavior in complex agent-based systems.

After finishing my undergraduate degree, I worked for two years at Toyota's North American headquarters for engineering and administration. I developed an interest in pattern recognition and machine learning as a result of a long-term forecasting model I created, which as since then been used in strategic planing for North American logistics. During my time at Toyota I led the development of an enterprise information system, through which I gained experience in collaborating in cross-disciplinary teams, acting as a project manager, and guiding software development. After the completion of this project, Toyota provided me with the opportunity to return to full-time study. Thus, I resigned from my job and committed full-time to my Masters of Science in Quantitative Analysis (MSQA) program, which I had been pursuing part-time while I worked.

I chose the MSQA program at UC because of its emphasis on understanding and controlling dynamic systems using mathematical models and programming tools such as GAMS, MATLAB, R, Arena, and scripting languages. The program provides a solid foundation in statistical methods and analysis, regression, optimization modeling and analysis, probability, simulation, and data mining. Because my ultimate goal lies in studying cognitive science, I have been supplementing my MSQA studies with other subjects that I think will be germane to to the subject; namely neuroscience, computer science, mathematics, and cognitive science itself.

Three paradigms have influenced my thinking on cognition. They are:

  1. The memory/prediction framework for modeling the function of biological neural networks using Bayesian networks and matrix transformations (from Paul Churchland and Jeff Hawkins),

  2. the parallel terraced scan approach to finding solutions to abstract problems which have a high degree of ambiguity using simple agents as the mechanisms of exploration (from Douglas Hofstadter and Melanie Mitchell),

  3. and the idea of using market mechanisms to allocate resources (such as control of sub-agents or tools to manipulate the environment) to sub-cognitive agents which compete for activation (from Eric Baum and Marvin Minsky).

I believe that these three paradigms share a common kernel. It is my hope to help unify them into a coherent theory of cognition, creativity, concept-formation, and decision making. Moreover, I hope to be able to generalize the result to be applicable to systems outside of the case of biological brains, to include both human-designed artificial intelligence and naturally occurring supra-human phenomena like institutions and markets.

My current thoughts on the subject are as follows. We can observe abstract structural similarities between biological neural networks and economic networks, and I suggest that this similarity in structure implies a similarity in function. My hypothesis is that human markets can be modeled as complex systems of interacting agents (where institutions and individuals can be considered agents), and that individual human minds can be modeled in a similar way (where sub-cognitive brain processes can be considered agents). I believe that exploring this isomorphism can be aided by using the theoretical and computational tools from financial engineering and integrating them with neuroscience. If there is indeed such a deep symmetry between minds and markets, the lessons learned from the agent-based approach to mind could be used to facilitate the operation of healthy and well-functioning markets, as well as to provide new paths of exploration into the mechanisms of analogy, creativity, and general intelligence in biological brains.

More broadly, I am interested in studying the representation of concepts in vector space. Specifically, I would like to explore how concepts are created through abstraction, how concepts map to language, and how concepts are manipulated to perform acts of creativity. I think that studying higher math will be instrumental to this pursuit, including advanced linear algebra, tensor calculus, statistical learning, belief propagation, and Bayesian inference. I would also like to develop my programming skills to a more masterful level so that I can make effective use of this knowledge.

Another field that I would like to become involved with is the study of neural plasticity and long-term-potentiation. I studied this topic with Dr. Bickle at UC, and I think that modeling these mechanisms will be very useful in creating time-continuous Bayesian-inference agents for interacting with the environment. I would welcome an opportunity to do work with neuroscientists, both in the lab and in data-interpretation and modeling.

In terms of my academic career, I am interested both in research and teaching. I take pleasure in exploring analogies between concepts, and I am driven by a sense of scientific integrity to test them for validity. I also love sharing ideas and knowledge, and I think that teaching would help me develop new ways of thinking about old ideas. Moreover, I'm eager to put myself in an environment of diverse intellectuals who can help me find flaws and uncover hidden gems in my ways of thinking.

Lastly, I will return to the topic of why I believe advancement in cognitive science is important. As humanity develops increasingly complex technologies and cultural institutions, the task of understanding them and their interactions will become correspondingly complex and further out of the reach of any given individual or organization. As complexity grows and human understanding shrinks proportionally, the risk of unforeseen catastrophe (eg: our recent financial crisis) will loom larger. If such threats are to be avoided, understanding must be built into the systems themselves. I would like to explore what this means and how it can be accomplished, both for my personal satisfaction and because I believe in its grand, over-arching importance.

We're incredible math

Imagine doing a few million matrix transformations, where each matrix has several thousand dimensions.

You just did exactly that, whether or not you understood what I meant. Neurons form networks, and the places where they connect to each other store information by changing their magnitude, like the elements of a matrix. Neurons themselves sum their inputs, and taken together the output of groups of neurons are matrix transformations of their inputs. So whether or not you can do formal math; you're doing nothing but math whenever your brain is functioning.

The line from the Ghost in the Shell closing song "she's incredible math" (speaking of the main character, to which the characterization more obviously applies) brought the thought to mind. And that was brought to mind by the conversation I just had with my thesis adviser (by the way, I have a thesis topic and an adviser now) where I shared with her my ideas on the mind-market conceptual connections.

Professor Yu has agreed to be my thesis adviser, and she's given me some good direction already. My project is on bankruptcy prediction, and its an offshoot of the class I had with her this summer. In short: my project uses the momentum of measures of firm's health to contextualize their status at a given point in time to help predict whether or not they are at risk for failure.

Saturday, November 14, 2009

So People are Talking About Rand again, Eh?

The surge in popular media mentions of Ayn Rand has got me reflecting on my history with her ideas. Now that I'm asking to be taken seriously by the academic establishment, does making my positions on a topic unpopular in that field serve my interests? The positions exist, and it would be a breach of integrity to pretend that they don't just because they're likely to be poorly received.

"Be responsible for your self; don't tolerate or participate in the abuse of others, and don't tolerate or participate in the abuse of yourself." In paraphrased form, that is the kernel of Rand's inspiration. It's a clear and timeless message that didn't originate with her, but found powerful expression in her writing. It's the idea that defines what it means to be "Liberal" in the original sense, and it would be difficult to state a contrary position that doesn't rely on racism, sexism, imperialism, or some other form of exploitation-ism.

That nugget has been the important part for me, and it has been the part that I've focused on. I would venture to guess that the negative impression that many budding intellectuals have of Rand stems from her not "playing nice," from some Hansonian status-motivation that causes people to be resistant to affiliating with someone who has caused herself to be disliked by a broad class of others, regardless of what she actually has to say.

With that said, some of what Rand has to say (especially about other thinkers and writers) is misguided and ignorant. She derides Hayek and Friedman, who are two of the most powerful voices in support of the same ideas she cherished. But who cares? You don't have to take it all as a "packaged deal." Agreeing with a person on one thing does not commit you to the loyal support of everything else they ever say. Rand was a flawed human being? Ha. Show me an unflawed one. I might even suggest that she deserves some credit for being authentic, for not hiding the aspects of herself that others are bound to dislike just as a matter of social convenience.

Also, I think people are hung-up on her gender. It seems "off" when a woman is as brash and bold as she was, but pretty standard when men exhibit the same characteristics. The fact that Rand isn't celebrated by the feminist movement is, I suspect, solely the product of her distaste for the ideas that get associated with modern feminism.

Other people's views aside: Rand inspired me with in her love of human accomplishment, her conviction, her pursuit of rationality, and her passionate belief that she could positively affect the course of human history. Those things I will take with me; the rest was ephemeral.

Sunday, November 8, 2009

Another Figurine

I had started on a male torso last night also. I finished it today, behold:

I left in the oven too long and got a burned stripe on the front. The color was quite nice, and I realized that cooking it longer would be an effective way to give it dark black skin. So: I turned up the heat and cooked each side. I love the result, and its a nice contrast to the pale-skinned female. I don't know that any particular ethnicity has that mix of maroon and dark black in their melatonin, but it looks good and matches the wood on my board.

With these black and white miniatures made, it seemed obvious to put them on the chessboard. They are just the right size, and they make a perfect king and queen. I think I might make a complete set in the same spirit. I'll just need to figure out what makes good analogs to the other pieces while keeping them distinct enough.

Here's an idea: go with the reproductive motif that the nudes hint at. Might be a little lewd by conventional standards, but hey, why not? Pieces might be designed as follows:

Bishop: Phallus
Knight: Uterus
Rook: Egg
Pawn: Sperm

Saturday, November 7, 2009

Google Books

I should be able to read the full text of books I already own in hard-copy on Google Books. Barring that, I should at least be able to search the full text and have the page number where my search string appears displayed. Currently searches only return three random occurrences of the search string.

Sunday, November 1, 2009

The importance of failure

In contemplating the connections between computation in neural networks and in economic networks, the question of failure arises.

"Market efficiency" does not mean that prices are always correct, or that every agent is always acting with perfect rationality. It does mean that all information available to the system is incorporated into prices, through the action of agents behaving rationally with their given small piece of information about the environment.

It may not always be clear whether an agent is acting rationally. Their information may come from unreliable sources, while their processing is rational, and this will result in failure. Likewise, they may have good information and bad processing, which will also result in failure. Therefore, failure should propagate backwards through a network until it reaches an agent in the latter condition.

Ideally, this agent would be corrected or removed, and all the other agents that follow it should be able to resume functioning with minimal penalty (unless part of their decision making involves where they get their information from; in which case they should be penalized for choosing a poor source of information). If the failure propagates back to an entity that is not allowed to fail, what happens? The information that the failure would contributes to the economy is lost.

The point is: failure is an important tool for developing economic efficiency. I mention this, of course, because of the current discussion over economic agents that are "too big to fail," and the idea of creating agents that cannot legally be allowed to fail (like social security and the impending health care program). Because they cannot be allowed to fail, they can't experiment with novel ways of incorporating information, which means they cannot develop and contribute to economic efficiency. And since the world is dynamic, their world-model and processing is likely to be wrong eventually (no matter how conservative it is), and so we get stuck with inefficient, broken economic agents that we can't get rid of.

And back to the original point: how do natural neural networks deal with failing elements?