As promised when I initially wrote about artificial intelligence (and specifically about Eureqa), I have been playing around with economic data and have come up with some interesting results. After running dozens of experiments using different combinations of data and allowing Eureqa to use different computational operations, I have come to a fascinating conclusion: in terms of what helps the AI fit equity performance data points to a model based on all data available to it, it seems equities don't 'care' about anything but how we, the consumers, think about the economy, regardless of how well- or ill-informed we are about the economy. Give the AI GDP data, CPI, unemployment, CD rates (to provide opportunity cost), even prior year stock performance and price-to-earnings data: if you also provide consumer confidence data, it will systematically discard every other data type save that one. In some simulations with solutions that have similar complexity and fit, it might also use Fed Funds rate and/or CD rates, which I see as opportunity-cost stand-ins. But for the most part, it just wants to know, "How do you feel about the economy, regardless of how well or poorly it is actually performing and regardless of how much you even really understand it?"
As I said in the prior post, I am far from convinced that such data-mining can offer realistic predictive models. But ironically, the fact that the AI prefers the least rigid, least 'rational' data type, in fact makes me less skeptical about its predictive power. My reasoning is this: if the AI chose models that were based strictly on 'hard' data such as CPI, etc., I would suspect it was simply data-mining and that the results would be useless outside the confines of the already-given universe of data points, since equities markets are inherently irrational and are driven by things far less quantifiable than, say, GDP; but the very fact that it chooses the least 'rational' data type, consumer confidence, tells me that it may indeed be coming up with a decent predictive model, since it seems fitting that it has chosen the one data type that combines rigid metrics but applies them to decidedly 'soft' data (that is, consumer sentiment).
So the proof will be in the pudding, I guess. But that pudding will take quite a while to cook, so don't hold your breath. Meanwhile, for what it's worth, I will shortly add some of the predicted values from various experiments. Then we'll sit back and see what happens!
My next project is GDP. I want to see what data types the AI most prefers for predicting the performance of the US economy.
Showing posts with label Eureqa. Show all posts
Showing posts with label Eureqa. Show all posts
27 January 2011
09 January 2011
Fun with Artificial Intelligence
I have been having a ball for the past week playing with Eureqa, an AI I mentioned in an earlier post. Eureqa is the brainchild of Profs. Schmidt and Lipson at Cornell. When it comes to data, my main fascination has always been economic data, so I have been toying with trying to find models that best account for why equities markets move in one direction or another. As I said in that previous post, there are far too many irrational factors involved in the movement of equities markets to come up with the S&P 500's answer to E=MC². Even if you managed to come up with a more or less reasonable predictive model, the theoretical economic characteristic of so-called 'perfect knowledge' eventually becomes not-so-theoretical economic reality; people begin operating under this new spotlight; next thing you know, the model is dead precisely because everyone knows about it and therefore behaves in ways not predictable by the model (since this new knowledge and resulting behavior are themselves major new variables).
Quite aside from the fact that eventual knowledge of a good model would itself make the model obsolete, is the fact that there is a HUGE difference between an equation that explains data and an equation that reveals cause and effect for data. Just ask all the people who have wasted good time and money 'data-mining' the history of equities markets. A perfect example is O'Shaughnessy's 'What Works on Wall Street'. The author dug through decades of data on the stock market and came up with elaborate models showing what would have been extremely effective ways of making money....assuming one had the knowledge of the entire period, but had gained that knowledge at the beginning of the period studied. It's amazing to me that an internet search of this man still pulls up almost universally positive, glowing articles and interviews, despite the fact that the mutual funds that he opened in the 1990s, funds entirely built on his 'research', were abject failures. He managed to spin this somehow, get out of mutual funds, and open a private wealth management company. This allowed him to continue making money and claiming he was right all along, but in fact freeing him to use completely unrelated methods of investing (since he isn't required to divulge his techniques). So he is undoubtedly a gifted marketer, and obviously even a good money manager...as long as he isn't following his own advice.
If this is all still (quite understandably) rather abstruse, I'll illustrate with a metaphor. Imagine you stand outside on the street corner and observe the weather and the passing of cars and people. Three out of ten days it rains. On those days, you notice people wearing raincoats. You also notice there are no open convertibles. Data-mining your way to 'good' equations tells you that an absence of convertibles and a presence of raincoats cause it to rain. That's an example of mixing up cause and effect. In another example, imagine that you observe that on the days it rained, you observed ten percent more people named Jane. Aha! An excess of Janes is causing rain! No, this is just coincidence.* I could go on, but for more examples of these kinds of problematic reasoning, there's a far better resource: read Crimes Against Logic by Jamie Whyte. This book is one of my top fifteen all-time favorites, not so much because it taught me anything I didn't already know (though it did that, too), but because he so eloquently and clearly expressed ideas I had known well but that I had been unable to articulate.
However, an inability to find well-performing predictive models for equities markets, doesn't mean that feeding such data in Eureqa is itself useless. By watching how Eureqa treats all the different variables, you start to see how they interact and which ones haven't even a correlative relationship with equities market performance. For example, Eureqa rarely seems to 'care' much for inflation. There appears to be very little correlation. BUT, it does 'care' quite a lot about the Fed Funds rate, which is essentially the public policy reaction to inflation. It also 'likes' CD rates, which might be a decent stand-in for opportunity costs, though that implies some cause-and-effect (CD rates are low -> opportunity cost of foregoing them in favor of equities is low -> I will buy equities -> everyone does same -> equity prices rise)**, which requires a heavier burden of proof, one that I am far from meeting. And employment? Almost always tosses that out as irrelevant very quickly. But it 'loves' consumer confidence, which suggests that while the markets don't 'care'*** about how many people are out of work, they care very much about how confident people feel in the economy (which is presumably in turn driven by how many of them have jobs, though not directly). But again, there is no straight cause-and-effect here. You can't say Consumer Confidence = y ergo stock performance will = z as an exact function of y.
Early days yet, but so far, so fun! After I get bored with this round of experiments, I think I'll move on to GDP.
-----------------------------------------------------------------------------------
Footnotes:
*Don't even get me started on people who say 'I don't believe in coincidences.' Do you have ANY idea what kind of universe we would live in WITHOUT A HUGE LOT OF COINCIDENCES? Randomness permeates the very fabric of existence. The 'problem' is that our human brains have evolved with this incessant need to find patterns. I say this facetiously because of course that very same 'problem' is doubtless one of the very core elements of our intelligence, not to mention a key explanation to our very survival as a species. But it does have the unfortunate side-effect of making us see Jesus in breakfast food far too often.
**This introduces an intriguing interplay itself. Perhaps the opportunity cost (in form of CD rates or T-bills) must reach a certain threshold before prompting consideration of equities, but that consideration is in turn colored by the confidence one has in the markets and the overall economy (as measured by U of M consumer confidence index?), and that interplay in turn drives the degree to which investors commit to equities, thus determining the demand for (and therefore value of) those equities. Add in a dash of price-to-earnings data (i.e., the 'real' cost of 'buying' the earnings behind an equity) and you might just have some soup worth tasting.
***Please forgive the anthropomorphic words here. And if you are a lefty like me, do not fall into the temptation of attributing 'feelings' to markets. That a market does not move in reaction to a tragedy like high unemployment, does NOT mean that the people who make up those markets do not care about unemployed people. The phenomenon is merely an observed outcome of the aggregate behavior of the people acting in the market, not the 'evil' intent of any group of people within the market. I enjoy demonizing Wall St fat-cats as much as the next liberal, but do so for their individual behaviors, not those of the markets in which they act.
Quite aside from the fact that eventual knowledge of a good model would itself make the model obsolete, is the fact that there is a HUGE difference between an equation that explains data and an equation that reveals cause and effect for data. Just ask all the people who have wasted good time and money 'data-mining' the history of equities markets. A perfect example is O'Shaughnessy's 'What Works on Wall Street'. The author dug through decades of data on the stock market and came up with elaborate models showing what would have been extremely effective ways of making money....assuming one had the knowledge of the entire period, but had gained that knowledge at the beginning of the period studied. It's amazing to me that an internet search of this man still pulls up almost universally positive, glowing articles and interviews, despite the fact that the mutual funds that he opened in the 1990s, funds entirely built on his 'research', were abject failures. He managed to spin this somehow, get out of mutual funds, and open a private wealth management company. This allowed him to continue making money and claiming he was right all along, but in fact freeing him to use completely unrelated methods of investing (since he isn't required to divulge his techniques). So he is undoubtedly a gifted marketer, and obviously even a good money manager...as long as he isn't following his own advice.
If this is all still (quite understandably) rather abstruse, I'll illustrate with a metaphor. Imagine you stand outside on the street corner and observe the weather and the passing of cars and people. Three out of ten days it rains. On those days, you notice people wearing raincoats. You also notice there are no open convertibles. Data-mining your way to 'good' equations tells you that an absence of convertibles and a presence of raincoats cause it to rain. That's an example of mixing up cause and effect. In another example, imagine that you observe that on the days it rained, you observed ten percent more people named Jane. Aha! An excess of Janes is causing rain! No, this is just coincidence.* I could go on, but for more examples of these kinds of problematic reasoning, there's a far better resource: read Crimes Against Logic by Jamie Whyte. This book is one of my top fifteen all-time favorites, not so much because it taught me anything I didn't already know (though it did that, too), but because he so eloquently and clearly expressed ideas I had known well but that I had been unable to articulate.
However, an inability to find well-performing predictive models for equities markets, doesn't mean that feeding such data in Eureqa is itself useless. By watching how Eureqa treats all the different variables, you start to see how they interact and which ones haven't even a correlative relationship with equities market performance. For example, Eureqa rarely seems to 'care' much for inflation. There appears to be very little correlation. BUT, it does 'care' quite a lot about the Fed Funds rate, which is essentially the public policy reaction to inflation. It also 'likes' CD rates, which might be a decent stand-in for opportunity costs, though that implies some cause-and-effect (CD rates are low -> opportunity cost of foregoing them in favor of equities is low -> I will buy equities -> everyone does same -> equity prices rise)**, which requires a heavier burden of proof, one that I am far from meeting. And employment? Almost always tosses that out as irrelevant very quickly. But it 'loves' consumer confidence, which suggests that while the markets don't 'care'*** about how many people are out of work, they care very much about how confident people feel in the economy (which is presumably in turn driven by how many of them have jobs, though not directly). But again, there is no straight cause-and-effect here. You can't say Consumer Confidence = y ergo stock performance will = z as an exact function of y.
Early days yet, but so far, so fun! After I get bored with this round of experiments, I think I'll move on to GDP.
-----------------------------------------------------------------------------------
Footnotes:
*Don't even get me started on people who say 'I don't believe in coincidences.' Do you have ANY idea what kind of universe we would live in WITHOUT A HUGE LOT OF COINCIDENCES? Randomness permeates the very fabric of existence. The 'problem' is that our human brains have evolved with this incessant need to find patterns. I say this facetiously because of course that very same 'problem' is doubtless one of the very core elements of our intelligence, not to mention a key explanation to our very survival as a species. But it does have the unfortunate side-effect of making us see Jesus in breakfast food far too often.
**This introduces an intriguing interplay itself. Perhaps the opportunity cost (in form of CD rates or T-bills) must reach a certain threshold before prompting consideration of equities, but that consideration is in turn colored by the confidence one has in the markets and the overall economy (as measured by U of M consumer confidence index?), and that interplay in turn drives the degree to which investors commit to equities, thus determining the demand for (and therefore value of) those equities. Add in a dash of price-to-earnings data (i.e., the 'real' cost of 'buying' the earnings behind an equity) and you might just have some soup worth tasting.
***Please forgive the anthropomorphic words here. And if you are a lefty like me, do not fall into the temptation of attributing 'feelings' to markets. That a market does not move in reaction to a tragedy like high unemployment, does NOT mean that the people who make up those markets do not care about unemployed people. The phenomenon is merely an observed outcome of the aggregate behavior of the people acting in the market, not the 'evil' intent of any group of people within the market. I enjoy demonizing Wall St fat-cats as much as the next liberal, but do so for their individual behaviors, not those of the markets in which they act.
03 January 2011
Yet another step towards the singularity...
A few years back, I remember reading an article about Vernor Vinge and his case for the technological singularity. I found it fascinating, but soon it crept to the back of my mind. I didn't see much evidence that it was approaching anytime soon. But recently, some breakthroughs in science and a wonderful new book by Mark Stevenson, have made me revisit the idea. In its simplest expression, it basically just says that at some point in the not-too-distant future, accelerating returns will result in a watershed moment, after which humankind will be so changed that all of our current assumptions about even the most fundamental concepts will be swept away, leaving us in a world so completely different from the one we had come to know as a species, that it will be essentially impossible to predict from this side of the singularity. So when will this point be reached? It is not too surprising that no two people agree and even less so that many people think the whole concept is bunk. But people like Ray Kurzweil seem to feel that many people alive today will live to see it. I am not sufficiently convinced even of the validity of the idea just yet, never mind having an opinion about timing. But after reading books like Mr. Stevenson's and seeing some of the mind-blowing advances happening in so many fields, it is certainly something that won't be creeping off again to the back of my mind any time soon.
To cite just one (albeit very powerful) example, consider Eureqa, a program developed by Professors M. Schmidt and H. Lipson at Cornell. Mr. Stevenson visited the team at Cornell and discusses it in his book. Basically, this AI 'program' - calling it a program seems akin to calling Mt Everest a 'mound of dirt' - takes your data and derives principles on which such data are built. That may sound rather dry and dull, but consider this: it figured out Newton's Laws of Motion based on data it was fed. In a few hours. So imagine brilliant careers in research distilled down to a few hours. Then consider that of course such minds won't retire after they feed an AI like Eureqa some great data and get some cool new fundamental laws. They will keep going. So imagine Newton figuring out his laws in a day and then going for another 40 years or so. (Well, OK, he DID keep going for another 40 years or so, but you get the point.) Starting to see how accelerating returns might be leading us somewhere unrecognizable?
If you're one of the select few scientists with access, you can feed...oh, wait, what?! Eureqa is a free download. Anyone in the world can use it. I have.* It's laid out like Excel. I'm no scientist and I doubt I will come up with any Earth-shattering theorems with Eureqa. But imagine this tool in the hands of thousands of brilliant researchers around the world.
Hold on to your hats, folks. It's gonna get wild.
................................................................................................
Footnotes
*I love playing with economic data, so I fed it 30 years worth of such data to see what equations it would come up with. This was just to amuse myself, mind you: one must be careful to distinguish between deriving laws from data gotten from the natural world v just plain data-mining. My exercise was essentially the latter. In other words, any equation derived from something as erratic as economic data will serve just one purpose: more or less accurately predicting a data point within the universe of data already provided to the AI. So if I got an equation for predicting, say, the value of the S&P 500 based on CPI and consumer confidence, all I could be certain of would be that the equation could more or less accurately 'predict' the values for 1987, using the other 29 points of data within its universe. Still, it may serve to give a general sense of trends.
To cite just one (albeit very powerful) example, consider Eureqa, a program developed by Professors M. Schmidt and H. Lipson at Cornell. Mr. Stevenson visited the team at Cornell and discusses it in his book. Basically, this AI 'program' - calling it a program seems akin to calling Mt Everest a 'mound of dirt' - takes your data and derives principles on which such data are built. That may sound rather dry and dull, but consider this: it figured out Newton's Laws of Motion based on data it was fed. In a few hours. So imagine brilliant careers in research distilled down to a few hours. Then consider that of course such minds won't retire after they feed an AI like Eureqa some great data and get some cool new fundamental laws. They will keep going. So imagine Newton figuring out his laws in a day and then going for another 40 years or so. (Well, OK, he DID keep going for another 40 years or so, but you get the point.) Starting to see how accelerating returns might be leading us somewhere unrecognizable?
If you're one of the select few scientists with access, you can feed...oh, wait, what?! Eureqa is a free download. Anyone in the world can use it. I have.* It's laid out like Excel. I'm no scientist and I doubt I will come up with any Earth-shattering theorems with Eureqa. But imagine this tool in the hands of thousands of brilliant researchers around the world.
Hold on to your hats, folks. It's gonna get wild.
................................................................................................
Footnotes
*I love playing with economic data, so I fed it 30 years worth of such data to see what equations it would come up with. This was just to amuse myself, mind you: one must be careful to distinguish between deriving laws from data gotten from the natural world v just plain data-mining. My exercise was essentially the latter. In other words, any equation derived from something as erratic as economic data will serve just one purpose: more or less accurately predicting a data point within the universe of data already provided to the AI. So if I got an equation for predicting, say, the value of the S&P 500 based on CPI and consumer confidence, all I could be certain of would be that the equation could more or less accurately 'predict' the values for 1987, using the other 29 points of data within its universe. Still, it may serve to give a general sense of trends.
Subscribe to:
Posts (Atom)