Episode 290 - Generational Turnings versus Technological Change
Max talks about how Strauss-Howe generational theory and the turnings interact with technology as he reviews the book "The First Turning: A Vision of America and a World at Peace" by Carol Engler.
Distribution of the Week: The Wishart Distribution
Links
The First Turning: A Vision of America and the World at Peace by Carol Engler
Related Episodes
Episode 75 - The Market Loves You: Jeffrey Tucker on the nature of Innovation
Episode 12 - Virtual Friends and Enemies, Building Products that look out for us with Marissa Chacko
Episode 174 - Jeremy Kauffman, CEO of Odysee and LBRY for Decentralized Video
Episode 153 - Decentralizing Before our Eyes
Episode 250 - The Bird Flips
Transcript
Max Sklar: You are listening to the Local Maximum episode 290.
Narration: Time to expand your perspective. Welcome to the Local Maximum. Now here's your host, Max Sklar.
Max: Welcome, everyone. Welcome. You have reached another Local Maximum. Feeling good this Tuesday night here in August where it's still light out. So very nice. All right, what are we going to talk about today? Remember just how optimistic we used to be about the future of technology? That era certainly predates this podcast — probably was around 2010 to 2013.
It's amazing how pessimistic we were about the economy in 2008, 2009, but how optimistic we were about technology in the early part of the 2010s. I think a lot of that was because of the mobile revolution. And some of it let's be fair, some of it was because of low interest rates. And we got to fund all these cool companies like Foursquare where I got to work. And some of them were good things, but maybe some of them went over the top by the time I started this podcast.
I would like to go back to some of my early episodes to see what I said exactly. But I know that I was already saying, man, some of these things have not lived up to its promise. I think actually, if I can go onto the archive, localmaxradio.com/archive, in the middle of recording this, if I can go into one of the early ones, I bet it's the one about, you know, what? Virtual friends and enemies. Episode twelve.
That is an often overlooked episode of The Local Maximum, where I spoke to our product manager, Marissa Chaco at Foursquare about how sometimes products are not built with the user in mind. And so actually, I'm going to post to that one because that kind of straddles the techno-optimism/techno-pessimism divide. And also that one needs to get a little more attention because that was episode twelve, folks.
I mean, yes, a lot of people go back and listen to zero one two, but not that many people were listening at the time. So twelve doesn't get the attention it deserves. And if I remember correctly, it was a long time ago, but Marissa was great in that one.
So I think we're too pessimistic today. I think there's a tendency to have a boom and a bust cycle with these things. And I'm going to review an interesting book that I read recently. I read it on the plane over to France, had time for that. And that was written in an earlier time. And sometimes it's helpful to listen to what people were writing at an earlier time. And I've also had this book on my shelf for like, two years, and I finally got around to it. So that's great.
So first, let's review Strauss-Howe Generational Theory. Yeah, yeah, this is one of those episodes. Strauss-Howe Generational Theory. That is the theory about the cycles of history which, when undisturbed, the kind of personalities of the different generations. It doesn't mean everyone your age has the same personalities, but there's a certain type of attitude and a certain type of approach to life.
Maybe you could think of it as a probability distribution over attitudes and over approaches to life. And that probability distribution changes as different generations come and go. And it doesn't just change randomly. I mean, maybe sometimes it changes randomly, but it seems like you have a generation lasting maybe 20 years. You think that maybe 25 years at most. And you think there are four and a half generations around at any given time.
And so the adult generations, they kind of imprint their outlooks on the younger generations, but they don't just make a carbon copy of themselves in the younger generations. I mean, in some ways they do. They teach us and then they put us through schools and whatever.
But the younger generations are going to observe the world. They're going to observe what's going on around them, and they're going to pick and choose what to pick out of that. And they're going to kind of react to that. So the idea is that the generations who are alive affects the generations who are so called youngsters.
And then this, for some reason, causes a four stage cycle where every lifetime about it's not that you can't live more than four generations. Plenty of people have lived five generations. Some people have even lived six generations. But the fact is that the generations making the imprinting are the three generations above. And then they make a kind of a copy, or not a carbon copy, but a similar generation to the one that's just above them.
So why is this? Why are there four cycles? Why is it they kind of make an analogy to the seasons, and I think it's a good analogy, you know, like spring and the seasons are not the four generations, but the four patterns that you get during the different time periods. So the four generations have archetypical names, like, for example, the millennial generation. We're a hero generation. Hell yeah, we're the heroes.
But when the heroes generation are at young adulthood, that means that you're in a crisis. And when the artist generation is in adulthood, there's a first turning, which we're going to talk about today, which is the high, et cetera, et cetera.
So why are there these four seasons of history? And I think, well, look back to why there are four seasons. The season that it is is because of the tilt of the Earth. Forgive me if I'm not being too specific on this. I haven't looked at my astronomy in a long time. And so it depends upon where the Earth is in its cycle around the sun, if you can think of it.
The Earth makes a circle, an oval around the sun. And it's very natural, interestingly enough to divide a circle into four pieces. You might think, why is it natural to divide a circle into four pieces. You could divide a circle into any number of pieces, and it's very even. But the fact is that a circle is a two dimensional object. And if you think of a unit circle, it kind of goes through. If you think of, like, a semicircle above a line and then another semicircle the other half below the line.
So there's like, two halves there. And then if you think there's a vertical line, the y axis, there's two halves there. And so you're either positive and negative on the x axis or positive and negative on the y axis. And so that makes four possibilities. There's positive, positive, positive, negative, negative, negative, negative, positive. So I think two dimensional objects make four. Like, it's reasonable to divide them or natural, I guess, to divide them up into four parts.
And what makes these circles? Why does the Earth, let's say, go in a circle around the sun? Let's call it a circle. It's because the laws of gravity create these differential equations or are described by these differential equations that create circles. Those are the same differential equations that create springs, that create sine waves, so on and so forth. And so I think that something very similar is happening with the generations.
One differential equation might be on the measure of how open society is to new ideas. And then maybe if society is more open to new ideas, that imprints something on the new generation that kind of dampens that. If society is too close to new ideas, that sort of imprints something on the new generation to try to open it again. And so you don't kind of go into a black hole on either side of openness or closedness. You have some sort of oscillating system.
So that's one way to think about Strauss-Howe Generational Theory. I'm not sure exactly. I was kind of thinking about this the other day. I'm not sure exactly how I would set up these differential equations. Some people call it like, oh, are you going to be radical? Are you going to be moderate? That's kind of a 40 year cycle, it seems.
Although, interestingly enough, the seasons are also radical-moderate in like, a 180. You could think of, like, spring and fall as moderate, but winter and summer as radical. But what are the actual differential equations? That's a good question. Is it openness and closedness or is it kind of forward thinking or backward thinking? I'm not quite sure. So I kind of want to look at that more, especially as I got Neil Howe's new book, the Fourth Turning is here.
But I didn't want to bring that big, thick, nice new book that I bought from Barnes and Noble. I didn't want to bring that on the train and ruin it. I mean, even this other book, the First Turning that I got by Carol Engler, that's what we're going to talk in a bit. I brought it on the train. On the train to France. The plane to France. And it got a little bend in here. So I don't like getting bends in my books, so I didn't want to bring that one.
But I was able to kind of brush up on someone who's thinking about this topic and get through it. So Howe came out with his book, The Fourth Turning is here. I want to read through it. Honestly. I want to have him on the show. I haven't reached out yet. I hope he'd be open to it. I know he's going on some podcasts. Maybe he'll go on this little show, The Local Maximum.
But in the meantime, I read this book. The First Turning: A Vision of America and A World At Peace. This is by Dr. Carol Engler, 2013, a professional in the field of education over three decades. And she talked a lot about an educational outlook in there a bunch. And it was written in 2013. It was almost a time capsule to the way people were thinking ten years ago. And I think that the author made one major mistake, which is like the thesis of the book, but she made a major mistake in prognostication.
But otherwise I think it was worth reading. For me, it was a quick read, and I got some really interesting ideas from it. So why don't we get the misprognostication out of the way first? This is what I think happened. The author, Dr. Engler, claims that the first turning is going to come sooner this year. It's kind of like the groundhog where we're going to have a short winter.
So she says the first turning will come sooner this year, not this year. I think it was called the saculum in the book. Not the Turning, but the four turnings together, like the whole cycle. So she claims the first turning is going to come sooner in this cycle because of the acceleration in technology that we're seeing is going to accelerate the timeline. Now, I think it's pretty clear this didn't happen.
Clearly, Neil Howe, without reading his book, I only read the title, The Fourth Turning is Here, and the book came out in 2023. So clearly he didn't think that this happened. But it's a common misconception, I think, about the turnings, and I think I've heard this before, and I think this is something to keep in mind when you talk about technology predictions and social predictions. When I first started the Local Maximum the show, I was making technology predictions. I've always been interested in technology predictions.
We did our first tech retreat in 2015. That was in the winter of 2015. And Aaron came out to, we met up in one of those cabins upstate, and it was like ten below. We were predicting the future of technology, and we're talking about Ray Kurzweil and accelerating returns. And I kept thinking, man, I would love to predict social trends, but I just can't. I just don't have the framework for it. Strauss and Howe gave me that framework to some degree.
So I think this is a misconception of the turnings that people often link them to the rate of change in society, the rate of technological you know, the rate of change in society, particularly technological change. But other changes as well certainly has been increasing. It's been accelerating. And we've seen that with, well, first the mobile revolution, and then we've seen that with, I think, crypto and Web 3. And we've seen that with AI, with generative AI, ChatGPT recently. It's changing so fast.
But if we go back to that 1997 book, Strauss-Howe pointed out that the social moods of the generations aren't really linked to the rate of growth in technology. It's really more a change in the focus of how the technology is used and disseminated in that day. So, for example, in a fourth turning, like today, you might be more likely to see censorship and propaganda. We're getting there for sure. We're getting that for sure. In the third turning, it's kind of everything goes.
And for those of you who remember, the internet. I kind of got the tail end of the internet in the 90s, but I certainly had the internet in the was really anything goes. It was a wild place. People could do whatever they want. You go on Napster before those all got shut down. I think LimeWire was shut down when, 2010, 2011? That's an interesting story, actually.
I remember I had, like, one recruiter because I was in NYU at the time, and he was like, I got this really cool company. It's something you've heard of, and you should go work there. It's like a name that everyone knows, so you should go work there. I was like, all right. Sure. He's like, so would you like to hear it? Because once you hear it, there's something about these recruiters where if they tell you, then they get the credit for it or whatever. Didn't matter in this case. And he's like I was like, okay, go ahead. I'll sign your thing. You tell me.
He's like, it's LimeWire. And I was like, all right. Cool. It was one of those sharing technologies. You remember, like Napster, people in the future listening to this, or people very young listening to this. There used to be these P2P file sharing software products online, and you could just kind of download all sorts of things from all sorts of people. Like, you could download music. You could download movies. You didn't have to pay for any of that. So apparently a bunch of people decided that was wrong.
But anyway, I remember the recruiter was so excited, and he told me about this opportunity as if I would jump up for joy. And I kind of was like, sure, whatever. Because I know that even if the tech is cool. A lot of companies are kind of grinds to work for us. It's like, let's hear more. And then I remember, so I didn't think much of it. And then I remember the same recruiter called me back a few months later, and he was like, hey, it's a good thing I didn't get you this, sorry. LimeWire shut down. And I was like, okay. But anyway, hey, bad for LimeWire.
Something not similar, but similar in the sense of government shutting them down. So it happened to LBRY, or LBRY Inc earlier this year. They're still LBRY, there’s still LBRY Token, there’s still Odysee. But that just goes to show how back in the third turning, this “anything goes” software was more tolerated than today in the fourth turning.
It doesn't really matter how fast the rate of change is in technology, whatever the technological level is at the time, the social moods of the day will direct how that technology is used. So if anything, I think that we're going to see the cycle, the Strauss-Howe cycle elongate not shorten. I think it's going to take longer because the older generations are living longer and longer.
We're supposed to be done with the silent generation. Even our president is kind of I don't know. Is he a boomer? He's half boomer, half silent. He's a silent boomer. That means it'll take more young people to grow up and to change the social mood. So Dr. Engler predicted the first turning will happen in about 2015. In reality, she's writing in 2013. She kind of saw the fourth turning as being 2008 to 2013. I think that the cycle has elongated, and I think I'm pretty sure Strauss and Howe still say that 2008 is when it started.
But I think that 2013, and I think a lot of people, if you think about it, yeah, trust me, a lot of people agree with me. I don't know. I'm curious. I think people would agree with me that 2013 is when we saw these huge cultural shifts. So I think, unfortunately, Dr. Engler predict, hey, this is the end of the fourth turning. I was like, Nah, we're just getting started. Unfortunately, you were just watching the final throes of the third turning.
I mean, that's how I see a lot of Foursquare technology today, unfortunately, that I was working on in 2011, 2012. It was like, people are like, what? You're just going to check in? How could you have an app that's that fun? And it's like, yeah, it's a third turning. We can do whatever we want, but we still very much saw the Internet as a place that was your adult playground. I probably shouldn't say that it was your playground to do whatever you want in, but by 2014, not so much.
So that said, I think that Engler talks. The main focus of the book is how technology will bring about an early first turning. But I don't think we should throw out the book because I think that the book allows us to kind of gives us kind of a framework in thinking about how the technology of our day will be used by the upcoming first turning, perhaps in the 2030s or the 2040s, maybe even the late 2020s. We'll see.
So what is a first turning? That's the next one. How does Howe describe the first turning? I hope he covers it in his new book. I'll read it. So on the pro side, they describe it — the First Turning — as if there's like a new civic order in place. This is like the 1950s or this is kind of the Gilded Age in America in the late 1800s, where Reconstruction is over.
Unfortunately, all of the inequality and Jim Crow that it was coming up, that was kind of left over after the Civil War, that wasn't fixed. That doesn't get fixed in a first turning because you don't fix big issues in the first turning. So that's a downside. But there's a new civic order in place. I think once reconstruction ended in the United States, for example, the threat of Civil War, like, people didn't know in the 1870s whether the Civil War was going to start again. It was very scary.
So I think by the end of Reconstruction, by 1877, it was like, no, we're good. There's not going to be a civil war. So there's a new civic order in place. People aren't constantly trying to undermine each other's points of view. Power isn't constantly trying to undermine power. But at the same time, there's sort of these bounds in place that are in the middle of the road. And so you get kind of a boredom, perhaps.
So they describe it as a time when people start taking more risks again. So that's a good thing, but they also say that the sense of shame is at its highest and society becomes very conservative. So I assume that means that people start taking economic risks again, but they take less social risks.
Seems like people have become a bit numb after the crisis, and they don't want to fight. They just want to live their life, it seems. And sometimes they have to fight anyway, because sometimes first turnings do have wars, but they're kind of echo wars, as they call them. So I think Dr. Engler, when calling for a first turning, she's sort of using it as a device to point to the positive attributes of the first turning. And I think that means more investment and better community.
So it's investment in community and more sort of a civic attitude. So I kind of look at it that way. So back to that book, back to the first turning. There was a lot about social media starting revolutions via the Arab Spring, but notice this is only happening in one part of the world. Social media did not create revolutions here in the west, here in the US. And so it's used in campaigns, but pretty predictably when looking back. This only happened just before this book was published in 2013.
So yes, social media led to social change. But today, in many ways, particularly around 2019, 2020, it became a reactionary force. But there are some bright spots today in social media that can be pointed to as maybe not so influential today, but might be influential ten years down the line or five years down the line. First is the decentralized social networks that we're seeing. One that I'm looking into recently is Jack Dorsey's Blue Sky, because that is technically decentralized.
I know I had Jeremy Kauffman on the program. Now I've got to look up again where I've had Jeremy Kauffman, he was the CEO of LBRY at the time, or Odysee and LBRY, which was doing decentralized video. Again, they ran into some totally unjustified legal troubles with the SEC. But still, that's a piece of tech. And I looked into it. It was a really cool piece of tech, and that was a piece of tech that kind of inspires me.
So there's the idea of these decentralized social networks. There's alternative social networks, too, that kind of had to provide some arena where people who are being censored in the mainstream had to go. And there were a lot of false starts on that. So for example, let me go back to I have an idea, decentralizing before our eyes. Episode 153.
That was the episode that came out just after Parlor was censored. It was a social network that was censored by Amazon shutting down their servers. It seems like that kind of spurred people into action. So that's not going to happen. Next time this happens, I feel like some of these marginalized groups will be prepared or it'll be some larger company like Rumble or something that is involved.
So they're going to have a much better. so we have some of these decentralized social networks. We have these alternative social networks. We have Elon Musk buying Twitter, which is a really big deal because I have so many episodes on that too. Episode, let's go back to it. Episode 250, The Bird Flips. That's a pretty good number. 250. Got to figure out what to do about my 300th episode.
And so he kind of exposed a lot of what the trust and safety of Twitter had done and what the government had done with social media. And that's why people don't like Musk these days. And then I think there's a lot of changes with people working on Web 3.0 and the Metaverse. We'll kind of see how that plays out. That's still kind of up in the air.
But again, it's how technology is used. And I think I see in these technologies, these are not going to save us tomorrow. These are not going to come by overnight. But this is the type of technology that might be used in a first turning where it's not everything goes and there are guardrails, but there's also some relief valve as well where you could have different communities and not go crazy where somebody disagrees with you on the Internet.
Or you could still go crazy if someone disagrees with you on the Internet, but you're not going to be able to shut them all down. So you might be able to shut them out of your life, but you might not be able to shut them all down. And another thing so, okay, so that's one of the things about social media.
So this is the perfect example of what Strauss and Howe are talking about in in 97, where it's not what the technology is. It's not the technology affects society, it's society affects how the technology is used. Another thing she mentions a lot because of her background in education is the massive open online courses. She mentions the machine learning course with Andrew Ng, which I watched.
So it's interesting that the field of machine learning is actually at the forefront of this, where you could go on to these websites. That was at Stanford. I know Yale, my alma mater, had a bunch of open courses at Yale. You also remember there was services like Coursera and Udemy, people that used to talk about this a lot then; we were very, very hopeful about that tech ten years ago.
Today I feel like we realize we've gotten into a funk because these are not talked about enough. I think we realize that with social media we have these casino-like products really winning out over these educational products. Maybe that's inevitable. I hope not, but it really is a shame. But we're still hopefully open to the possibility of having AI that could teach us interactively.
So maybe in the 2030s, maybe we'll have another look at these products as real vehicles for learning and not just online services for vehicles for propaganda, whether it be political propaganda or corporate propaganda. I'm not just talking about propaganda where they're trying to sell you on a way to live or like a political view, but also advertising as propaganda, which I don't like.
Maybe I should do a whole episode on this. I was kind of watching some old ads the other day and how like, oh, they just want you to make associations with their ads to positive things. And that's a good thing. But I don't know. There's a certain way to do it that's kind of nasty. We'll see what- I don't really have very clear thoughts on these advertising, but I kind of put that in the bucket of they're trying to convince me of something that's not necessarily in my interest.
So what does the first turning have in store? If I ever get a chance to talk to Neil Howe, I will certainly ask this. I will certainly look for it in the book when I read it. We have to understand that the first turning isn't necessarily a good thing. You could get good things done in the fourth turning that you get done in the first turning. The fourth turning is not necessarily a bad thing. Both have their ups and downs.
But if their theory is correct and they were pretty good at predicting the fourth turning in the third turning from ‘97. So if they're correct, society should get a bit more orderly. The divisive social movements that you see today, that can just be nuts at times. I mean, maybe depending on what side you are, you think the other side is nuts. But those divisive social movements will go away, but kind of the collectivism will not.
So if I were to pin some hopes on it, I would hope that we would place more value on long term investments as we should now. That includes education, software, energy investments, government, et cetera, science.
If I were concerned, I think that freedom of speech and thought might not be able to be fully restored in the first turning. So that would suck. So if anything, perhaps we can hope for these things to get restored but with guardrails of, like decorum replacing guardrails of content. I don't think it's going to be a free for all like the 90s or whatever, but this all might be too much to ask for. Let's hope that we get some of it.
All right. So I'm also reading, and this also has to do with Donald Hoffman, the Case Against Reality. I was going to say that also has to do with advertising, interestingly enough, The Case Against Reality. It's philosophy, it's physics, a little bit of advertising in that book. It's pretty cool. So I'm looking forward to finishing that and then I'll start Neil Howe's book. And today we have got a special segment.
Narration: And now the probability distribution of the week.
Max: It's probably not surprising to you that it's a probability distribution of the week. That's the only segment we have so far. Maybe I should start making other segments because I feel like we are almost done with the segment of probability distribution of the week. But this is going to be a good one today.
It's a little complicated, it's a little bit of a complicated distribution. So I'm not going to draw out the equation. I'm just going to try to talk about a little bit how I think about it and why it's interesting to me. So this is the Wishart distribution or the inverse Wishart distribution. There are several ways of looking at it.
When you have a normal distribution, like a symmetric curve, then hey, it's the same going up the hill and it's the same going down the hill. But if you have a distribution that's not the same going up the hill as the same going down the hill, then you have an inverse version of that.
The inverse Wizhart distribution or the regular wizard distribution, that is a distribution on vectors. Well, what is that a distribution on? It's interesting, it's hard to describe and so maybe I'll get to it in a little bit, but it's a distribution over a matrix, which is crazy enough. It's a distribution over a positive definite matrix.
Well, what does that mean? I'm not going to define that for you, but I'm going to try to show you what direction this is going in. So phase distribution is going to be hard. It'll require you to think multidimensionally, but it has a pretty cool use, which I'll show you. So let's review some of our previous probability distributions.
First, we have the normal probability distribution. So for those of you, hopefully you have some background in probability. If you're listening to podcasts, some of you don't. So you might have just come for the fourth turning stuff.
But we have a normal distribution, which is like the normal one you get. It's like kind of a bell curve. There's that middle where everyone is, and then there's the outliers to the right and the outliers to the left. And for those of you who do even very basic statistics, it has a mean, it has an average, a center, and it has a standard deviation, which kind of measures how far away the data is from the mean.
And you could estimate this mean and standard deviation from the data. So the mean is just you just take the average of that data and for the standard deviation, there's kind of a formula for it, something like squared distances, but that's the observed mean and the standard deviation of your of your data set. It doesn't tell you how well you know those values.
So for example, if I know I have a normally distributed data and I get 20 pieces of information, and I take the normal that's my observed mean, that might not be the mean that I'm pulling from, probably is not. So there's difference between observed and actual. So we don't know those values from the data, even if that distribution is normal. And for that you need a mathematical tool and you guessed it, Bayesian Inference. We always talk about this program.
See, you picked some distribution over the mean. You start with no data, but you take a guess as to what the mean is. And that guess comes in the form of probability distribution. Oh, I have some belief about where the mean lies. Could be some very uninformed prior. And then you take some distribution over the standard deviation as well, which is a positive number. So it has to be some kind of distribution that works over positive numbers, like, let's say the gamma distribution.
And then you have those update as you collect more data. And what you get over time is a distribution of all possible normal curves. So let's talk about the standard deviation a little bit. Usually the standard deviation is the one that's used in mathematics. It's denoted by the Greek letter sigma. It's not really the natural value. There's also something called the variance, which is the square of the standard deviation.
Interestingly enough, in like so many of the formulas in mathematics, when you have standard deviation, it's always sigma squared, sigma squared, sigma squared. Well, guess what? You might be talking about the variance. And then it would just be variance, variance, variance. So for some reason, standard deviation is used more often than variance.
And then there's something else called the precision, which is one over the variance, the inverse of the variance. And this might be even more natural than the variance, the precision. If you think about the variance, if you have a very high variance, then that means that data can be very far away from the mean. But if you have a very high precision — precise — that means that the data is very close to the mean. If you have a very low precision, that means the data is all over the place and you don't know where it's going to go.
So, interestingly enough, if you want to get a conjugate prior for the standard deviation, you really should use the precision and you should really get a conjugate prior over the precision. Conjugate prior — for those of you who are uninitiated Bayesians — means that let's just say that it's probability distributions that are very easy to work with mathematically. Let's just say that.
So it's interesting that the conjugate prior over the precision is the gamma distribution, which we've covered before that's distribution on positive numbers. And so usually when you set these things up, you place a distribution over the precision and the distribution is the gamma.
So we have a gamma distribution over the precision and over the mean. We actually put another normal distribution over the mean as what we think the mean is. But this time it is a very wide, very wide variance, very low precision over that one.
There's two precision. There's the precision over the distribution you think you're measuring, and then there's the precision over your belief as to what the mean is. Two very different things, and collect lots of data. And if you collect lots of data, eventually you'll estimate some normal curve, and that won't change very much. But your normal curve representing the belief over the mean of that normal curve becomes very small and tight, almost like a Dirac distribution approaching it.
Okay, if that makes sense. So we have a normal distribution over the mean, a gamma distribution over the precision. That's the same as having an inverse gamma distribution over the variance. With me so far? Maybe not. All right, but does this generalize to higher dimensions? Guess what? It does.
So remember that in a single dimensional gaussian that's normal distribution, you basically have the standard deviation is two points. You have one standard deviation below the mean one standard deviation above the mean and you have like 60 something percent of all the data in the middle. That's that middle chunk.
Well, when you generalize two dimensions, that standard deviation is no longer range. It's like an oval. And then three dimensions, it's some like warped spherical shape around where the data is. And this is instead of one number representing the variance. Now you have something called the variance matrix, the covariance matrix.
Because it's not just like you have data is going near and far to the mean. It could be near and far in different dimensions. It could be like in one dimension the data is really close to the mean. In another dimension the data is really far from the mean. Or it could be like the oval could be turned. So it's like far from the mean along this line, but not along that line, et cetera, et cetera.
So there's a lot more ways to configure it. There's a lot more parameters when you have a multivariable Gaussian. And so two dimensional, you could think of it as some oval, three dimensional, some like, warped spherical shape. And so this is instead of the variance, you get the covariance matrix.
And the variance matrix always has a property. It can't be any old matrix. It has to be symmetric because the covariance between dimension A and dimension B is the same as the covariance between dimension B and dimension A. And it's also positive semidefinite. It's got positive numbers in it. We don't have to get into that. And so of course there's precision as well, which is just the inverse of the variance. So that's also a positive semidefinite matrix.
Turns out there's a really great distribution over positive definite matrices that can be used. In this case, it's over positive definite, but the matrices are semidefinite. Let's not even try to details, details, details.
So anyway, that great distribution is the Wishart distribution. A normal Wishart is the conjugate prior over the mean and precision of a multivariate normal. The normal inverse Wishart is the conjugate over the mean variance of the multivariate normal.
So what does this all mean practically? Let's take a step back. How can we use this? Imagine that we're gathering a lot of data and all of this data has many dimensions and we can compute a Gaussian for it. We can compute like a multidimensional normal distribution. Great. Just like we can normally we have the mean and we have the variance. There's a formula for it and we could kind of figure out what that is observed. Great.
But better yet than looking at the observed distribution we can compute the normal Wishart distribution over all possible Gaussians and then sample from that. And that would be so much better because then we'll know what to expect for future data. It's like, okay, do I really know this data very well or do I not know it so well? And that would answer that question.
The observed data doesn't take into account how few data we've collected. If we've only collected a few points, then you don't really know where the mean and the standard deviation is. And this problem gets much, much worse with higher dimensions. It becomes a larger and larger problem. The curse of dimensionalities. This is much more important in the multidimensional case.
So I would love to use this system. I haven't actually used it in any of my models yet, but I'm sure you could do it in Pi MC-3 and Pi MC-4. I'd love to see examples. Anyone out there use it? I'd like to know the inverse Wishart distribution or the Wishart distribution. Have you used it to estimate a normal distribution? I would like to know.
All right. That's some advanced grad school type stuff. So I'm glad that you stuck around with me for that. Other than that, what are we going to do in the next couple of weeks? I know Aaron is off on a trip. I'm not sure, I don't remember where he went this time. But when he comes back, I'm going to start lobbying him to do another news update because I know we need to do that.
And we need to also go over more and more about this new constitution that I'm writing. I know we did three episodes on it, but I changed a lot since then. It's really interesting to build these things because I feel like you don't really appreciate the constitutions and even corporations until you try to make the rules yourself. So I changed my minds on a few things. So maybe we'll go over that. That was always very interesting to do.
Maybe a news update and certainly all these books I'm reading that I've told you about before, including The Case Against Reality and hopefully the Fourth Turning soon. Let's see if I can get some of the authors on. I don't know. All right, remember to join the locals, Maximum Locals.com, particularly if you really like what we do here. And that's all I have for today. Have a great week, everyone.
That's the show. To support the Local Maximum, sign up for exclusive content and our online community at maximum.locals.com. The Local Maximum is available wherever podcasts are found. If you want to keep up, remember to subscribe on your podcast app. Also check out the website with show notes and additional materials@localmaxradio.com. If you want to contact me, the host, send an email to localmaxradio@gmail.com. Have a great week.
