**Wolf: 00:06 **Hey everybody, it is another episode of Runtime Arguments where we talk about programming and technology and hopefully related stuff enough to reach everybody. I’m Wolf, and I am joined, as always, by my close, close, I would even say best friend Jim. Say hi, Jim. Hello, Wolf. How are you doing? I’m doing good, thank you. And I hope the same for you. I’m doing too much. in today’s episode, Jim’s done some looking around. We’re gonna talk about data centers. But let’s start off with how’s your week? How’s your week, Jim?
**Jim: 00:49 **Oh, I as always I had a busy week, which which is good, very productive. If our listener will look back at our second episode on the transition to the data center, where I moved from a small data center in Southfield, Michigan to th…
**Wolf: 00:06 **Hey everybody, it is another episode of Runtime Arguments where we talk about programming and technology and hopefully related stuff enough to reach everybody. I’m Wolf, and I am joined, as always, by my close, close, I would even say best friend Jim. Say hi, Jim. Hello, Wolf. How are you doing? I’m doing good, thank you. And I hope the same for you. I’m doing too much. in today’s episode, Jim’s done some looking around. We’re gonna talk about data centers. But let’s start off with how’s your week? How’s your week, Jim?
**Jim: 00:49 **Oh, I as always I had a busy week, which which is good, very productive. If our listener will look back at our second episode on the transition to the data center, where I moved from a small data center in Southfield, Michigan to the Azure Cloud. There’s a whole episode about doing that. Anyway, at the time when I did that, I set it up where we use Postgres and I let Azure manage my Postgres database and they manage the replica and it’s all really nice and really easy. Turns out it’s really expensive. Still I saved a lot of money by moving out of the data center, but it costs a lot of money. And even at the time that I did the episode, I mentioned that at some point I was going to take control of the database myself. And there’s a lot of reasons for it. One of them is the cost. The other one is when Azure manages your database with replication, they are completely in charge of that and you can’t touch it. You get to connect to the database and do your tables and queries and all that kind of stuff. But you don’t get to create additional replicas, which I like to do. It’s very handy when I want to take a snapshot of a production database. I always keep a replica of myself running. And when I want to take a snapshot, all I have to do is break that replica and promote that to a new standalone database and then start creating a new replica. Well, you can’t do that when Azure manages your database. So I finally switched it and I had to increase the size of my server on Azure and I run my own instance of Postgres on it. And that was really simple. Just had to suck all the data out of the managed database and load it up into my database and start it up and running, and then I created a replica on another machine somewhere else, and so everything’s happy. And it turns out that the database was about something like 73% of my total cost when they were managing it. So 27% was networking and the virtual machine and all that kind of stuff. I increased the size of my VM to do this. So obviously those costs are gonna go up a little bit, but in the end, I’m saving about 75%. Well, there’s another reason I’m saving some money too, and that is I switched from a pay as you go plan, which is how you get started with whether it’s Amazon or Azure or Google, you just pay every month for whatever services you happen to use. But they have this managed plan thing, or maybe managed isn’t the right word, but if I can commit to 12 months of a certain size VM. And when you commit to 12 months, the discount is about 40 percent. If you commit to 36 months, the discount is 60 percent. So I went three years because this is a pretty stable business, the stuff that we’re doing. And you know, hell, I had a data center account for 12 years, I didn’t mind the three-year contracts there. So anyway, my costs now are about 25% of what they used to be, and I’m very, very happy. And side benefit, the database is about twice as fast as it was. The customer loves it, everybody’s happy. So I don’t know, I’ve already talked too long on this. If you’re interested in the whole Azure data center migration, go back to episode one and check that out. And I’m happy, the customer is happy, everybody’s happy. That’s what you want, right? So yeah, so I was busy. How about you?
Wolf: 05:02 My week, my week is good. I actually do have a couple things. I don’t usually have anything interesting to say, but a couple things. one is my jobby job, our company, which builds maps for high-resolution maps for semi-autonomous cars. We’re at CES. That’s pretty exciting. Oh yeah. I happened to do I got a lot of good important stuff done for work for my jobby job. One of them is that the architecture that I designed and implemented to improve the way we work on certain kinds of things, particularly how we read, write, and manipulate data from the database. Net speed improvement of 36%, and it’s not even completely implemented. That was very satisfying to hear. I liked that. And third, I like any software engineer, scratch my own itch. I have a couple of interesting things that I care about and I put a little bit of time into. One is this starting, I guess, in 2022, I stopped using GNU STO to manage my dot files. GNU STOW is a link farm manager. If you use it, and there’s a ton of good dot file managers. GNU STOW assumes that you probably have a Git repo or something, and then in your home directory or your home directory.config file, wherever, it will put simlinks that point back into your actual real source of truth files. And Git Stow does this really nice thing where if you’ve got a file, and that file is named, for instance, I’m gonna spell it D-O-T-B-A-S-H-R-C. Dot bash rc. If you have a file that is named that, then the sim link, if you say the right parameter flag, option flag, will be named like it’s supposed to be named with an actual period, a dot sign, dot bash rc in the right place. All the good things happen. GNU Stow is great for that, but it’s had this bug forever, and the bug is that renaming facility only works for files, not for directories. So you can’t apply it to dot config, for instance. And it’s got some other things that I personally disagree with. So in 2022 I wrote just enough code of my own to do the thing I wanted, but not have that bug. And over the course of the last little while I’ve been improving it and improving it and improving it. And it is called .x d O T X, four letters long. It is available from PyPI. It’s a standard Python package that actually provides a command line interface. So if you install the package, you’ll have the command line interface. Don’t use pip. Install it with UV. That’s the way you do things these days. It’s almost ready. I well, I think it might be ready. It’s almost ready to announce. I’ve just got stuff to verify. So I’ve been using .exe. I like it. I’d like to I’d like other people to try it. Tell me what you think. I want it to be useful. It’s in a field of very useful and capable existing tools. I picked kind of a middle range. We’ll see. The other thing, much smaller to announce is I read an article maybe five months ago, don’t know when, from a guy who had some very opinionated, specific ideas on syntax coloring schemes for your editor. His ideas disagree with a lot of people. The theme he made that implements these ideas is called alabaster. The basic ideas are that the stuff you already know and expect shouldn’t stand out. The stuff you actually care about should stand out, and you shouldn’t use very many colors. When you are talking about this theme to other people, you ought to be able to name every color and what it does. It shouldn’t be like walking into a candy store. He’s made this theme for so many different editors, and there are people who like it. But I use Helix. I really like Helix a lot, and there just wasn’t one that did what I wanted. So I finally put together my own Helix theme. It’s in my dot files. I haven’t made it be a separate thing. It probably needs some changes to account for Helix user interface elements and some other things like that. And I only made the light version, not the dark version, so I know I’m only talking to 15% of the population, but it’s a start. The name of the file is wolf alabaster light.toml. Those are separated by hypens.
**Jim: 11:15 **That’s right.
**Wolf: 11:19 **Okay, it’s absolutely usable. I’m looking for comments. I’m ready to make it better. And again, I think once I get some feedback, it’ll be ready to announce, and I probably need to make a dark version as well. So yeah, I scratched my own edge. That was the thing I did.
**Jim: 11:39 **We’ll we’ll include that’s in your GitHub account. We’ll include links to both of those in the show notes. Because you’ve got that, you got a lot of the cool stuff in your in your GitHub account. So we’ll we’ll include the links to that. And as for themes and colors and stuff, I’m kind of colorblind, not completely, but colorization or syntax coloring and stuff, I go pretty basic. I do use colorized syntax, but not to the extent that you do, because there’s just things I just can’t see. And it’s it’s weird. It’s not that I can’t tell two colors apart, it’s sometimes the contrast. For instance, I cannot see red on black or black on red. I just can’t see it. So I picked a very simple color scheme and I stick with it.
**Wolf: 12:36 **So when I look over at my co-worker’s machine, he’s using a dark theme, as so many kids, kids do. Kids, yeah, kids like me. It is like looking at some location in Spyro the Dragon. It is nighttime with a zillion different colored jewels. Complete to me, completely unhelpful. I was gonna say the word incomprehensible, but there’s just so many colors, like what do they mean? How do you find that you missed return? I don’t even know. So here’s the thing: it’s opinionated. If you read the article and you think that guy had a point, might be worth trying. I don’t know. Why don’t we move into feedback and follow-up? The first thing I want to talk about is our last episode was questions for which I was completely unprepared. I did a lot of things. By design.
**Jim: 13:43 **By design. We try to always be prepared for these, but this was a case where I was asking you questions and you didn’t know the questions up front. That’s all.
**Wolf: 13:52 **Yes. And one of those questions, they were mostly about what is X? And sometimes X was pretty easy for me, and one of the questions, X was outside my scope of knowledge. Now I did make a couple of guesses, and of those two guesses, my first one was close, but not close enough for me. The question was, what is generative testing? And let me tell you what generative testing is. A lot of times when you look at a function or a process, what you care about is if I’m going to put in these things, if I’m gonna start out with a square, if I’m working with this tree, if I happen to know that the fabric I’m using is red, any of those. You stick them into your code, and then you expect a specific answer out of that code. This is the usual case. We often call this unit tests. If it turns out that the kind of input and output we’re describing are much larger, like the user types hello, and then a whole bunch of stuff happens and the program ends up doing something at the other end that’s usually bigger than one function, we call that integration testing. Generative testing is when let’s look at a specific function or class or however it is that you’ve designed your program. There are things that you have decided are always true. Whenever I’m talking about this with my team, I use the phrase invariance. Back in the Eiffel days, I learned words like preconditions, invariance, and post-conditions. Wheeza certs for some of these things. But an invariant is a thing that’s always true. For instance, maybe you have designed a class and an invariant of that class, it acts like a list. An invariant of your class is that that list is always sorted. That’s always true. Now it happens that in the middle of a function, when you’re inserting some new value or deleting, it may be that in the middle of a function it’s not true just for a minute. But outside of a function, your invariant is always true. So that’s how invariants work. Generative testing says let’s throw in all kinds of data, every possibility. It’s like that joke about QA testers. QA tester walks into a bar, orders a beer, orders zero beers, orders infinity beers, orders 1.3 beers, orders negative beers. Like that that is what generative testing is. It says I’m gonna make up a whole bunch of data. Some of it is hopefully going to be edge cases. All of it is designed to see if I can get you to break your invariant. I’ve vastly simplified this, and there’s a zillion different ways to accomplish it. I didn’t tell you any of them, I just described what it is, but that’s the answer to the question.
**Jim: 17:40 **Okay.
**Wolf: 17:42 **To me it sounds like testing. Right. I also want to mention a really, really big change for us, and that is we now have transcripts, text transcripts of our episodes. We started with the most recent episode. We’re gonna try to go back and create, provide, ensure they exist for all previous episodes. Maybe this will help with searching. We don’t yet have any reasonable interface. Maybe Google or whatever your search engine is is gonna do the right thing. We’re working on it. But transcripts is a thing for us. Did you have any feedback and follow-up, Jim?
**Jim: 18:32 **Oh, I think you pretty well covered it. You know, I gave a little bit of feedback on episode one, but I’ve got nothing else. I’m kind of walking.
**Wolf: 18:40 **I do want to add one thing. The work I’ve been doing over the last week or so involved tons of refactoring, and the fact that I had good tests and good coverage made that refactoring a breeze. Anyway, from there, let’s actually do a show. Let’s talk about data centers. Yeah. I do want to ask you one weird question before we even start, Jim.
**Jim: 19:12 **Yeah.
**Wolf: 19:12 **And that is episode one, you moved from the data center to a data center. You said the cloud back then. But I guess these are different. There’s a data center that will rent you rack space, yeah. And then there’s data centers that do something anyway.
**Jim: 19:32 **Yeah, you know what? I’ll get into that. That’s that’s part of what we’re gonna talk about. So yeah, today we’re talking about data centers. Before getting too far into it, though, I just have a couple of things I need to say. First of all, I’ve mentioned Marlin in the past. He’s our friend that works at Meta. Brilliant engineer. When I have problems, he’s the guy. So I thought, hey, Meta is big and data. Centers, right? Let me ask him a bunch of questions. So I sent him a handful of questions, and he he promptly gave me miles of information and it it was great. So thanks to Marlon. And as always, if I say anything right during this episode, it’s because he was there backing me up. And if I say anything wrong, it’s not Marlon, it’s me. So it don’t blame it on him. I’ll gladly take the blame. And the other thing is I’m gonna be talking about a lot of numbers, sizes of things and power usage and stuff. None of the companies involved want to really give us specifics. So we don’t have numbers direct from the companies, so these numbers are really not gonna be really accurate. But they’re they’re a guide to some of these things. You know, I had to do a lot of Google searching and looking at basic information from companies that aggregate this kind of stuff. So so the numbers I’m using, they might be off, but they’re not that far off, right? I mean, maybe hopefully less than an order of magnitude off, but you get a good idea by the numbers that I use. So it just don’t hold me to them as as actual fact. So having said that, The whole reason the this idea came up, There’s a new data center going in in Celine, Michigan. That’s a little community just south of Ann Arbor. It happens to be where we have lunch every Saturday, Just a few miles away from this construction site. And last week, just before lunch, we went for a little drive and drove past it. And this is a brand new project. This was just approved by the zoning board in Celine Township, like in the middle of December. And they’ve already broken ground on it now. We went by what was it, December 3rd? and there were there were lots of heavy machinery there, chewing up dirt and moving it around and you know, doing what the big boys do with with all those trucks and things. anyway, it’s full steam ahead for that. This is a massive, massive project, a $7.4 billion project. It’s part of the Stargate project by Oracle Open API and related digital. Stargate that’s what did I say? You said open API, but you meant open AI. That’s because I’ve been working on API stuff lately. Yes. Open AI, artificial intelligence. It’s part of the Stargate project, which there it’s a nationwide thing. They’re building data centers all over the place. There’s a huge one down in Texas. anyway, it’s part of that. Huge footprint, 575 acres. That’s that’s almost a square mile. Three buildings. Each of these buildings is 555 550,000 square feet. It’s the largest economic project in Michigan’s history of any kind.
**Wolf: 23:15 **Now, when you say that, yeah, you mean it’s bigger in terms of money and work than those bridges? Yeah.
**Jim: 23:25 **Than like the Mackinac Bridge. Yeah. Or the Gordy Howe Bridge. Yeah. It’s the largest economic project. So I guess the Gordy Howe Bridge that separates the new bridge that separates Detroit from Windsor, Ontario. I mean, that’s a big economic project. I should have checked to see how how much that costs, but apparently this is bigger in terms of dollars to spend and impact on Michigan. And of course, you know, we’re in Michigan, so we care a little bit about that. This thing is estimated to require about 1.4 gigawatts of electricity.
**Wolf: 24:03 **Just to be clear, yeah, that is bigger than 1.21 gigawatts. That’s awesome. That’s right.
**Jim: 24:11 **That’s it’s a little bit bigger, you know, that and the 88 miles an hour, right? It’s more electricity than the Fermi 2 nuclear plant can produce. That only produces 1.1 gigawatts. So it’s gonna take a lot of electricity to run this thing. It’s about the energy that a million homes would use. And I it just floors me that I mean you know it’s only 575 acres. All right.
**Wolf: 24:48 **Just to fill me in, yeah. When you say 1.4 gigawatts, yeah. 1.4 gigawatts every day, every month, every year.
**Jim: 25:01 **Gosh, I don’t know. I don’t know. Maybe it’s gigawatt hours? I don’t know. It’s a lot of watts.
**Wolf: 25:13 **Okay.
**Jim: 25:14 **I should check that and make sure I understand that properly. It’s an awful lot of electricity. The project itself is about 2,500 construction workers. That’s pretty good. 450 permanent on-site jobs, and I believe those jobs will be like people replacing hard drives, right? Or, you know, the basic maintenance to keep those machines running. They’re expected to open this thing in 2027. So we’ll check on it occasionally, driving by and see see what they’ve done. I think they’re gonna be moving dirt around for a few months at least. That’s the project that sort of spurred this conversation today. let’s get into what is a data center.
**Wolf: 26:07 **What yeah, what exactly is a data center?
**Jim: 26:13 **Yeah, yeah. It’s a physical facility, Housing an organization’s critical IT infrastructure, servers, storage, networking, all that kind of stuff. The a data center would provide cooling power, internet connectivity, redundant systems, all the kind of stuff you need to run a bunch of computer services. That’s that’s kind of what a data center is.
**Wolf: 26:43 **We can So this says a thing to me, and you can tell me to shut up because you’re gonna talk about it in a little bit. But over the years I hear tons of CEOs, and who knows when a CEO says something, what that really means, talking about how they’re gonna have data centers on oil rigs or on barges or underwater or in space. The thing you just said, and I know you’re gonna talk about the connection to the internet, what things is the data center itself responsible? And one of those is the pipe, the pipe that goes to the internet, and it seems like every data center has to have a big fat pipe. They do.
**Jim: 27:38 **They do, and yeah, we’ll get to that. I know you’re anxious to find that out because it’s it’s interesting, right? Because that’s something I’ve always wondered. And again, thanks for Marlin. We’ll get to that in a little bit. So there are types of data centers. Let’s let’s start with the enterprise data center that’s owned and operated by a single company to run their data services, right? Banks might have their own data center, people that don’t want to put their stuff up in a public cloud, or maybe they just want control of everything about their servers. There’s co-location, and that’s that’s the type of data center that I was in for 12 years. And that is basically they rent me rack space. I go in there with my servers, I bolt them up into the rack, I plug in the internet that they provide me, I plug in the power connections, and I say connections because they would give me two power feeds, and all of my servers had two power supplies. So I would plug in, that would give me the power, and they would provide the cooling and the security and all that kind of stuff. That’s a co-location site. Those are pretty popular, they’re they’re all over the place. And then there’s cloud, right? That’s kind of what we’ve we we’ve seen more of. Those are the those companies like Google and AWS and Azure and several others. They give you on-demand computing resources. You can easily spin up new virtual machines or shut them down or or increase the size of them or whatever. Think of cloud basically as using other people’s computers. That’s what that is. And then we get into the big stuff, the hyperscale. That’s what they’re building in Celine, that’s what they’re building in lots of other places. Hyperscale, it’s just extremely large cloud data centers. And then kind of smaller but widespread is the edge data centers. Think of like Cloudflare or Akamai, where they’re they’re your sort of edge connection to the services of the big providers. They handle caching for you and stuff like that. So th those are the types of data centers. There are lots of data centers out there. As of late 2025. That’s some of the latest estimates that I have. Worldwide, there’s over 12,000 data centers. And what what’s what’s kind of weird is most of those are in the United States. There’s over 5,400 data centers in the U.S. Michigan, where we’re at, they’ve got about 50, but that’s growing. That’s not counting the new one we talked about, and several other new projects that are in the area.
**Wolf: 30:30 **In fact, do we Do we have like we have more of X than any other like we’ve got a ton of enterprise or they’re all clouds or I do we have any kind of idea what the makeup is?
**Jim: 30:45 **No, not really. I’m sure that information is out there, but I think I think we can attract more data centers in this area because we have the resources. We have decent weather, you know, we’re not out in the desert somewhere. And we have relatively cheap land. It’s a lot of farmland that that is. And no earthquakes so much. No earthquakes, no mudslides, lots of fresh water, all those kinds of things. So yeah, we’ve we’ve got over 50, and I think that corridor along US 12 in Saline is going to see more data centers once this one’s up and running. The United States, as I mentioned, they have the most. But the UK has 499, Germany has 487, China has 381. That’s a lot, but doesn’t come close to what the U.S. has. The state of Virginia, they have the most data centers of any location. They’ve got 560 data centers in in Virginia that’s closer to Washington, D.C. of the reasons are they’ve got great connectivity. They’re they’re connected to everything, so it works out really well. And the state has has made an investment in that connectivity. They’ve given tax incentives to the data centers and the and the phone companies and everybody providing data to increase that to encourage it. Whether I would like to be in a state with 560 data centers or not, I don’t know. Seems like they’re probably taking up a lot of space. I know the data center I use on Azure that’s US East 2, that’s in Virginia. And Amazon’s got space there, Google’s got space there, everybody does. I mean, 560 centers, that’s a lot of a lot of stuff there. Texas, California, Illinois, they’ve got a lot of data centers. And like I said, Michigan, they’re ramping up. We’re we’re gonna we’re gonna become a major player, I think. and again, it’s because we’ve got the resources, we’ve got the water and the and the space and the weather. All right. So that’s pretty good.
**Wolf: 33:02 **So these things sound expensive. And so my thinking is, you know, I don’t own one, but the people who do are probably gigantic. Who owns them? Who are the players? Who are the big guys?
**Jim: 33:20 **Well, I’ve got a little list here. Meta. they’ve got like 30 data centers, and they handle things like Facebook, Instagram, WhatsApp, and a bunch of other stuff. They’re meta, what what are they? Oculus, all that kind of stuff. Google, they’ve got between 110 and 130 data centers. They got YouTube and search and all the things that they do. Microsoft’s got over 400. I was kind of surprised they had that many. Amazon’s got over 900, but you think about how big Amazon is. You know, first off, they’re retail. That’s the whole reason they they started AWS is they they needed the compute power to handle all their retail business and they needed to scale up at Christmas time. But then the rest of the year they didn’t need to be that big, so they started renting space out. They saw a business model there and it became AWS. So yeah, 900 centers. Oracle’s got 150, Cloudflare is over 330, Akamai 4300. That’s those edge servers that we talked about. Linode. Linode is really popular in the in the Linux space, the open source guys. A lot of people I know use Linode. They only had 11 data centers, but then they got acquired by Akamai. So I who knows what the count is now.
**Wolf: 34:39 **I have to express a personal opinion as a former happy Linode user. Yeah, former happy, yeah. Former happy. That’s my opinion. Former happy, and then Akamai, and now okay, you go on.
**Jim: 34:57 **Anyway, China Telecom. That’s a big one. You know, we don’t have real accurate numbers from them, but the estimate is they’ve got one of the largest data centers with 10 million square feet. Now we talked about about 1.6 million, 1.65 million square feet for the one in Celine. They’ve got 10 million square feet in their data center in the China Telecom Inner Mongolia Information Park. So you can imagine that’s pretty big. I think China’s got a lot of a lot of real estate there that they can they can afford to do this with. Take away. So let me ask you this.
**Wolf: 35:36 **Yeah. okay, a data center is big. But a data center is just the building and the lines and the power and the it’s full of the thing that actually matters, the servers. How many servers are in a data center?
**Jim: 35:54 **Well, an average data center, you know, we’re not talking about the hyperscale centers, but the average one is about two thousand to five thousand servers. I think in the in the data center I was in in Southfield, Michigan, there was probably a thousand servers there. and that was just one center out of a complex of three. So, you know, they had a few thousand servers. A hyperscale data center can exceed a million servers. And the rough estimate that I got, kind of a rule of thumb was for each one million square feet, it’s about two point six million one U servers. 2.6 million. 1U, that means that that’s a measurement in rack space that’s an inch and a half tall and 19 inches wide. It goes into a standard 19-inch rack. So I think you can fit what 48 1U servers in a rack, something like that. Anyway, 2.6 million servers.
**Wolf: 36:58 **All right. So this immediate I’m gonna I’m gonna push a question on the stack for later. Okay. Okay. Because I think this matters, and then I’m gonna immediately move to my next my next problem. That question is obviously they’re not just using ordinary networking inside the building. I’ve heard the word fabric when people talk about connecting servers and stuff like that. So that’s something for you to talk about later. Yeah. Maybe not even later in this episode, maybe later, like in feedback or follow-up. But they’re huge and there’s a zillion servers in them. What all goes into building one? What do you have to do?
**Jim: 37:46 **You know, it’s it’s it’s much more complex than I thought, you know, the center in Celine, they like I said, they broke ground on it. They’ve nobody’s saying this for sure, but they’ve probably that that sort center has probably been in the works for a year in terms of acquiring the property, getting the zoning approved, doing all the land usage studies and getting past all these regulations for the government. So it could be like six to twelve months just to get to the point where you get shovels in the ground. Then you’ve got kind of parallel to the to the building of the building. It can take about a year to actually construct the building, you know, moving the dirt around, putting the concrete down, putting the structure up. That can take about a year, but in the meantime, you’ve got other things to consider, like power infrastructure. While they’re doing this, the power company, and in this case it’s Detroit Edison DTE. I mentioned that a few times later. They’ve got to ramp up their power infrastructure to feed power to this place. And then there has to be water availability. They’ve got to pipe in the water. They use massive amounts of water. And it’s important for cooling. It has to be fresh water. Yeah, for cooling. It has to be fresh water because salt water will just corrode things and they don’t want that. and there are ways to deal with that. But for the most part, the current technology, it’s a lot of fresh water.
**Wolf: 39:22 **And I just want to make sure they don’t use up the fresh water, they don’t make it dirty or anything like that. Like it’s still fresh water, right? Like people always talk about consuming water. Yeah, you that’s not really how it works, right?
**Jim: 39:41 **In in my head, it was always like, well, yeah, like like I think you’re thinking right now, yeah, they’re using water, but it doesn’t disappear. Well, you know, it kind of does. It it they use it for cooling, it ends up evaporating, so it goes up into the atmosphere and then it comes back down eventually as rain. But all this water has to go through treatment. You know, it’s it’s drinking water, it’s the same. Water that that you and I would drink out of the tap. So it’s got to go through processing centers. So all this water, you know, while it may end up back in the lakes, it’s gotta be processed for use. So it’s it’s it’s it’s infringing on the water that we have, that the residents have. So it it does impact that. So it’s a consideration. Yeah, I didn’t either. I just thought, hey, the water just, you know, it’s a it’s a it’s a closed system, right? It’s a loop, it just keeps coming back. But there there’s work involved. So yeah. So the big thing, you know, once they get this the building built, it might take a year, might have taken six months to a year before that just to get all the stuff in place so they could build the building, and then they get the power going, then they have to populate the thing, right? So you got three times five hundred and fifty thousand square feet. Well, you know, it’s not a big open space. You gotta, I mean, it is a it starts off as a big open space, so this that’s not useful. It isn’t useful until you start putting racks in there and computers in there. And imagine that if you’re if you’re putting 2.6 million computers in a one million square foot spot, you know, and if you do the math, 1.6 million square feet, so what are we closer to 4 million servers possibly? Now maybe they’re not filling it up that much, but it’s a couple of million servers for sure. Where do you get these servers? You know certainly the companies like IBM and Dell and Supermicro they all offer computers like that, but people seem to be moving or companies seem to be moving to something more like an open compute platform where it’s it’s open compute, as I understand it, it’s like an open source specification for building servers and stuff. And one of the cool things that they do, certainly power distribution within the building is really, really important. And you know, we’re all used to computers that have a transformer in there that take the 120 volts and transform it down to five volts and maybe 3.3 volts. So every computer would have a transformer in it, changing the electricity to DC and the lower voltages. Well, these open compute platforms they don’t take 120 volts. They’ll take five volts and 3.3 volts, and that’s what you distribute around the building. So that the transformers are big transformers, you know, somewhere in in the area of the building, but they just distribute the five volts and the 3.3 volts. You know, these servers they don’t need to be fancy commercial servers, they’re fairly bare bones, you know, they do exactly what they need to do. And even the cooling on the individual servers, they don’t need that much cooling because this the site has massive cooling. The airflow is directed past all the servers and there’s a hoisted. It’s what it’s been hoisted. Yeah, like in your cooling hoisted. Right, right. That’s the optimization technique that they you know they did a dash O2 to optimize everything that they could. So the cooling and the and the and the power and everything is is is all handled so that what gets to the computer is exactly what the computer needs. Doesn’t need these big heavy hot transformers, so you know what they can do, they can fit more computing power in a smaller space because it, you know, look at your computer. How much of the space in your computer is the power supply? It’s a lot, right? So anyway, other things they have to put in the data centers, tons and tons of networking equipment. And you talked about fabric networking. I believe that’s part of it. But it’s just an awful lot of a lot of wires and bits flying around. And the big companies there are Juniper and Cisco and Arista, and there’s lots of small players as well. And the other thing is, you know, especially for companies like Google, like YouTube and Facebook with all the stuff they have, is where do you store all the data? So there’s tons of storage, big companies, it’s very competitive. There’s Dell EMC, there’s IBM, there’s HPE, I think that’s part of HP. There’s NetApp, Hitachi, many others. So there’s lots of stuff going on there. And it’s a it’s it’s a it’s a nightmare.
**Wolf: 44:37 **Tons of equipment.
**Jim: 44:39 **Yeah.
**Wolf: 44:39 **A lot of that equipment is non-controversial. But there are things in this world that even normal people well, gamers I don’t mean I guess they’re not really na normal people. There’s stuff in this world that we discuss. I don’t necessarily care about some juniper router or whatever it is, but I do care about the fact that my laptop has a hundred hundred and twenty-eight gigs of RAM or whatever. So there’s all these different things going on in the industry. How does these the creation of these data centers and whatnot, how does that interrelate with the rest of the industry?
**Jim: 45:27 **Well, we’re seeing that now, you know. Go try to buy a RAM stick for your for your computer. You know, for those of us on Apple where the RAM is built into the chip, it hasn’t impacted us yet. But trying to build a traditional x86 server and you want to put some SD or AM in it, the prices are through the roof. I’ve heard that the like prices tripled recently. And that’s because these AI data centers are sucking up all the RAM and CPUs and GPUs. You can barely buy an Nvidia card these days because they’re all going to to data centers. So I mean if you can buy it, it’s gonna be a lot more expensive than than it used to be. So that’s impacting.
**Wolf: 46:11 **So this instantly makes me want to ask a question, and maybe you’re not prepared for this question, but that is we’re building data centers, especially these hyper centers, for businesses that aren’t making a profit, that we don’t know if they’re gonna be around, we don’t even know if the job they’re doing is real, that all this AI stuff. Yeah. They’re asking for memory. They’re even asking memory manufacturers to build new fabs to make more memory that we don’t know they’re ever going to need. They’re doing it with money they’ve borrowed from places that are expecting to get that money back. There’s an entire circular economy with respect to, well, specifically Nvidia, but other places too. My question is the combination of these data centers and the amount of RAM and therefore increase in the cost of RAM for everyone else who actually does need it and write then, and CPUs and GPUs, how is this not just straight up gambling? Isn’t this the stock market? What is this?
**Jim: 47:32 **It sure feels like it, doesn’t it? I think you know there everything comes in cycles, and I feel like this is gonna be one of those huge cycles where the cost of stuff right now is incredibly high because everybody’s investing in in AI data centers. And you know, people talk about how it’s going to crash, and yeah, I think there’s gonna be a bubble burst. You know, AI is still gonna be useful, it’s still gonna be out there, but the people putting money into it aren’t gonna be happy because they’re not gonna get the rate of return they were expecting. So there’s there’s gonna be a giant sucking sound coming out of these places that are putting all this money into AI data centers. So what what I expect to happen is in a couple of years, RAM is gonna be dirt cheap, right? It’s gonna come back around because there’s gonna be so much capacity for making RAM and CPUs and GPUs, and there’s not gonna be the demand for it. I could be wrong. It could be that you know the demand is gonna grow exponentially. I don’t see that.
**Wolf: 48:39 **And let me say, this is not you and me saying in any way AI sucks. That’s absolutely not true. AI does useful things. I think for a lot of people it’s a problem of expectations. They think AI does one thing, but that’s not actually the thing that AI does. I use quad code every day. I think it’s incredibly useful. But this brings me on to my next question for you. The thing I am super excited to hear more about, and that is let’s talk about the pipe. Tell me how data centers get their internet. I want to know more.
**Jim: 49:20 **Well, here’s the thing to to ask the question, how does a data center connect to the internet? That’s the wrong thing to say. Because you know what? They don’t connect to the internet, they become part of the internet, right? They’re growing the internet. It’s not like you and I connecting to the internet. They they have connections to the internet, certainly, because we need to access that. They have lots of connections. They connect to all the big providers, That you know, Verizon and those guys that provide big fat pipes. The big thing that they do is they they talk to each other. These data centers have all these interconnected connections. What’s the word? It’s not fabric, it’s a mesh. It’s like a giant mesh network. They’re all talking to each other. So the data that they share back and forth doesn’t have to go out on the public internet. So they don’t have to pay the transit cost for that. They they just have to pay for a dedicated line between the data centers and they they send data around all they want. And then finally, to get out to us, yeah, they have to have a big, big fat pipe. And the kinds of connections they have, either between data centers or between them and the actual internet, is fiber. And according to my buddy Marlin, those fiber cables can be up to an inch in diameter, the bundle, and the bundle can have over 500 fibers in it, and each fiber can carry an ungodly amount of traffic. So think about an inch thick fiber cable, what it can do. That’s what you get.
**Wolf: 50:59 **For everybody who’s listening right now, I just want you to understand that the thing Jim just described, that is, if you look in the dictionary under quote, big fat pipe, unquote, they’re gonna have a photograph of that. Yeah. That’s what it is.
**Jim: 51:18 **It’s you have you have fiber in your house, right? You got it a year or so ago. Did you get a chance to see what the fiber itself looked like? It’s probably just like two little strands, right? Maybe it’s protected by a black It’s inside a sheet.
**Wolf: 51:34 **It is no thicker than an Ethernet cable and a little thinner than than Coax.
**Jim: 51:40 **Yeah, yeah. And thick, but let me ask you the next thing.
**Wolf: 51:45 **When they first started talking about this new data center in Celine, there were crowds. People showed up, they blocked cars, they protested. The community didn’t like it. And as I understand, that’s not unusual. Lots of places the community doesn’t like the idea of introducing a data center. To me, I can see why a community would say, yeah, don’t bury your nuclear waste here. That makes sense. Yeah. What’s wrong with the data center?
**Jim: 52:19 **Well, first off, a lot of people just don’t want any change, right? And that’s sort of what I thought before doing the research on this topic. People just don’t want any change. And you know, for me it was no big deal because it’s it’s miles away from where I live and in, you know, kind of close to where you live. But they oppose it because you know there’s massive energy and water consumption. We talked a little bit about that already, and I’ve got more coming. That that leads to higher utility costs. There’s an environmental impact, right? You know, they’re they’re taking farmland that was nice green, you know, fertile land, and they’re building buildings on it. There’s concerns about the electric grid stability, you know, if they’re sucking lots of power that that residents would like to have. There’s noise pollution, or they can be. I’m curious, you know, come 2027 when it’s up and functioning. I want to go by there and stand there for a few minutes and see if I can hear it. These are not attractive buildings. They’re they’re architectural eyesores, right? It doesn’t amount in that many local jobs. Like, you know, I mentioned that that huge one in Celine, 450 jobs when it’s all said and done. That’s not a lot of jobs. You know, it this thing replaced a farm. I have no idea how many farmer jobs there were. You know, farming had become largely automated, you know, so you could do farming with fairly few people. So maybe it’s more jobs than the farm. But it’s not, you know, it’s not 60,000 jobs. You know, it’s it it it’s just not that great. And it can have a negative effect on on the property values. So yeah, I sort of get why they they do this, why they protest it. You know, the communities they feel that the data centers are are taking more than they’re giving back. And I believe that’s true. You know these these data centers serve at the at the at the pleasure of of the companies that need them, and they don’t they don’t offer much locally. but yeah.
**Wolf: 54:31 **A thing you said which instantly got my hackles up, was raise utility rates. Which I assume is gonna is talking about, for instance, electricity. I don’t want to pay more for electricity, but my immediate question is if DTE is coming down here and increasing the stability and capacity of the grid, if they’re going to trouble to make this area have more electricity available to satisfy this data center, why would my power bill go up? Well, shouldn’t it go down?
**Jim: 55:20 **You know, this is this is another thing. I was educated during the research. And we’ve got a good friend of the podcast podcast, Dave. Dave knows this stuff really, really well. So I reached out to him to try to get some clarification, and he provided me with some some great information. first off, the electric companies they what do they do? They generate electricity and they deliver it to their customers. We are their customers, businesses are their customers, right? It takes a huge investment in power plants, transmission lines, and local distribution for all that to work. It wouldn’t make sense for multiple companies to be in the same space delivering the same thing because they’d need multiple data, multiple power plants, multiple lines. It would just be a big mess. Sure, there would be competition, maybe the rates would come down, but it’d be a big ugly mess to have all that stuff out there. So that that’s not good. So power companies like Detroit Edison, they’re allowed to run as a monopoly. And that means they have guaranteed business. You have no choice, you have to pick DTE for your electricity. I do too. Other residents in in southeastern Michigan, they can pick consumers energy, but if they can pick that, that’s the only thing they can pick. They can’t pick DTE. It’s a monopoly. But by being a monopoly, They have agreed to be regulated by a public utility commission. In Michigan, it’s called the MPSC, Michigan Public Service Commission. All rate increases are approved by the MPSC, and it seems like that would protect us, right? Well, they they they seem to always approve DTE’s rate increases and consumers’ rate increases. So that’s I don’t know if that helps, right? And they earn what’s called a regulated rate of return. It’s on their on the capital that they have. And the capital in their case is the power plants and the and the transmission lines. And they earn between nine and ten percent on that. It turns out they don’t earn a profit on the energy that they produce. They they sell that to us for cost, which sounds like a great deal, right? But no, it’s it’s the delivery. If you ever look at your electric bill, there’s there’s a couple of numbers on there, and one of them is the cost for generating the electricity electricity for the power that you used. And then there’s the cost of delivering it to you, and that’s often much higher than the cost of the electricity. And that’s where they make their profit.
**Wolf: 58:04 **They make their profit on the power that’s sounds it sounds exactly like the and maybe reversed, but it sounds exactly like printer cartridges. Like part of the part of the colour.
**Jim: 58:22 **Yeah, you’re yeah. Yeah, you’re paying for the thing that uses all the ink. You’re you’re you’re paying for the ink, right? you think you’re just paying for a printer, but no, you and that’s a one-time cost. It’s the ink that gets you forever. And that’s the way it is with electricity. Yeah, yeah, you got to pay for the power you use. You pay for that at their cost, but the delivery is where all the all the profit is. And the power company gets this regulated rate of return. They’re guaranteed to get this profit on all of the infrastructure that they have. And you know what? That makes them really, really happy to build more infrastructure. So they love it when something like the data center comes in Michigan. That’s a chance for them to increase their infrastructure, increasing their profit. It’s crazy. These things are expensive. And you know, they’re allowed to make a profit on it because of their investment in that infrastructure. The thing is, we pay for that infrastructure. So when they want to upgrade their their their infrastructure, we have to pay for that. It’s in the form of surcharges on our electric bill. And then we gotta pay constantly for the for that regulated rate of return. We’re paying for that out of our own pockets. And that’s the thing that really, really bothered me. more than the water issue. This this really sort of gets under my under my skin a little bit. And meanwhile, large new customers like. Like the data center that they’re building. They they negotiate a a discounted rate. Do we have any power to negotiate a discounted rate? No, we pay whatever they you know, whatever the MPSC says we’re gonna pay in these big data centers. They get to pay a reduced rate. You know, maybe not forever, but for a period of time. Yeah. That’s another thing. This isn’t strictly related to data centers, but solar. You know, I’ve always heard that, yeah, put solar on your house, and if you if you generate enough electricity, you can sell it back to the electric company. I’ve got friends that have done that. And first of all, they’re not really selling it back to the power company, they’re not getting a check from DTE for the power that they gave them, they’re just getting a reduction of their electric costs. A reduction maybe down to zero, probably not, but maybe to zero, but no further. You know, if they if they need a gigawatt of electricity, or they’re never gonna need that. You know, if they need a a couple of hundred kilowatts of electricity and they can produce you know 10 kilowatts of electricity and they only need five, that that’s not a profit for them. It just means they pay a zero bill. They they never get a check back. And to make it worse, you know, you look at things like apa