AI Waste

I have Opinions™ about artificial intelligence. I’m not wholly against it, but I do feel that the current over-enthusiastic investment and circular dealing in the industry is a toxic aberration. We are spending trillions of dollars on technology that produces a few billion dollars worth of questionable benefit, all while consuming vast amounts of power, water, and technology.

Questionable benefits

Large language model (LLM) generative AI is currently the only style of AI that is making money. This is what OpenAI (ChatGPT), Anthropic (Claude), Google AI and others are using.

All of these systems work basically the same way. They ingest vast amounts of data and generate a mathematical probability model based on a set of complex rules: this is a slow and computationally obscenely expensive process involving something like 10^23 power floating point operations.

The probability model predicts the most likely ‘match’ words to follow a given set of previous words. When you interact with, say, ChatGPT you are interacting with this model, not the model generation process- this is a computationally much less expensive process. The end result is something that does a surprisingly good job of simulating human conversation and answering certain types of questions. Variations on this model produce images or programming code.

The question is how much all this computing and generating is actually “worth”. What will someone pay to generate some code or an image that can’t be copyrighted? The answer thus far is “Not very much”, at least compared to the expenditures. Trillions of dollars have been spent on the computational infrastructure for AI, and only a few billion dollars in profit have been the result.

Most of the financial “growth” demonstrated by AI has come from circular dealing. This is when an AI company like Anthropic pays a hardware manufacturer like Nvidia billions for data centre processors, and Nvidia invests billions in Anthropic stock. The process is more complex and less obvious than that, but the end result is that real consumers aren’t really driving the huge investments in AI: the industry is basically pulling itself up by its own bootstraps and not in a good way.

Much of the investment in AI is in the form of data centres that house tens of thousands of high performance dedicated processors like the Nvidia Blackwell architecture cards. These types of processors are very specialized to particular types of extremely parallel work loads that scale well such as large language model processing. Processors like this require large physical sites to reside in, cooled to chilly temperatures to keep them running well, and consume vast amounts of electricity.

Power waste

AI processing data centres typically consume in the tens of megawatts of power per facility. This power has to come from somewhere, and quite often it comes directly from the same over-stretched power grids that feed our homes and other industries. Planting an AI data centre at a site is the power equivalent of placing an electrical blast furnace for smelting steel at that site, but nothing of significant value comes out the other end.

Some data centres overcome their power requirements in places with inadequate grid power by building their own power generation facilities. In Alberta, for example, some of the data centres are using on-site natural gas fired generators for power. The source of the power is an obvious issue in this case regarding greenhouse gases and pollution than needs to be dealt with.

An alternative power source that is being considered for some data centre facilities is nuclear power in the form of a small modular reactor (SMR). These are still in the early days of commercialization, but the basic idea is that a self-contained reactor can be dropped off in a couple of large shipping containers and provide a significant portion of the power required for a single data centre without impacting the local power infrastructure. The downside, of course, is the management of nuclear refuelling and spent fuel disposal. Solving these problems is not fully resolved by the vendors of the SMR devices.

Water waste

Data centres of any kind require cooling, and AI specific data centres are no exception. Various solutions are used, including evaporative water based cooling, closed loop cooling, and hybrid cooling that uses a mix of systems. Most of the data centres currently being built in BC and Alberta are using closed loop systems which are very efficient in their use of water or glycol coolant and require very little of the local water infrastructure.

Some of the older data centres used evaporative cooling and these consumed vast quantities of water to maintain the required temperatures. Water is a precious commodity in some states, particularly Texas where a great deal of strain has been placed on limited water supplies by inefficient data centres.

Technology waste

Every piece of computer technology has a limited functional life. A processor may still function five years from now, but it will be several times slower for a given unit of power than a new processor. So it is a near certainty that whatever high end processor is placed in one of these data centres will be functionally obsolete within three to five years.

The sheer volume of processors and their cost means that there is a continuous stream of cash flowing into AI data centres. For example, Telus is building three data centres in BC. They are each going to have about 20,000 Nvidia processors that cost somewhere between $20,000 and $40,000 a piece. Doing the math and assuming a $30,000 average cost, that means Telus will be replacing $600,000,000 worth of processors per data center every three years. They have three of them in BC alone.

All of that processing power has a dollar cost, but is also has an industrial cost. The technology industry that produces RAM and solid state storage used in those processing units is completely consumed just keeping up with AI demand. This has resulted in the costs for normal consumers like you and I for these components going up by a factor of four or five times (400-500%) over the past year or so. There is no recovery from these high prices in the near term because the profligate investment in AI infrastructure shows no signs of abating.

Not all negative

I get some enjoyment out of interacting with AI. For example, the image for this post was generated by ChatGPT and without that tool I likely wouldn’t have created such a graphic. And there is some business value to AI as well: code is generated that otherwise might have been too expensive to produce, and in some cases developers are made more efficient as a result.

There are also less clear benefits like AI driven help engines for particular products and services. Generally I find these to be of questionable utility: they suffer from the fact that they don’t have the same size of source data to draw upon as some of the larger models and thus come across as less “intelligent”. It is important to note that the intelligence is all just smoke and mirrors in any case, but a huge source of input data leads to a more effective simulation.

Building and running a data centre involves some employment benefits as well. No where near as much as you might think, though: maybe a thousand or so people work on the construction phase, so that’s pretty good. But that reduces to as little as fifty people when the data centre is operational and you can rest assured that every possible step to make certain these are menial workers paid the minimum amount is taken.

Netting it out, I feel we are spending too much on AI and ignoring some of the real costs it entails. The impact of the data centres is just one such cost: the total collapse of the stock market that might come if investor confidence wanes is another. I would like to see the investment in current AI technologies made more rational, but it seems improbable that this will happen in the near term.

The header image for this post is AI generated using ChatGPT