Measuring the Accuracy of Climate Models
An example of a climate model, in this case NOAA's GFDL CM2.1 model.
![]()
A study by meteorologists from the University of Utah shows that current climate models are indeed quite accurate and a valuable tool for seeking solutions on reversing global warming trends. The study will be published today in the Bulletin of the American Meteorological Society. I expect to see some interesting comments from our readers in regards to this.
Co-authors Thomas Reichler and Junsu Kim measured how well climate models actually do their job in simulating climate. They compared the output of about 50 national and international climate models that were developed over the past two decades against observations for present climate.
Excerpt from the press release........
"Coupled models are becoming increasingly reliable tools for understanding climate and climate change, and the best models are now capable of simulating present-day climate with accuracy approaching conventional atmospheric observations," said Reichler. "We can now place a much higher level of confidence in model-based projections of climate change than in the past."
Other observations from the study......
--Models used for the IPCC (Intergovernmental Panel on Climate Change) paper have reached an unprecedented level of realism.
--Climate models built in the U.S. are now some of the best worldwide. (This reminds me of what you typically hear from the American auto industry.)
Here is the link to the full study (pdf. file)
By the way, after almost 4 weeks, my wife and new daughter are home! Hopefully, things will slowly get back to normal now. It was one crazy March! Brett



Comments (39)
What a surprise! The funding beneficiaries agree that they should continue the benefits.
As I noted previously, the CPC winter prediction was exactly inverted from results. Same for the Met's UK forecast last summer.
Roger Pielke Jr. made a great comment recently about how well the IPCC models do - after they reverse fit them to the past data. It reminds me of a stock analysis program I tried once which predicted the past behavior of the stock market almost perfectly.
None of the models predicted this. Temperatures have been declining for 10 years and are barely higher than 1940 or 1880.
http://www.cru.uea.ac.uk/cru/climon/data/themi/g17.htm
Posted by Patrick Henry | April 4, 2008 12:34 PM
Here are two articles from Science News:
One from December 12. 2007: New Study Increases Concerns About Climate Model Reliability
Then the one on this blog: Climate Models Look Good When Predicting Climate Change
http://www.sciencedaily.com/releases/2008/04/080402100001.htm
http://www.sciencedaily.com/releases/2005/09/050922015634.htm
So much for these 'studies', the above, I believe, illustrates the very problem with the whole AGW scenario:
Who is saying what? And just how reliable is it? In what way are these models accurate? What is their track record? Obviously if the projections of those who are making the predictions is Zero, the models must not be so reliable, and neither are those climate soothsayers making the projections.
Typical AGW: Some uncertainty about our future climate is because there are processes and feedbacks between different parts of the Earth that are not fully understood. These are difficult to include in the models until we understand them better. Today, scientists are conducting research to learn more about how some of the less well-known processes and feedbacks work. For example, the effects of clouds on climate is known to be a large, however it is not fully understood and so scientists are researching clouds to ensure that climate models are as accurate as possible. Scientists work to ensure that natural processes are represented in climate models as accurately as possible, so that models can be used to make predictions of future climate that are as accurate as possible.
Most of the uncertainty in these predictions of future climate is not related to natural processes. Instead, it is uncertain how much pollution humans will be adding to the atmosphere in the future. Innovations that stop or limit the amount of greenhouses gases that are produced, laws and rules that change the amount of pollutants that are released, and how the growing human population lives in the future are all somewhat unknown. To deal with this, climate models are often run several times, each time with different amounts of pollution and development by humans.
According to the IPCC, most climate scientists agree that while climate models are not perfect, they are currently pretty good and better models would not change the conclusion that Earth�s average temperature is warming.
October 18, 2005: http://www.sciam.com/article.cfm?id=climate-model-predicts-ex
Excerpt: "Climate change is going to be even more dramatic than we previously thought," says Noah Diffenbaugh, who reported his team's findings in the October 17 online edition of the Proceedings of the National Academy of Sciences. Until now, the fastest computers crunching climate data and using the most sophisticated computer codes have produced only a grainy image of the country's climate picture. For one thing, the codes--which incorporate data describing hundreds of natural variables such as ocean currents, cloud formation, vegetation cover and the amount of greenhouse gases expected to exist in the atmosphere--have left out such factors as soil moisture, how much sun snow-cover reflects, and to what extent mountains thwart weather systems. What is more, previous models calculated the data by breaking the country into a grid of 50-kilometer squares, a resolution that can miss climatic nuances.
Some of these models projecting up to a 7 degree increase in temperature by the end of this century, how reliable is this? If, as the High Priestess of AGW, Heidi Cullen contends, just a 1 degree increase portends dire straits for the world, then with 7 degrees foretold, I had better loose my Skeptic tinfoil hat, and become a Hysteric.
The typical answer to "How Accurate Are Computer Models?" is "Scientists are continually testing and improving the capabilities of existing models as more data is gained on the environment."
Yeah, sure, but How accurate are they NOW?
Posted by Steve Rowland | April 4, 2008 1:22 PM
Brett: Welcome back and congratulations!
Hmmm, accurate climate models. That should be helpful in understanding our changing climate. But will it be accepeted by the doubters? Stay tuned for another exciting episode of "As the denier world turns - or doesn't - cus it might be flat!" Anyway, I find that I am greatly reassured. Good luck and keep on denying!
Posted by kevinag | April 4, 2008 1:41 PM
Off subject (as 2 of my pother posts today were):
The 'Alternative Fuels' post is moving down the blog so I though I would post this here:
http://biz.yahoo.com/ap/080403/corn_at_6.html?.v=6
Corn Hits $6 a Bushel on Tight Supplies:
Excerpt: Moreover, cold, wet weather in parts of the U.S. corn belt may force farmers to delay spring planting, potentially sending prices even higher.
Excerpt: While corn growers are reaping record profits, U.S. consumers can expect even higher grocery bills -- especially for meat and pork -- as livestock producers are forced to pass on higher animal feed costs and thin their herd size.
"Higher corn prices is going to affect meat prices. If you're feeding with $6 corn, you'll definitely have some (cost) pressure," Kub said.
In addition, corn and corn syrup are used in an array of products, meaning the price of everything from candy to soft drinks will eventually go up, analysts say. It's the latest dose of bad news for U.S. consumers, who are already struggling with higher food costs from record increases in the price of wheat, soybeans and other agriculture products.
Posted by Steve Rowland | April 4, 2008 1:56 PM
Brett, I don't have a comment for the blog, Patrick Henry said it best in the first post.
I just wanted to say welcome home and it's good to hear that you guys are all doing fine! Reply: Thanks Saly, we appreciate that!
Posted by saly | April 4, 2008 3:08 PM
Behold the wonder of the Engish Language:
--Models used for the IPCC (Intergovernmental Panel on Climate Change) paper have reached an unprecedented level of realism.
--Climate models built in the U.S. are now some of the best worldwide. (This reminds me of what you typically hear from the American auto industry.)
Hooray!!! We said something that sounds good but is empty inside!!
Unprecedented levels? Are you serious? OMG for real? No way!!!! In the case of climate models, an unprecented level would be to finally achieve the rank of "complete and utter crap" which is the unprecedented level. It has never, ever risen above the level of "pure, shamless, who gives a crap, useless junk"
And now we have the best models here in the US!!!
REJOICE!!! Our crap is better than your crap. Seriously, it takes linguistic ability to say so little with so many words.
Posted by Veets | April 4, 2008 3:26 PM
I just bought property in Colorado because the oceans are going to cover most of the United States in ten years. If you skeptics were as smart as me, you would be doing the same thing. Beware of global warming.
Posted by kevin | April 4, 2008 3:51 PM
Another Friday another chuckle.
This paper is filled with so many caveats that one wonders how this stuff gets published.
Again these types of articles are fine for astrophysics or in a field that doesn’t demand we either lower the world’s standard of living or commit suicide by the billions.
Here the writer admits they can’t get "past climate" to work because of uncertainties and it would be too complicated to try and get it right.
“….information about past climate carries large inherit uncertainties, complicating the validation process of past climate simulations (e.g., Schmidt et al. 2004).
Apparently it is much easier to use adjusted information to make it work. Even at that they say
“….there is a lack of reliable and consistent observations for present climate, and some climate processes occur at temporal or spatial scales that are either unobservable or unresolvable.”
Question just what is reliable?
This quote is an exercise in obfuscation
“…model agreement with observations of today‟s climate is the only way to assign model confidence, with the underlying assumption that a model that accurately describes present climate will make a better projection of the future.”
Once can be confused with this statement. As the AGW side has said, “Climate is over an extended period of time while weather is short term.” So what are the models predicating?
When words like “Assumption” is used one thinks about the old adage that says “ASSUME usually means you make an ASS of U and ME.
The authors must mean the models can predict weather but not Climate!
Another exercise in “if you can’t dazzle them with brilliance baffle them with BS”
“suggest that an increasing level of confidence can be placed in model based predictions of climate. This, however, is only true to the extent that the performance of a model in simulating present mean climate is related to the ability to make reliable forecasts of long-term trends."
Let me get this straight their models can only predict what just happened but not what will happen in the future! My is that convenient! What we have is a “Just past present weather history model with no guarantee that anything predicted in the future will spew out anything remotely reality!”
With that statement we all must be just jumping with confidence and running to buy Al Gores carbon credits.
I submit that this quote could sum up the entire paper:
“good model performance evaluated from the present climate does not necessarily guarantee reliable predictions of future climate!
And people want to change the world on research like this?
Read, learn, Listen and think for your self. You don’t need a doctorate to smell baloney.
Posted by ted | April 4, 2008 4:17 PM
Hi Guys,
this just irk's me every time they toot their own horn.
Joanne Simpson equated them to weather models.
http://climatesci.org/2008/02/27/trmm-tropical-rainfall-measuring-mission-data-set-potential-in-climate-controversy-by-joanne-simpson-private-citizen/
Roger Pielke Sr. write up on GCM�s is more to the point.
http://climatesci.org/category/climate-models/
This is how I feel about them.
http://www.youtube.com/watch?v=8_RKPGS2vwM
Posted by Jim Arndt | April 4, 2008 4:24 PM
Steve Rowland
All good points about the state of soaring food prices. Some of its also due to the high price of diesel fuel that truckers need to carry such cargo....This all makes me wonder if the current trend of using food in place of oil continues, will we soon be calling farmers, "Big Farmers" and big oil companies will finally lose their "big" adjective?
Posted by Goldfinger | April 4, 2008 4:54 PM
Hi Brett, glad your family's ordeal is over - real life begins again now!
An admittedly quick perusal of the article seems to show that it basically says models are better than they were but there are still so many variables that are difficult to model that they, quote, "hope" things really are better.
Posted by Dave Andrews | April 4, 2008 4:57 PM
Today here where I live computer models showed heavy rain throughout the day, actual weather a little rain in the morning and pastly sunny through the rest of the day..hmm...can not predict one day and it predicts the future, years down the road. Has anyone noticed in the past 10 years that the weather where you live has warmed, I haven't. Perhaps this is the computer model Ted Turner was looking at when he stated like it was the gospel in 8 years, I mean 30 years the earth will be blank degrees warmer, horrible droughts, no crops, human life has to turn to cannabalism and on and on. FIRE!! Same deal here to, he should be thrown in jail for this ignorance, but hey maybe some older folks or uneducated younger ones will listen and buy some things and live their life to make some of those kinds of people more money, thats the goal behind it anyway. kevinag, if you recall the "skeptics" are suppose to be the minority and the minority years ago believed the earth was round as the majority believed it was flat so go figure how that one turned out and someone so sure of global warming why then on the one blog I believe the Joe Bastardi one I had a few simple questions that I asked and wanted some believers to answer me to make me believe your side you didn't attempt to, why? Early this morning where I live we had some freezing rain but this afternoon a warm front pushed through and temps warmed to 60, thank goodness for this global warming or we might have had a heck of an ice storm.
Posted by Josh Brenneman | April 4, 2008 5:08 PM
Brett,
You think those fellers can computer model my "TUNNELS" to see what effects on climate they will have? BTW good luck to you and your family!!! Thank you.
Reply: Yes, but they will need lots of $$$$$$$.
Posted by Patrick Cyclonebuster | April 4, 2008 5:11 PM
I work with and formulate air quality dispersion models all the time in my place of work and I can say with experience there is no perfect model. There is allways some better way to simulate known dispersion modeling events based on some theoretical correlation of the measured data variables. You always try to incorporate the latest known dispersion parameters through model algorithms, but you cannot trully mimic the full effect of the complexities within the atmospheric environment. There are just too many variables and an ever increasing set of options to evaluate.
The accuracy of air quality models have increased dramatically in the past 20 years due to increased understanding of the atmospheric dispersion components within the environment. The increase in model capacity and speed has also aided in that change.
Models are critical however, in a feedback fashion in order to understand the combined effects of various components and factors that relate the the overall prediction of downwind impact of the emitted pollutants. Without these air quality models we could not fully understand the true impact of a source since we could not practically place monitors in every position within what would be considered a model domain.
Models have changed the way we understand the movement and dispersion of manmade pollutants in our localized and regional areas, and the enhancement and improvement of the climate models are just a larger version of that fact.
Posted by Dennis Hlinka | April 4, 2008 5:44 PM
It is good to know these models are getting better. Surely they predicted the severe winter in China, and the fact Central Asia was hit even worse.
This brings up a sensitive subject. Considering these models are so much better, it seems wrong that the modelers kept this information to themselves. I am sure the Chinese are less than happy we didn't warn them. Or how about the poor nation of Tazikistan? With most of its electric power coming from hydro, they should have been warned that these new and improved models could forsee hydro would be useless, with rivers basically frozen solid.
http://www.eurasianet.org/departments/insight/articles/eav022008b.shtml
Shepherds tend to be tough and astonishingly weatherwise, yet all through Central Asia there are reports of shepherds enduring extreme hardship, and of frostbite costing them fingers and toes and hands and feet and hundreds of lives.
According to this report the new and improved climate models have achieved a level of certainty where they must have seen this extreme cold coming. Therefore the blame falls squarely on the modelers for not warning the innocent.
The modelers should be taken to task. Why did they withhold this information? Was it because they repress any and all information about cooling? Or was it because their models in fact totally blew the winter forecast, and they in fact had no information to withhold?
Posted by Caleb | April 4, 2008 6:39 PM
"A study by meteorologists from the University of Utah shows that current climate models are indeed quite accurate and a valuable tool for seeking solutions on reversing global warming trends."
I don't see why you reach this conclusion. (Reply: Which "you" are you talking about? It was not my conclusion, it was the mets. at the Univ. of Utah.)The study concludes the models are becoming more accurate - I would hope so! - but this is a relative term. A poor model is still better than a terrible model. The authors also admit severe shortcomings in their study. It's a bit premature for jumping to conclusions such as the one above.
Posted by Tom | April 4, 2008 6:47 PM
So what happened with christy's recent paper showing that nearly all the models were way off? Am i correct in saying that this paper (this thread) uses data from 1979 to 99 and is out of date? Anyway.. first official admission by the BBC the global warming has stopped wait for more soon...
http://news.bbc.co.uk/2/hi/7329799.stm
Posted by vg | April 4, 2008 7:50 PM
I just bought property in Colorado because the oceans are going to cover most of the United States in ten years.
kevin,
That is odd. Al Gore just bought an expensive home at sea level in San Francisco. It must have slipped his mind for a minute that he expects it to be underwater soon.
Reply: He must be right on the water if his house is at sea level in that region.
Posted by Patrick Henry | April 4, 2008 8:55 PM
I just cant believe how some people on here still buy this garbage. It amazes me how low the IQ's are in the united states. It also mazes me that accuweather actually posts this garbage also. Reply: Your free to your opinion. I don't think it is garbage, even if you don't agree with it.
That cant even model my forecast from day to day. It was supposed to rain all week, surprise not one drop. (Reply: the story is about climate models, not weather forecast models. Weather forecast models, while far from perfect, have come a long way since I started my career almost 20 years ago. Without the models, weather forecasting would be a lot less accurate, believe me! Where do you live? I ask because I would like to see that forecast for rain all week.
Brett how about stepping out and admitting how rediculous this all is, Your job would survive, maybe..
Posted by mark | April 4, 2008 10:12 PM
Hi Brett,
Gore's new house is supposedly on Fisherman's Wharf - which is built on fill dumped into the Bay. You can't get much closer to sea level than that. Reply: Yes, I remember it was that fill that led to the high damage to certain parts of San Fran in the 1989 quake.
Posted by Patrick Henry | April 4, 2008 11:47 PM
Um... OK. So the model fits the data. Glad they managed that.
But what if the data starts looking dodgy? Will the model still fit after they ... er .. fix the data?
M
Posted by Mark Alger | April 4, 2008 11:58 PM
Brett,
Look at the trillions of $$$$$$$$$$$$$$$ they save in the near future. An investment well worth it. Reply: Maybe so.
Posted by Patrick Cyclonebuster | April 5, 2008 8:30 AM