Post by marchesarosa on Nov 23, 2011 10:19:11 GMT 1
Prof Ross McKitrick of Guelph University has produced a critique. Here are the main points. You can find the full document here
www.rossmckitrick.com/uploads/4/8/0/8/4808045/mckitrick-ipcc_reforms.pdf
Summary
The name “Intergovernmental Panel on Climate Change” (IPCC) is used
to describe both an administrative entity and an assessment report-
writing process; the former is the technically-correct usage but the latter is
common. The administrative entity consists of three layers. The top one is a
plenary Panel comprising delegates from 195 member states. It oversees a
30-member Bureau (with accompanying Secretariat) that executes most of
the operations of the IPCC. In particular, the Bureau oversees three Working
Groups that produce assessment reports on climate change science and
policy issues. When the assessment reports are accepted by the Panel they
are deemed “IPCC Reports.”
The IPCC has collected many accolades over the years, but criticism
has also grown over whether its assessment reports are as objective and
comprehensive as they ought to be. Three concerns make it particularly
timely to consider reforms to the IPCC process. First, while the IPCC has
long had critics, their number is growing and their ranks include new
members who have in the past been advocates on its side. Second, the
IPCC plays a very influential role in the world, and it is imperative that its
operations be unimpeachable. Yet the oversight mechanisms of the IPCC
simply do not appear to be adequate to assure this. Third, there is a wide
misunderstanding about the IPCC assessment process, such that it is often
considered more formal and rigorous than it actually is.
This report reviews the IPCC procedures in detail and points out a number
of weaknesses. Principally, the IPCC Bureau has a great deal of arbitrary
power over the content and conclusions of the assessment reports. It faces
little restraint in the review process due to weaknesses in the current rules.
And the government delegates who comprise the plenary Panel provide
what appears to be largely passive and ineffective oversight.
The scientific assessment process is thus characterized by the following deficiencies:
a) an opaque process for selecting lead authors
The Bureau has, effectively, a free hand in picking Coordinating Lead
Authors, Lead Authors and Contributing Authors of the report.
Past Lead Author selections have been criticized by other Lead Authors as
being overly dominated by political considerations.
Coupled with the deficiencies in the peer review process, this opens up the
possibility that the IPCC Bureau can pre-determine the conclusions of the
report by its selection of Lead Authors.
b) the absence of any binding requirement for incorporating the full range
of views
The language in IPCC procedures requiring comprehensiveness of the
reports is vague and inadequate.
Recommendations for improving this aspect of IPCC procedures were
shelved during the Task Group process created to deal with reform
recommendations received from an outside agency in 2010.
c) Intellectual conflicts of interest
Lead Authors regularly review their own work and that of their critics,
thereby operating in an intellectual conflict of interest.
A large number of Lead Authors, including ones connected to half the
chapters in the Working Group I report and all the chapters in the Working
Group II report, are employed by or serve as advisors to environmental
activist organizations.
Since Lead Authors have the final say over the published text, the
participation of adversarial reviewers partway through the assessment
process does not mitigate the bias created by this situation.
d) loopholes and gaps in the peer review sequence
Lead Authors can defeat the review process either by overruling reviewers
or by waiting until after the close of expert review and then rewriting the
text. Material changes to important sections of text have been made in this
way in past assessment reports.
Reviewers are not assigned to specific sections of the report, hence there is
no guarantee that a section will be subject to any independent scrutiny, let
alone a level commensurate with its importance to the overall conclusions.
Government review is virtually non-existent. About 90 percent of countries
in the IPCC did not submit any review comments on the last assessment
report and, of the comments received, half were from only two countries.
Likewise only a handful of countries provided written comments on the
recent Task Group recommendations for reforms of IPCC procedures. The
existence of a 195-member plenary Panel thus creates a false impression of
extensive oversight activity.
The InterAcademy Council Report
A recent review of IPCC procedures was undertaken by the InterAcademy
Council, a body jointly sponsored by national academic societies that
conducted a review of IPCC procedures in 2010. It touched briefly on some
of these issues, but none of the subsequent procedural reforms adopted
by the IPCC addressed them. The procedural revisions made in response
to the InterAcademy Council review largely ignored the real problems in
the IPCC, especially those related to intellectual conflicts of interest. The
process instead gave evidence of considerable indifference on the part of
the Panel regarding its supervision of the Bureau.
After going through a number of case studies to illustrate these problems, I
present a series of recommendations to fix the IPCC assessment process.
Recommendation 1: An objective and transparent Lead Author selection
procedure.
Recommendation 2: A transparent Contributing Author recruitment process.
Recommendation 3: Appointment of an Editorial Advisory Board and
identification of potentially controversial sections.
Recommendation 4: Explicit assignment of both section authorship and
reviewer positions.
Recommendation 5: Adoption of an iterative process to achieve a final text
under the joint supervision of authors, reviewers and editors.
Recommendation 6: Adoption of a procedure for seeking technical input
when necessary from outside the list of authors and reviewers during the
assessment process.
Recommendation 7: Due diligence regarding key supporting papers and full
disclosure of all data and methods used to produce original IPCC Figures
and Tables.
Recommendation 8: Immediate online publication of the full report upon
finalization, prior to production of summary.
Recommendation 9: Production of Summary by Ad Hoc group appointed by
the Panel based on recommendations from the Editorial Advisory Board.
Recommendation 10: Release of all drafts, review comments, responses and
author correspondence records within 3 months of online publication of the
full report.
Recommendation 11: That the nations involved in the IPCC Panel begin
these reforms at once, and if such a process cannot be initiated then those
national governments that seek objective and sound advice on climate
change issues should withdraw from the IPCC and begin the process of
creating a new assessment body free of the deficiencies identified herein.
www.rossmckitrick.com/uploads/4/8/0/8/4808045/mckitrick-ipcc_reforms.pdf
Summary
The name “Intergovernmental Panel on Climate Change” (IPCC) is used
to describe both an administrative entity and an assessment report-
writing process; the former is the technically-correct usage but the latter is
common. The administrative entity consists of three layers. The top one is a
plenary Panel comprising delegates from 195 member states. It oversees a
30-member Bureau (with accompanying Secretariat) that executes most of
the operations of the IPCC. In particular, the Bureau oversees three Working
Groups that produce assessment reports on climate change science and
policy issues. When the assessment reports are accepted by the Panel they
are deemed “IPCC Reports.”
The IPCC has collected many accolades over the years, but criticism
has also grown over whether its assessment reports are as objective and
comprehensive as they ought to be. Three concerns make it particularly
timely to consider reforms to the IPCC process. First, while the IPCC has
long had critics, their number is growing and their ranks include new
members who have in the past been advocates on its side. Second, the
IPCC plays a very influential role in the world, and it is imperative that its
operations be unimpeachable. Yet the oversight mechanisms of the IPCC
simply do not appear to be adequate to assure this. Third, there is a wide
misunderstanding about the IPCC assessment process, such that it is often
considered more formal and rigorous than it actually is.
This report reviews the IPCC procedures in detail and points out a number
of weaknesses. Principally, the IPCC Bureau has a great deal of arbitrary
power over the content and conclusions of the assessment reports. It faces
little restraint in the review process due to weaknesses in the current rules.
And the government delegates who comprise the plenary Panel provide
what appears to be largely passive and ineffective oversight.
The scientific assessment process is thus characterized by the following deficiencies:
a) an opaque process for selecting lead authors
The Bureau has, effectively, a free hand in picking Coordinating Lead
Authors, Lead Authors and Contributing Authors of the report.
Past Lead Author selections have been criticized by other Lead Authors as
being overly dominated by political considerations.
Coupled with the deficiencies in the peer review process, this opens up the
possibility that the IPCC Bureau can pre-determine the conclusions of the
report by its selection of Lead Authors.
b) the absence of any binding requirement for incorporating the full range
of views
The language in IPCC procedures requiring comprehensiveness of the
reports is vague and inadequate.
Recommendations for improving this aspect of IPCC procedures were
shelved during the Task Group process created to deal with reform
recommendations received from an outside agency in 2010.
c) Intellectual conflicts of interest
Lead Authors regularly review their own work and that of their critics,
thereby operating in an intellectual conflict of interest.
A large number of Lead Authors, including ones connected to half the
chapters in the Working Group I report and all the chapters in the Working
Group II report, are employed by or serve as advisors to environmental
activist organizations.
Since Lead Authors have the final say over the published text, the
participation of adversarial reviewers partway through the assessment
process does not mitigate the bias created by this situation.
d) loopholes and gaps in the peer review sequence
Lead Authors can defeat the review process either by overruling reviewers
or by waiting until after the close of expert review and then rewriting the
text. Material changes to important sections of text have been made in this
way in past assessment reports.
Reviewers are not assigned to specific sections of the report, hence there is
no guarantee that a section will be subject to any independent scrutiny, let
alone a level commensurate with its importance to the overall conclusions.
Government review is virtually non-existent. About 90 percent of countries
in the IPCC did not submit any review comments on the last assessment
report and, of the comments received, half were from only two countries.
Likewise only a handful of countries provided written comments on the
recent Task Group recommendations for reforms of IPCC procedures. The
existence of a 195-member plenary Panel thus creates a false impression of
extensive oversight activity.
The InterAcademy Council Report
A recent review of IPCC procedures was undertaken by the InterAcademy
Council, a body jointly sponsored by national academic societies that
conducted a review of IPCC procedures in 2010. It touched briefly on some
of these issues, but none of the subsequent procedural reforms adopted
by the IPCC addressed them. The procedural revisions made in response
to the InterAcademy Council review largely ignored the real problems in
the IPCC, especially those related to intellectual conflicts of interest. The
process instead gave evidence of considerable indifference on the part of
the Panel regarding its supervision of the Bureau.
After going through a number of case studies to illustrate these problems, I
present a series of recommendations to fix the IPCC assessment process.
Recommendation 1: An objective and transparent Lead Author selection
procedure.
Recommendation 2: A transparent Contributing Author recruitment process.
Recommendation 3: Appointment of an Editorial Advisory Board and
identification of potentially controversial sections.
Recommendation 4: Explicit assignment of both section authorship and
reviewer positions.
Recommendation 5: Adoption of an iterative process to achieve a final text
under the joint supervision of authors, reviewers and editors.
Recommendation 6: Adoption of a procedure for seeking technical input
when necessary from outside the list of authors and reviewers during the
assessment process.
Recommendation 7: Due diligence regarding key supporting papers and full
disclosure of all data and methods used to produce original IPCC Figures
and Tables.
Recommendation 8: Immediate online publication of the full report upon
finalization, prior to production of summary.
Recommendation 9: Production of Summary by Ad Hoc group appointed by
the Panel based on recommendations from the Editorial Advisory Board.
Recommendation 10: Release of all drafts, review comments, responses and
author correspondence records within 3 months of online publication of the
full report.
Recommendation 11: That the nations involved in the IPCC Panel begin
these reforms at once, and if such a process cannot be initiated then those
national governments that seek objective and sound advice on climate
change issues should withdraw from the IPCC and begin the process of
creating a new assessment body free of the deficiencies identified herein.
