anuary 18, 2013, 11:28 PM
Statistical Diplomacy in North Korea
By Carl Bialik
My print column
examines an initiative to teach statistics to university students and
government workers in North Korea, a country with an authoritarian
government that many researchers in the region think issues the fewest,
and least reliable, statistics in the world.
The Pyongyang Summer Institute in Survey Science and Quantitative Methodology began with about 250 students last summer, taught by 13 instructors from the U.S. and Europe. This summer organizers hope to have 30 instructors, about 250 university students and 100 North Korean government workers, taught in classes hosted by the Pyongyang University of Science and Technology, the country’s first private university, opened in 2010.
“Capacity building in the area of statistics is helpful to governments everywhere because quality data collection leads to informed policy decisions in all areas of development such as agriculture, education, etc.,” Justin Fisher, a PSI instructor last summer and chair of Statistics Without Borders, an international outreach group of the American Statistical Association that supports PSI, wrote in an email. “For example, how can you fund a country of hospitals efficiently if you don’t have data on morbidity and mortality? And while gathering the data seems like the first step, the step before that is for a country to have statisticians trained in sampling, survey methodology, and data analysis in order to gather this data.”
The institute, known as PSI, is attempting to boost the quality of statistical practices and survey research in a country with a poor track record in the field, researchers say. Nicholas Eberstadt, a researcher at the American Enterprise Institute, a conservative Washington, D.C., think tank, said he once visited government statisticians in North Korea and they told him “they were producing rubber statistics that could be stretched or shrunk according to need.”
One sign of how hard it is to gather reliable statistics and conduct surveys in the country: Gallup has polled in 162 countries, including in Somaliland and Iran, but not North Korea — also known as the Democratic People’s Republic of Korea, or DPRK. “We’d sure like to add them to the list,” said Jon Clifton, a partner at Gallup. He added, “We’re just not sure at this point that it’s feasible.”
Daniel J. Schwekendiek, an economist at Sungkyunkwan University in Seoul, said the country is facing “a quantitative and qualitative deficit of statisticians and technical experts.” He added, in an email, “While other nations in the Eastern Bloc trained statisticians also because they arguably needed experts to ‘fine-tune’ these statistical publications for political reasons, North Korea simply ceased to publish data, thereby avoiding external and internal criticism on statistical inconsistencies and saving many resources to run statistical bureaus.”
This could create challenges for PSI’s mission, said Scott Snyder, senior fellow for Korea studies at the Council on Foreign Relations. “With a centrally planned economy that seems to treat all numbers as top secret, one wonders whether trainees would have many opportunities to use these skills to their full potential,” Snyder said.
Still, on balance researchers — including skeptics of North Korean statistics — expect the institute to do more good than hard. “In theory, some skills could also be used to further unconstructive regime objectives, but the risk seems small compared to the upside of having some specialized capacity in North Korea that could be used for positive purposes, even including improving North Korean collection of useful statistics on basic population and demographic information in North Korea,” Snyder said.
Added Marcus Noland, director of studies at the Peterson Institute for International Economics in Washington, D.C., “It’s probably a good thing for reasons that have little to do with statistics. Greater interaction contributes to a certain amount of cognitive dissonance and questioning of the system.” Noland said that the statistical training could prove useful down the line. “If there were ever abrupt political change in North Korea, would be good to have a cadre of people who know what the bond market is, or what seasonal adjustment of statistics is. If you have people who have some familiarity with these concepts, you can start to build the new system much more rapidly.”
A man who answered the phone at North Korea’s mission to the United Nations in New York said, “We don’t accept any questions from the media,” and hung up when asked for his name.
The institute’s organizers steer well clear of politics. “PSI stays away from controversial courses,” said Yena Lee, co-founder of PSI. She added, “Through dialogue over these nonpolitical issues, we hope to pave the way to greater scholarly and professional engagement with DPRK and to long-term sustainable science diplomacy.”
“I had to change some of my ‘go-to’ examples about political polling, but discussion of politics and religion were off limits,” said Fisher, who in addition to — and unconnected to — his work in North Korea is a statistician at the U.S. Government Accountability Office. “Our purpose was one of science diplomacy and we all viewed it as an educational humanitarian mission.”
Governmental constraints extend to instructors while they’re in the program — for instance, barring them from leaving campus unaccompanied. Rules also kept students from PSI and PUST from being interviewed about the program. “That would not be allowed” by the North Korean government, said Norma H. Nichols, director of the International Academic Affairs Office at the institute’s host university, PUST. “If I were in Pyongyang and had the students nearby, it would still be a near impossibility to get permission even for one of them to participate with me in a Skype call.”
Instructors spoke positively about their students in the stats classes. The institute’s director, Asaph Young Chun, described how in one of his two classes last summer, students had to share textbooks because there weren’t enough to go around. “Most of the students now know by heart what survey is about, why pretest is essential and how data analysis should be planned in advance,” Chun said. And most, Chun said, “were so engaged, responsive, and interactive, I observed. They did not hesitate asking questions when in doubt or when I gave them opportunities to ask questions.”
Outside of class, students “wanted to sit and talk about our lives and who we were,” said René M. Paulson, founder of the statistical-consulting firm Elite Research LLC in Carrollton, Texas, and assistant director of PSI.
These sorts of positive interactions lead Fisher to believe that greater cross-cultural understanding and stronger bonds are the more likely outcome from the institute than are any misuses of statistical knowledge to produce faulty data. “What’s more probable is that North Koreans will meet Americans through this cultural exchange and say to themselves, ‘That American was really nice. He took the time to learn how to say “rice” in Korean and he likes soccer too and (despite what I read in my daily paper) he didn’t strike me as an aggressive imperialist.’ And maybe that future leader of DPRK uses the training to affect change in his country. I’m not a Korea expert or a political scientist or even a betting man. But I’d bet on optimism over pessimism in this case.”
Classes, and many of these out-of-class conversations — about soccer and other more leisurely topics than correlation and regression — were conducted in English. Schwekendiek had his doubts about how well that would work, saying, “In the past, North Koreans learned English by simply memorizing words in dictionaries, and not really through communication or extensive reading. Another problem is that North Koreans are not familiar with the relevant jargon.”
However, a member of the PUST faculty said, “We have been impressed with the standard of English which the majority of students have, when they arrive at PUST, and the level they are able to achieve through their hard work during our English-language courses.”
After one summer of work, the program remains a work in progress. It has many influential supporters and advisers, including members of the American Statistical Association. Several of the members of its advisory board said they had only recently learned of the institute and weren’t yet prepared to comment on its operations. “I agreed to provide advice about the challenges posed by North Korea but made it clear I have no expertise in their academic work,” said David Alton, a member of the U.K.’s House of Lords and chair of a group of British legislators who are promoting relations with North Korea.
Organizers say they have learned from the first summer of the program and will make some changes, including bringing in government workers to take some classes, and trying to bring instructors for longer periods so students don’t have to get used to too many instructors for one class.
“I tend to be an optimistic realist, so I am hoping that what was done with our students who should be the leaders of tomorrow will have an impact in the future,” Nichols said.
Lee also is optimistic. “Can statistics be misused?” she asked, then answered, “Yes, as can all data in any field by anyone. But development of statistical sampling methods in the early 20th century in the U.S. didn’t so much create a nation of damned liars as it did progress in fields in private, public, and academic sectors. We have no reason to believe our DPRK counterparts would act otherwise. Improving education or health programs simply requires evidence-based decisions, and twisting the numbers stands to benefit no one. The long-term gains in collecting accurate statistics through better statistical training far outweigh the possible short-term gain of lending legitimacy to falsified statistics.”
Schwekendiek worries about what happens if North Korea does achieve desired gains in statistical capacity. “Is North Korea really interested in opening up, or is it just a free-rider trying to benefit from foreign human capital to improve its economy and scientific knowledge?” he asked. “Training statisticians and businessmen in an international environment might also turn out counter-productive in the long run for the international community to get a foot in the currently open North Korean door. Once the DPRK acquires expertise and educates enough experts, the government might close down these programs and expel all foreign educators.”
*****
The Pyongyang Summer Institute in Survey Science and Quantitative Methodology began with about 250 students last summer, taught by 13 instructors from the U.S. and Europe. This summer organizers hope to have 30 instructors, about 250 university students and 100 North Korean government workers, taught in classes hosted by the Pyongyang University of Science and Technology, the country’s first private university, opened in 2010.
“Capacity building in the area of statistics is helpful to governments everywhere because quality data collection leads to informed policy decisions in all areas of development such as agriculture, education, etc.,” Justin Fisher, a PSI instructor last summer and chair of Statistics Without Borders, an international outreach group of the American Statistical Association that supports PSI, wrote in an email. “For example, how can you fund a country of hospitals efficiently if you don’t have data on morbidity and mortality? And while gathering the data seems like the first step, the step before that is for a country to have statisticians trained in sampling, survey methodology, and data analysis in order to gather this data.”
The institute, known as PSI, is attempting to boost the quality of statistical practices and survey research in a country with a poor track record in the field, researchers say. Nicholas Eberstadt, a researcher at the American Enterprise Institute, a conservative Washington, D.C., think tank, said he once visited government statisticians in North Korea and they told him “they were producing rubber statistics that could be stretched or shrunk according to need.”
One sign of how hard it is to gather reliable statistics and conduct surveys in the country: Gallup has polled in 162 countries, including in Somaliland and Iran, but not North Korea — also known as the Democratic People’s Republic of Korea, or DPRK. “We’d sure like to add them to the list,” said Jon Clifton, a partner at Gallup. He added, “We’re just not sure at this point that it’s feasible.”
Daniel J. Schwekendiek, an economist at Sungkyunkwan University in Seoul, said the country is facing “a quantitative and qualitative deficit of statisticians and technical experts.” He added, in an email, “While other nations in the Eastern Bloc trained statisticians also because they arguably needed experts to ‘fine-tune’ these statistical publications for political reasons, North Korea simply ceased to publish data, thereby avoiding external and internal criticism on statistical inconsistencies and saving many resources to run statistical bureaus.”
This could create challenges for PSI’s mission, said Scott Snyder, senior fellow for Korea studies at the Council on Foreign Relations. “With a centrally planned economy that seems to treat all numbers as top secret, one wonders whether trainees would have many opportunities to use these skills to their full potential,” Snyder said.
Still, on balance researchers — including skeptics of North Korean statistics — expect the institute to do more good than hard. “In theory, some skills could also be used to further unconstructive regime objectives, but the risk seems small compared to the upside of having some specialized capacity in North Korea that could be used for positive purposes, even including improving North Korean collection of useful statistics on basic population and demographic information in North Korea,” Snyder said.
Added Marcus Noland, director of studies at the Peterson Institute for International Economics in Washington, D.C., “It’s probably a good thing for reasons that have little to do with statistics. Greater interaction contributes to a certain amount of cognitive dissonance and questioning of the system.” Noland said that the statistical training could prove useful down the line. “If there were ever abrupt political change in North Korea, would be good to have a cadre of people who know what the bond market is, or what seasonal adjustment of statistics is. If you have people who have some familiarity with these concepts, you can start to build the new system much more rapidly.”
A man who answered the phone at North Korea’s mission to the United Nations in New York said, “We don’t accept any questions from the media,” and hung up when asked for his name.
The institute’s organizers steer well clear of politics. “PSI stays away from controversial courses,” said Yena Lee, co-founder of PSI. She added, “Through dialogue over these nonpolitical issues, we hope to pave the way to greater scholarly and professional engagement with DPRK and to long-term sustainable science diplomacy.”
“I had to change some of my ‘go-to’ examples about political polling, but discussion of politics and religion were off limits,” said Fisher, who in addition to — and unconnected to — his work in North Korea is a statistician at the U.S. Government Accountability Office. “Our purpose was one of science diplomacy and we all viewed it as an educational humanitarian mission.”
Governmental constraints extend to instructors while they’re in the program — for instance, barring them from leaving campus unaccompanied. Rules also kept students from PSI and PUST from being interviewed about the program. “That would not be allowed” by the North Korean government, said Norma H. Nichols, director of the International Academic Affairs Office at the institute’s host university, PUST. “If I were in Pyongyang and had the students nearby, it would still be a near impossibility to get permission even for one of them to participate with me in a Skype call.”
Instructors spoke positively about their students in the stats classes. The institute’s director, Asaph Young Chun, described how in one of his two classes last summer, students had to share textbooks because there weren’t enough to go around. “Most of the students now know by heart what survey is about, why pretest is essential and how data analysis should be planned in advance,” Chun said. And most, Chun said, “were so engaged, responsive, and interactive, I observed. They did not hesitate asking questions when in doubt or when I gave them opportunities to ask questions.”
Outside of class, students “wanted to sit and talk about our lives and who we were,” said René M. Paulson, founder of the statistical-consulting firm Elite Research LLC in Carrollton, Texas, and assistant director of PSI.
These sorts of positive interactions lead Fisher to believe that greater cross-cultural understanding and stronger bonds are the more likely outcome from the institute than are any misuses of statistical knowledge to produce faulty data. “What’s more probable is that North Koreans will meet Americans through this cultural exchange and say to themselves, ‘That American was really nice. He took the time to learn how to say “rice” in Korean and he likes soccer too and (despite what I read in my daily paper) he didn’t strike me as an aggressive imperialist.’ And maybe that future leader of DPRK uses the training to affect change in his country. I’m not a Korea expert or a political scientist or even a betting man. But I’d bet on optimism over pessimism in this case.”
Classes, and many of these out-of-class conversations — about soccer and other more leisurely topics than correlation and regression — were conducted in English. Schwekendiek had his doubts about how well that would work, saying, “In the past, North Koreans learned English by simply memorizing words in dictionaries, and not really through communication or extensive reading. Another problem is that North Koreans are not familiar with the relevant jargon.”
However, a member of the PUST faculty said, “We have been impressed with the standard of English which the majority of students have, when they arrive at PUST, and the level they are able to achieve through their hard work during our English-language courses.”
After one summer of work, the program remains a work in progress. It has many influential supporters and advisers, including members of the American Statistical Association. Several of the members of its advisory board said they had only recently learned of the institute and weren’t yet prepared to comment on its operations. “I agreed to provide advice about the challenges posed by North Korea but made it clear I have no expertise in their academic work,” said David Alton, a member of the U.K.’s House of Lords and chair of a group of British legislators who are promoting relations with North Korea.
Organizers say they have learned from the first summer of the program and will make some changes, including bringing in government workers to take some classes, and trying to bring instructors for longer periods so students don’t have to get used to too many instructors for one class.
“I tend to be an optimistic realist, so I am hoping that what was done with our students who should be the leaders of tomorrow will have an impact in the future,” Nichols said.
Lee also is optimistic. “Can statistics be misused?” she asked, then answered, “Yes, as can all data in any field by anyone. But development of statistical sampling methods in the early 20th century in the U.S. didn’t so much create a nation of damned liars as it did progress in fields in private, public, and academic sectors. We have no reason to believe our DPRK counterparts would act otherwise. Improving education or health programs simply requires evidence-based decisions, and twisting the numbers stands to benefit no one. The long-term gains in collecting accurate statistics through better statistical training far outweigh the possible short-term gain of lending legitimacy to falsified statistics.”
Schwekendiek worries about what happens if North Korea does achieve desired gains in statistical capacity. “Is North Korea really interested in opening up, or is it just a free-rider trying to benefit from foreign human capital to improve its economy and scientific knowledge?” he asked. “Training statisticians and businessmen in an international environment might also turn out counter-productive in the long run for the international community to get a foot in the currently open North Korean door. Once the DPRK acquires expertise and educates enough experts, the government might close down these programs and expel all foreign educators.”
*****
Putting Statistics to Work in a Land of Illusions
Wanted:
statisticians to teach their craft in what many experts call the
country with the least reliable statistics in the world.
The Pyongyang Summer Institute in Survey Science and Quantitative Methodology last year began teaching students at North Korea's first private university about such topics as probability, correlation and survey methodology. More than 250 students, mostly in their 20s, learned from 13 instructors from the U.S. and Europe. This summer, the institute hopes to have 30 teachers instructing 250 students and 100 government workers.
Several students sought grants to apply
the knowledge in year-long projects, such as using statistical
techniques to create a plan for an ice-cream business in Pyongyang, or
to study college students' television-watching behavior, according to
the institute's director, Asaph Young Chun.
In the U.S., these could be the unremarkable activities of an introductory stats course at a college summer program. In North Korea, they are extraordinary events that could have positive effects, but also come with risks, North Korea researchers say. Since the country's establishment in 1948, its authoritarian regimes have made it state policy to conceal many statistics from the outside world, and to falsify some of the numbers they do release, according to researchers.
"Trying to describe North Korea statistically
for a statistician is like [climbing] Mount Everest," said Nicholas
Eberstadt, a demographer at the American Enterprise Institute, a
conservative Washington, D.C., think tank. "It's the last place on earth
where one can't find regular, reliable data on a national scale."
Organizers of the summer statistical institute, known as PSI, say their goal is education and promoting engagement between scientists from different countries, without any political intent. "We are there to serve them, to learn from them," Dr. Chun said of the institute's students, who last summer were drawn from the institute's host school, the Pyongyang University of Science and Technology, which opened in 2010.
Some researchers familiar with North Korea's official statistics worry that training young people in proper statistical techniques won't achieve its goal, or will even backfire. Marcus Noland, director of studies at the Peterson Institute for International Economics in Washington, D.C., who has experience in similar training programs in arbitration and economic policy in North Korea, said graduates of the program likely "will never be able to put their new skills to use," because the government isn't equipped or willing to let them do so.
There is also the risk graduates could be tapped by the government "to better disguise statistical fabrication," Dr. Noland said. Past statistics issued by the North Korean government looked, to Dr. Noland and others, like they were falsified. For instance, there were far more children born during the country's severe 1990s famine and counted in the 2008 census—just the second since the country was formed in 1948—than researchers would expect based on famine's typical effect on female fertility, Dr. Eberstadt said.
Many stats aren't released at all, leaving outside researchers to use whatever data they can find to make educated guesses. Estimates for the famine's death toll at the time varied from 220,000 to 3.5 million. Among the numbers the Central Intelligence Agency World Factbook's North Korea page lists as not available are the unemployment rate, the inflation rate, military expenditures and education expenditures.
A man who answered the phone at North Korea's mission to the United Nations in New York said, "We don't accept any questions from the media," and hung up when asked for his name.
Other countries have statistical data that are incomplete, but "normally they fall into the category of extremely poor or failed states," Dr. Noland said. "Our statistical understanding of North Korea is very poor as a matter of state policy."
The country ceased publishing statistical yearbooks a half century ago, said Daniel J. Schwekendiek, an economist at Sungkyunkwan University in Seoul. Dr. Schwekendiek said he generally finds what sparse North Korean data exist to be of good quality. What he finds lacking is any skilled analysis of the raw numbers, which the institute could help improve.
Instructors who taught last summer said students were engaged and open to learning.
"The biggest standout for me was the dedication of the students," said René M. Paulson, founder of the statistical consulting firm Elite Research LLC in Carrollton, Texas. They grasped mathematical concepts quickly and understood English well, she said, but needed more help learning how to apply the concepts to real-world statistical gathering and analysis.
Funded in part by the International Strategy and Reconciliation Foundation, a small Washington, D.C., nonprofit that focuses on North Korea, the institute has an annual budget of about half a million dollars, Dr. Chun said. The group sent an email this week to members of statistical organizations—including the American Statistical Association, which helps run the institute through its international outreach arm—soliciting instructors for the summer. The institute made clear it wouldn't be able to fund most travel costs. Dr. Chun expects 60 to 70 applications for 30 spots.
Even several skeptics of North Korean statistics think the institute is, on balance, a plus. "It seems to me there is nothing but good that can come out of statistical training, exposure to the outside world and all of that, in the long run," Dr. Eberstadt said.
—Learn more about this topic at WSJ.com/NumbersGuy. Email numbersguy@wsj.com.
The Pyongyang Summer Institute in Survey Science and Quantitative Methodology last year began teaching students at North Korea's first private university about such topics as probability, correlation and survey methodology. More than 250 students, mostly in their 20s, learned from 13 instructors from the U.S. and Europe. This summer, the institute hopes to have 30 teachers instructing 250 students and 100 government workers.
Associated Press
Since North Korea's
establishment in 1948, its authoritarian regimes have made it state
policy to conceal many statistics from the outside world, say
researchers. Here, North Koreans shown last month leaving the statues of
late North Korean leaders Kim Il Sung, left, and Kim Jong Il in
Pyongyang.
In the U.S., these could be the unremarkable activities of an introductory stats course at a college summer program. In North Korea, they are extraordinary events that could have positive effects, but also come with risks, North Korea researchers say. Since the country's establishment in 1948, its authoritarian regimes have made it state policy to conceal many statistics from the outside world, and to falsify some of the numbers they do release, according to researchers.
The Numbers Guy blog
Organizers of the summer statistical institute, known as PSI, say their goal is education and promoting engagement between scientists from different countries, without any political intent. "We are there to serve them, to learn from them," Dr. Chun said of the institute's students, who last summer were drawn from the institute's host school, the Pyongyang University of Science and Technology, which opened in 2010.
Some researchers familiar with North Korea's official statistics worry that training young people in proper statistical techniques won't achieve its goal, or will even backfire. Marcus Noland, director of studies at the Peterson Institute for International Economics in Washington, D.C., who has experience in similar training programs in arbitration and economic policy in North Korea, said graduates of the program likely "will never be able to put their new skills to use," because the government isn't equipped or willing to let them do so.
There is also the risk graduates could be tapped by the government "to better disguise statistical fabrication," Dr. Noland said. Past statistics issued by the North Korean government looked, to Dr. Noland and others, like they were falsified. For instance, there were far more children born during the country's severe 1990s famine and counted in the 2008 census—just the second since the country was formed in 1948—than researchers would expect based on famine's typical effect on female fertility, Dr. Eberstadt said.
Many stats aren't released at all, leaving outside researchers to use whatever data they can find to make educated guesses. Estimates for the famine's death toll at the time varied from 220,000 to 3.5 million. Among the numbers the Central Intelligence Agency World Factbook's North Korea page lists as not available are the unemployment rate, the inflation rate, military expenditures and education expenditures.
A man who answered the phone at North Korea's mission to the United Nations in New York said, "We don't accept any questions from the media," and hung up when asked for his name.
Other countries have statistical data that are incomplete, but "normally they fall into the category of extremely poor or failed states," Dr. Noland said. "Our statistical understanding of North Korea is very poor as a matter of state policy."
The country ceased publishing statistical yearbooks a half century ago, said Daniel J. Schwekendiek, an economist at Sungkyunkwan University in Seoul. Dr. Schwekendiek said he generally finds what sparse North Korean data exist to be of good quality. What he finds lacking is any skilled analysis of the raw numbers, which the institute could help improve.
Instructors who taught last summer said students were engaged and open to learning.
"The biggest standout for me was the dedication of the students," said René M. Paulson, founder of the statistical consulting firm Elite Research LLC in Carrollton, Texas. They grasped mathematical concepts quickly and understood English well, she said, but needed more help learning how to apply the concepts to real-world statistical gathering and analysis.
Funded in part by the International Strategy and Reconciliation Foundation, a small Washington, D.C., nonprofit that focuses on North Korea, the institute has an annual budget of about half a million dollars, Dr. Chun said. The group sent an email this week to members of statistical organizations—including the American Statistical Association, which helps run the institute through its international outreach arm—soliciting instructors for the summer. The institute made clear it wouldn't be able to fund most travel costs. Dr. Chun expects 60 to 70 applications for 30 spots.
Even several skeptics of North Korean statistics think the institute is, on balance, a plus. "It seems to me there is nothing but good that can come out of statistical training, exposure to the outside world and all of that, in the long run," Dr. Eberstadt said.
—Learn more about this topic at WSJ.com/NumbersGuy. Email numbersguy@wsj.com.
A version of this article appeared January 19,
2013, on page A2 in the U.S. edition of The Wall Street Journal, with
the headline: Putting Statistics to Work in a Land of Illusions.
沒有留言:
張貼留言