Abstract: This paper responds to criticisms levied at my previous research paper on “gray” household income in China. In 2010, I published a paper estimating the actual income of high-income urban residents in China. Results indicated that per-capita disposable income for the wealthiest 10 percent of households in 2008 was roughly 139,000 yuan, rather than the 44,000 yuan indicated by official statistics. This suggested an aggregate gray income of 9.3 trillion yuan for China’s urban residents in 2008, mainly dispersed among high income groups. In 2012, Luo Chuliang et al. published a criticism paper, arguing that flawed methodology and analysis exaggerated gray income and household income gaps. This paper both responds to these criticisms and reasserts my original claim that official statistics seriously underestimate both gray income and income inequality in China.
Key words: gray income
JEL: D31, P26, P36, R29
After its publication in 2010, my paper1 on gray income in China spawned much discussion and debate. One such paper, written by Luo Chuliang, Yue Ximing, and Li Shi, argued that my estimate of gray income in China “beyond common sense” and that its underlying methodology was “deeply flawed.” They argued that based on such flaws, they “could not accept” my research as valid2. While I welcome criticism and hopes that my research will stimulate continued discussion of the topic, due to the nature of Luo et al.’s criticism, I consider it necessary to defend my earlier research.
1. Were Gray Income and Income Gaps Exaggerated?
My previous paper states that official statistics severely understate China’s actual household income and income gaps. First, high-income households tend to underreport their total income (and especially gray income), and official survey statistics do not reflect this omission. Second, official survey data lack a sufficient sample size for high-income households. Both will lead to systematic deviations in estimates of actual income. The paper, therefore, was intended to estimate and correct for such deviations. Indeed, this effort was preliminary and may, to some extent, still underestimate the per capita income of high-income households.
Having recognized these two problems in official survey data, Luo et al. argued that since my research addressed only the first problem (i.e., the problem of income understatement) and did not correct for the second (i.e., insufficient sample size), my estimate of actual income would be inflated “beyond belief”. In the words of these authors: “The estimates defy common sense and should be questioned.”
It is interesting to note that Luo et al. found issue with my conclusions first, and only subsequently criticized my methodology and analysis. It is also interesting to note that they lack in evidence, criticized me with vague and subjective expressions like “unbelievably large” and “beyond the pale”.
Of course, it is necessary to determine whether my estimates were indeed unbelievably large and beyond common sense.省略 launched an online poll asking users if they believed that the paper had overstated actual household income. Between the publication of the paper in 2010 and May 14, 2011, 7,918 people participated in the poll. Results showed that while 10.3 percent believed the paper had overstated gray income, 83.7 percent believed that the estimates were, in fact, accurate. While the estimates may defy common sense for Luo et al., they are very much within the realm of possibility for the majority of the public.
One can also assess the legitimacy of these estimates by simply examining the discrepancies between different official statistics. First, according to household income statistics from the National Bureau of Statistics (NBS), China’s aggregate disposable household income in 2008 was 13 trillion yuan. This figure is 5.2 trillion yuan lower than disposable household income figures on NBS Flow of Funds Accounts, which are based on the National Economic Census. Since the National Economic Census had a broader coverage, we can reasonably assume that there are significant omissions in household survey data.
Second, various savings and investment NBS statistics indicate that China’s aggregate household savings in 2008 was roughly 11 trillion to 11.5 trillion yuan, other than 3.5 trillion yuan that implied by the official household survey data. Adding household consumption to it, disposable household income increases to 22.1 to 22.6 trillion yuan, rather than the 13 trillion yuan calculated using household survey statistics. How does one explain the discrepancy of nine trillion yuan? Furthermore, a similar degree of discrepancy is present in statistics for multiple years. In 2009, for example, this figure exceeded 10 trillion yuan.
Third, if we examine figures for commercial housing sales, private car sales, and overseas tourism, it becomes clear that household spending levels far exceed affordable levels for disposable household income as reported by the NBS.
Even without more examples, one may derive a reasonable judgment from these evidences on whether my income estimation is exaggerated.
2. Analytical Techniques
In addition to rejecting my conclusions, Luo et al. also scrutinized the paper’s research methodology. The research, they argued, “lacks a sound methodological foundation,” “lacks a reasonable logical basis,” and shows a range of problems from using a model with weak explanatory power to inaccurate figures to discrepancies between results calculated using different approaches. Some of these criticisms are self-contradictory, while others have distorted the original intent of the report. Here, I respond to specific criticisms intended to paint the results as “unbelievable” and “ridiculous.”
(1) With regards to my paper’s “lacking a sound methodology foundation” (i.e., lacking a precursor in the existing literature), this relates to a more fundamental question: how does one evaluate the soundness of a research methodology? In economics research, many problems can be studied using existing methodologies that are well-developed and recognized in the field. For some others, however, there is no established and mature approach to follow. While I acknowledge the value of peer-reviewed approaches, the soundness of a research methodology depends first and foremost on whether it can effectively address the outstanding question rather than on whether or not it is present in existing research.
(2) I admit that there is a valid criticism on the relatively low R2 in the model. Notably, this is also the only well-founded criticism among more than ten presented by Luo et al. I admit that this is a defect in the study and still needs improvement. On this matter, however, Luo et al. still failed to note two important points: first, estimated values for most variables reach high statistical significance (at 1 percent level). This confirms the expected relationship between explanatory and explained variables. Second, in order to test the reliability of results, I conducted cross-validation using different approaches and data, and the results were consistent with those estimated by the model. These statistical tests compensate for the relative low value of R2 and prove that the model’s estimates are reliable.
(3) Luo et al. ranked their own urban household samples by income and grouped them by their cumulative Engel’s coefficient. They then calculated per capita incomes for the income groups and compared them with my estimates. Luo et al. identified an error rate of 3 percent to 6 percent for most income groups, with this error rate reaching 16 percent for the highest income group. They thus concluded that my initial analytical approach was impractical.
I contend, first of all, that it is not unacceptable for estimates using different data to exhibit differences of 3 percent to 6 percent. In contrast to this, as mentioned above, there is a difference of roughly five trillion yuan between two official sources of the 2008 NBS household income statistics, i.e., the household survey data and the 2008 National Economic Census data. In this case, the error ratio was as high as 30 percent (40 percent if household data are used as the base). It is curious that although these critics use NBS household survey statistics frequently in their research, they never question such significant and obvious error ratios in their own studies.
Secondly, Luo et al. used their data (which are a part of the NBS survey data) at face value and compared the grouping results to my estimates, although they were not unaware of the unreliability of self-reported income of high income households in the NBS data set. Thus their comparison, especially for high-income households, is not a valid way to criticize the methodology I used.
Finally, Luo et al. employed an incorrect calculation method in their comparison. Instead of using the average Engel coefficient for the groups of households, they used the cumulative Engel coefficient. This method in fact gives different weights to households at different income levels, which naturally produces certain error.
(4) The critics calculated per capita incomes and Engel’s coefficients for different provinces using their own sample data and found that, for various income groups in Jiangsu Province, Engel’s coefficients and incomes were nearly twice as high as those in Gansu Province. Luo et al. then used this to conclude that estimating actual income using the Engel coefficient method is “ridiculous”. It is impossible to verify whether their calculations are correct because they did not provide original data and specific calculation methods. However, from the available information, we find a couple of problems:
First, the grouped data used in Luo et al.’s calculations exhibit an abnormal relationship among their Engel coefficients. Data from both my survey and the NBS data indicate a regular decline in the Engel coefficient as income levels rise. This pattern can be well explained by household economics. The second, third, and fourth groups in the critics’ Jiangsu sample, however, have higher Engel coefficients than the first group (with lowest incomes). Additionally, the sixth group has a higher Engel coefficient than the fifth group, and the seventh group has a higher Engel coefficient than both the fifth and sixth groups. In the Gansu sample, the third group has a higher Engel coefficient than the second group, and the sixth and seventh groups have higher Engel coefficients than the fifth group. Such abnormalities indicate either data quality problems or calculation mistakes. Based on such problematic data of their own, in addition to their incorrect method in calculating Engel coefficient, their criticism on the Engel coefficient method that I have employed is not valid.
Second, the Engel coefficient is related not only with income levels but also with certain other variables such as educational level, family size, employment ratio of households, and regional differences in consumer habits. In my study, these variables are controlled in the model and the outcome is adopted as the final result, whereas the group-comparison method without control these influential variables was used only as a reference. Using the grouped comparison method alone to produce abnormal results, as did by the critics, does not reflect the whole picture.
(5) My original study observed that high-income households intend to understate their income, but less intend to understate their consumption. Stated consumption, rather than stated income, therefore, is a much more reliable base for calculating actual income and household inequality. Furthermore, when household consumption is understated in some cases, spending on food is found generally understated as well. The Engel coefficient represents the ratio between the two; proportionally, when they are similarly understated, the errors can offset each other, leave the ratio unbiased. I thus believe that the Engel coefficient method for income estimation is a reasonable approach.
These explanations are not so complex, but Luo et al. seem to have ignored these technical issues and chosen instead to accuse me “do not trust the NBS income data but trust its consumption data, thus falling into an antinomy”. Upon close inspection, however, this criticism does not stand up.
3. Survey Methodology
The critics also listed many concerns with my survey methodology. Many of these arguments, however, were not in a clear logic or even self-contradictory, especially the following:
Their first criticism aims to prove that my survey data is less reliable than the NBS survey. The critics asserted that the professionalism of contracted surveyors “usually” cannot be guaranteed and that “at the very least [they are] no more professional than NBS surveyors.” They may not exhibit “professional ethics” (i.e., they may falsify questionnaire answers). Furthermore, according to Luo et al., the typical sociological survey method I referenced states that the method “must fulfill a precondition: all individuals are equivalent”; otherwise, the survey “may not be conducted appropriately”.
These logics are unusual. When the critics assume that the surveyors worked for our survey are prone to violations of professional ethics, they seem to assume that NBS surveyors are free from similar problems, without providing any evidence to support the two assumptions. Moreover, in social science researches, there is no such a necessary precondition “all individuals [be] equivalent” or, for that matter, that any two individuals be identical.
Secondly, Luo et al. argue that my survey method “exhibits no substantial difference from the NBS survey” and, therefore, “cannot avoid the problems extant in the NBS household survey.” The authors go on to say that the approach “is not distinctive” and “does not seem to be more advanced.” The survey samples “show no significant difference in income composition” from the NBS samples. Finally, the investigation using an enquiry approach is “very close” to the bookkeeping survey results of the NBS.
While it appears that these statements aim to prove that my survey methodology is no better than that of the NBS, they in fact contradict their previous criticism levied at the paper: that is, my survey method is less reliable than the NBS survey method. Our survey methodology, indeed, “is not distinctive” but only took every possible measure to ensure the data reliability. It behooves me to point out that my sampling method was designed for avoiding data bias that happened in the NBS survey, and the questionnaires are filled anonymously in order to obtain more truthful data. Confidentiality agreements were carefully executed. In designing the survey methods and questionnaires, I took account of psychological reactions from respondents, so that if respondents attempted to understate their incomes, such intention may be identified using self-reported itemized spending data. Survey contractors in each city were selected with caution, required qualifications and experience, and were given training and guidance in each location. Problems that arose in the course of survey were addressed in a timely way, and the survey results were subject to strict quality inspections. In my opinion, this survey may have exceeded the quality standards of normal official surveys.
It is necessary, however, to answer some questions raised by Luo et al. and some others:
(1) The surveyors surveyed acquaintances with whose financial situation they were familiar. Did this approach actually increase reliability of the collected data and avoid income underreporting?
Critics expressed disbelief, and arguing that this may increase the tendency of respondents to understate income. This, however, is only a subjective assumption. According to survey results, middle and high income sample groups with equivalent consumption patterns reported much higher income levels than their NBS counterparts. Clearly, the survey method and support measures are conducive to increasing the authenticity of data.
(2) Did an anonymous survey approach increase the possibility of falsification?
Anonymous surveys allow respondents to feel less pressure to disclose their income level, and help surveyors gather more accurate information. Admittedly, anonymous surveys by their nature do leave more difficulties for quality assurance. Without appropriate quality control and inspection measures, then, the possibility for falsification does increase. In order to address this problem, we selected partners that had established a clear record of trustworthiness. Surveyors were selected based on requirements for education level, work ethics, survey experience, and standing within the community. Avoiding falsification and ensuring data authenticity was a key theme in training sessions, and enhanced supervision and double-check measures were implemented. During the strict quality checking process, questionable questionnaires were eliminated. These measures helped to avoid data distortion resulting from the use of anonymous surveys.
(3) How were the questionnaires inspected?
Verification of the survey process. After completing each questionnaire, surveyors were required to fill a postscript to describe their relationship with the respondents, how well they know the respondents, the survey date, a summary of the interview process, a description of respondents’ attitudes towards the survey, and finally an assessment by surveyors themselves as to the reliability of key information in the questionnaires. These documents helped the survey supervisors to control survey quality and ensure information authenticity. Some unqualified questionnaires were eliminated during this process.
Data verification. Questionnaires were assessed for the integrity of the information collected, their internal logical consistency among the general information provided, and logical consistency between reported income and reported spending. The latter two controls aimed to preclude false information provided by some interviewees, as well as the possibility of falsification by certain surveyors. Generally speaking, it is difficult for those who provide false information and also ensure internal consistency and coherence throughout a questionnaire. After running data verification procedures, 714 out of 4,909 questionnaires were eliminated, 15 percent of the total. Such rigorous inspections ensured overall reliability of survey results.
4. A Final Note on the Spirit of Academic Debate
Academic discussions should uphold objectivity as the highest standard. Some criticisms expressed in Luo et al.’s paper, however, failed to follow these norms. Here, I address two examples:
(1) Attempting to show my study being unacceptable, Luo et al. made two statements that allegedly come from my original paper. First, they said that I had “taken all understated income as gray income”; and second, they said that according to my estimation, “gray income of the lowest income households grows fastest.” Such criticism is unfounded. My original paper makes a clear differentiation between “hidden income” and “gray income”: the former refers to income not reflected in household income statistics, while the latter refers to income that “cannot be clearly defined and is considered legal or illegal”. The paper places estimates for China’s 2008 hidden household income at 9.3 trillion yuan and gray income at 5 trillion yuan. In terms of distribution, the paper’s conclusion is also very clear: more than 80 percent of hidden and gray income is concentrated within the wealthiest 20 percent of urban households. Estimates for low earner household income differed slightly from the NBS figure, but this generally constitutes an ordinary statistical error rather than gray income.
Luo et al., however, mistook this discrepancy as the “gray income of the lowest earning group” and, furthermore, used this figure to calculate “the rate of gray income growth for the lowest earning group”. This departs far away from my original paper.
To derive this unbelievable conclusion, the critics seem intentionally ignored the basic result from my modeling work but used the group-comparison result from 2008 that was not finally adopted by me for its larger error. These critics then compared the 2008 error term of the lowest income group with the 2005 error term to derive a “gray income growth rate”, in order to show how it “unreliable”. This kind of argument is not in an honest way.
(2) Luo et al. also criticized my estimates based on the observation that the resulting Gini coefficient for China would be as high as 0.685, a figure which “overshadows all Latin American countries”. In other words, they labeled my analysis and methodology “ridiculous” because the class divides in China cannot be that stark.
However, the argument itself has specific problems. First, Luo et al. altered my original estimation in their calculation of Gini coefficient. My original paper estimated the difference between urban household per capita income and the NBS statistics for 2005 and 2008 to be 1.78 and 1.90 times, respectively (see Table 8 of the original paper). Luo et al. misstated these figures as 2.53 and 2.25 times, respectively. This difference alone would cause great deviations in the calculation of Gini coefficient.
Secondly, to calculate the nation-wide Gini coefficient, an integrated data set for both rural and urban households is necessary. However, the NBS urban and rural household surveys have always produced two separated data sets, and the original data have never been published. My paper only recalculated grouped urban household incomes other than rural incomes. It is impossible to calculate the national Gini coefficient using data from these urban income groups alone. Therefore, no calculation of the Gini coefficient in my original paper. Luo et al. claim to have “converted” my original data and “consolidated” with their own rural samples in order to arrive at a Gini coefficient of 0.685. These authors, however, did not clarify how urban grouped statistics can be “converted” into original sample data and “consolidated” with rural sample data. They offered an equation but no explanation of the equation’s meaning, theoretical basis, calculation process, and notations of variables. Moreover, “converting” categorical data back into sample data is like turning cooked noodles back into a stalk of wheat: this kind of “calculation” for Gini coefficient is theoretically untenable and problematic. Such a calculation is therefore insufficient to undermine the methodological and analytical basis of my original study.
Lastly, regarding the twin problems of gray income and large income inequality, we should note that these are world-wide problems in many countries which are not only unique to China. Thus, research on gray income should be of international relevance. We should neither, however, disavow its existence in China nor gloat over its existence in other countries. This matter, like all of those in academia, should be studied objectively.