A. Introduction
In 2009 Statistics South Africa (Stats SA) introduced extensive changes to the methods of the consumer price index (CPI). These methods brought the South African CPI into line with international standards.
Stats SA’s approach to managing the CPI is to continually explore ways to improve the accuracy and quality of the index. To this end, two reviews of the CPI were initiated. One is an assessment of the extent to which the CPI conforms with the nine quality dimensions of the South African Statistical Quality Assessment Framework (SASQAF). This work is currently under way and is expected to be completed early next year.
The second was a peer review of the CPI by an international CPI expert. Mr David Fenwick was appointed to conduct the evaluation. Mr Fenwick was previously the head of Price Statistics at the Office for National Statistics in the United Kingdom and was the editor of the Practical guide to producing Consumer Price Indices published by the United Nations. Mr Fenwick also recently participated in the International Monetary Fund (IMF) review of the Australian consumer price index.
B. Specifications of the review
The brief given to the reviewer by Stats SA was the following.
The review should determine:
1. The extent to which the CPI adequately reflects changes in consumer prices in South Africa;
2. The extent to which the methods used in compiling the CPI comply with accepted international practices;
3. The suitability of the methodology to measure consumer price inflation in the South African context;
4. The robustness of the operations of collecting, processing, compiling and disseminating CPI information;
5. Recommendations for improvements.
The review took place in February 2011 with the report being submitted to Stats SA in March. The exercise consisted of:
1. review of documentation outlining the methods followed in collecting and compiling data for the CPI,
2. observation of price collection in two different provinces,
3. meetings with users of the CPI from the South African Reserve Bank (SARB), National Treasury and the private sector, and
4. observation of the capturing, editing and compilation processes.
The method for the evaluation broadly followed that outlined in the Practical guide to producing Consumer Price Indices.
C. Key Findings
The primary finding of the review endorses the changes introduced into the CPI over the past few years.
Specifically, the report concludes that:
1. “Statistics South Africa’s Consumer Price Index is generally of a good quality and adheres to many of the relevant international standards and to recognised good practices in index construction and in statistical production more generally.”
2. “The statistical foundation of the South African CPI has not been stronger than it is now.”
3. “Statistics South Africa is committed to maintaining a high level of professionalism and transparency, and this is appreciated by users. It also shows a high level commitment to collecting good quality data, to methodological soundness and to publishing reliable indices.”
D. Recommendations for improvement
The aim of commissioning the review was to identify opportunities for further improvement of the South African CPI. To this end the review made 26 recommendations for possible improvement. Of these 13 concern the methods employed to calculate the CPI and the remaining 13 are focussed on improving the operational efficiency and effectiveness of the collection and editing processes.
E. Methodological recommendations
Each recommendation is presented below and is followed (in italics) by a response on whether and how it will be implemented. The recommendations are grouped into short-, medium- or long-term proposals. Short-term means that it can be implemented before the end of March 2012; Medium-term means that it will be implemented before the end of March 2013 (as part of this round of reweighting); Long-term means that it will be implemented from April 2013 onwards.
Short-term actions
1. Sample design
Variances of the prices and the price relatives should be re-calculated and another sample optimisation exercise carried out to coincide with the results of the 2010/11 IES. The possible extension of the CPI to rural areas will also have a bearing.
The sample design and allocation of price quotes should better take into account the methodological requirements for the production of regional indices, depending on the purpose of these indices. This is currently being done on an annual basis.
2. Review of basket
The availability of data from the 2010/11 Income and Expenditure Survey (IES) should be taken as an opportunity for a fundamental review of the CPI basket. This is in addition to the recommendations relating to the coverage of items purchased by tourists, rural areas, markets & street-traders and to the continuous and systematic monitoring of the weights and basket.
This will be done.
Medium-term actions
3. Linking of indices
Chain-linking should be used when new weights (& basket) are introduced on the 2010/11 IES results becoming available. This will involve dual pricing in the link month & the calculation of new price relatives.
This will be done.
4. Quality adjustments
Statistics South Africa introduces at the first opportunity implicit quality adjustment (as in the Stage 1 Harmonised Consumer Price Index (HCPI)) and in the longer-term considers possible areas of the index where explicit methods should be used. Implicit quality adjustment should be the default method.
Quality adjustments are to be introduced for second-hand motor vehicles, cell phones and selected electronic appliances at the next reweighting. Other items such as clothing, household appliances, new vehicles and housing rents will be considered for the long term.
5. Scope of the CPI
Stats SA should consider the implications of moving to a “domestic” concept. We do implicitly follow the domestic concept (meaning we cover all expenditure in the borders of the country irrespective of the nationality of the spender). Expenditure of nationals abroad is excluded from the scope of the CPI. To properly implement this we need to estimate the expenditure of tourists while in South Africa. We will explore sourcing data from the tourism satellite account. We will formally state in the revised methods and sources that we follow the domestic concept.
6. Coverage of price survey
Subject to issues raised about the CPI being used for multiple purposes, rebalances the sample of goods and services priced so that it better reflects tourist expenditure.
Tourist expenditure is covered through hotel and restaurant prices. Curios and tourist T-shirts are currently being included in the HCPI survey. In the long term we will explore the possibility of including entrance to tourist sites such as historical monuments, national parks etc.
7. Coverage of rural areas and the informal sector
a) Statistics South Africa examines the consequences of excluding the expenditure of rural households and the prices they pay and takes a view as to the cost-benefits of their inclusion in the future, drawing on and building on the price information they already collect on a regular basis for the Agricultural Marketing Council.
b) Rural CPIs should be based on purchases and prices paid in rural areas. The rural food prices Statistics South Africa currently collect for the National Agricultural Marketing Council can be a starting point but the rural basket needs to be balanced to reflect to expenditure of the rural population. In this connection the underlying conceptual issue also needs to be addressed of whether the rural CPIs are trying to measure the change in the cost-of-living in rural areas or the cost of goods and services purchased in rural areas.
c) Research should (also) be undertaken to inform decisions on whether markets (in rural areas) and street-traders should be included in the CPI. In principle a CPI should cover these.
Firstly, the current IES will include information on place of purchase which will allow us to determine which products rural households purchase where. We are hoping to use the prices of the 25 different food items in the calculation of the Rural CPI from 2013. This will affect only the CPI aggregates for rural areas and total country, as the Headline index is for All Urban Areas.
Secondly, we are researching possible methods for including food prices from informal traders in to the CPI from 2013.
8. Tariff pricing
Statistics South Africa should undertake a methodological review of its approach to tariff prices and introduce weights.
We are researching this with a view to implementation.
Long-term actions
9. Coverage of weights
Statistics South Africa researches the impact of excluding institutional households with a view to their subsequent inclusion in the index. Stats SA’s master sample does not cover institutional households (although worker hostels are included). Engagement will be entered into with the methodology division on this issue as it is not directly in our control. We do not believe that any significant bias in the weights arises from this omission.
10. Updating of weights
Statistics South Africa considers options for the more systematic monitoring of CPI weights and baskets; and introduces operational rules, including thresholds, for the introduction of newly significant and new types of goods and services between each re-basing.
Implementation of this recommendation is difficult given the broad manner in which the products are described and the detailed level at which weights are determined. However, we would rather treat this with more frequent reweighting than by tinkering on the margins in between.
11. Sample design
Statistics South Africa investigates introducing: stratification by outlet type on receipt of information from the 2010/11 IES; a systematic local enumeration of shopping locations to introduce improved controls for the purposive selection of outlet in locations; continuous and systematic monitoring of the outlet sample to take account of e.g. new shopping centres.
The stratification of the outlet sample is a longer-term exercise that will require dedicated funding and project management. While this approach is certainly the ideal, we do not believe there is any significant bias arising from our purposive sampling methods. We will conduct desk research on the possible methods of stratifying the outlet sample and then make a decision.
12. Owner-occupied housing
Statistics South Africa re-investigates the use of imputed rents and whether it produces a significantly different result, compared with the net acquisition approach, in terms of inflation as measured by the CPI. It should do this by using the results of the pilot survey of housing developments which was run for a year.
If the two approaches result in a significant difference in measured inflation then Statistics South Africa should consider further developing its data sources and moving to a net acquisition approach, which is more compatible with an index constructed as a microeconomic indicator.
The pilot survey for net acquisitions needs to run for at least two more years before a clear decision can be made on this matter. The biggest problem with the survey to date is the small number of prices that we manage to obtain. As the South African CPI is a dual purpose (macro economic indicator and cost of living measure) it is not clear that net acquisitions is preferable.
13. Using best practices from the HCPI: Long term
Statistics South Africa introduces into the CPI relevant best practices adopted in the Harmonised Consumer Price Index (a SADC/ COMESA project). This should be done as soon as practicable.
The HCPI has a few methods that are different to the South African CPI. All are covered in recommendations above.
F. Operational recommendations
The following recommendations are aimed at improving the operational effectiveness and efficiency of the price collection, data editing and compilation process.
Short-term actions
1. Product expertise
To help facilitate this Statistics South Africa should consider introducing a “Commodity Specialists” function at Head Office e.g. by allocating this role to staff already working on the CPI – for instance to those involved in editing and compilation. These staff would collate market information on a particular COICOP grouping such as food or clothing to help inform and make recommendations on the updating of the CPI basket.
This will be done. Synergies with the PPI and other product-based surveys will be explored.
2. Price collection form
Statistics South Africa should in the short term:
a) Review the design of the Pricing Form with a view to streamlining it. E.g. using tick boxes for price collectors to confirm no changes in item characteristics and for some “field messages”.
b) Asses the value and accuracy of some of the information being collected and take a view on whether it is worth collecting. E.g. some “field messages” (again).
c) Look to improving the accuracy of the pre-printed information, particularly where it counts, and provide price collectors with more guidance on how to complete the forms.
The price collection form is designed to ensure high quality collection. Getting the price collector to actually write the price, rather than tick that it is the same, is designed to prevent short cuts being taken which may result in an incorrect price being recorded.
We will review the requirements of the field message and decide if changes should be made that will improve efficiency but not compromise quality. Errors in the pre-printed information on forms result from data capturing errors. We have recently introduced a rigorous quality control system for data capturing and results show that the quality of capturing is improving. Price collectors do receive training each year and more often if specifically required. The price collection manual will be reviewed and updated as part of the update of the basket.
3. Editing procedures
In the short term, Statistics South Africa should review its current editing and validation procedures with a view to ensuring that resources are directed at the most problematic areas which have the biggest potential impact on the index. This should be informed by the routine monitoring of “outcomes” at each editing stage e.g. the error rates of price collectors; the error rates for forms submitted to Head Office after the supervisor has quality assured the completed pricing forms – all classified by the type of error and its potential impact on the published indices. This should build on the work already being done.
The editing procedures were reviewed and updated during 2010. Investigation will take place during 2012 to identify further areas for improvement.
4. Management structure
The creation of a “Production Manager” role with responsibility for all quality and performance management issues relating to all operational processes associated with the production of the CPI from data collection, to data capture to editing and compilation (but not analysis or methodology). The role would cover responsibility and authority for all production issues along the statistical value-added chain and would also be responsible for some staff management and training (delivery, co-ordination and management).
We have implemented this recommendation.
5. Quality control
The data capture of centrally-collected prices should always be checked by a second person, together with the calculation of the corresponding indices.
This will be implemented.
6. Validation of results
Statistics South Africa continues to check each month that the CPI is largely consistent with other price statistics and with national accounts and also with general expectations from market reports, media articles etc. and also undertakes more rigorous and in-depth research and analysis. This should be done by strengthening the analytical function.
A clearance document which accompanies the monthly release does compare the CPI figures with other relevant and available information. The research unit does produce analytical documents which discuss the credibility and validity of the CPI results. Output of the research unit depends on task prioritisation and resource availability.
7. Price collection training for Head Office Staff
Staff at Head Office should be given experience of price collection and consideration should be given to Head Office taking responsibility for the monthly collection of prices at a nearby shopping area.
This will be implemented during 2012.
Medium-term actions
8. Turn over of high tech items
Statistics South Africa should review the SPD (Structured Product Descriptions) & its replacement strategy for high-tech high-turnover items to satisfy itself that the item sample is not subject to undue aging.
This will be reviewed.
9. Quality management
The extension of Quality and Performance management to all Head Office activities – this would include compilation as well as data capture checking and data editing.
Activities could build upon the monthly indicator meetings which are held to discuss issues that have arisen from the current month’s production and to take a forward look at issues relating to the next month.
Staff should be encouraged to actively participate in these meetings, possibly broadening the membership.
We do monitor the performance of data capturing and editing with a view to improving quality. The system is currently being refined.
10. Team building and communication
Consideration should be given to team building initiatives for price collectors and their supervisors e.g. visits to Head Office (or more regular visits by Head Office to the regions) and monthly newsletters which not only raise operational issues but also give news from Head Office & the latest CPI figures.
A training/team building session normally takes place each year. More regular visits to the field staff will be undertaken within the constraints of funding. We agree with the proposal of the newsletter. We have struggled in the past to identify a suitable resource to take responsibility for this but will prioritise this.
11. Digital data collection
Statistics South Africa should investigate the possibility of introducing handheld computers for collecting prices. This option would: increase quality control at the point of data entry; help identify anomalies and ensure that prices are correct; increase speed & improved timeliness of the CPI; lead to a more efficient, better use of resources.
We are currently in the planning phase for this and intend to pilot during 2012.
12. Rental survey
Statistics South Africa should re-consider whether personal visits every quarter are necessary. After a personal visit for initiation, the next three quarters’ prices should be collected either via email or over the telephone - timing by prior arrangement. If the latter, an email should be sent listing the properties for which rents are required to allow the respondent to dig out the paper work before the telephone call. This seems to have worked for auditors.
The “desirability of the location” (up-market/standard/other) should be one of the price-determining variables used in the initial selection process.
The “additional” services included in the rent, such as water and electricity,
should be excluded prior to the calculation of imputed rents. Using the telephone for follow up interviews of letting agents will be piloted during 2012.
Using desirability of location as a sample selection criterion presupposes that we know the distribution and number of different types of homes, which we do not. The data from the 2011 Census will be analysed to assess whether this might be practical in the future.
Where possible the additional services are excluded from the rental price. However, in many rental agreements these costs are included but not specified.
Long-term actions
13. External ISO quality assessment
Look at the possibility of an ISO 9000 assessment carried out by trained external auditors. This would entail a detailed check of whether the procedures which are laid down are followed in practice (i.e. that all staff involved in the CPI actually do what they say they do and should do). ISO 9000 is an audit of the processes not of the people and provides a list of noncompliance’s for action. Statistics South Africa may want to consider applying formally for the ISO “kite mark” but should note that this entails regular reassessments which may not add so much value as the first assessment. It should also be noted that from a presentational perspective, it may be difficult to withdraw from the ISO “kite mark” process once awarded.
The CPI is currently undergoing an assessment in terms of the SASQAF. This has been a time consuming processes and the value for the CPI operations is yet to be proven. We do not see a need for further external validation of the quality of the index at this point.
Recommendations from independent peer review of the consumer price index and Stats SA’s responses to these
