|Year : 2012 | Volume
| Issue : 4 | Page : 463-468
A quick inexpensive laboratory method in acute paracetamol poisoning could improve risk assessment, management and resource utilization
S.M.D.K. Ganga Senarathna1, Shalini S Ranganathan2, Nick Buckley3, S.S.S.B.D. Preethi Soysa4, B. M. Rohini Fernandopulle2
1 Department of Pharmacology, Faculty of Medicine, University of Colombo, Sri Lanka; South Asian Clinical Toxicology Research Collaboration; Pharmacy Program, Department of Medical Education and Health Sciences, Faculty of Medical Sciences, University of Sri Jayewardenepura, Sri Lanka; School of Pharmacy, Curtin University, Perth, Australia
2 Department of Pharmacology, Faculty of Medicine, University of Colombo, Sri Lanka
3 South Asian Clinical Toxicology Research Collaboration, Sri Lanka; Faculty of Medicine, University of New South Wales, Sydney, Australia
4 Department of Biochemistry and Molecular Biology, Faculty of Medicine, University of Colombo, Sri Lanka
|Date of Submission||29-Aug-2011|
|Date of Decision||27-Feb-2012|
|Date of Acceptance||30-Apr-2012|
|Date of Web Publication||3-Aug-2012|
S.M.D.K. Ganga Senarathna
Department of Pharmacology, Faculty of Medicine, University of Colombo, Sri Lanka; South Asian Clinical Toxicology Research Collaboration; Pharmacy Program, Department of Medical Education and Health Sciences, Faculty of Medical Sciences, University of Sri Jayewardenepura, Sri Lanka; School of Pharmacy, Curtin University, Perth, Australia
Source of Support: None, Conflict of Interest: None
Objectives: Acute paracetamol poisoning is an emerging problem in Sri Lanka. Management guidelines recommend ingested dose and serum paracetamol concentrations to assess the risk. Our aim was to determine the usefulness of the patient's history of an ingested dose of >150 mg/kg and paracetamol concentration obtained by a simple colorimetric method to assess risk in patients with acute paracetamol poisoning.
Materials and Methods: Serum paracetamol concentrations were determined in 100 patients with a history of paracetamol overdose using High Performance Liquid Chromatography (HPLC); (reference method). The results were compared to those obtained with a colorimetric method. The utility of risk assessment by reported dose ingested and colorimetric analysis were compared.
Results: The area under the receiver operating characteristic curve for the history of ingested dose was 0.578 and there was no dose cut-off providing useful risk categorization. Both analytical methods had less than 5% intra- and inter-batch variation and were accurate on spiked samples. The time from blood collection to result was six times faster and ten times cheaper for colorimetry (30 minutes, US$2) than for HPLC (180 minutes, US$20). The correlation coefficient between the paracetamol levels by the two methods was 0.85. The agreement on clinical risk categorization on the standard nomogram was also good (Kappa = 0.62, sensitivity 81%, specificity 89%).
Conclusions: History of dose ingested alone greatly over-estimated the number of patients who need antidotes and it was a poor predictor of risk. Paracetamol concentrations by colorimetry are rapid and inexpensive. The use of these would greatly improve the assessment of risk and greatly reduce unnecessary expenditure on antidotes.
Keywords: Acute poisoning, paracetamol concentration, risk assessment
|How to cite this article:|
Senarathna SG, Ranganathan SS, Buckley N, Soysa SP, Fernandopulle BR. A quick inexpensive laboratory method in acute paracetamol poisoning could improve risk assessment, management and resource utilization. Indian J Pharmacol 2012;44:463-8
|How to cite this URL:|
Senarathna SG, Ranganathan SS, Buckley N, Soysa SP, Fernandopulle BR. A quick inexpensive laboratory method in acute paracetamol poisoning could improve risk assessment, management and resource utilization. Indian J Pharmacol [serial online] 2012 [cited 2021 Jun 17];44:463-8. Available from: https://www.ijp-online.com/text.asp?2012/44/4/463/99305
| » Introduction|| |
Acute paracetamol poisoning is an emerging problem in Sri Lanka. In recent years, there have been dramatically increased numbers of admissions with acute paracetamol poisoning to the National Hospital of Sri Lanka. There were only 35 cases in 2003 and 515 cases in 2005.  A similar marked increase has also been noted over the same period in North Central Province hospitals. 
Acute paracetamol poisoning does not cause many early clinical signs. Hepatotoxicity develops more than a day after the acute ingestion. Therefore risk assessment is usually based on the history of ingested dose of paracetamol and serum paracetamol concentrations in relation to the time of ingestion. Clinical signs are only useful at a much later stage. 
Most evidence-based guidelines recommend measuring serum paracetamol concentration for risk assessment and it is the standard practice of most emergency departments in the developed world for patients with a suspected paracetamol overdose.  Patients with serum paracetamol concentrations above a line on the nomogram are considered to have a risk of hepatotoxicity. Two distinct but similar nomograms are in common use; one is the standard treatment line in the UK and starts at 200 mg/l at 4 hours.  The standard US and Australian line is parallel to this line but 25% lower. , Estimation of serum paracetamol concentration is not yet available in the state sector hospitals in Sri Lanka or in many other resource poor health services throughout the developing world. Risk can then only be assessed from the history of ingested dose. The threshold dose used to indicate the need for antidotes was >150 mg/kg up to 2007,  but recently this was increased to >200 mg/kg in Sri Lanka.  There are no data to support using the dose reported by patient alone in assessing the risk of hepatotoxicity.
Patients with a risk of hepatotoxicity are treated with an antidote, N-acetylcysteine or methionine. These antidotes, especially N-acetylcysteine, are expensive. For example, it costs US$140 to treat a 60 kg patient. Thus there are substantial savings possible from using more accurate risk assessment to correctly identify patients who require antidotes.
There are different methods of measuring serum paracetamol concentrations, which could be adopted. The common methods used in the estimation of paracetamol include use of immunoassay  High Performance Liquid Chromatography (HPLC),  Gas Liquid Chromatography (GLC),  and UV spectrophotometric methods. ,
HPLC is considered as the preferred method for estimation of paracetamol due to its high sensitivity, precision, and freedom from interference.  Both HPLC and GLC cannot be widely adopted in resource-limited settings due to high capital expenditure on the purchase of the instrument, high running and maintenance costs, and the time taken for the procedure. The immunoassay method is used in private laboratories in Sri Lanka; however, the cost per test is considerable by local standards (US$18-22).
Spectrophotometric methods were the first methods used in estimation of paracetamol.  With the emergence of new analytical methods, the spectrophotometric methods were mostly abandoned. However, we considered these methods may still be suitable for resource limited settings. On review of the different spectrophotometric methods utilized in the estimation of paracetamol, the colorimetric method established by Glynn and Kendal appeared to be a suitable method as it could be performed rapidly and there is minimal interference from other color producing substances. ,
Hence, this study in Sri Lankan patients with acute paracetamol poisoning aimed to:
- Examine the accuracy of using reported dose ingested to assess the need for antidotal therapy.
- Determine the accuracy of paracetamol concentrations estimated by colorimetric method compared with HPLC, the reference method.
- Examine the accuracy of risk categorization by the colorimetric method in regard to the need for antidotal therapy.
| » Materials and Methods|| |
A cohort of consenting patients with a history of acute paracetamol poisoning was studied at the National Hospital of Sri Lanka. Ethical approval was obtained from the National Hospital of Sri Lanka and the Faculty of Medicine, Colombo Ethical Review Boards. Verbal consent was obtained from the patients.
Patients were included into the study if they could provide reasonably accurate estimates of the amount of paracetamol ingested and the time of acute ingestion. Patients who had immediately vomited the tablet particles were excluded from the study. The first 100 patients who met the inclusion criteria were studied. Blood (5 ml) was taken for serum paracetamol concentrations at 4 hours postingestion, or at admission if presenting later than this. The amount of paracetamol ingested (mg/kg) and the time of acute ingestion were recorded.
Blood samples were centrifuged and serum sent for analysis to two laboratories. One laboratory used the reference HPLC method  and the other laboratory used the test colorimetric technique. , Instruments were calibrated and methods were revalidated in the laboratory before the estimation of patient samples.
This was carried out on the basis of Glynn and Kendal method with a few modifications to the volumes of reagents and concentrations in order to increase the absorbance obtained. ,
Reagents and instruments: A Shimadzu UV 120-02 UV visible spectrophotometer (Shimadzu Corporation, Kyota, Japan) was used to read the absorbance. Centrifugation of samples in centrifuge tubes was done in a Kubota 6500 centrifuge (Kubota Corporation, Tokyo, Japan). A Gallenkamp spinmix (Gallenkamp Service organisation, Wesy, Sussex, RH, 102RE UK) was used for mixing.
Procedure: Test, standard, and blank samples of 1 ml were transferred into centrifuge tubes containing trichloroacetic acid (15%; 1 ml). After a brief vortex, the tubes were centrifuged for 3000g for 5 minutes. Supernant (900 μl) was pipetted out into a test tube and sodium nitrite (15%; 1 μl) was added followed by hydrochloric acid (0.5 ml; 6N). The tubes were closed using a rubber bung and the contents were allowed to stand for 2 minutes. Sulfamic acid (18%; 1 ml) was added carefully followed by sodium hydroxide (15%; 2.5 ml). The absorbance was measured at 430 nm against a reagent blank using a spectrophotometer.
A calibration curve was obtained and inter-batch (between day precision) and intra-batch variation (within day precision) and accuracy were calculated to validate the two analytical methods in both laboratories.  The Receiver Operating Characteristic (ROC) curve was used to evaluate the usefulness of the history of ingested dose (across the range) in estimating the risk of hepatotoxicity (an area under the ROC curve = 0.5 indicates no discriminatory value, >0.75 is generally regarded as a threshold for clinically useful information, while 1 indicates a perfect test).  In doing that paracetamol level by HPLC plotted on the nomogram was considered as the gold-standard predictor of risk of hepatotoxicity. The figures were created using GraphPad Prism v 5.00 for Windows, GraphPad Software, San Diego, CA, USA (www.graphpad.com).
The correlation coefficient was used to measure agreement between paracetamol concentrations by HPLC and colorimetric methods. Cohen's Kappa was used to quantify agreement with respect to risk categorization on the nomogram (the standard treatment line used in Sri Lanka is identical to that used in the UK). Kappa 0.2 to 0.4 is considered fair, 0.4 to 0.6 is considered moderate, >0.6 is considered good while kappa = 1 indicates complete agreement.  Sensitivity, specificity, and predictive values were also derived to compare the risk categorization by dose ingested >150 mg/kg and colorimetry. 
| » Results|| |
Characteristics of the study population
A 100 patients with a mean age of 22 ± 6 years and mean body weight of 47 ± 11 kg were included in the study. The male to female ratio was 1:3, the mean reported ingested dose of paracetamol was 343 ± 225 mg/kg and the median time between acute ingestion and blood collection was 7 hours.
Establishment of analytical methods in the laboratory
The peak area ratios for standard serum paracetamol samples were plotted against the respective concentrations to obtain the calibration curve for the HPLC method. The curve was linear over 1-300 mg/l and the correlation coefficient was 0.998 for the HPLC.
The absorbance for standard serum paracetamol concentrations were plotted against the respective concentrations to obtain the calibration curve for the colorimetry method. The calibration curve was linear from 10 to 300 mg/l and the correlation coefficient for the standard curve was 0.98. Both test methods had less than 15% intra- and inter-batch variation as shown in [Table 1] and therefore both test methods fulfill standard validation criteria for precision and accuracy. 
|Table 1: Intra- and inter-batch variation and accuracy for colorimetric and HPLC methods|
Click here to view
History of ingested dose as a diagnostic tool
The ROC curve for reported ingested dose of paracetamol is shown in [Figure 1]. The area under the ROC curve was 0.578. Neither the conventional dose point of >150 mg/kg, nor any other point provided clinically useful information in terms of risk assessment.
|Figure 1: ROC curve for history of ingested dose of paracetamol in predicting concentration above nomogram line by HPLC|
Click here to view
When paracetamol levels by HPLC is taken as the reference, a threshold dose of >150 mg/kg had very low specificity and low positive predictive values [Table 2], meaning many patients would be judged incorrectly to be at risk and might unnecessarily be given antidotes.
|Table 2: Sensitivity, specifi city and predictive values of colorimetry and dose >150 mg/kg [HPLC concentration above UK nomogram line (standard treatment line) taken as gold standard]|
Click here to view
Comparability of paracetamol concentration by colorimetry as a diagnostic tool
The correlation coefficient between the serum paracetamol concentrations by HPLC and by colorimetric method was 0.85 [Figure 2]. The agreement on risk categorization on the nomogram using serum paracetamol concentration by the two analytical methods was good (Kappa = 0.62). The number of patients having paracetamol concentrations above the higher risk line of the nomogram was 27 by the HPLC method and 31 by the colorimetric method. The colorimetric method had a high negative predictive value indicating only few patients would not receive treatment when indicated using this method [Table 2].
|Figure 2: Correlation between paracetamol concentrations by HPLC and colorimetry (dotted line = line of unity)|
Click here to view
This is further demonstrated in [Figure 3]. Each pair of serum concentrations of paracetamol (by HPLC and colorimetric methods) are plotted against time in relation to the two nomogram treatment lines. The reference serum concentration by the HPLC method is represented by a horizontal line. The vertical line then indicates the extent and direction of difference. It can be seen that there were only six instances where HPLC serum concentration was over the UK nomogram and the respective colorimetric concentration was below (these cases represented as bold lines). Only two of these were also below the lower treatment line.
|Figure 3: Serum concentration of pairs of HPLC and colorimetric paracetamol assays from 100 patients in relation to standard and lower nomogram treatment line. Horizontal lines indicate HPLC result. Vertical lines the difference. The six differences between methods potentially leading to under-treatment are shown in bold.|
Click here to view
The time between blood collection and the final report was 180 minutes for the HPLC method and 30 minutes for colorimetric method. The estimated cost of measuring paracetamol concentration using the HPLC method was US$20 compared with US$2 for the colorimetric method.
| » Discussion|| |
Modifications to Glynn and Kendal's colorimetric method have been published by Mace and Walker, 1976; Archer and Richardson, 1980; Barker and Jacobs, 1982; Longlands and Wiener, 1982; and Shihana et al., 2010. ,,,, All these modifications are minor. For example, [Table 3] shows the difference among the three of these. Validation of the colorimetric method used in our study showed a r2 = 0.98 (0- 300 mg/l), which is comparable to that reported by Shihana et al., 2010 (linearity between 0 and 400 mg/l concentration with r2 = 0.99).  Commercial diagnostic assay test kits for quantification of plasma paracetamol have also been developed based on Glynn and Kendal's chemical colorimetric method. , Our study has further proven the reliability of this simple colorimetric method in the estimation of plasma paracetamol by showing a good agreement with that of the HPLC assay results. This colorimetric test is reliable, reproducible, and can be rapidly performed in the most basic laboratory. It is very inexpensive and substantial savings could result from making this test more widely available and reducing the unnecessary use of expensive antidotes.
|Table 3: Comparison of reagent volumes and concentrations used for colorimetric assay compared with Glynn and Kendal, 1975 and Shihana et al., 2010|
Click here to view
We found the ingested dose of >150 mg/kg had very poor specificity (5%) despite high sensitivity (89%). As a consequence of this low specificity, by this criterion the majority of our patients were judged to be at risk of hepatotoxicity despite not having any significant risk according to the nomogram. These patients still received antidotes estimated to cost around US$140 per patient. In countries such as ours with limited health budgets the amount potentially saved per person by using this test is more than the annual health expenditure per capita. 
The area under the ROC curve is widely used as a measure of a diagnostic test's discriminatory power. An area under the curve of 0.58 of the ROC curve indicates that history of ingested dose of paracetamol given by the patient is not useful as a risk assessment tool in acute paracetamol poisoning.  This indicates that there is little point in modifying the ingested dose cut-off point from the currently used dose of >150 mg kg. However, the high sensitivity and negative predictive value indicates this may still be regarded as a reasonable threshold indicating the need for further evaluation with a colorimetric paracetamol concentration.
The interpretation of the performance of diagnostic tests (including history) depends greatly on the context. Previous studies from the developed world have usually focused on negative predictive value and high sensitivity of ingested dose. For example, the study by Waring et al., in 2008 concludes that the stated dose ingested is a reliable indicator for risk of subsequent toxicity because the negative predictive value was very high (97%). This was despite a positive predictive value of only 32% for their preferred threshold of 12 grams.  This confirmed many previous reports that those reporting doses less than between 150 and 200 mg/kg or between 8 and 12 grams ingested had very low risk.  The assumption in these studies is that a confirmatory test is usually available and that unnecessary recommended use of the antidote is without great consequence. We confirm these findings of the high negative predictive value and low positive predictive value of history of ingested dose in a Sri Lankan context (although both values were lower than reported previously). However, the consequences of a low positive predictive value (18%) in this setting are substantial. This currently means that in many hospitals, four out of five patients are transferred to hospitals several hours away and receive an expensive antidote unnecessarily. The cost of this practice to the health system was estimated to exceed 1.4 million Sri Lankan rupees in 2004 only for those patients treated at the National Hospital of Sri Lanka. 
In acute paracetamol poisoning management antidotes should be administered within 8 hours of the acute ingestion to obtain the maximum efficacy. , Therefore the colorimetric method has an advantage of producing results more rapidly, as well as being ten times less expensive than the HPLC methods. Thus, even where HPLC facilities are available, the colorimetric method might be more useful for routine paracetamol estimation in the developing countries.
There were six patients with HPLC concentrations above the standard nomogram treatment line who were judged at risk by HPLC and not at risk by the colorimetric method [Figure 3]. However, this would be reduced to just two if the lower treatment line was used to assess colorimetric assays. Using the more widely used nomogram with a lower treatment line (starting at 150 mg/l at 4 hours) would provide a greater margin of safety. This would increase by around 10% the number of people deemed to require antidotes but would still lead to substantial reduction in the current unnecessary expenditure on antidotes.
When colorimetric assay results were assessed using the lower treatment line, only 37% of the patients required antidote treatment. A study by Shihana et al., in a different setting in Sri Lanka found similar results in this regard, the number of patients requiring antidotal therapy being only 42%.  Both studies showed the use of simple colorimetric techniques rather than history of ingested dose in assessing risk could result in substantial reduction in unnecessary expenditure on antidotes.
| » Conclusions|| |
Paracetamol concentrations measured by the colorimetric method were rapid, reliable, and inexpensive. They had good agreement with the concentrations measured by the HPLC method. The method is suitable for risk assessment in acute paracetamol poisoning in the Sri Lankan and other developing world settings. History of dose ingested alone greatly over-estimated the number of patients who need antidotes. Therefore, using this test for risk assessment instead of the history of ingested dose will prevent much unnecessary treatment and lead to substantial cost savings even if the lower Rumack-Matthew nomogram line is used.
| » Acknowledgement|| |
The authors thank Professor Andrew Dawson for providing useful initial comments and reading and commenting on the final copy and Professor Nick Freemantle for providing advices on statistics. The authors would also like to thank all the consultant physicians at the National Hospital of Sri Lanka for allowing their patients to be involved in the study and Mrs. WDC Ratnayake, Mr. HH Ajith Lionel, and Mr. KSJ Kolambage for assisting to measure serum paracetamol concentrations. South Asian Clinical Toxicology Research Collaboration (SACTRC) is funded by the Wellcome Trust/National Health and Medical Research Council International Collaborative Research Grant GR071669MA.
| » References|| |
|1.||Senarathna SM, Sri Ranganathan S, Dawson AH, Buckley N, Fernandopulle BM. Management of acute paracetamol poisoning patients in a tertiary care hospital. Ceylon Med J 2008;53:89-92. |
|2.||Fernando R. Management of poisoning. Sri Lanka: National Poison Information Centre, National Hospital of Sri Lanka, State Printing Corporation; 2007. |
|3.||Hartington K, Hartley J, Clancy M. Measuring plasma paracetamol concentration in all patients with drug overdoses; development of a clinical decision rule and clinicians willingness to use it. Emerg Med J 2002;19:408-11. |
|4.||Paracetamol information centre in collaboration with the British Association for Accident and Emergency Medicine and the Royal College of Paediatrics and Child Health. In: Guidelines for the management of paracetamol overdose. London: Paracetamol Information Centre; 1999. |
|5.||Daly FF, Fountain JS, Murray L, Graudins A, Buckley NA. Guidelines for the management of paracetamol poisoning in Australia and New Zealand - explanation and elaboration a consensus statement from clinical toxicologists consulting to the Australasian poisons information centres. Med J Aust 2008;188:296-301. |
|6.||Rumack BH. Acetaminophen hepatotoxicity the first 35 years. Clin Toxicol 2002;40:3-20. |
|7.||Fernando R. Management of poisoning. Sri Lanka: National Poison Information Centre, National Hospital of Sri Lanka, State Printing Corporation; 2002. |
|8.||Jones AL, Jarvie DR, Simpson D, Prescott LF. Comparison of assays for measuring plasma paracetamol. Br Med J 1998;316:475. |
|9.||Sood SP, Green VI. Routine methods in toxicology and therapeutic drug monitoring by high-performance liquid chromatography. I. Rapid method for determination of acetaminophen in plasma, including a STAT procedure. Ther Drug Monit 1987;9:248-54. |
|10.||Chambers RE, Jones K. Comparison of gas chromatograpic and colorimetric method for determination of plasma paracetamol. Ann Clin Biochem 1976;13;433- 34. |
|11.||Glynn JP, Kendal SE. Paracetamol measurement. Lancet 1975;1:1147-8. |
|12.||Flanagan RJ, Braithwaite RA, Brown SS, Widdop B, Wolff de FA. Basic analytical toxicology, monographs-analytical and toxicology data. England: Geneva World Health Organization in collaboration with the United Nations Environmental Programme and the International Labour Organization: Macmilian/Clays; 1995. p. 193-5. |
|13.||Soysa P, Kolambage S. Rapid HPLC/UV method for analysis of urinary and plasma/serum paracetamol concentrations. J Natn Sci Foundation Sri Lanka 2010;38:131-7. |
|14.||Moffat AC, Osselton MD, Widdop B, editors. Clarke's analysis of drugs and poisons, ed 6. London: Pharmaceutical press; 2006. |
|15.||Fan J, Upadye S, Worester A. Understanding Receiver Operation Characteristics (ROC) curves. Can J Emerg Med 2006;891:19-20. |
|16.||Landis JR, Koch GG. An application of hierarchical kappa type sta'tistics in the assessment of majority agreement among multiple observers. Biometrics 1977;33:363-74. |
|17.||Loong TW. Understanding sensitivity and specificity with the right side of the brain. Br Med J 2003;327:716-9. |
|18.||Mace PF, Walker G. Salicylate interference with plasma Paracetamol method. Lancet 1976;2:1362 |
|19.||Archer CT, Richardson BA. An improved colorimetric method for the determination of plasma paracetainol. Ann Clin Biochem 1980;17:45-6. |
|20.||Barker DE, Jacobs G. Paracetamol estimation: A new approach to reducing salicylateinterference. Ann Clin Biochem 1982;19:120-4. |
|21.||Longlands MG, Wiener K. Minimisation of salicylate interference in the Glynn and Kendal paracetamol procedure. Ann Clin Biochem 1982;19:187-90. |
|22.||Shihana F, Dissanayake D, Dargan P, Dawson A. A modified low-cost colorimetric method for paracetamol (acetaminophen) measurement in plasma. Clin Toxicol (Phila) 2010;48:42-6. |
|23.||Lancer Acetaminophen Rapid Stat Diagnostic Kit instruction, publication no. 2015201. |
|24.||Sigma Technical Bulletin No. 430 (11-82). |
|25.||WHO (World Health Organization). 2007a. Core Health Indicators 2007 [Database online]. Geneva: WHO statistical Information system; 2008. [Last updated in 2008 May] |
|26.||Waring WS, Robinson OD, Stephen AF, Dow MA, Pettie JM. Does the patient history predict hepatotoxicity after acute paracetamol overdose? QJM 2008;101:121-5. |
|27.||Senarathna SM, Sri Ranganathan S, Fernandopulle BM. Cost-outcome description of management of patients with acute paracetamol poisoning [Abstract]. Ceylon Med J 2008;53(Suppl. 1):53. |
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3]
|This article has been cited by|
||Reported ingested dose of paracetamol as a predictor of risk following paracetamol overdose
| ||Y. Leang,D. M. Taylor,P. I. Dargan,D. M. Wood,S. L. Greene |
| ||European Journal of Clinical Pharmacology. 2014; |
|[Pubmed] | [DOI]|
||Recent trends in diagnosing poisoning in domestic animals
| ||Russo, R., Restucci, B., Severino, L. |
| ||Journal of Animal and Plant Sciences. 2013; 23(2): 657-665 |