Numeric Data Entry and Rounding

Original Document

Scanned document (image-only PDF)

Extracted Text

Searchable text extracted from PDF

1.0 Purpose 
 
 The purpose of this procedure is to define the process for rounding numbers and significant
 figures determination for data reporting. 
 
 2.0 Scope 
 
 This procedure describes the application of significant figures for the proper data reporting.
 
 3.0 Responsibility 
 
 3.1 It is the responsibility of all employees recording data to comply with this procedure for
 significant figures and rounding. 
 
 3.2‘ It is the responsibility of each Supervisor/Manager, Data Reviewer, and/or QC to ensure
 
 that this procedure is properly followed. 
 
 4.0 Definitions 
 
 4.1 Significant Figures — The number of significant figures in a measured quantity is the
 number of digits that are known accurately, plus one that is in doubt. Zeros that appear
 
 to the left of the first non-zero digit are placeholders and are not considered significant.
 
 Zeros are located to the right of the first digit may be considered significant.
 
 4.2 QC — Quality Control 
 
 5.0 References 
 
 N/A 
 
 
 

[SOP 

 Standard Operating Procedure SOP No | Rev 
 A-101 4 Page 2 of 4 
 Numeric Data Entry and Rounding 
 
 6.0 Procedure 
 
 6.1 Significant Figures 
 
 6.1.1 A significant figure is any of the digits 0 through 9, except leading zeros and
 
 some trailing zeros. 
 
 6.1.2 Consider any non-zero digit as significant, e.g. 6.28 contains three significant
 
 figures. 
 
 6.1.3 Consider any zero located between non-zero digits as significant regardless of
 
 the position of the decimal point. 
 
 Example: 3.048 contains four significant figures. 
 
 6.1.4 Consider leading zeros as placeholders to locate the decimal point. A leading
 zero is any zero located to the left of a non-zero digit except when it is between
 
 two non-zero digits. 
 
 Example: 0.0789 contains three significant figures and two leading zeros.
 
 6.1.5 It is not usually possible to determine if a trailing is significant by inspection of
 the result. A trailing zero is any zero that follows a whole number. For example,
 
 for the number 750, the zero may be significant but since it cannot be deleted,
 
 there is no way to make this determination without additional information. If
 this number were reported as 750.0, it contains four significant figures.
 
 6.2 Rounding 
 
 6.2.1 Observed or calculated values usually contain more significant figures than are
 
 in the established acceptance criteria or specification. 
 
 6.2.2 Round off an observed or calculated final result to the number of places that is
 
 in agreement with the established acceptance or specification before the result is
 compared to the acceptance criteria or specification and reported.
 
 

[SOP 

 Standard Operating Procedure SOP No | Rev 
 A-101 4 Page 3 of 4 
 Numeric Data Entry and Rounding 
 
 6.2.3 Numeric data should be recorded to the same number of significant digits that
 the specification is stated. 
 
 Example: Yield specification is 95.0% - 105.0%. Record value to one (1)
 decimal place, i.e. 98.5% or if the component is 0.4268kg the weight
 
 should be reported to the fourth decimal place, e.g. 0.4268kg (4
 
 significant figures). 
 
 6.2.4 For final results, when rounding off, only one digit in the decimal place to the
 
 right of the last place in the established specification should be considered.
 
 Example: If the specification is 98.0% - 110.0% and the observed or calculated
 
 value is 98.045, consider only 98.04 when making rounding
 decisions. The number reported will be 98.0%. 
 
 6.20) If the digit in the decimal place to the right of the last place in the established
 specification is less than 5, it is eliminated and the preceding digit is unchanged.
 
 Example: The specification states 95.0% - 105.0% but the observed or
 calculated value is 98.04%. The number reported will be 98.0%.
 
 6.2.6 If the digit in the decimal place to the right of the last place in the established
 specification is 5 or greater, the preceding digit is increased by one.
 
 Example: The specification states 95.0% - 105.0% but the observed or
 
 calculated value is 98.05%, the number reported will be 98.1%. (or)
 The specification states 95.0% - 105.0% but the observed or
 
 calculated value is 98.88, the number reported will be 98.9%. (or)
 The specification states 95.0% - 105.0% but the observed or
 
 calculated value is 98.99, the number reported will be 99.0%.
 
 

[SOP 

 Standard Operating Procedure SOPNo | Rev pela 
 Numeric Data Entry and Rounding oa : 
 
 7.0 Revision History 

| Rev | Date | Description of Changes | CCR # | By |
|-----|----------|------------------------|-------|----|
| 0 | 06/09/10 | New. - - I 05/13/13 Scheduled review: changed the format; updated SOP. 13-0376 V. Iltcheva | - | - |
| 2 | 09/09/15 | Scheduled review: updated SOP format. | 15-0567 | V. Iltcheva |
| 3 | 01/02/19 | Scheduled review: no changes made. | 19-0003 | K. Burris |
| 4 | 06/02/22 | Scheduled review: updated format and logo. CC- | 22-0249 | K. Burris |