Understanding Measurement & Precision
Help Questions
ACT Science › Understanding Measurement & Precision
Which measurement represents an overestimation of precision?
3.45678 m
3.4 m
3.456 m
3.45 m
Explanation
Overestimation of precision occurs when measurements are recorded with more decimal places than the instrument can reliably measure. The measurement 3.45678 m with five decimal places likely represents false precision unless using an extremely precise instrument. Most common measuring devices cannot justify this many significant figures. Appropriate precision should match the instrument's actual capability, typically fewer decimal places for standard measuring tools.
The instrument shown can measure to a precision of:
$1 , \text{kg}$
$0.01 , \text{kg}$
$0.1 , \text{kg}$
$10 , \text{kg}$
Explanation
The precision of an instrument depends on its smallest readable division or scale increment. Based on typical scale designs, an instrument measuring to $0.1 , \text{kg}$ precision would have gradations marked at $0.1 , \text{kg}$ intervals, allowing readings to be estimated to the nearest $0.1 , \text{kg}$. This level of precision is common for many laboratory balances and scales. An instrument with $0.01 , \text{kg}$ precision would require much finer gradations and more sophisticated design.
A lab group measures the length of a metal rod using a metric ruler. The ruler shows millimeter marks (smallest division = 1 mm). The rod's left end is aligned at 0.0 cm and the right end falls between 12.3 cm and 12.4 cm, about halfway.
Based on the measuring instrument shown, the measurement should be recorded to the nearest:
0.001 cm
0.1 cm
0.01 cm
1 cm
Explanation
The metric ruler has millimeter marks as its smallest division (1 mm = 0.1 cm), so measurements should be recorded one estimated digit beyond that smallest division. With 1 mm markings, the measurement should be estimated to the nearest 0.01 cm (hundredths place). Choice C correctly identifies 0.01 cm as the appropriate precision level. The rod end falling halfway between 12.3 and 12.4 cm would be recorded as approximately 12.35 cm.
A student times a reaction three times using two different stopwatches. Stopwatch X shows time to the nearest 0.1 s; Stopwatch Y shows time to the nearest 0.01 s. The data table lists the recorded times.
Which measurement set shows the greatest precision?
Stopwatch X: 12 s, 12 s, 12 s
Stopwatch Y: 12.3 s, 12.4 s, 12.3 s
Stopwatch Y: 12.34 s, 12.36 s, 12.35 s
Stopwatch X: 12.3 s, 12.4 s, 12.3 s
Explanation
Stopwatch Y shows the greatest precision because it records time to the nearest 0.01 s (hundredths), as evidenced by measurements like 12.34 s, 12.36 s, and 12.35 s. Precision refers to the smallest unit an instrument can measure, not the consistency of repeated measurements. Stopwatch X only measures to 0.1 s precision, making it less precise regardless of how consistent the readings are.
A sample’s mass is measured on a digital balance that displays 12.340 g. Based on this display, the measurement should be recorded to the nearest:
0.1 g
0.001 g
0.01 g
1 g
Explanation
The digital balance displays 12.340 g, showing three decimal places, so measurements should be recorded to the nearest 0.001 g (thousandth of a gram). Digital instruments display all digits they can reliably measure, and the last digit shown represents the instrument's precision limit. Since the display shows milligrams (thousandths), the balance can distinguish between 12.340 g and 12.341 g. Recording to only 0.01 g or 0.1 g would discard precision the instrument provides.
Two instruments are available to measure a solution’s volume: (1) a beaker with marks every $10,\mathrm{mL}$ and (2) a graduated cylinder with marks every $1,\mathrm{mL}$. A student follows the usual rule of recording one estimated digit beyond the smallest marked division.
Which measurement shows the greatest precision?

$32,\mathrm{mL}$
$32.0,\mathrm{mL}$
$32.0,\mathrm{mL}$ (from a cylinder with $1,\mathrm{mL}$ marks)
$32.00,\mathrm{mL}$
Explanation
The graduated cylinder with 1 mL marks offers greater precision than the beaker with 10 mL marks, as it allows estimation to 0.1 mL compared to 1 mL for the beaker. Precision is evaluated by the smallest marked division on each instrument and the rule of including one estimated digit beyond it, leading to more decimal places for finer graduations. The correct answer, 32.0 mL from the cylinder, shows the greatest precision because it reflects estimation to 0.1 mL consistent with 1 mL marks. Choice C implies unrealistic precision to 0.01 mL, which neither instrument supports.
A student measures the time for a cart to travel a fixed distance using two timing methods. Method 1 uses a handheld stopwatch readable to 0.01 s; Method 2 uses a phone video analyzed frame-by-frame at 30 frames/s (time step $\approx 0.033$ s). The student records one trial from each method.
Which recorded time is most consistent with appropriate precision for the method used?
Video (30 fps): 2.34 s
Stopwatch: 2.34 s
Video (30 fps): 2.340 s
Stopwatch: 2.3 s
Explanation
The handheld stopwatch is readable to 0.01 s, allowing time measurements to be recorded with two decimal places. Precision is determined by the instrument's smallest division; for the stopwatch, 0.01 s enables 2.34 s, while the video's 0.033 s frame step limits precision to about 0.03 s, making finer readings inappropriate. The recorded time 2.34 s for the stopwatch reflects appropriate precision because it aligns with the device's resolution for a single trial. Option C assigns 2.34 s to the video, which overstates precision given the coarser time step.
A student measures the diameter of a wire using a micrometer. The sleeve scale reads $2.5,\mathrm{mm}$, and the thimble aligns at $0.23,\mathrm{mm}$. The micrometer’s smallest marked increment on the thimble is $0.01,\mathrm{mm}$, and the student records one estimated digit beyond that increment.
Which recorded diameter is most appropriate?
$2.73,\mathrm{mm}$
$2.7,\mathrm{mm}$
$2.730,\mathrm{mm}$
$2.7300,\mathrm{mm}$
Explanation
The micrometer's thimble has increments of 0.01 mm, allowing for diameter measurements to the nearest 0.001 mm by estimating one digit beyond the smallest marked division. Precision is assessed by adding the sleeve reading (2.5 mm) to the thimble reading (0.23 mm) and including an estimated digit, resulting in three decimal places. The correct answer, 2.730 mm, appropriately reflects this as it sums to 2.73 mm with an added zero for estimation. Choice D adds an extra zero, implying unsupported precision to 0.0001 mm.
A student times a reaction using two different stopwatches.
Which measurement shows the greatest precision?

Stopwatch Y, because it records to the nearest 0.01 s.
Both are equally precise because their averages are similar.
Stopwatch X, because it has fewer digits and is easier to read.
Stopwatch X, because its values are closer to 12.3 s.
Explanation
Stopwatch Y records to the nearest 0.01 s (hundredths place) while Stopwatch X only records to 0.1 s (tenths place). Precision refers to the smallest increment an instrument can measure, determined by the number of decimal places in the readings. Stopwatch Y provides measurements with more decimal places, indicating greater precision. Choice B correctly identifies that Stopwatch Y is more precise due to its finer resolution, regardless of how close the average values are.
Which measurement shows the greatest precision?
0.1234 kg
0.12 kg
0.1 kg
0.123 kg
Explanation
Precision is determined by the number of significant figures or decimal places in a measurement, with more decimal places indicating greater precision. The measurement 0.1234 kg has four decimal places, making it the most precise among the options. This indicates the measuring instrument can detect differences as small as 0.0001 kg. Measurements like 0.12 kg or 0.1 kg have fewer decimal places and therefore represent lower precision.