Digital Oscilloscope Selection: Resolving Nanovolt Small Signals with 12-Bit Resolution
Direct Answer & Executive Summary: A detailed guide to selecting true high-resolution oscilloscopes for low-amplitude signals, power-rail ripple, and sensor measurements.
Last updated: 2026-09-22
Quick answer
A 12-bit oscilloscope can improve the usable vertical detail of low-amplitude measurements, but converter bits alone do not create a meaningful nanovolt measurement. The actual result depends on full-scale range, front-end noise, probe loading, bandwidth, input termination, shielding, acquisition mode, and the DUT. Select true high-resolution hardware when the measurement requires more code density and low baseline noise at the required bandwidth; use averaging and digital high-resolution modes carefully when transient capture remains important.
1. What vertical resolution actually means
An ideal 8-bit converter has 256 code levels; an ideal 12-bit converter has 4,096. At the same full-scale setting, the nominal quantization increment of the 12-bit converter is sixteen times smaller. That helps the display and measurement engine represent small changes without making the user expand the signal until a larger waveform clips.
The converter calculation is only one component. With a 1 V full-scale range, the ideal code increment is roughly 3.9 mV at 8 bits and 0.24 mV at 12 bits. The instrument's actual noise, offset, gain error, probe noise, and environmental pickup may be larger than either value. Do not turn this calculation into a claim that the scope can measure a 0.24 mV signal accurately under every setting.
2. Native high resolution versus processed high-resolution modes
Some oscilloscopes use an ADC with more native bits. Others produce a high-resolution display mode by averaging or filtering multiple samples from a lower-resolution converter. Both approaches can be useful; they are not equivalent. A processed mode may reduce displayed noise but can reduce bandwidth, alter the response to a short transient, or require a repetitive signal.
| Selection question | Native high-resolution acquisition | Processed high-resolution mode |
|---|---|---|
| Code density | Defined by converter architecture | Derived from acquisition/filtering method |
| Non-repeatable glitches | May be retained if capture path supports them | Can be smoothed or missed |
| Bandwidth | Verify at selected resolution/rate | Verify the mode's bandwidth penalty |
| Acceptance evidence | Noise and ENOB under stated conditions | Filter, record length, and repeatability documented |
Ask the vendor for effective number of bits or noise performance at the sample rate, bandwidth, channel count, vertical scale, and termination that resemble your work. A high bit number quoted at an unusually narrow bandwidth does not necessarily help a fast power stage.
3. Probe and connection geometry often dominate
Low-level signals are easily changed by the measurement connection. A passive probe can impose capacitive load; a long ground lead can form a loop antenna and add ringing; a 50-ohm termination can load a high-impedance source; an unshielded adapter can collect mains hum. Before specifying a premium oscilloscope, define the source impedance, allowable loading, signal reference, voltage to earth, expected bandwidth, and safe probe type.
For high-side or floating measurements, use a differential or isolated probe rated for the actual differential and common-mode voltage. Do not use the ground clip of a normal earth-referenced probe as a convenient second signal lead. The resulting short can damage the DUT and create a safety incident.
4. Applications that justify the extra detail
True high-resolution acquisition can be valuable for low-level sensor conditioning, power-rail ripple, current-shunt work, audio/precision analog evaluation, and measurement of small deviations superimposed on a larger signal. In switching power work, it should be paired with a suitable differential probe, short connection geometry, and a stated bandwidth limit. In sensor work, low noise may be more valuable than multi-gigahertz bandwidth.
For a broader scope selection workflow, read How to Choose an Oscilloscope. For a power-specific setup, see oscilloscope selection for switching power supply testing.
5. Build an acceptance test that reflects the DUT
Require a baseline-noise screenshot and waveform export using the proposed probe, termination, vertical range, bandwidth limit, sample rate, record length, and channel count. Then test a representative low-amplitude signal or calibrated source. Record repeatability, trigger setting, compensation/zero procedure, and the exact algorithm used for any averaging or filtering. This gives the buyer a defensible comparison between models from Tektronix, Keysight, Rohde & Schwarz, Siglent, Rigol and other suppliers.
Use the AI Blueprint when you can describe the smallest signal, frequency content, voltage environment, and capture duration but need help translating them into a probe-and-scope specification.
Sources for technical review
- Tektronix, Tools to Boost Oscilloscope Measurement Resolution
- Tektronix, ABCs of Probes Primer
