precision
PrintThe measure of the stability of an instrument and its capability to give the same measurement over and over again for the same input signal.
Precision is generally considered to consist of a short-term (repeatability) and long-term (reproducibility) component, as given by:
Precision = 1- |Xn - Av(Xn)|/|Av(Xn)|
where Xn = the value of the nth measurement
Av(Xn) = the average value of the set of n measurements
For instance, if you are monitoring a certain constant voltage, let’s say 1 V, and you notice that your measured value changes between measurements, let’s say by 20 m V, then your measurement precision is:
Precision = (1- 20 m V /1V) *100 = 99.998%
This specification is most valuable when you are using the voltmeter to calibrate a device, or when you are performing relative measurements.
%
Additional References
Helpful Web Sites:
Reader Comments | Submit a comment »
Legal
This tutorial (this "tutorial") was developed by National Instruments ("NI"). Although technical support of this tutorial may be made available by National Instruments, the content in this tutorial may not be completely tested and verified, and NI does not guarantee its quality in any way or that NI will continue to support this content with each new revision of related products and drivers. THIS TUTORIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND AND SUBJECT TO CERTAIN RESTRICTIONS AS MORE SPECIFICALLY SET FORTH IN NI.COM'S TERMS OF USE (http://ni.com/legal/termsofuse/unitedstates/us/).