Timebase setting and frequency resolution

The FFT converts a time domain waveform into a frequency domain spectrum. The theoretical frequency resolution is calculated as reciprocal of duration of the time domain waveform. Figure 1 shows this relation at a 5ns/div timebase setting that provides 50ns as duration of the waveform. The reciprocal of the 50ns becomes 20MHz as shown as ∆f in setting dialog box in Figure 1. Spectrum as result of FFT is as shown in the figure.

Figure 1:

Frequency resolution @ 5ns/div

Of course, if timebase setting is changed, the frequency resolution is also changed. Figure 2 shows the same waveform and spectrum but at 10ns/div.

Figure 2:

Frequency resolution @ 10ns/div

Window function

“Rectangular” window is used in Figure 1 and Figure 2 to maximize frequency resolution. Those measurements were done on a 500MHz sinusoidal signal that has single line spectra at 500MHz. The FFT assumes that the captured waveform is repeated forever. In this case, 500MHz signal has just 50 cycles in the 100ns time window. So that end of the waveform and beginning of the waveform can be connected smoothly. Then FFT treats this waveform as continuous a sinusoidal signal and provides single sharp spectra. This operation is shown in Figure 3.

Figure 3:

FFT operation on signal with integer number of cycle. There is no discontinuity at the junction.

However, this condition is not found in general. Most actual signals do not give an integer number of cycle in the time window. Generally in this condition, a signal cannot be treated as a continuous sinusoidal signal as discussed above. For instance, if the signal frequency was 495MHz, it gives 49.5 cycle in 100ns time windw. Then a large discontinuity exists at the connection point. This discontinuity impacts on the spectrum result of FFT operation. Energy of the discontinuity at the connection point is spread out in a very wide frequency range that makes side lobes in the spectrum. With a 100ns time window, the frequency bin (∆f) of the FFT is 10MHz as seen in Figure 2. The 495MHz component is just between 490MHz bin and 500MHz bin. So the energy of the signal is split into two bins, then the peak of the spectral line is lowered this is called ‘picket fence’ effect. This operation is shown in Figure 4.

Figure 4:

FFT operation on signal with non-integer number of cycle

To reduce side lobe and picket fence effect, a window function is applied. Figure 5 shows effect of Hanning window. Window function is shown in red on upper left grid over the signal in yellow. The signal is multiplied by the window function. The result is shown in blue on lower left grid. FFT of the blue waveform gives spectra in pink on lower right grid. It is obvious that the window function reduces side lobe effect compared with FFT result of without window function in yellow on upper right grid.

Figure 5:

Effect of window function (Hanning)

There are several window functions available in the market for different applications. Hanning (Von Hann) is the most common), Hamming, Flat-top and Blackman-Harris are compared in Figure 6 on 500MHz sinusoidal signal. Rectangular provides the narrowest spectra, while Flat-top gives the widest spectra.

Figure 6:

Comparison of window function on 500MHz sinusoidal signal

Figure 7 shows same comparison but on 495MHz sinusoidal signal. Rectangular shows the worst case side lobe effect while Flat-top keeps almost same shape of spectra. It should be obvious that side lobes and frequency resolution of the spectra are a trade-off.

Figure 7:

Comparison of window function on 495MHz sinusoidal signal

Picket fence effect is compared in Figure 8. Peak values of spectra with different window are plotted from 450MHz to 550MHz with 500kHz step. Flat-top weighting keeps almost the same value along with the frequency range, while Rectangular shows about a 4dB variation.

Figure 8:

Comparison of window function peak variation from 500Mhzto 600MHz

FFT Algorithm

The Cooley-Tukey Algorithm is widely used for FFT operations. It provides the fastest calculation speed on data records which are a power of two in length. However, it requires a reduced number of data points relative to the digital oscilloscope’s memory length. In Figure 9, spectra of 500MHz sinusoidal signal at 10ns/div timebase setting is shown. Scope acquires the signal at 20GS/s (50 ps resolution) 2,000 points. The next lower power of two to 2,000 is 1,024. So, the software takes 1,024 points at its center (As shown in light blue rectangular) and applies FFT on the data. Time window of 1,024 points is 1,024 x 50 = 51.2 ns that the 500MHz signal takes 25.6 cycles. Then it shows side lobe and leakage effect as Figure 9.

Figure 9:

FFT result with “Power2” algorithm (Truncation)

It results in a 19.35125MHz as shown in setting dialog box. In Figure 10, LeCroy’s other algorithm called “LeastPrime” is used. It can process data with whole memory length as combination of power of two and power of five. In this example, 2,000 point is 24 x 53 . So that, “LeastPrime” does not need to be truncated and gives minimum as 10MHz as shown in figure 2. To utilize the whole data record in the scope memory using the power of 2 algorithm, data filling is used. The next higher power of two to 2,000 is 2,048. To apply FFT on 2,048 points of data, 48 points of the data must be filled. To avoid discontinuity, last point and first point of the 2,000 point wave form are connected in the 48 points as shown in figure 10 as orange line. It gives finer as 9.76563MHz. However, it treats the signal as burst signal with 100ns duration and 102.4ns repetitive period. So that, truncation is recommended in general, Data filling is recommended for burst type of signal.

Figure 10:

FFT result with “Power2” algorithm (Data Filling)


There is common misunderstand on relationship of memory length and frequency resolution. When a signal has short duration, it gives fewer data points. The FFT result of the signal also has fewer points with coarser frequency resolution. In this situation, users expect an interpolation operation to improve the frequency resolution. They may apply interpolation in time domain. Using the same time window, the frequency resolution cannot be improved. Interpolation in time domain creates a higher effective sampling rate that gives a wider frequency range as shown in Figure 11. On the left hand side shows FFT on data with short duration. On the right hand side shows FFT on data interpolated in time domain. Right middle trace is full range of FFT as a result of FFT on interpolated waveform. It has much wider bandwidth than its original one in left. All additional points in time domain were consumed in higher frequency region. The spectrum in same frequency region is identical as lower right trace that is zoom of full range of spectrum in interested frequency range. Interpolation in time domain does not help anything in the interesting frequency region.

Figure 11:

Interpolation in time domain

Figure 12 shows effect of interpolation in frequency domain. It cannot make spectra narrower, but it may improve the precision of reading of peak. Actual signal frequency is 955MHz. The interpolated spectrum gives 955MHz as P2, while the Original spectrum gives 951MHz as P1.

Figure 12:

Interpolation in frequency domain

FFT on Selected Region of Waveform

Sometimes a specific region of waveform of interest for spectral analysis. Figure 13 shows two methods to analyze specific region of waveform.

Figure 13: FFT on specific region of waveform

On the left hand side FFT is applied on “Zoom” trace that expands the specific region. It produces a similar situation as the FFT on waveform with short duration discussed previously. On the right hand side, a gating signal is produced as shown on the upper right trace and it is uses as a mask waveform. Masked waveform is shown as right middle trace. FFT result of the masked waveform is shown as lower right trace that gives much precise frequency resolution. However, this situation is similar to “Data Filling” discussed above. Frequency component of the signal is identical. Light blue trace over FFT result of the masked waveform is copy of FFT result of zoomed trace. The FFT output of the masked waveform is interpolated more effectively.

This situation is similar to FFT with “Zero Filling” discussed above. Using the “Hanning” window instead of a rectangular pulse for a gating function could be used for a better result as Figure 14. FFT on “Zoom” trace is now use “Von Hann” window on the left for comparison.

Figure 14: simulated “Hanning” Window for FFT on specific region of waveform


The FFT is a very well known calculation offered in many digital oscilloscope’s as standard function. Hopefully, this paper may help obtain the most meaningful data.