How does the attenuation algorithm in the NFA work?

Once calibration ([Calibrate]) is complete the NFA attenuator algorithm will set the attenuator to the best value for the noise level coming in. If you have a noise source connected directly to NFA input the best value of attenuator will be 0dB.

In the case when your DUT is a high gain DUT and your minimum attenuation is set to > 0dB under [Correction] {Input Calibration} {Min RF Atten} or [Correction] {Input Calibration} {Min Microwave Atten}, this 0 dB attenuation happens to be uncalibrated, and it warns the user of such a condition. See the image for RF range (< 3 GHz) immediately after calibration when the minimum attenuation is set to > 0dB. It would look the same for the microwave range (3 GHz-26.5 GHz). Image for NFA FAQ

In the RF range (<3GHz) the NFA auto ranges its attenuators. If the user, after calibration, uses the high gain DUT for measurement, the auto ranging attenuation setting would then set for more attenuation in place and the user would get correctly calibrated results back.

The attenuation can also be set to fixed attenuation under [Sweep] {Manual Measurement} {RF/Microwave Att} fixed and enter a value under [Sweep] {Manual Measurement} {More 1 of 2} {Fixed RF Att}

However, when working in the microwave frequency range of 3.0 GHz to 26.5 GHz, the NFA does not auto-range the attenuators. Hence, in case of a high gain DUT, you will have to manually set the microwave input attenuation to avoid overdriving the NFA under [Sweep] {Manual Measurement} {RF/Microwave Att} fixed and enter a value under [Sweep] {Manual Measurement} {More 1 of 2} {Fixed Microwave Att}