Hold LFO/Envelope Value After Releasing Key

When I map an LFO or envelope to the filter cutoff, as soon as the key is released, the value jumps back up the spectrum. This creates a zwoop sound at the release of each note which is not what I want. The modulation seems to no longer apply as soon as the note is released.

What I would like to have happen is, when I press a key, the low-pass filter slides down the frequency spectrum following the envelope that I have set to modulate it. Then, when I release the key, I want the frequency value of the low-pass filter to remain at the value it was until I press a key again.

Is there a way to accomplish this? Am I missing something obvious?