Experimental Study of MLSE Receivers in the Presence of Narrow-Band and Vestigial Sideband Optical Filtering

We report on experimental investigations of real-time maximum likelihood sequence estimation (MLSE) in the presence of narrow-band optical filtering, using 10.7-Gb/s non return-to-zero on-off keying (NRZ-OOK) and a fiber grating filter with 6.25 GHz bandwidth. Compared to standard threshold detection, the MLSE eliminates a 10-3 error floor due to narrowband optical filtering and results in a 2-dB optical signal-to-noise ratio (OSNR) penalty at a 10-3 bit error ratio. Furthermore, we demonstrate the ability of the MLSE to simultaneously compensate for narrow-band optical filtering and chromatic dispersion. Finally, we investigate the effect of narrow-band filter frequency detuning and show that the vestigial side-band (VSB) effect observed in standard threshold detection disappears in the presence of the MLSE.