This puzzle appears to be a classic lossy ISI problem. We work with these all the time. So much so that we developed a line of high pass filters that are the inverse function of the typical bessel loss of a transmission line. We plot the S-parameters and mark the loss point of the fundamental frequency of the expected data rate. We then use a measured model of this filter part with the channel measurement to show the expected eye using this filter.
This is very easy and took about 3 minutes, 10 minutes tops if you count installing the model library. This is not the tightest solution but is still very effective and the simulation is done from measured models on both ends. Our experience is that 10dB channel loss budget are pretty standard but this one is 15dB. The simulated TDR of the trace shows an impedance discontinuity that is most likely a via stub that pushes the loss down and out of budget.
Editor’s Note: Sanjeev’s solution to the puzzle he posed is here.