Target Display Mode with iMac 2013 - only works properly per reboot

Hi all. This is yet another version of this “popular” issue.

So, here’s the thing. I have the Apple product dongle and cable between the two (thunderBolt 3 to Thunderbolt2 conversion). I reset SMC and reboot. Cmmand-F2 works the first time. If I Command-F2 is pressed again to stop Target Display Mode that works too. From then on, if Command-F2 is pressed the iMac display is barely visible with no way to increase brightness. The MacBook is aware it is there because Display Arrangement shows it as an extra display. However the touchBar only shows Brightness Control for just the MacBook this time. That is one symptom. The other one is that - after a while (!) the Command-F2 does indeed start TDM but only for a few seconds before it reverts back.