So I thought I finally got it. I managed to send bytes over spi to the atmega, I converted sid dumps to appvars, but then disaster struck...
Sometimes the atmega is in a timer interrupt when a byte should be processed by the SPI interrupt. Can I just enable interrupts inside the timer2 interrupt or would this cause problems?

Timer2 updates the SID emulator at a sample rate of 16kHz.

Edit: Solved it temporarily by polling the SPI instead of using interrupts for it.

I am no Atmega interrupt expert, but enabling one interrupt only while inside a second interrupt sounds like a dangerous proposition to me. Did you have any ideas for a longer-term fix?

Yes, I just poll the SPI registers now. This way data transfer is slower, but more reliable. I just need to add a little bit of delay in between writes on the calculator side. I could also just write an interrupt routine on the calculator that writes one byte at a time.

Well, TI calcs do that and other z80 systems, too. (ON handling, NMIs, etc.) Prizm afaik does it (More general, SuperH), and I am guessing PCs with IRQs. The list could go on with priority-based interrupts.