Okay - for everybody who is - like me - a bit stupid and runs into the same problem, here is the fix: Choose only one soundpack for your track and then start to work. Sorry for littering the forum. ;-)
Hey Tipota, thanks for this report.
This is a memory issue that can vary in frequency between iOS devices and between different NOISE soundpacks, depending on their size and the amount of memory each soundpack requires. To avoid having to re-load as much as possible, we recommend reducing the number of tracks running NOISE in GarageBand and re-loading when the prompt appears, as this ensures device memory is then allocated as efficiently as possible.
Whilst we don't have any set date on any potential improvements to this behaviour, we appreciate your feedback - if you encounter any further issues, please don't hesitate to contact the support team.