@tenderlove for Mastodon we started using security advisories to get a private fork and it improved things a little bit. At least we get a branch per version, and CI runs on them.