I have rules for myself.

1. If I change a major app (productivity or otherwise) I have to keep using it for minimum 1-2 years, unless it is so badly broken that I cannot (losing data, major bug, etc.) I don't really learn all of the tricks and how to make my system as efficient as possible if I switch before the one year mark.

2. If I see a shiny new app that I like, I need to identify the "shiny" features that are really attracting me and decide (a) are that/those shiny features enough to make a major impact on my productivity or do I just want to play with them? (b) if it would make a major impact to my productivity is there a way to use that feature/a similar feature in my current system?

Switching apps takes proper planning and is a big commitment. Lots of hours to set something new up. Lots of effort to get everything transferred from the old system to the new system. That's a big deficit that needs to be filled by the improvements (hopefully) found in the new system.