First of all, the vast majority of tourists rarely if ever do pure sightseeing trips. Most will plan their holidays around beach activities or winter sports. While it's technically possible to do both in Poland, it's obviously not a prime destination for either.
Second, among those who do culture trips Poland will rarely be the first choice. Its rife with history and sights, but not quite in the same league as say France or Italy.
Third, Poland lacks a proper hot spot city. People tend to maximize their "touristic efficiency" by going for huge ones first: London, Paris, Rome, etc and branching out later. Warsaw is very dull and most of Poland's charming sights are spread out across the country and are more suited to roadtrip-style tourism, which isn't everyone's cup.
Fourth, it's not marketed very well. Czechia was already known as a trendy tourist spot 20 years ago and it's because they put a lot of work into making it happen. For Poland, I'd reckon more people have seen "invest in Eastern Poland" memes than any kind of genuine promotional material.
Lastly, visiting Poland just doesn't sound very sexy. It might be delightful and it might be good for your, but just like having steamed broccoli for lunch, there's not much oomph to it.