Would they have actually thought the other group to be more different than other groups of their own "species"? (I'm actually not sure you can really call these human subtypes different species when they could and did reproduce with fertile offspring, but from what I gather the species concept overall just breaks down at certain points when you poke at it anyway.)
I mean, sure when you look at Neanderthal reconstructions they look somewhat different from us, but not different like a chimpanzee, and modern humans among ourselves also have quite a lot of variation (though I'm not sure how much of that had developed at what time), not just skin color, hair and eye color but facial features, size etc. leading to the whole "human races" construction, and the difficulty to get to a consensus that we are in fact all fundamentally the same. So if you are an early Homo Sapiens and see a Neanderthal would you think them really "other" or just somewhat "other" (i.e. like some unrelated other clan of Homo Sapiens)? They also had tools, fire and probably art too after all.
Of course we probably won't ever know, and afaik they haven't found sites of mixed groups that might be a sign that the mixing was even somewhat consensual. And there doesn't seem to have been that much mingling considering both kinds shared Europe several thousand years.
So then another interesting question is of course whether the later Neanderthals were aware that they were slowly going extinct. I imagine they must have been on some level, as their groups got rarer and modern humans more numerous, but then each group might think it a local problem?
I feel like there should be more speculative fiction about this. The examples with Neanderthal appearances I've read were either older (like Auel) and/or more like SF with alternate timelines where they didn't die out (like Neanderthal Parallax) and such.