The most of the rape in the series is mostly implied off-screen and we're just shown that the goblin's female captives have been abused by them. There's only really around 4 really explicit rape 'scenes', The Fighter in first goblin cave (the one that everyone is about), the group of female adventurers in the goblin forest base, Goblin Slayer's sister and his village, and finally maybe the Sword Maiden's backstory. While all the adaptations I know of do have rape in story, it's wildly overplayed by some people that assume that it's just full of it.
This has left the impression to some people that have just heard of the show to label it off as an edgy it is really edgy and that it's filled of rape. Just recently the first episode of the anime adaptation was released and the show has already come under fire on it's rather now infamous rape scene of one of the minor characters. So if you haven't heard about Goblin Slayer, it's a pretty straight forward story about a guy named Goblin Slayer (all the character names are refer to their occupation) and he really fucking hates goblins.
(Disclaimer, I haven't read the LN for Goblin Slayer and I've only read the manga to up to the sewer attack.)