The 57-year-old took over at Portman Road in 2012 and successfully guided the Tractor Boys away from relegation worries, before leading them to their first playoffs campaign in 10 years in the 2014-15 season.
Town have made an underwhelming start to the new campaign, however, winning just three of their initial 11 fixtures to sit 16th in the Championship table.
"I heard the reaction after the last game. If they want to get shot of me, and they want change and want to do something else, I can't affect that," McCarthy told the Ipswich Star ahead of the side's trip to Blackburn Rovers on Saturday.
"I'm not bothered. All I can do is try and get back to when they weren't calling for my head – and that's by getting results.
"If we go to Blackburn, get something, then beat Burton on Tuesday night, suddenly we'll have a different feeling. I do prefer it when they are cheering and not jeering, and when they are applauding us rather than booing."
Town had failed to score in more than 400 minutes of football heading into the international break.