The Three Lions spurned numerous chances during the game but came up trumps in injury time when Kane headed home from close range following Harry Maguire's flick-on from a corner.
"It's massive," Kane told BBC Sport at full time in Volgograd. "I'm so proud of the lads. It's tough. We played so well especially in the first half and we could have scored a few more. We kept going. It's a World Cup, you go to the last second. I'm absolutely buzzing.
"We've done it for a while (had good resilience) since the gaffer has been here - he's instilled it into us. We've got a great bond off the pitch so it's great to see it on the pitch. We'll get onto the plane happy tonight.
"We could have had a couple of penalties, especially when you look at theirs. A few corners, they were trying to grab, hold and stop us running. Maybe a bit of justice to score at the back post at the end. That's football, that's the ref. It showed good character to get on with it.
"I'm really proud. We did a lot of work on training field and were itching to play. Get the nerves and excitement out of the play."
England play Panama, who lost 3-0 to Belgium earlier on Monday, in their next Group G game on Sunday.