Jurgen Klopp's side emerged victorious in an end-to-end game at the King Power Stadium after Simon Mignolet saved Jamie Vardy's second-half penalty to keep the scores at 3-2.
Mohamed Salah and Philippe Coutinho had given the visitors a two-goal lead on Saturday before Shinji Okazaki pulled one back for the Foxes.
Henderson then made it 3-1 but Vardy scored then missed a penalty as Liverpool brought a run of four games without a win to an end.
Speaking to 'BT Sport' after the game, Henderson said: "We dug in right until the end and showed great character to see the game out. We can do better but it's still an improvement.
"I felt we defended well for the majority of the game and obviously Si (Mignolet) made a brilliant save from the penalty.
"We played some brilliant stuff, especially first half when we were two goals up, which we deserved.
"I think overall we're delighted with the result in the end as well as the performance and we're looking forward to the next game now."
The defeat meant Leicester have only managed one win from their opening six Premier League games but boss Craig Shakespeare believes his side should have taken something from the King Power Stadium.
"Very, very disappointed not to come away with something," Shakespeare said.
"Ultimately because we were searching for that second goal it cost us. We left ourselves too open.
"The third goal was always going to be hard to come back from. But try we did and we could have got a point in the end."