Rovers are 23rd in the league standings after 20 games and in danger of being relegated to the fifth-tier National League, which is largely semi-professional.
Horseman was sacked after Tuesday's FA Cup second-round loss at Blackpool.
Deeney, who spent the majority of his playing career at Watford and led them to Premier League promotion twice, joined Rovers as a player-coach earlier this season from Birmingham City.
"I know the fans will have many questions, which will be answered as soon as possible. We will not hide from the task at hand and will be open and honest along the way," Deeney said in a statement.
"One thing I can assure everyone is that we will work our very hardest to bring this wonderful club back to success.
"This is an exciting new role with many challenges that I'll embrace and there's a lot of work for us to do, which has already started."
Deeney's first game in charge will be at home against Gillingham on Friday.