The news comes exactly a year after the Italian helped secure the club their first Premier League title.
A statement from the club read: “This has been a difficult decision for the owner, chairman and board to make.
“And it is the outcome of a planned end of season review process that has been brought forward in light of recent speculation and out of respect for Roberto and his extensive contributions to the football club.
“Despite everyone’s best efforts, the club has failed to achieve any of its stated targets this year, with the exception of qualification for next season’s Uefa Champions League.
“This, combined with an identified need to develop a holistic approach to all aspects of football at the club, has meant that the decision has been taken to find a new manager for the 2013/14 season and beyond.”
Chairman Khaldoon al Mubarak said: “Roberto’s record speaks for itself and he has the respect and gratitude of Sheikh Mansour, myself and the board for all of his hard work and commitment over the last three and a half years.”
It has been widely reported that Malaga coach Manuel Pellegrini has been lined up as a replacement for Mancini.
But the Chilean insisted he had not spoken with City and said no agreement had been reached.
Assistant manager Brian Kidd will take charge for the remaining two games of the season and the post-season tour to the US.