With the victory, the NBA-leading Bucks (37-6) extended their winning streak to five games.
Kemba Walker scored a game-high 40 points to lead the Celtics, who took the floor without Jaylen Brown. The 6-foot-6 forward suffered a sprained right thumb in a 13-point loss to Detroit on Wednesday.
“We saw a play late in the first half and we think that might have been when it happened,” Boston coach Brad Stevens said. “Maybe it got worse as the game went on. He didn’t say much about it at halftime. Certainly, after the game it was sore. Today, more sore.”
Stevens said they will do more tests on Brown’s thumb on Friday.
Marcus Smart, who started in place of Brown, had 24 points.
Antetokounmpo, who also had seven assists, had his 35th double-double of the season.
After trailing by as many as 27, the Celtics outscored Milwaukee 36-22 to pull within 127-123 with 38 seconds to play. But Smart missed a 3-pointer and Middleton knocked down one of two free throws to seal the victory.
Antetokounmpo scored the first basket of the night and the Bucks never trailed.
Milwaukee hit seven of their first 10 shots, including their first four attempts from 3, and led 36-20 after the first quarter. Brook Lopez had three of those 3-pointers and had 11 points in the opening quarter. Lopez finished with 16 points.
Milwaukee set a new season-high with 76 points in the first half. The Bucks had a 76-58 lead at the break.
Boston stormed out in the second half, cutting Milwaukee’s lead to 87-81 with 5:16 left in the third. But the Bucks ended the quarter on a 19-6 run.
Gordon Hayward, who finished with seven points on 1-of-10 shooting for the Celtics, scored his only field goal on a 3 at the end of the third.
Donte DiVincenzo had a career-high 19 points off the bench for Milwaukee.